How is the payload distributed across Multilink PPP connections

Question

A site I support uses 3 T1s setup with multilink PPP. They are trying to use Jive which is a hosted VoIP provider, with horrible results. I want to know how packets are distributed between the individual T1s since I think this may help explain what is going on.

SNMP monitoring of the multilink interface shows that they have available capacity, but their VoIP test calls are horrible. It acts as if there is a huge amount of jitter and dropped packets. Though simple tests with PING/Iperf to not show jitter/latency that is as bad as you might expect given the call quality.

How are packets distributed between the multiple T1s. I assume that it isn't the same as Ethernet bonding.

If the multi-link PPP is an issue what can I look at on the routers that will show this? Can it be corrected? The routers are Cisco, I believe they are 2800s, but I would have to double check.

Answer 1

Oi...lemme dredge up those long-unused neurons to remember how Multilink PPP works.

During the LCP phase when the first link is brought up during a non-multilink PPP session, the two sides negotiate the MRU (Maximum Receive Unit) for each direction of the link...basically each side tells the other how big of a packet it can handle being sent to it.

With Multilink PPP, they negotiate that, but they also negotiate, when the first link is brought up, an MRRU, or Maximum Reconstructed Receive Unit.

Because Multilink PPP added extra frame header space, the creators were concerned about Path MTU problems, so they decided that Multilink would be able to fragment and reassemble packets on either end of the Multilink PPP session.

So, yes, unlike Ethernet bonding, its not just a matter of balancing frames across the multiple links...the frames can actually be fragmented and reassembled. This can cause a jump in latency and jitter.

On T-1's you should be able to configure each side with a significantly larger MRU and MRRU than any packets that would likely need to cross the link, and if the MRU is as large as, or larger than the MRRU, then you shouldn't see any fragmentation and reassembly occuring. Hopefully this will get the latency and jitter under control and help out your VoIP behavior. You will likely need to adjust this configuration on both sides of the links, though, as each direction is negotiated independently.

In general, I wouldn't expect VoIP packets to need to be fragmented, though...they tend to not be big enough to need that. Worth a shot to check your MRU and MRRU sizes on the links and the Multilink session, maybe they're way out of whack and causing problems.

Answer 2

(haven't used MLPPP since the days before VoIP. in fact I'm looking at an archived config from 2003)

The only thing I can add:

interface Multilink X
  no ppp multilink fragmentation

Each member interface has tx-queue-limit 26; I'm sure I did that for a reason.

Can it be corrected?

If you can get both Cisco ends to do it... try per-packet load balancing:

interface Serial0/0/4:0
 ip address 198.xx.xx.210 255.255.255.252
 ip load-sharing per-packet
 no fair-queue
!
interface Serial1/0/5:0
 ip address 198.xx.xx.22 255.255.255.252
 ip load-sharing per-packet
 no fair-queue
!
ip route xxx.xxx.201.0 255.255.255.0 198.xx.xx.21
ip route xxx.xxx.201.0 255.255.255.0 198.xx.xx.209

That works even better (at least between Cisco's.) In this case, we were forced to do it this way because they were on different linecards. (7500's cannot [read: don't, they get punted to the RSPs] do MLPPP across VIPs)