I have been struggling on getting reliable connectivity to China (AS4837 China Unicom Backbone, AS4134 China Telecom Backbone, AS4538 China Education and Research Network Center)

On most links I have used I see 300ms+ RTT with high jitter and 50% packet loss is not rare at all. Some links even has RTT over 1000ms during busy hours.

Chinese ISPs tend to send non-premium traffic to oversubscribed links.

I am trying to work around it by having multiple servers in US and China, and hope each US server have acceptable connectivity to at least one server in China (e.g. AS6939 Hurricane Electric with AS4134 China Telecom Backbone). For a US customer that wants to access resources in China, they connect to my servers in US and my servers make the "routing decision" which server in China to connect to. I am running the transport in some sense.

Is this feasible? At this point I cant buy premium links in China due to monetary and licencing constraints (even CDNs find it hard to get reasonable priced bandwidth, not to mention the difficulty to obtain an telecom operator licence in China).

What kind of routing options do I have to make routing decisions in my transport network? How can it scale if I mesh site-to-site VPNs?

Any encryption I can use? AFAIK OpenVPN does not work on AS4837 China Unicom Backbone because TLS handshakes are getting dropped by China's GFW. Double encapsulation with things like stunnel works for now but it really hurts performance (traffic encapsulated with tcp within tcp).

BGP peer with upstreams with good connectivity to China and some BGP traffic engineering sounds like the right approach but that's far over my budget for now.

My goals is to optimize residential users' experience accessing content far away. Am I looking in the right direction to solve the problem?

This is the challenge...RTT between US and China residential connections


I am not the content, I just want to provide transport.

Edit1:AS4538 China Education and Research Network Center is IPv6 enabled and IPv6 performance there is usually bettbetter than IPv4. Connectivity between those 3 Chinese ISPs can be very poor sometimes(300ms+ RTT with high packet loss to another ISP in the same city, slower than China ISP1-US-China ISP2). And my users can be on any of these 3 ISPs.

  • why don't you just get a China server that uses BGP connectivity? – lamp_scaler Jun 9 '13 at 13:56
  • I would look to peer with those three ISPs in China directly. – jwbensley Jun 11 '13 at 10:33
  • They have very restrictive peering policy and prices are outrageous. – sdaffa23fdsf Jun 12 '13 at 2:38

Seems potentially wide topic. But I interpret it as two questions:

  1. How to use jitter/packet loss/delay as best path selectors
  2. Practical solutions to encrypt this traffic

For first question, there are external commercial solutions which monitor links and optimize your BGP, benefit of them is that they are mostly vendor neutral, as long as you do BGP, they should work. I cannot recommend any, as I've not used such solution personally.
I hope someone else will give rundown of what is available in the market and which is recommendable product.

Then there is Cisco's PfR (was colled OER before), which will use link which most closely satisfies your performance requirements. It is quite nice, while completely proprietary and not available throughout their routing portfolio.

For encryption question, I'm not really sure what to tell, I think IPSEC links we have there to connect some remote factories to customer's L3 MPLS VPN just work, without GFW interfering.

It might be useful to include in your post rough pps/bps limit what need to be supported, what equipment you are today using and how much CAPEX you have to spend on the new solution.

  • Rhetorical: How can the GFW allow encrypted tunnels? That would make it gFW. – generalnetworkerror Jun 9 '13 at 8:59
  • 'G' as in great. Not sure what 'g' in your context would mean. But maybe the reason why we're not affected is because it's transmission (L2) link (via Russia), not global L3 INET. But really interesting question, I think it would deserve its own post, where/how does GFW interfere. – ytti Jun 9 '13 at 9:03
  • Little 'g' as is not-so-great if the Great FireWall allows encrypted tunnels to bypass their filtering. And posting about GFW (or PRISM) will surely bring someone to your digital doorstep. – generalnetworkerror Jun 10 '13 at 6:41

Not the answer you're looking for? Browse other questions tagged or ask your own question.