2

I have a home server in New Zealand and a remote server in Amsterdam of the following configuration:

NZ to Amsterdam network diagram

I have a simple test script:

#!/bin/bash

for i in `seq 1 20`;
do
    scp root@remoteserver:~/testfile.txt .
done

Example output:

testfile.txt                                  100%   10MB 764.9KB/s   00:13
testfile.txt                                  100%   10MB 810.6KB/s   00:12
testfile.txt                                  100%   10MB 810.5KB/s   00:12
testfile.txt                                   48% 4960KB  11.6KB/s   07:36 ETA

testfile.txt is 10MB of random data. It seems that approximately 1/4 of the transfers will be locked in at around 10KB/s. The rest will average around 800KB/s. Now, I certainly don't expect to saturate a 1 Gigabit pipe with a 300ms RTT (and an untuned TCP stack), so the 800KB/s cap is to be expected. What isn't expected is the random streams that complete at 10KB/s. Can anyone explain this behaviour and perhaps suggest a solution?

A few bits of extra context:

  • The local and remote server are both running Ubuntu. I have tried different versions to no avail.
  • This has only been a problem for the last couple of weeks. Prior to that, I never encountered such strange behaviour.
  • Either the RTT or the length of the link seem to have an impact on this. The same behaviour is not witnessed on a server operating out of Australia (i.e. ~60ms RTT).
  • Switching to a different European country doesn't seem to make a difference (i.e. data centers in France and Germany have the same problem).

I'm asking on Network Engineering because I believe this may come down to the infrastructure of the internet or my network as opposed to some configuration issue at either endpoint.

  • I wonder if this is a case of congestion control algorithms getting confused by random packet loss. IIRC TCP increases speeds exponentially until the first packet loss is detected but only linearly after that. – Peter Green Nov 3 '18 at 7:20
  • "this may come down to the infrastructure of the internet" Unfortunately, questions about networks you do not directly control are off-topic here, as are host/server configurations. If you believe it is your own network, then you need to provide (at least) a good description or diagram of the network, the network device models, and the network device configurations. Speculation and guessing are off-topic here. See the Network Engineering Question Checklist for guidance, then edit your question to include enough information. – Ron Maupin Nov 3 '18 at 9:46
  • Did any answer help you? If so, you should accept the answer so that the question doesn't keep popping up forever, looking for an answer. Alternatively, you can provide and accept your own answer. – Ron Maupin Dec 25 '18 at 9:51
2

One of my clients has traffic between UK, Ireland, and Singapore; frequently there are spectacular slowdowns of the kind you describe. My understanding is that a lot of traffic between Europe and Australian and New Zealand goes through Singapore, and there can be considerable congestion there. (Even if not the case for you, you may have analogous issues.)

  1. Routing: One problem was caused by poor routing of the UK site. Routes transitting through NTT were taking approx 190 ms RTT and with relatively low variation, through PCCW were taking 330 ms with very high variation up to about 1000 ms. Additionally there apparent bandwidth limitations. (For comparison, AWS Singapore to AWS London 165 ms; theoretical minimum 105 ms.) Only sometimes was that related to this client's traffic. Attached graph for illustration (measuring from AWS Singapore to client's site in UK). Suggestion: log RTT and look for patterns, examine route (with traceroute) carefully

enter image description here

  1. Server throttling: Another problem has to do with the servers in Europe. Some rented virtual private servers have limits which are triggered by usage. I've certainly seen very severe throttling on very cheap VPS, on network traffic, on CPU usage, and on "disk" throughput. Suggestion: repeat tests on expensive VPS or physical hardware.

Perhaps these suggestions will give you a direction to try.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.