I just had an argument with a colleague of mine and thought I'd just reach out to the experts on this. Here's the scenario. We were using a website that measures your connection's speed. We tested using a server that is far from us (We are in Malaysia and the server was in US). It was around 2 Mbps. Then we tried with a server in Singapore and it was much faster (around 15 Mbps). My colleague believed it's because of the physical distance while I don't think it matters. My understanding is once you have done the initial handshake and the data flow has started, it doesn't matter where the server is located and the result should be almost the same. Am I missing something here? How does it really work?
My colleague believed it's because of the physical distance while I don't think it matters. My understanding is once you have done the initial handshake and the data flow has started, it doesn't matter where the server is located and the result should be almost the same. Am I missing something here? How does it really work?
Both of you were right at some point in history, but your understanding is mostly correct... today :). There are a few factors that changed between the older answer your friend gave, and the capabilities we have today.
- TCP Window Scaling
- Host buffer tuning
The difference in the results you saw could have been affected by:
- Packet loss
- Parallel TCP transfers
TCP Window Scaling: The bandwidth-delay effect
As your friend mentioned, older implementations of TCP suffered from the limits imposed by the original 16-bit receive window size in the TCP header (ref RFC 793: Section 3.1); RWIN controls how much unacknowledged data can be waiting in a single TCP socket. 16-bit RWIN values limited internet paths with high bandwidth-delay products (and many of today's high bandwidth internet connections would be limited by a 16-bit value).
For high RTT values, it's helpful to have a very large RWIN. For example, if your path RTT from Malaysia to the US is about 200ms, the original TCP RWIN would limit you to 2.6Mbps.
Throughputmax = Rcv_Win / RTT
*Throughputmax = 65535*8 / 0.200*
Throughputmax = 2.6Mbps
RFC 1323 defined some "TCP Options" to overcome these limitations; one of those TCP Options is "window scaling". It introduces a scale factor, which multiplies the original RWIN value, in order to get the full receive window value; using window scaling options allow a maximum RWIN of 1073725440 bytes. Applying the same calculations:
Throughputmax = Rcv_Win / RTT
*Throughputmax = 1073725440*8 / 0.200*
Throughputmax = 42.96Gbps
Keep in mind that TCP increases RWIN gradually over the duration of a transfer, as long as packet loss isn't a problem. To see really large transfer rates over a high-delay connection, you have to transfer a large file (so TCP has time to increase the window) and packet loss can't be a problem for the connection.
Internet circuits across the Pacific Ocean get pretty congested at times. Some of my family lives in Taiwan, and we routinely run into issues when we use Google Talk with them. I often see packet loss over 0.5% when I ping their DSL line from the US; if you're seeing anything like 0.5% loss to the "slower" server, it very easily would limit throughput on a single TCP socket.
Parallel TCP streams
FYI, some speed test websites use parallel TCP streams to increase throughput; this may be affecting the results you see, because parallel TCP streams dramatically increase throughput in case you have some packet-loss in the path. I have seen four parallel TCP streams completely saturate a 5Mbps cable modem that suffered from 1% constant packet loss. Normally 1% loss would lower the throughput of a single TCP stream.
Bonus Material: Host Buffer tuning
Many older OS implementations had sockets with limited buffers; with older OS (like Windows 2000), it didn't matter whether TCP allowed large amounts of data to be in-flight... their socket buffers were not tuned to take advantage of the large RWIN. There was a lot of research done to enable high performance on TCP transfers. Modern operating systems (for this answer, we can call Windows Vista and later "modern") include better buffer allocation mechanisms in their socket buffer implementations.
Short answer: Yes, distance has an effect on single stream bandwidth.
The internet has evolved means of limiting that effect... delayed ACK, window scaling, other protocols :-) But physics still wins in the end. In this case, it's much more likely to be general network congestion over so many hops -- it only takes a single dropped packet to kill a TCP stream.
While there already excellent answers to this, I'd like to add: no, speed is not necessarily affected by distance and yes, very often speed is affected by distance are both true.
Why is that?
Strongly simplified, the longer the distance, the more "hops" are involved on the way through the Internet. The maximum bandwidth is determined by the slowest hop and concurring traffic. With increasing distance and a somewhat random distribution of hop speeds, the probability for getting slower overall speeds is increasing. Additionally, physics gets in the way and increasing latency may also slow down the link.
But this is not to be taken for granted. Technology allows us to build a planet-encompassing connection of nearly any desired bandwidth. However, bandwidth and distance are enemies and both dramatically increase the cost of the connection, again making it less likely to exist just for the connection you might need right now.
Of course, this is oversimplified but in reality, this situation is what you very often find. And then again you don't when there's a surprisingly fast connection or a distribution proxy just around the corner - but when everything's instant we rarely think about the speed of the Internet...
According to Andrew Martin the answer is yes