why TCP window size is restricted to 1GB even with TCP window-scaling option?

Question

TCP window size = 16 bits , TCP scale factor size = 14 bits. So, overall the max TCP window size can be 2^30 = 1GB.

For networks where the latency is high like 500 milli-second. 1GB of TCP window size is also not enough to fully utilise the link's bandwidth. For instance, consider a link with 500 milli-second RTT and bandwidth of 40Gbs(ie; 40/8 = 5 Giga-Bytes/Second). BDP = (500 * (10 ^ -3)) * ((40 *10^9)/8) = 2.5 GB.

Hypothetically, we need a TCP window size of 2*2.5 = 5GB (multiplied by 2 because generally the kernal meta-data reqires half the size of TCP buffer) to achieve the line-rate of 40gbs.

So, the max TCP window size of 1GB is still not enough. So, why restrict the TCP window size to 1GB ?

Answer 1

500 ms RTT is very much on the high side. On a very long backbone actually spanning half the globe I wouldn't expect an RTT larger than perhaps 250 ms, probably more like 200 ms or even less. And you likely won't find any real 40G connectivity across carriers.

Also, as Ron M. has pointed out, 40G is normally used with an aggregating link, carrying a large number of simultaneous TCP connections and other traffic. It's not really relevant whether a single TCP connection could saturate that link.

What is relevant is that your application requirements are met and there's no congestion on the key links. Also, if your application actually requires such a throughput you could simply run multiple TCP connections. (And the situation becomes "worse" with up to 400G links being currently available and up to 1600G coming up).

Effectively, the 1 GB maximum limits the throughput of each TCP connection to 1 GB / RTT, regardless of a potentially higher path bandwidth. That really becomes a problem with extremely high RTTs, e.g. in future communication between Earth and Moon or even between planets, but these are limited by being wireless anyway.

Answer 2

RFC 1323 TCP Extensions for High Performance, 1.3 Using TCP options indicates that TCP window size is restricted to 1GB bcs of backward compatibility and overhead considerations.

1.3 Using TCP options

  The extensions defined in this memo all use new TCP options.  We
  must address two possible issues concerning the use of TCP
  options: (1) compatibility and (2) overhead.

  We must pay careful attention to compatibility, i.e., to
  interoperation with existing implementations.  The only TCP option
  defined previously, MSS, may appear only on a SYN segment.  Every
  implementation should (and we expect that most will) ignore
  unknown options on SYN segments.

Transmission Control Protocol (TCP) is a connection oriented transport layer protocol assuming packet loss and provides features such as flow-control, connection establishment, and reliable transmission of data.

Remember that overall link/path utilization is best achieved at the application layer, not the transport layer protocol. If you want to maximize link utilization, you might utilize multiple TCP connections.

With respect to RFC 7323 TCP Extensions for High Performance which defines the TCP Window Scale (WS) option

The maximum receive window, and therefore the scale factor, is determined by the maximum receive buffer space. In a typical modern implementation, this maximum buffer space is set by default but can be overridden by a user program before a TCP connection is opened. This determines the scale factor, and therefore no new user interface is needed for window scaling.

The scale factor is limited to a power of two and encoded logarithmically, so it may be implemented by binary shift operations. The maximum scale exponent is limited to 14 for a maximum permissible receive window size of 1 GiB (2^(14+16)).