The short answer is no, that's not the limit.
A TCP Port field is 2x bytes and holds a quantity of 65536. This number limits the amount of addresses a server can have. But this doesn't limit the number of clients to ~64k. Each TCP Packet has two Port fields one for the destination and one for the source (as well as two IP addresses).
A given TCP connection is a tuple of the source and destination, each with IP address and Port number. The destination (the server side) remains fixed, but the source address (the client side) can vary over both Port AND IP Address.
Consider:
- Server IP - 100.0.0.1 (Fixed)
- Server Port - 80 (Fixed)
- Client IP - 0.0.0.0 - 255.255.255.255 (32-bit Range)
- Client Port - 0 - 65535 (16-bit Range)
Yes, a client (or office) with a single IP address, can only connect to your server 65535 times concurrently, but if that client (or office) had multiple IPv4 addresses, they could connect many multiples of that more.
Typically, there are millions of IP addresses in use across many client devices, and if they were to only use a single source port to connect to your single server port, then already you can see the potential to go beyond the 65536 number.
The theoretical mathematical limit of the IP/TCP protocol is 2^32 * 2^16. Practically the number of IP addresses is lower - you would need to subtract for some reserved IP blocks. Also practically, the number of client-side ports is lower because a typical client computer will be running multiple applications connecting to other servers too reducing the TCP port pool, but this pool is seldom used up - once a TCP session has ended the Port number is available in the pool again.
Note: For IPv6, the amount of IP addresses goes way up, but the Port field for TCP remains the same size.