I work for a VoIP service provider and I am working an issue with a customer who has a cable internet connection that's throwing me for a loop.
He has a single block, which we'll pretend is 70.141.15.0/29, with the gateway at .1 and routers at .2 and .3. Both routers are connected to his cable modem which is, to the best of our knowledge, set to whatever cable providers imagine "bridge mode" is.
I am pinging both these routers simultaneously from the same box, which is a linux system connected to fiber from (probably) Level(3). So needless to say, nobody on the planet knows how many nodes there are between here and there. But check out the ping results.
To the first router:
64 bytes from 70.141.15.2: icmp_seq=2637 ttl=47 time=45.0 ms
64 bytes from 70.141.15.2: icmp_seq=2638 ttl=47 time=39.2 ms
64 bytes from 70.141.15.2: icmp_seq=2639 ttl=47 time=37.3 ms
64 bytes from 70.141.15.2: icmp_seq=2640 ttl=47 time=46.1 ms
64 bytes from 70.141.15.2: icmp_seq=2641 ttl=47 time=45.8 ms
64 bytes from 70.141.15.2: icmp_seq=2642 ttl=47 time=46.5 ms
64 bytes from 70.141.15.2: icmp_seq=2643 ttl=47 time=40.9 ms
From the second:
64 bytes from 70.141.15.3: icmp_seq=631 ttl=239 time=54.7 ms
64 bytes from 70.141.15.3: icmp_seq=637 ttl=239 time=40.5 ms
64 bytes from 70.141.15.3: icmp_seq=638 ttl=239 time=40.3 ms
64 bytes from 70.141.15.3: icmp_seq=639 ttl=239 time=38.4 ms
64 bytes from 70.141.15.3: icmp_seq=640 ttl=239 time=44.9 ms
64 bytes from 70.141.15.3: icmp_seq=641 ttl=239 time=38.4 ms
64 bytes from 70.141.15.3: icmp_seq=642 ttl=239 time=38.8 ms
Check out the TTL values. Does this make sense? These devices are directly adjacent to each other, plugged into that modem via separate switch ports. How can one appear to have nearly 200 more hops? From pinging other sites I'm getting the impression that TTL is just not implemented the way I think it is. I doubt there are 200 hops between me and 4.2.2.2, or woot.com, yet I get under 50 TTL results from both of those.
One of these routers (the one with the higher TTL) is from Fortinet, while the other one is a custom Linux-based device. I'm pretty sure the Forti has a home-rolled network stack, while the Linux box uses whatever came with the source tarball the hardware devs downloaded. Is it likely that ICMP echo is implemented in a bizarre form on one of these and deliberately sends all replies with a TTL of 50?
I also notice that one of the only sites I can find that replies with sane-looking TTL is slashdot, and I can imagine that their servers and routers might be a little less "whatever we found in the garage" than the average website, which sort of makes me feel like I might be on the right track with that last supposition.
To sum up: does TTL on ping mean anything reliable whatsoever?