0

The checksum used in Ethernet packets cannot 100% detect errors. If that is the case, how can I ensure that some file I have transferred from one computer to another is uncorrupted?

Thanks

  • Nothing can prevent errors in network transportation. You can reduce the error possibility to a vanishingly small number with error detection at different network layers (data-link, transport, application). The problem is to have small, fast checks at the layers. Even if you had a checksum that is a complete copy of the data, there is still the possibility that the data and copy of the data could have the same error. The odds of that are very small, but you double the data transferred, and that is impractical. – Ron Maupin Oct 4 at 13:34
3

The Ethernet FCS's purpose is to detect rather simple errors and drop damaged frames as soon as possible (ie. do not forward corrupt frames). It cannot guarantee 100% integrity (actually, nothing can really do that). However, since the FCS is checked on each switch hop, errors can't pile up and it's more effective than it sounds. Also, an Ethernet frame is limited to 1500 bytes payload and chances for multiple errors canceling each other out are very low.

When a better integrity is required it's up the higher-layer protocols to do that. They often use checksums for their PDUs as well, e.g. TCP or (optionally) UDP. Of course, an application can even put an integrity check into the application-layer protocol or the transported object itself. Another option is to transmit a good checksum (e.g. SHA256) through a side channel - put it on a web page or in an extra file besides the data file so the receiver can verify the integrity.

  • People occasionally ask why our product packaging "double compresses" things. (tar : gzip : encrypt : gzip again) After all, any attempt to compress the encrypted payload is just going to make it bigger. We use it as a means of integrity check. We could sign it, but we can't be sure everything will have the tools to check it -- we know it has gzip, or it wouldn't be able to unpack it. – Ricky Beam Oct 4 at 17:02
  • Most compressors have a store method that simply stores/encrypts a file without attempting compression - saves quite a bit of time. Additionally, one strong checksum should be fine. – Zac67 Oct 4 at 18:16
  • But then you have to publish a checksum, and someone would have to manually check it. Running gzip takes care of that. – Ricky Beam Oct 4 at 21:07
1

If an error slips past the Ethernet checksum then it may be detected by the TCP/UDP checksum. The Ethernet checksum is 32-bits and the TCP/UDP checksum is 16 bits, so if the checksums were ideal and independent random oracles, the chance of a corrupt packet passing both would be 1 in 248

Unfortunately the checksums used in Ethernet and IP are far from ideal random oracles. Also not all of the network path is protected by the checksums. The Ethernet checksum is stripped off and re-generated at every router hop. The TCP checksum was end to end in an old-school IP network, but has to be modified by NAT devices.

On a reliable network with small volumes of data and few packets getting corrupted in the first place, the protection is probablly good enough that users are unlikely to notice issues, but as the data volume grows or the application is operated over questionable networks the possibility of corrupted data becomes more real.

Mathematically, no checksum can detect 100% of errors, but it is possible to get close enough that it doesn't practically matter.

As a rule of thumb if a protection method is good enough to protect against malicious modification then it is more than good enough to protect against any reasonable accidental modification. So often the easies solution is to use something like TLS, which provides protection against both accidental and malicious modification.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.