2

I am building an image server. If a client uploads an image, is it possible for image to be corrupted when it reaches the server from its journey from client to server? If yes, is there some TCP mechanism to fix this issue (e.g. parity)?

1
  • are you faced such case when you transfer this photo from camera to photo server
    – Gadeliow
    Commented Nov 11, 2015 at 7:43

3 Answers 3

7

I am building an image server. If a client uploads an image, its it possible for image to be corrupted when it reaches the server from its journey from client to server ?

Possible? Certainly, but before you get worried, lets get into this a bit more.

Ethernet contains a CRC value to help prevent corruption. IPv4, TCP, and UDP all use checksum values to help prevent corruption (note: IPv6 does not). CRC and checksum are two different methods to calculate the data is unchanged. So a single IPv4 packet will have a CRC and two different checksum operations performed on it.

Any single bit error introduced in the data will be detected by both methods. The issue comes when you have multiple bit errors in the data. It is possible to have multiple bit errors that will provide a valid CRC. It is also possible to have multiple bit errors that will provide a valid checksum. However, it is very unlikely (but yes, possible) to have multiple bit errors that provide both a valid CRC and checksum.

In all cases, when corruption is detected it causes the data to be dropped. TCP contains a mechanism for retransmission of dropped segments, no normally you experience no data loss when it is in use.

However if UDP is used instead of TCP, if there is corruption detected any any of the three levels (Ethernet, IPv4 or UDP), then the data is dropped and it is up to a higher level process (i.e. the application, file format, etc) to detect and remedy the lost data.

All of this so far is based purely on the network, but applications transferring data often have some means of checking data integrity as well. Further some file formats may have some means of validating integrity as well, especially if they are compressed.

So again, yes it is possible, but unlikely. You are probably just as likely to introduce data corruption when your computer or server is reading/writing data to disk or memory. I will also note that data corruption in image files will generally either cause a serious problem or be unnoticeable.

If yes, is there some tcp mechanism to fix this issue like parity etc ?

As I detailed, TCP (as well as UDP, IPv4, and Ethernet) all have ways to detect errors in the data. Only TCP has a mechanism for retransmission, so if TCP is not used, the data is dropped. The good news is that most file transfers do use TCP.

Beyond that, it would be dependent on the application to fix errors. For instance, most compressed data has some sort of error detection/correction. So compressed file formats often have some measure of this built into the file. Also if the data sent across the network is compressed (for example a browser and server with HTTP compression enabled), there will be a measure of this involved.

And that doesn't include any measures built into the application itself, which should be concerning itself with data integrity even when handling data with the the same computer/server.

3

There is a CRC in an ethernet frame header which will drop a frame if it detects corruption.

IPv4 has a header checksum to detect corruption in the IP header (IPv6 doesn't have this).

TCP has a checksum for the entire TCP segment to detect data corruption, and TCP guarantees delivery and out-of-order reassembly.

UDP has a checksum, but it is a fire-and-forget, best-effort protocol with no expectation of delivery or reception, so you would need to build that into the application layer if you use UDP.

0

You cannot guarantee it with certainty that the image stored on the receiving server is the exact same as on the sender, as some errors are not detected by TCP's CRC mechanisms

As has been responded, jpg and other image format tolerate a reasonnable amount of errors, but the fact is that your image can be stored wrong "for life", so every time you display it back to the user it contains the error.

One way to prevent this is using an md5 comparison, which is used when downloading big iso images of operating systems for example. But unfortunately for uploads that would be too cumbersome for your users.

Another way is to use https (TLS), which will garantee you no errors end-to-end, as the decryption would fail in case of a wire error uncaught by lower layers.

The best way to prevent errors if you can't force https, is probably to encourage gzip compression of your user's uploads, which if using Apache, you can activate using SetInputFilter DEFLATE

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.