Checksums do not match when I download file over http

Question

Just for the test I did the following:

Downloaded a page from Wikipedia via wget on remote server and named the file test.dat. Then I executed md5sum.

I download the file via HTTP and check it back on my home computer. The checksum does not match.

When I download the same file via SCP the checksum matches.

When I compare test.dat and test.dat.original in diff viewer I get these kind of differences.

When I download the file via 3G (not using my mobile phone, not my home WiFi) there is no such problem.

Any idea what might be the problem and what to look for ?

Edit by KoCMoHaBTa

Actually me and gotha are performing the test. The problem is in my network and here is full description of the problem.

From several days I have experienced issues with downloading iOS apps from the appstore on all of my devices - they downloaded but iOS does not let them to install and gives an error.

Then I started digging around. In short what it came out is that every time I download a file from internet - it gets altered.

I have tried with anything, applications, RAR files, ISO files. They never match the provided MD5 and SHA1 check sums.

The test we have performed was intended to see what is happening. The changes are always actually at random place - they are never the same.

Before umping into what I have discovered, here is my home network configuration:

1. I have a modem provided from my ISP - Cisco EPC3208
2. Then I have a D-Link router - Dir-810L, flashed with DD-WRT (the only build available for it)
3. I have 3 iOS devices and a Windows PC
4. I own the router from about 2-3 weeks and this problem started 3-4 days ago.
5. Before this router I had D-Link Dir-300, again flashed with DD-WRT - never had any problems with it.

So digging even more:

1. Modem -> Router -> Device (Wi-Fi or Lan) - the problem occurs
2. Modem -> Windows PC (LAN) - > the problem does not occurs
3. iOS Device (Tethering over Cable) -> Windows PC -> Router (LAN to WAN) -> iOS Device (Wi-Fi) - the problem does not occur
4. iOS Device (Tethering over Wi-Fi) -> any device - the problem does not occur

So at this point it appears that the problem occurs when the router is connected to the modem. So I'm not sure if my ISP or my router is doing something bad. Or my ISP is doing something bad, which my router does not like :)

Anyway, we are trying to figure out what is the origin of the problem. Here is a link to test files - Test Files This archive contains the same file downloaded when I was connected directly to the modem and when using my router. It also contains WireShark dumps of the network for both cases.

At first point, when using my router, everything appears OK to me in WireShark, but the file is modified.

The interesting thing is when you look at the dump when I was connected directly to the modem - a large number of "duplicate" and "out of order" packets appears, but the file was delivered as expected.

Any thoughts on this?

Answer 1

Looks like an issue handling out-of-order and/or duplicate packets at the router, perhaps an issue with that build of DD-WRT when processing large transfers. The problem occurred using wget but not scp because most authenticated protocols will handle discrepancies at the application layer as they occur—scp runs over SSH which includes integrity protection—while raw HTTP relies on the integrity of the transmission as reconstructed at the presentation layer. In any case, I do not believe this to be an issue of information security.