Detecting malware hidden in P2P traffic

Question

Consider a network situation involving (only) Peer to Peer traffic in medium sized network.

For the sake of convenience, a simple P2P application like DC++ may be assumed, although I am aiming at a broader set involving torrents etc. too.

Peer A does a search for file qwerty.mp3, and then starts pulling the file, in pieces, from different Peers- B, C, D and E.

Now say Mr. C is the one who has shared this file amongst Peers, and he is a malicious peer and the file contains a malware/trojan etc.

Since the file is being downloaded in pieces from different Peers, it is unlikely that a standard Intrusion Detection system will be able to detect it since most of them involve signature-based detection techniques.
And most likely when the pieces are re-assembled at the system of Mr. A, his anti-virus will flag off a warning about the malicious content.

So, my question:
At the network level, what approach an admin may take so that innocent peers like A are protected from such situations?

Kindly do not suggest actions or measures relating to the end user. Whatever has to be done, must be done at the network level.

Please note that I am looking at this more from a research point of view and it is perfectly alright to not to point me towards existing techniques used in production environments but share links for research papers and give answers which tell of an open-ended techniques.

Answer 1

P2P traffic has in fact have directly impacted the performance of most of the IDS/IPS and there is a significant increase in the false positive. The issues lies with the behavior of P2P traffic which is lot alike malicious traffic.

For the second part of your question the most idealistic approach that can be adopted at network level is DPI (Deep packet inspection). The idea is to segregate P2P traffic from the normal traffic as much as possible, though DPI is not applicable on encrypted P2P traffic. In the next phase signatures are applied on this segregated P2P traffic for separating malicious for benign P2P traffic.

Source: What Is the Impact of P2P Traffic on Anomaly Detection

Answer 2

Many P2P software will create hashes of each piece to prevent a malicious entity from modifying the pieces in-flight. When your P2P client gets a piece, it verifies it's hash. This will prevent one malicious entity modifying a piece of a valid file, but it won't help you if the file itself (qwerty.mp3) is malicious.

If the file itself is malicious, a simple solution is to just run an anti-virus scan against the final file, and see if there are any problems. This will satisfy your signature/anomaly based detection (depending on what the Anti-virus does)

Answer 3

Perhaps you could run the p2p programs on a proxy, download the whole file (this is the first time you're going to see it and be able to fully virus scan it... the p2p transmission is irrelevant), then scan it and serve it to the user.

Easy way: web UI for people to search, request & download. Hard way: intercept from p2p clients and pretend to be the only peer whilst in fact downloading from loads of other peers, not letting much file hit the downloader until you've scanned it

Answer 4

This is similar to IP fragmentation attack. By fragmenting IP datagram into smaller pieces, one kind of DOS attack is done, as far as I know. You can google it. This is the 4th result in google but OffensiveSecurity is generally one of the best in security issues.

Hope this helps