Why didn't any firewalls catch the flame malware siphoning out all of that data? I thought there was firewall software that looks for "suspicious" activity?
At the very basic level Firewall is a device that applies security policy by looking at packet's header. Firewalls are not made to look at the payload of the packet.
IT industry has split the responsibility of firewalling and IDS/IPS into two separate devices due to performance considerations and complexity. Some of the latest firewalls can do both, but they do not have the complete ability to examine each payload of each packet for each known or unknown threat. Their ability is limited due to performance considerations. Hence a firewall cannot flag every suspicious thing.
Malwares, viruses, worms reside in the packet's payload even though that packet might comply with that firewall's security rule, and the firewall will allow it.
IDS/IPS has the role to examine each payload, either by "Pattern matching" or "Anomaly based".
Pattern matching is based of matching patterns of bits in the payload to a set of signatures (or definitions). Firewalls don't do this. To detect flame, the IDS/IPS vendor has to provide a signature to detect it.
Anomaly based is comparing suspicious behavior with learned normal behavior of that application in the past. Again firewalls can't do this. To detect flame, IDS/IPS has to learn the normal behavior first. The IDS/IPS vendor has to provide an update to the device for behavior of flame.
Host based IDS/IPS can do a very good job in identifying and stopping an unknown threat. Example, Cisco CSA agents can detect and stop an unknown threat because it monitors behavior of system's processes very closely.
That's a very good question. Will's answer is accurate as far as firewalls go. Another part of the equation is why didn't any intrusion prevention systems (IDS/IPS) catch it?
My first-hand knowledge is limited to Cisco's IPS solution, but I'm sure most of them work similarly. They can be configured to scan outgoing as well as incoming traffic, but they rely on signatures to profile and match traffic that are updated regularly, much like anti-virus definitions. As long as Flame was an unknown threat, there were no signatures added to match against its traffic. Now that it is known, I'm sure there are updates that will detect it.
There are some IPS/IDS products that claim to work "heuristically" and can catch "zero-day" or brand new exploits. Given the level of sophistication used by the Flame malware, I'm surmising that its creators simply bought samples of heuristic IPS systems and tuned the malware to avoid detection by testing it against real world gear. It is pretty clear at this point that a national government with serious resources was involved so that sort of expense is not unlikely.
There's a number of reasons:
- Firewalls are traditionally only Layer 3 aware; provided the source/destination/port/protocol matches an allowed policy, the traffic should be allowed. The contents of the data/payload doesn't matter.
- IPS/IDS are signature based and effectiveness varies, the other answers explain this well.
- Flame uses SSL & SSH tunnels to encrypt traffic in transit. Unless these security controls are configured to MITM, decrypt and inspect; they will have no visibility of the data being exfiltrated.
- DLP products are possibly the best bet, as they should be configured to look for business specific information which may have been caught with decryption of the traffic. However the effectiveness of these products still vary.
You could argue that with sufficient log analysis capabilities (resources and tools [SEIM]), the threat could have been detected. In practice, it would probably have been lost amongst the masses.
Most modern firewalls are "stateful"- they will not allow outsiders to come into your network UNLESS you sent something to that outsider FIRST. Many firewalls (such as the Win7 software firewall) will detect programs sending information out and ask you if you really want that program to send information out.
Many people turn off the firewalls, or just click "yes" without realizing what they're doing, or don't even have a computer that has a firewall...
Remember the only secure computer is the one with no electricity in it, the weakest link in computer security is the keyboard/chair interface.
It's also worth noting that targeted computers were not necessarily protected by a firewall because they were protected by an air gap (i.e. not connected to the Internet at all). Flame not only got itself onto target computers via USB drives, it also got its data off those computers via USB drives.