Are older viruses removed from virus definition files?

Question

Antivirus software receives constant updates in the form of virus definition files, containing the patterns used to identify viruses. It is a feature present on most (if not all) major AV for at least the last decade and half.

The frequency and size of updates has only grown over the years, and I wonder if in order to limit the size of the updates the AV companies remove older viruses from their databases.

If so, how likely is it that a virus old enough to have been removed from definition files, but not old enough to be obsolete would easily spread, uncontrolled and undetected.

Answer 1

Detection for a piece of malware is never removed from a mainstream AV.

Detection for old or rare malware is not removed mainly because AV benchmarks and clients seeing one AV missing detection while the others have it.

Let's say a signature is added for "Malw" malware but then the persistent malware writer makes subtle changes to avoid that specific detection. So variants start trickling in: Malw.1, Malw.B, Malw.X.32768, ... . The AV company then starts seeing a pattern and produces a smarter signature that finds a common factor and detects all of the variants.

This smarter signature might be:

• the same classical byte pattern but on something that doesn't change between variants
• a regular-expression-like pattern
• a combination of byte patterns and metadata about the executable
• features extracted from emulating the malware, like packers, loading patterns and behavior

This smarter signature is slower than the simple byte pattern but when there are a lot of variant signatures, the overall performance will favor the smart signature. When the performance is better with the smart signature, the simple signatures will receive a lower priority or will be deleted. That means Some malware will lose specific detection although some AV engines will output the detection with multiple signatures, not just stop at the first detection.

Periodical re-scanning of the entire collection of malware is performed to check if some changes in the AV have lead to malware not being detected or false positives have been introduced by smart signatures.

Answer 2

It depends on the malware as well as the vendor, but the primary incentive for reducing the number of signatures is performance.

A few reasons why a malware detection rule might be removed:

• It only works on a platform that is no longer supported by the AV engine, e.g. old DOS / 16-bit PE malware.
• The malware is so old that it no longer poses any threat (e.g. it just crashes on first run), due to threat mitigations introduced in OS patches that 99.99% of machines will now have installed.
• The malware is no longer in distribution, and no instances of it have been detected for years.
• The malware is very rare and has a minimal impact, but the signature required to detect it is quite CPU / disk intensive.
• The specific signature has been superseded by a generic signature which still catches that malware.

Keep in mind that these are only possible reasons. The vendor may choose not to delete something, or may have development / testing reasons for keeping an old definition. Often old malware is used to test for basic protection when AV systems are reviewed.

So, whilst there's no concrete answer to your question (different companies have different ideas) it's not difficult to see when / why an AV suite might have a few old definitions pruned from its database now and then.

Answer 3

I wonder if in order to limit the size of the updates the AV companies remove older viruses from their databases.

They are not.

Virus definition stay over time and as such take space. Antivirus structure their signature database for management purpose and and permit us, technical people, to be able to manage, repair, change and backup them since many years.

If so, how likely is it that a virus old enough to have been removed from definition files, but not old enough to be obsolete would easily spread, uncontrolled and undetected.

Being able to protect the integrity of the virus database is part of the security itself and a necessary monitoring activity (imho) 1. I think it could explain a bit why antivirus program are sometime so deeply rooted to machines they 'protect' and why it also happen that antivirus screw the operating system.

As IT sec, it could be part of your duty to manage risk upon factor such as compatibility and abandonware, etc. Since antivirus can screw-up just like any other software, they must be managed and so their virus definition either.

Excluding those signatures could reduce disk usage, reduce antivirus engine corruption occurrence, fragmentation, etc, and the risk would be managed, etc, just like distributing operating system/software/firmware update/installation, etc, over the network. Not all updates are useful for all computer park, etc.

Examples Norton Nod32