1

Accoording to HowToGeek:

Your antivirus software relies on virus definitions to detect malware. That’s why it automatically downloads new, updated definition files – once a day or even more often. The definition files contain signatures for viruses and other malware that have been encountered in the wild.

My question is, wouldn't the storage space required to hold such signature databases be quite large in size? However, this doesn't seem to be the case as daily AV updates don't seem that big, nor does the installation download.

Are these signature definitions just not as big as I'm imagining them to be?

Improve this question
1
  • 2
    How long is a piece of string ! The answer depends on which AV and how that AV works !
    – Little Code
    Commented Dec 10, 2016 at 9:13

3 Answers 3

Reset to default
0

ClamAV's database size from their update server is 109.1MB as of today. These signature databases are created and stored in such a way as to compress them, as you can imagine.

ESET says their updates (performed 2-3 times a day) are 60KB in size on average (though can be bigger).

Installations would only include the binaries to install the program, and the program would contact the update server to retrieve the latest database, also as you would imagine.

These databases are the intellectual property of the AV vendor and represents one of the unique values the vendor provides with the product, so they will not be forthcoming with all the details about how they manage their databases. But open source programs provide a glimpse into one method for how they might work.

Improve this answer
0

With the latest advanced, persistent & Polymorphic malware and the kind of attributes that they come with, for every 10 lines of Malicious code, the AV signature code goes till 1000 lines.

With such a correlation, an average AV definition DB can be in few hundreds of MBs in size !! However, now most of them are moving towards Behavioral detection as it is impractical to have heavy signature database !!

Thanks

Improve this answer
-1

There are various methods employed by AV vendors to make the database size smaller. One is compression techniques, wherein the vendor will compress the database and make them available for download to users.

Second is generalization, a technique used in machine learning. They use pattern-matching or regular expressions for this. Suppose a vendor has 4 signatures of the following format:

aaaaa
aaaab
aaaac
aaaad

Instead of having separate definitions for each of the above, these 4 definitions can be generalized to: aaaa?

Here, ? being a wild-card character, which represents any one character. This technique significantly reduces the size of the database.

Here lies one caveat, though. Suppose the AV product finds aaaax in some legitimate software. Naturally, it is also flagged as a malicious program(in reality, though, it is not). This gives rise to false-positives. In such cases, the AV vendor will give an exception to aaaax in their next update of database, so that it does not get red-flagged in future.

There might be other techniques as well, which might be proprietary, but the above 2 techniques are generally widely-used ones.

As requested in comments, here are the links for the generalization techniques:

Wikipedia

Intego Search for Generic Detection

Pattern Matching in GPUs

Avira

Improve this answer
5
  • Do you have any reference for the 'generalization' section of your answer? You start of saying "they might use" then at the end, you say that it is a generally used technique. A source for your aaaa? example would be helpful.
    – schroeder ♦
    Commented Dec 10, 2016 at 9:26
  • @schroeder: I just created that example to explain the things. Some of the links for pattern matching and generic detection have been added now. I said "they might use" because they can use either pattern matching(as in wild card character example) or switch to a more complex regex solution. But they are essentially the same. I'll edit the answer, though. Thanks!
    – pri
    Commented Dec 10, 2016 at 9:56
  • The problem is that your example is so specific and technical, one might think that is the actual method use (regex wildcards). Since it is the bulk of your answer, such a conclusion is easy to make. What you simply meant to say is that the signature doesn't need to match exactly, so that one signature can match a range of viruses. That would reduce the size of the database.
    – schroeder ♦
    Commented Dec 10, 2016 at 10:47
  • @schroeder: "the signature doesn't need to match exactly, so that one signature can match a range of viruses" Exactly my point! Just to explain this point I had to give that example.
    – pri
    Commented Dec 10, 2016 at 10:52
  • The way you wrote your answer, your example distracts from the reality.
    – schroeder ♦
    Commented Dec 10, 2016 at 11:24

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .