4

I wish to scan files that are bigger then 4GB. For example video files.

I noticed many antivirus software are limited to a maximum of 4GB (ClamAV for example).

So:

  • Is there a antivirus software that allow scanning of files bigger then 4GB?
  • If not, is there a work around that will allow me to pass this limitation?

Update: as suggested by @Purefan, I opened a GitHub issue at ClamAV

  • 2
    I think this is because most are written in 32 bit and so the maximum size is 4GB which can be used. I am not sure which tools are fully 64 bit and can scan larger files. – Daniel Ruf Dec 3 '15 at 9:02
  • 2
    @DanielRuf My thought exactly. I guess michael could download the source code and try to compile for 64 bit, or file a bug here github.com/vrtadmin/clamav-devel/issues – Purefan Dec 3 '15 at 9:04
  • 2
    32-bit executables can definitely scan files larger than 4GB. mmap takes an offset of type off_t (which is 64-bit) to specify where in the file you want to map from. The process can't map more than about 2.8GB at once, but it can definitely unmap and remap the next chunk. The equivalent in Windows is MapViewOfFile which also accepts a 64-bit offset. If instead the AV wishes to use traditional file functions (e.g. fopen) then they can define _FILE_OFFSET_BITS and use fseek64. Any restriction to 4GB by the AV is not due to technical restrictions of the operating system. – Polynomial Apr 26 '18 at 13:27
  • I answered something similar in SuperUser here. – Shane Andrie Jun 25 '18 at 19:32
  • If it were simply the file size limit then it would be trivial to recompile the code with -D_FILE_OFFSET_BITS=64. However given the way AV detection works it is likely they are maintaining a pointer as a 32 bit unsigned integer rather a off_t type.. – symcbean Jul 25 '18 at 14:32
-1

I have never tried myself, but checking in clamd.conf file I can see that the size of files can be set manually.

Changing the following values to "n" appears to be the workaround for the file size limit which sets it to unlimited:

-max-filesize=#n 
-max-scansize=#n 
BytecodeUnsigned yes

http://blog.clamav.net/2014/11/brief-re-introduction-to-clamav.html

However, I know that currently ClamAV has hard file limit to 4GB. However, from my understanding by setting to "n" along with BytecodeUnsigned to "yes" in config file will do the workaround and of course there are a few particulars to consider before doing that, particulars such as:

  • the amount of memory present/available on server should be way bigger than the file size because ClamAV is mapping the file into memory.
  • The swap file system, the size of swap fs should be calculated according to the ram size on server (just in case it hits the swap).

If the above to particular are not taken care of it with an extra attention ClamAV won't be able to scan or better, it will eat all of systems memory and therefore killed by OEM Killer.

  • Do you have a reference for this info? – Neil Smithline Dec 4 '15 at 0:25
  • 2
    the amount of memory present/available on server should be way bigger than the file size because ClamAV is mapping the file into memory. This statement is not true. You definitely can map a file into memory that's way larger than the amount of memory the machine has available. The size limit to mapped file is the virtual address space, rather than the amount of available memory. – Lie Ryan Apr 27 '18 at 0:06
  • As @LieRyan says, mapping something into memory does not require actually allocating memory. I can easily map a 50 GiB file on a system with literally 32 MiB of physical memory. – forest Jul 26 '18 at 1:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.