[Clamav-users] Large extremely compressed text files and clamd

[Dennis Peterson]

I have a user that is sending themselves e-mails containing attachments of
web server logs on the last few days of each month that appear to be very
well compressed. When this happens, clamd seems to overheat.
88457 root 59 0 319M 207M RUN 8:28 90.82% 90.82% clamd
Notice the amount of memory that clamd is occupying and the CPU utilization
%. Our mail server caps individual e-mails at 30MB total size. The
uncompressed attachment files take up over 200MB all combined.

Is it a requirement in your environment to scan all files regardless of
size? The preponderance of viruses if not all viruses come in much
smaller packages and so we don't scan files over a certain size (which
shall remain undisclosed).

As for performance on large files - I've seen large text files bog down
the clamd process in this way and this can be tested on a local machine
using Apache or syslog files for examples.

I had a similar problem with a large zipped wmv file crashing the
milter, too. It becomes a big problem if the process times out and the
response to the timeout is a tempfail back to the smtp client because
they will resend over and over to each MX until the message is accepted
or rejected. The work-around is to limit the size of files sent to the
scanner, but that comes with additional risk.

dp
_______________________________________________
Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net
http://lurker.clamav.net/list/clamav-users.html

[Tom Samplonius]

Dec 29 12:56:58 host /usr/local/sbin/amavisd[89023]: (89023-01)
(!)do_uncompress: Error running decompressor /usr/bin/gzip -d on p001,
exit 1
at (eval 68) line 562.
Dec 29 12:57:14 host /usr/local/sbin/amavisd[89023]: (89023-01)
(!)Decoding
of p006 (ASCII text, with very long lines) failed, leaving it
do_ascii: timed out

...

While there have been reports of long scanning times on big ZIP files containing text files, it seems that the above error is coming from Amavis, not ClamD. It does not look like the message is hitting clamd at all. I use Exiscan, not Amavis, but I know that some sites have to increase the Amavis timeout to 700 seconds, otherwise Amavis gives up too quickly.

But since ClamD has a ZIP engine built-in, why do you want amavis to extra the files to temporary file, when clamd could scan the file in place?

And what version of ClamAV are you running? 0.88.7 has improvements to the ZIP engine. 0.99.* is supposed to be much faster on ZIP files (but still too slow for some sites).

Tom
_______________________________________________
Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net
http://lurker.clamav.net/list/clamav-users.html