[clamav-users] signature processing order

[Axb]

Good day,

I've noticed that apparently third party (UNOFFICIAL) signatures get
applied before the official ones.

Depending on the signature types, we may never see any "official" sigs
hitting, ever.

Is there a scientific reason for this? (or am I missing something?)

If no, could it be made switchable (via clamd.conf) and --switch for
clamscan.

Thanks

Axb
_______________________________________________
Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

[Groach]

Out of interest, what does it matter? Why is it important that an
official CLAM definition stops the virus before the 3rd party definition
stops the same virus (if they both have the same criteria)? Surely a
goal is a goal and it doesnt matter who kicked the ball.

Post by Axb
Good day,
I've noticed that apparently third party (UNOFFICIAL) signatures get
applied before the official ones.
Depending on the signature types, we may never see any "official" sigs
hitting, ever.
Is there a scientific reason for this? (or am I missing something?)
If no, could it be made switchable (via clamd.conf) and --switch for
clamscan.
Thanks
Axb
_______________________________________________
https://github.com/vrtadmin/clamav-faq
http://www.clamav.net/contact.html#ml

_______________________________________________
Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

[Arnaud Jacques / SecuriteInfo.com]

Hello,

Post by Groach
Out of interest, what does it matter?

The question of Axb is interesting.
Such option could be used to remove signatures from 3rd party when detection is done
with official signatures from Clamav.
We do not need 4 different signatures in RAM to get the same sample detection.
--
Best regards,

Arnaud Jacques
SecuriteInfo.com

Facebook : https://www.facebook.com/pages/SecuriteInfocom/132872523492286
Twitter : @SecuriteInfoCom
_______________________________________________
Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

[Steve Basford]

Post by Groach
Out of interest, what does it matter? Why is it important that an
official CLAM definition stops the virus before the 3rd party definition
stops the same virus (if they both have the same criteria)? Surely a goal
is a goal and it doesnt matter who kicked the ball.

I have to agree :)

a) if you *really* want to know what sigs matched a sample you
can use clamscan -z, which gives you this sort of output...

caution_lizr_587777.zip: Sanesecurity.Foxhole.JS_Zip_1.UNOFFICIAL FOUND
caution_lizr_587777.zip: Sanesecurity.Foxhole.Zip_fs208.UNOFFICIAL FOUND

Ok, so scanning will continue until ALL matches are found in official and
3rd party sigs, which would take a bit longer to scan... but at least
you'd know.

b) You can use clamscan --official-db-only=yes to only use official ones

As for "removing" a 3rd party signature when official ones block it,
well... overall... it wouldn't really be a good idea.

Cheers,

Steve
Web : sanesecurity.com
Twitter: @sanesecurity

_______________________________________________
Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

[C.D. Cochrane]

Hmm, that's strange. I have noted exactly the opposite behavior. My customsig.ndb sigs
only get applied after official ClamAV detection has run. I know this because I am
always watching for my UNOFFICIAL FOUNDs to be replaced by official ones and I then
delete the related sig from my customsig.ndb. It does not happen often, but it does
happen (official detection, I mean)!
...Chris
 

Sent: Tuesday, May 24, 2016 at 5:54 AM
Subject: [clamav-users] signature processing order
Good day,
I've noticed that apparently third party (UNOFFICIAL) signatures get
applied before the official ones.
Depending on the signature types, we may never see any "official" sigs
hitting, ever.
Is there a scientific reason for this? (or am I missing something?)
If no, could it be made switchable (via clamd.conf) and --switch for
clamscan.
Thanks
Axb

_______________________________________________
Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

[Groach]

I dont understand why anyone would want to delete a signature from their
databases even if it is a duplicate. Consider this:

MAIN: signature "BadWilly" (no guesses what it might be trying to trap)
3rdParty signature "3rdBadWilly" attempting to catch the same virus

Ok, so now you have determined there are 2 viruses with the same
intewntion. So you delete one of them

Unknown to you, the one you deleted wasnt very good and doesnt actually
work as expected. (Whereas the deleted one weas good).

OR

You delete one, leaving one that was once proven effective...then tnat
same provider changes that defniition (agains leaving you without the
protection).

OR.... you delete signature (thinkning its redundant) then do a database
update and it gets restored again.

And you simply cant ask the providers to not include the definitions
'just because MAIN Clam has included it' because MAYBE there is a
customer that does like or update MAIN database (and actually likes to
rely solely on the 3rd party database).

Post by C.D. Cochrane
Hmm, that's strange. I have noted exactly the opposite behavior. My customsig.ndb sigs
only get applied after official ClamAV detection has run. I know this because I am
always watching for my UNOFFICIAL FOUNDs to be replaced by official ones and I then
delete the related sig from my customsig.ndb. It does not happen often, but it does
happen (official detection, I mean)!
...Chris

Sent: Tuesday, May 24, 2016 at 5:54 AM
Subject: [clamav-users] signature processing order
Good day,
I've noticed that apparently third party (UNOFFICIAL) signatures get
applied before the official ones.
Depending on the signature types, we may never see any "official" sigs
hitting, ever.
Is there a scientific reason for this? (or am I missing something?)
If no, could it be made switchable (via clamd.conf) and --switch for
clamscan.
Thanks
Axb

_______________________________________________
https://github.com/vrtadmin/clamav-faq
http://www.clamav.net/contact.html#ml

_______________________________________________
Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

[Arnaud Jacques / SecuriteInfo.com]

Hello Steve,

Post by Steve Basford
As for "removing" a 3rd party signature when official ones block it,
well... overall... it wouldn't really be a good idea.

Why ?

Clamav official signatures + all 3rd party signatures needs a lot of system RAM. Optimizing
our signatures to scan faster and use less RAM should be a priority. Am I wrong ?
--
Best regards,

Arnaud Jacques
SecuriteInfo.com

Facebook : https://www.facebook.com/pages/SecuriteInfocom/132872523492286
Twitter : @SecuriteInfoCom
_______________________________________________
Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

[C.D. Cochrane]

I guess it all depends on what you want from AV. I hope for 0 day email
detection. If my customsig or ClamAV official DB detect the virus in the
days and weeks AFTER the virus hit my inbox then I've already lost. I
never do full system file scans with ClamAV. I want incoming email
detection.

So, I keep hoping that any new official detection will be indicative of
a new 0 day algorithm, not merely a copy of the static signature I already
redundantly created. Insanity is doing the same thing over and over
hoping for different results :) Am I insane, or are the ClamAV sig writers?
...Chris

Sent: Tuesday, May 24, 2016 at 8:37 AM
Subject: Re: [clamav-users] signature processing order
I dont understand why anyone would want to delete a signature from their
MAIN: signature "BadWilly" (no guesses what it might be trying to trap)
3rdParty signature "3rdBadWilly" attempting to catch the same virus
Ok, so now you have determined there are 2 viruses with the same
intewntion. So you delete one of them
Unknown to you, the one you deleted wasnt very good and doesnt actually
work as expected. (Whereas the deleted one weas good).
OR
You delete one, leaving one that was once proven effective...then tnat
same provider changes that defniition (agains leaving you without the
protection).
OR.... you delete signature (thinkning its redundant) then do a database
update and it gets restored again.
And you simply cant ask the providers to not include the definitions
'just because MAIN Clam has included it' because MAYBE there is a
customer that does like or update MAIN database (and actually likes to
rely solely on the 3rd party database).

_______________________________________________
Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/cla

[Al Varnell]

Post by Arnaud Jacques / SecuriteInfo.com

Post by Steve Basford
As for "removing" a 3rd party signature when official ones block it,
well... overall... it wouldn't really be a good idea.

Why ?
Clamav official signatures + all 3rd party signatures needs a lot of system RAM. Optimizing
our signatures to scan faster and use less RAM should be a priority. Am I wrong ?

Because the signatures may not be identical and could be looking for two different things so that a variant of the original malware that could be caught by one sig will be overlooked by the other.

-Al-
_______________________________________________
Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

[Arnaud Jacques / SecuriteInfo.com]

Hello Al,

Post by Al Varnell
Because the signatures may not be identical and could be looking for two
different things so that a variant of the original malware that could be
caught by one sig will be overlooked by the other.

This can not happened with Securiteinfo.com sigs. We remove signatures when
Clamav official detect malwares, based on hundred of samples.

That's the only way to keep the number of signatures the lowest possible.

If 3rd party providers don't do that, they are condemned to have more and more
signatures everyday, until it breaks low memory systems.
--
Best regards,

Arnaud Jacques
SecuriteInfo.com

Facebook : https://www.facebook.com/pages/SecuriteInfocom/132872523492286
Twitter : @SecuriteInfoCom
_______________________________________________
Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

[Groach]

6 for one, half a dozen for the other.

(A great advert, though. Well done.)

Post by Arnaud Jacques / SecuriteInfo.com
Hello Al,

Post by Al Varnell
Because the signatures may not be identical and could be looking for two
different things so that a variant of the original malware that could be
caught by one sig will be overlooked by the other.

This can not happened with Securiteinfo.com sigs. We remove signatures when
Clamav official detect malwares, based on hundred of samples.
That's the only way to keep the number of signatures the lowest possible.
If 3rd party providers don't do that, they are condemned to have more and more
signatures everyday, until it breaks low memory systems.

_______________________________________________
Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml