Yahoo Groups archive

Milter-greylist

Index last updated: 2026-04-28 23:32 UTC

Thread

BotNet plugin

BotNet plugin

2006-12-25 by Federico Giannici

I'm new to this mailinglist so I don't know if this has already been 
proposed...

Have you considered to implement the tests done by the BotNet plugin of 
SpamAssassin?

http://people.ucsc.edu/~jrudd/spamassassin/Botnet-0.7.tar

It try to identify the botnets by looking at the DNS information of the 
last relay. It appears to me (and many others on SpamAssassin 
mailinglist) to be VERY effective. It gives some false positives so it's 
not a good idea to use for blacklisting, but it is very good for 
greylisting.


Thanks.

-- 
___________________________________________________
     __
    |-                      giannici@...
    |ederico Giannici      http://www.neomedia.it
___________________________________________________

Re: [milter-greylist] BotNet plugin

2006-12-25 by manu@netbsd.org

Federico Giannici <giannici@...> wrote:

> It try to identify the botnets by looking at the DNS information of the
> last relay. 

How does it works? regexp paterns for dynamic hosts? DNSRBL?

-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu@...

Re: [milter-greylist] BotNet plugin

2006-12-25 by Federico Giannici

manu@... wrote:
> 
> 
> Federico Giannici <giannici@... <mailto:giannici%40neomedia.it>> 
> wrote:
> 
>  > It try to identify the botnets by looking at the DNS information of the
>  > last relay.
> 
> How does it works? regexp paterns for dynamic hosts? DNSRBL?

It looks at the direct and reverse DNS informations of the last relay.

Here it is part of the documentation.
I found very interesting and effective the following rules: BOTNET_IPINHOSTNAME, BOTNET_NORDNS, BOTNET_CLIENTWORDS and BOTNET_SERVERWORDS.


Rule:   BOTNET_NORDNS
   The relay has no PTR record (no reverse dns).  This rule does NOT incur
a DNS check, as Botnet obtains this invormation from the rdns= field in
SpamAssassin's Untrusted Relays pseudo-header.

Rule:   BOTNET_BADDNS
   The relay doesn't have a full circle DNS.  Full circle DNS means that,
starting with the relay's IP address, going to its PTR record, and then
looking at the IPs returned from that hostname's A record, is the relay's
IP address in that group if addresses?  If it isn't, then there's probably
a DNS forgery.
   Note: BOTNET_BADDNS causes Botnet to do a DNS lookup.  This can be time
consuming for your SpamAssassin Checks.

Rule:   BOTNET_IPINHOSTNAME
   Does the relay's hostname contain 2 or more octets of its IP address
within the hostname?  They can be in decimal or hexadecimal format.  Each
octet can have leading zeroes, or a single separator character.

Rule:   BOTNET_CLIENTWORDS
   Does the relay's hostname contain certain keywords that look like a
client hostname?  They can be any keywords, but the included list is intended
to identify ISP end clients and dynamic workstations.

Rule:   BOTNET_SERVERWORDS
   Does the relay's hostname contain certain keywords that look like a mail
server hostname?  They can be any keywords, but the included list is intended
to identify exceptions to the BOTNET_IPINHOSTNAME and BOTNET_CLIENTWORDS
checks, that might indicate they actually are legitimate mail servers.

Rule:   BOTNET_CLIENT
   This rule duplicates the checks in BOTNET_IPINHOSTNAME, BOTNET_CLIENTWORDS,
and BOTNET_SERVERWORDS to decide whether or not the hostname looks
like a client.
   It is effectively (!serverwords && (iphostname || clientwords))
   See Botnet.variants.txt for a way to replace this a meta rule.

Rule:   BOTNET_SOHO
   This rule checks to see if the relay is possibly a SOHO (small office,
home office) mail server.  In this case, the sender's mail domain is examined,
and resolved.  First an A record look up is done, and if the relay's IP
address is found in the first 5, then BOTNET_SOHO hits.  Second, the same
check is done on the MX records for the domain, again limited to 5 records.
These checks are limited to 5 records because a SOHO domain is not likely
to have a large round-robin A record nor a large number of MX records.  In
order to avoid having this check used as a back-door by botnet coders, by
using a throw-away sender domain that has all of its botnet hosts in the
A records or MX records, BOTNET_SOHO only looks at 5 records.

Rule:   BOTNET
   This rule duplicates the checks done by the above rules.
   The intent is to flag a message automatically for quarantine or storage
in a spam folder if the message does have the fingerprints of a spambot
or virusbot, but does NOT have the fingerprints of a server.
   It is effectively (!soho && (client || baddns || nordns))
   See Botnet.variants.txt for a way to replace this with a meta rule, or
replace this with piece-meal rules.



Bye.

-- 
___________________________________________________
    __
   |-                      giannici@...
   |ederico Giannici      http://www.neomedia.it
___________________________________________________

Re: [milter-greylist] BotNet plugin

2007-01-04 by Oliver Fromme

Federico Giannici wrote:
 > manu@... wrote:
 > > Federico Giannici wrote:
 > > > It try to identify the botnets by looking at the DNS information of the
 > > > last relay.
 > > 
 > > How does it works? regexp paterns for dynamic hosts? DNSRBL?
 > 
 > It looks at the direct and reverse DNS informations of the last relay.
 > 
 > Here it is part of the documentation.
 > I found very interesting and effective the following rules: BOTNET_IPINHOSTNAME, BOTNET_NORDNS, BOTNET_CLIENTWORDS and BOTNET_SERVERWORDS.

In fact, I think the "SOHO" check is useful for automatic
whitelisting, and should be easy to implement.  Quote:

 > Rule:   BOTNET_SOHO
 >    This rule checks to see if the relay is possibly a SOHO (small office,
 > home office) mail server.  In this case, the sender's mail domain is examined,
 > and resolved.  First an A record look up is done, and if the relay's IP
 > address is found in the first 5, then BOTNET_SOHO hits.  Second, the same
 > check is done on the MX records for the domain, again limited to 5 records.
 > These checks are limited to 5 records because a SOHO domain is not likely
 > to have a large round-robin A record nor a large number of MX records.  In
 > order to avoid having this check used as a back-door by botnet coders, by
 > using a throw-away sender domain that has all of its botnet hosts in the
 > A records or MX records, BOTNET_SOHO only looks at 5 records.

In other words, this is what milter-greylist could do:
When a connection from 12.34.56.78 arrives and tries to
send mail from foo@..., then look at the A records
and MX records of bar.com.  If one of them matches the
connecting IP (12.34.56.78), then the mail is not grey-
listed but whitelisted.  It might also be useful to
combine that check with other ACLs.

The idea is that many small sites (e.g. soho) use the
same servers for incoming and outgoing mails, so the
MX record (or A record) of the domain is the same as
the one we receive their mail from.

Of course that rule doesn't guarantee that it's not spam,
but it's pretty certain that there's a real SMTP server
running on that machine, so it doesn't make sense  to
greylist it (because it will resend anyway), and so the
mail shouldn't be delayed.

Emmanuel, what do you think?

Another thing is the "BOTNET_IPINHOSTNAME" check.  It
does _not_ simply check whether the hostname matches a
regular expression that represents the IP number.
Instead, it checks if at least two octets from the
actual connecting IP address are contained in the
hostname (in decimal or hex).  That should reduce
the number of false positives considerably, compared
to simple regular expressions.

For example, if the IP 12.34.56.78 connects, then check
if at least any two of the octets (12, 34, 56, 78 or
their hexadecimal equivalents) appear inside the host
name.  I think the check could even be more tight by
checking for at least three octets instead of two.

Again ...  what do you think?  :-)

Best regards
   Oliver

-- 
Oliver Fromme,  secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing
Dienstleistungen mit Schwerpunkt FreeBSD: http://www.secnetix.de/bsd
Any opinions expressed in this message may be personal to the author
and may not necessarily reflect the opinions of secnetix in any way.

"It combines all the worst aspects of C and Lisp:  a billion different
sublanguages in one monolithic executable.  It combines the power of C
with the readability of PostScript."
        -- Jamie Zawinski, when asked: "What's wrong with perl?"

Re: [milter-greylist] BotNet plugin

2007-01-04 by Emmanuel Dreyfus

On Thu, Jan 04, 2007 at 05:29:39PM +0100, Oliver Fromme wrote:
[SOHO server]
> Emmanuel, what do you think?

[IP contained within DNS address]
> Again ...  what do you think?  :-)

I think that these are good candidates for using (and improving) the 
urlcheck interface. What about hacking that in PHP or perl?

-- 
Emmanuel Dreyfus
manu@...

Re: [milter-greylist] BotNet plugin

2007-01-04 by Oliver Fromme

Emmanuel Dreyfus wrote:
 > Oliver Fromme wrote:
 > [SOHO server]
 > > Emmanuel, what do you think?
 > 
 > [IP contained within DNS address]
 > > Again ...  what do you think?  :-)
 > 
 > I think that these are good candidates for using (and improving) the 
 > urlcheck interface.

I would prefer if milter-greylist supported them natively,
because I don't want to run a webserver.  It would be just
another (potential) point of failure, and I think it would
make the whole system less robust.  Therefore the urlcheck
interface is useless for me.

 > What about hacking that in PHP or perl?

Well, I don't like either of them.  :-)  I would prefer C,
Ocaml or Python (in fact I've written a milter in Python
recently, for syncing valid users on backup MX servers).

Best regards
   Oliver

-- 
Oliver Fromme,  secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing
Dienstleistungen mit Schwerpunkt FreeBSD: http://www.secnetix.de/bsd
Any opinions expressed in this message may be personal to the author
and may not necessarily reflect the opinions of secnetix in any way.

"[...]  one observation we can make here is that Python makes
an excellent pseudocoding language, with the wonderful attribute
that it can actually be executed."  --  Bruce Eckel

Re: [milter-greylist] BotNet plugin

2007-01-05 by manu@netbsd.org

Oliver Fromme <olli@...> wrote:

>  > I think that these are good candidates for using (and improving) the
>  > urlcheck interface.
> I would prefer if milter-greylist supported them natively,

Well, my concern is just that the config file has already become a real
mess. Fortunately I just did a large refactoring of the ACL code that
will make it easier to add features, but documentation will remain a
weak point.

How do you imagine that new features in the config file? 

-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu@...

Re: [milter-greylist] BotNet plugin

2007-01-05 by Matthias Scheler

On Thu, Jan 04, 2007 at 05:29:39PM +0100, Oliver Fromme wrote:
> The idea is that many small sites (e.g. soho) use the
> same servers for incoming and outgoing mails, so the
> MX record (or A record) of the domain is the same as
> the one we receive their mail from.

The problem is that spammers create such real looking setups, too.

	Kind regards

-- 
Matthias Scheler                                  http://zhadum.org.uk/

Re: [milter-greylist] BotNet plugin

2007-01-05 by Oliver Fromme

Matthias Scheler wrote:
 > Oliver Fromme wrote:
 > > The idea is that many small sites (e.g. soho) use the
 > > same servers for incoming and outgoing mails, so the
 > > MX record (or A record) of the domain is the same as
 > > the one we receive their mail from.
 > 
 > The problem is that spammers create such real looking setups, too.

No, not easily, because you need many servers to send
massive amounts of spam.  That's why they often use
botnets.

Apart from that, nobody is forced to use such an option
(if it existed).  If you think it lets too much spam
through, then don't use it.  ;-)

Personally, I tend to prefer false negatives over false
positives.  After all, greylisting is not the only measure
against spam, and if something gets through, the next
stage filter (e.g. content-based or whatever) can take
care of it.  YMMV, of course.

If the probability of a sending host having a real SMTP
server is fairly high, then it doesn't make sense at all
to waste a greylist entry for it and cause a delay,
because it will resend anyway.  It doesn't matter at all
whether it is spam or not.

Best regards
   Oliver

-- 
Oliver Fromme,  secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing
Dienstleistungen mit Schwerpunkt FreeBSD: http://www.secnetix.de/bsd
Any opinions expressed in this message may be personal to the author
and may not necessarily reflect the opinions of secnetix in any way.

"C++ is over-complicated nonsense. And Bjorn Shoestrap's book
a danger to public health. I tried reading it once, I was in
recovery for months."
        -- Cliff Sarginson

Re: [milter-greylist] BotNet plugin

2007-01-05 by Alan M. Evans

On Fri, 2007-01-05 at 05:19, Oliver Fromme wrote:

> If the probability of a sending host having a real SMTP
> server is fairly high, then it doesn't make sense at all
> to waste a greylist entry for it and cause a delay,
> because it will resend anyway. It doesn't matter at all
> whether it is spam or not.

Greylisting allows the spamming host time to be listed in the RBLs.

Re: [milter-greylist] BotNet plugin

2007-01-05 by Matthias Scheler

On Fri, Jan 05, 2007 at 02:19:26PM +0100, Oliver Fromme wrote:
>  > > The idea is that many small sites (e.g. soho) use the
>  > > same servers for incoming and outgoing mails, so the
>  > > MX record (or A record) of the domain is the same as
>  > > the one we receive their mail from.
>  > 
>  > The problem is that spammers create such real looking setups, too.
> 
> No, not easily, because you need many servers to send
> massive amounts of spam.  That's why they often use
> botnets.

They can have a lot of MX records which point to A records that resolve
to a lot of hosts. So it would perhaps make sense to check for such
things.

	Kind regards

-- 
Matthias Scheler                                  http://zhadum.org.uk/

Re: [milter-greylist] BotNet plugin

2007-01-05 by Techwolf

Oliver Fromme wrote:
>
> Personally, I tend to prefer false negatives over false
> positives.  After all, greylisting is not the only measure
> against spam, and if something gets through, the next
> stage filter (e.g. content-based or whatever) can take
> care of it.  YMMV, of course.
>
>   

That was my thinking when I set mine spam stuff up years ago and found
out about greylisting. My set up is:

Pregreet, sendmail.mc setting to get rid of ratware programs running on
botnets.
greylist, selfexplaining here. ;-)
clamav, milter-calmav to take care of the viris and phising mails.
spamassassasn, milter-spamassassan takes care of the ones that slip
through and is the last due to most resource one of them all and is set
up to flag/tag SPAM in the subject line only. And reject for very high
scoreing ones.

This setup has no false positves except for spammassassan and it can
only tag it with SPAM and send it on its way. If there is a technicial
glitch or whatever and letgit mail gets rejected, the sender ACTUALLY
gets an error message and not to /dev/null. So the sender can use
another channel, such as IM, IRC, phone, etc. to alert the sysdem admin
of a problem.

Its a lot easer to whitlist problem servers then to blacklist all the
spammer servers out there.

--Techwolf.

Re: [milter-greylist] BotNet plugin

2007-01-05 by Oliver Fromme

Matthias Scheler wrote:
 > Oliver Fromme wrote:
 > > > > The idea is that many small sites (e.g. soho) use the
 > > > > same servers for incoming and outgoing mails, so the
 > > > > MX record (or A record) of the domain is the same as
 > > > > the one we receive their mail from.
 > > > 
 > > > The problem is that spammers create such real looking setups, too.
 > > 
 > > No, not easily, because you need many servers to send
 > > massive amounts of spam.  That's why they often use
 > > botnets.
 > 
 > They can have a lot of MX records which point to A records that resolve
 > to a lot of hosts. So it would perhaps make sense to check for such
 > things.

Please read the explanation from my first mail.  It says
that it doesn't cover more than 5 records (MX or A).

So if a spammer enters dozens (or even hundreds or
thousands) of his spam machines as MX records for his
sender domains, then they won't match the "soho" check,
and he will not be whitelisted.

Best regards
   Oliver


-- 
Oliver Fromme,  secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing
Dienstleistungen mit Schwerpunkt FreeBSD: http://www.secnetix.de/bsd
Any opinions expressed in this message may be personal to the author
and may not necessarily reflect the opinions of secnetix in any way.

C++: "an octopus made by nailing extra legs onto a dog"
        -- Steve Taylor, 1998

Move to quarantaine

This moves the raw source file on disk only. The archive index is not changed automatically, so you still need to run a manual refresh afterward.