Milter-greylist

A growing number of spammers are using anti-spam tools like SPF and
DKIM (and even DNSWL) these days.  Therefore, specifying global rules
for behavior in response to passing SPF is dangerous.

I'd love the ability to specify specific actions for known
SPF-protected domains (e.g. whitelist_from_spf in SpamAssassin).

In other words, I want to do this:

########
list "bypass spfs" spf { \
    salesforce.com vbulletin.com paypal.com gmail.com \
}
list "bypass dkims" dkim { \
    ebay.com \
}
racl whitelist list "bypass spfs"
racl whitelist list "bypass dkims"
########

(This would allow the "broken mta" list to shrink considerably, even
if it's just Google.)  These actions are triggered by spf=pass and
dkim=pass respectively.


Another option to specify what response type would enable other
things.  Example follows:

########
list "bypass spfs" spf pass { \
    salesforce.com vbulletin.com paypal.com gmail.com \
}
list "bypass dkims" dkim pass { \
    ebay.com \
}
list "broken spfs" spf soft-fail { \
    example.com \
}
racl whitelist list "bypass spfs"
racl whitelist list "bypass dkims"
racl whitelist list "broken spfs"
racl greylist spf soft-fail  delay 30m autowhite 2d
########

So if trusted domain example.com is known to violate its own SPF
record (there are lots of these!), this skips that before hitting the
clause that would delay it.

Adam Katz <yegsa-yahoo@...> wrote:

> racl whitelist list "bypass dkims"

I beleive you cannot filter on DKIM at RCPT stage, since the information
used by DKIM to decide the message status is in the message headers.

But does the DKIM code work? I implemented it and asked for testers, but
so far nobody ever reported using it.

-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu@...

manu@... wrote:
> I beleive you cannot filter on DKIM at RCPT stage, since the information
> used by DKIM to decide the message status is in the message headers.

Yeah, I realized that soon after writing; the example should have said
dacl, or even better, should have stuck to just SPF.

> But does the DKIM code work? I implemented it and asked for testers, but
> so far nobody ever reported using it.

I have no idea.  I just upgraded to 4.3.2 and am now happily using p0f.
 Next step is SpamAssassin for some testing hopefully followed by the
removal of the stale (and buggy!) spamass-milter.



Getting back to my request ... is it possible to whitelist by SPF record
currently, or is it easy to implement?

Adam Katz <yegsa-yahoo@...> wrote:

> Getting back to my request ... is it possible to whitelist by SPF record
> currently, or is it easy to implement?

Perhaps something like this will do the trick?

list "spf" domain { gmail.com yahoo.com  paypal.com }
racl whitelist list "spf" spf pass
racl blacklist list "spf"

-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu@...

Hi,

We have been using this method for moths with milter-greylist 4.0 and
works very well. It was the way we found to emulate the spf status
selections in acl's, as version 4.0 doesn't have this function.

We use two lists, one with blacklist on the second line (for banks and
important stuff) and other with 20 min greylisting (for Gmail and
others). 

Regards,
Jorge


El dv 08 de 05 de 2009 a les 09:33 +0200, en/na manu@... va
escriure:

> Adam Katz <yegsa-yahoo@...> wrote:
> 
> > Getting back to my request ... is it possible to whitelist by SPF record
> > currently, or is it easy to implement?
> 
> Perhaps something like this will do the trick?
> 
> list "spf" domain { gmail.com yahoo.com  paypal.com }
> racl whitelist list "spf" spf pass
> racl blacklist list "spf"

manu@... wrote:
> Perhaps something like this will do the trick?
> 
> list "spf" domain { gmail.com yahoo.com  paypal.com }
> racl whitelist list "spf" spf pass
> racl blacklist list "spf"

Perfect!

Assuming DKIM works, could I do this:

list "verified dkim" domain { ebay.com gmail.com google.com }
dacl whitelist "verified dkim" dkim verify
dacl blacklist "verified dkim" dkim fail

(Or whatever the term is for verified rather than passed)


Also, what is the relative maturity of extras like geoIP, DKIM,
SpamAssassin, and p0f?  I use p0f happily, and I plan to start toying
with SA, but I'd like to know what I'm in for ;-)

Adam Katz <yegsa-yahoo@...> wrote:

> Also, what is the relative maturity of extras like geoIP, DKIM,
> SpamAssassin, and p0f?  I use p0f happily, and I plan to start toying
> with SA, but I'd like to know what I'm in for ;-)

I have been using geoIP and p0f for months without any problem. I cannot
comment on DKIM and SpamAssassin.

-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu@...

On Fri, May 08, 2009 at 12:33:53PM -0400, Adam Katz wrote:
> manu@... wrote:
> > Perhaps something like this will do the trick?
> > 
> > list "spf" domain { gmail.com yahoo.com  paypal.com }
> > racl whitelist list "spf" spf pass
> > racl blacklist list "spf"
> 
> Perfect!
> 
> Assuming DKIM works, could I do this:
> 
> list "verified dkim" domain { ebay.com gmail.com google.com }
> dacl whitelist "verified dkim" dkim verify
> dacl blacklist "verified dkim" dkim fail
> 
> (Or whatever the term is for verified rather than passed)
> 
> 
> Also, what is the relative maturity of extras like geoIP, DKIM,
> SpamAssassin, and p0f?

We use the SA module since it was imported.  Runs fine.  You should
consider [1]this patch when running SA from milter-greylist.



   Petar Bogdanovic



[1] http://tinyurl.com/rdns-none

Petar Bogdanovic wrote:
> We use the SA module since it was imported.  Runs fine.  You should
> consider this patch <http://tinyurl.com/rdns-none> when running SA
> from milter-greylist.

Ah, yes.  Thanks for reminding me.  That's postfix-specific, so since
I'm on sendmail, I guess a trial to determine if that issue surfaces
will be in order.

Any other snafus?

On Fri, May 08, 2009 at 06:25:17PM -0400, Adam Katz wrote:
> Petar Bogdanovic wrote:
> > We use the SA module since it was imported.  Runs fine.  You should
> > consider this patch <http://tinyurl.com/rdns-none> when running SA
> > from milter-greylist.
> 
> Ah, yes.  Thanks for reminding me.  That's postfix-specific, so since
> I'm on sendmail, I guess a trial to determine if that issue surfaces
> will be in order.

Yes, that would be good to know so please report back.



   Petar Bogdanovic

I created a wiki page about this:
http://milter-greylist.wikidot.com/postfix

On May 8, 2009, at 2:33 PM, Petar Bogdanovic wrote:

> On Fri, May 08, 2009 at 12:33:53PM -0400, Adam Katz wrote:
>> manu@... wrote:
>>> Perhaps something like this will do the trick?
>>>
>>> list "spf" domain { gmail.com yahoo.com  paypal.com }
>>> racl whitelist list "spf" spf pass
>>> racl blacklist list "spf"
>>
>> Perfect!
>>
>> Assuming DKIM works, could I do this:
>>
>> list "verified dkim" domain { ebay.com gmail.com google.com }
>> dacl whitelist "verified dkim" dkim verify
>> dacl blacklist "verified dkim" dkim fail
>>
>> (Or whatever the term is for verified rather than passed)
>>
>>
>> Also, what is the relative maturity of extras like geoIP, DKIM,
>> SpamAssassin, and p0f?
>
> We use the SA module since it was imported.  Runs fine.  You should
> consider [1]this patch when running SA from milter-greylist.
>
>
>
>   Petar Bogdanovic
>
>
>
> [1] http://tinyurl.com/rdns-none
>
>
> ------------------------------------
>
> Yahoo! Groups Links
>
>
>

Petar Bogdanovic <petar@...> wrote:

> We use the SA module since it was imported.  Runs fine.  You should
> consider [1]this patch when running SA from milter-greylist.

What about adding an option to enable it, so that we can have in
integrated without breaking other user's log parsing scripts?

-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu@...

On Sat, May 09, 2009 at 06:01:53AM +0200, manu@... wrote:
> Petar Bogdanovic <petar@...> wrote:
> 
> > We use the SA module since it was imported.  Runs fine.  You should
> > consider [1]this patch when running SA from milter-greylist.
> 
> What about adding an option to enable it, so that we can have in
> integrated without breaking other user's log parsing scripts?

I'm not sure if that should be implemented as an option.  The injected
Received:-header for that specific situation doesn't look like it was
generated by any MTA known to SpamAssassin (otherwise RDNS_NONE would
work).

Let's see how Sendmail headers look like when RDNS is not available.
We can then make that the default behaviour and add an option for
backward compatibility?



   Petar Bogdanovic

Petar Bogdanovic <petar@...> wrote:

> Let's see how Sendmail headers look like when RDNS is not available.
> We can then make that the default behaviour and add an option for
> backward compatibility?

There are certainly people out of there that made scripts to parse
milter-greylist log output. If you change the behaviour, this is nasty
for them. IMO the default setting should be backaward compatibility.

-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu@...

... Ok, I dug through some pieces of the sendmail code and did some tests on an emulated Ubuntu installation:  It seems that the way milter-greylist logs

... Minor correction. Index: spamd.c =================================================================== RCS file: /milter-greylist/milter-greylist/spamd.c,v 

Hi,

> On Sat, May 09, 2009 at 02:13:35PM +0200, manu@... wrote:
> > Petar Bogdanovic <petar@...> wrote:
> > 
> > > Let's see how Sendmail headers look like when RDNS is not available.
> > > We can then make that the default behaviour and add an option for
> > > backward compatibility?
> > 
> > There are certainly people out of there that made scripts to parse
> > milter-greylist log output. If you change the behaviour, this is nasty
> > for them. IMO the default setting should be backaward compatibility.
> 
> Ok, I dug through some pieces of the sendmail code and did some 
> tests on an emulated Ubuntu installation:  It seems that the way 
> milter-greylist logs unresolvable clients is consistent with that of 
> sendmail.
> 
> Sendmail, however, has not the same approach when it comes to 
> received headers:
> 
> 	log (rdns):	relay=some.host.name. [1.2.3.4]
> 	log (no-rdns):	relay=[5.6.7.8] [5.6.7.8]
> 
> 	hdr (rdns):	(some.host.name. [1.2.3.4])
> 	hdr (no-rdns):	([5.6.7.8])
> 
> I think we should move that tiny piece of code into spamd.c and make 
> the injected received header look like one from sendmail.  

Does this mean that you have to be running spamd to take advantage of that code?

Michael.

> Preliminary patch is attached.  I won't have a chance to test it 
> until somewhen next week.
> 
>    Petar Bogdanovic
> 
> P.S.
> 	It would be still very helpful if a sendmail-user
> 	could confirm my observations.
> 
> ------------------------------------
> 
> Yahoo! Groups Links
> 
> 
>

On Sun, May 10, 2009 at 09:57:07AM +1100, Michael Mansour wrote:
> Hi,
> 
> > On Sat, May 09, 2009 at 02:13:35PM +0200, manu@... wrote:
> > > Petar Bogdanovic <petar@...> wrote:
> > > 
> > > > Let's see how Sendmail headers look like when RDNS is not available.
> > > > We can then make that the default behaviour and add an option for
> > > > backward compatibility?
> > > 
> > > There are certainly people out of there that made scripts to parse
> > > milter-greylist log output. If you change the behaviour, this is nasty
> > > for them. IMO the default setting should be backaward compatibility.
> > 
> > Ok, I dug through some pieces of the sendmail code and did some 
> > tests on an emulated Ubuntu installation:  It seems that the way 
> > milter-greylist logs unresolvable clients is consistent with that of 
> > sendmail.
> > 
> > Sendmail, however, has not the same approach when it comes to 
> > received headers:
> > 
> > 	log (rdns):	relay=some.host.name. [1.2.3.4]
> > 	log (no-rdns):	relay=[5.6.7.8] [5.6.7.8]
> > 
> > 	hdr (rdns):	(some.host.name. [1.2.3.4])
> > 	hdr (no-rdns):	([5.6.7.8])
> > 
> > I think we should move that tiny piece of code into spamd.c and make 
> > the injected received header look like one from sendmail.  
> 
> Does this mean that you have to be running spamd to take advantage of that code?

Not [1]spamd but [2]spamd.  (man greylist.conf)



   Petar Bogdanovic



[1] http://www.openbsd.org/spamd/
[2] http://spamassassin.apache.org/full/3.2.x/doc/spamd.html

... New patch is attached.  Tested, seems to work. Petar Bogdanovic Index: spamd.c =================================================================== RCS

Petar Bogdanovic <petar@...> wrote:

> New patch is attached.  Tested, seems to work.

What do I add in ChangeLog, after all?

-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu@...

On Tue, May 12, 2009 at 05:28:49AM +0200, manu@... wrote:
> Petar Bogdanovic <petar@...> wrote:
> 
> > New patch is attached.  Tested, seems to work.
> 
> What do I add in ChangeLog, after all?

Cleanup spamd_rcvhdr(), adjust fake Received:-header and make it look
like a real one created by Sendmail.  This new header will finally
trigger the SpamAssassin RDNS_NONE check in the specific case where a
connecting client has no rdns mapping.