Milter-greylist

Hi,

I've created a patch which optionally adds a 'Received-SPF' header when using 'libspf2' (probably could support 'libspf' and version varients if there's interest). Avoids having a seperate milter to achieve  this. One example where this is useful is prior to 'spamassassin'  as it will use the header rather than duplicating the lookups etc.
--
Steven Hiscocks

Steven Hiscocks steven@... [milter-greylist]
<milter-greylist@yahoogroups.com> wrote:

> I've created a patch which optionally adds a 'Received-SPF' header when
> using 'libspf2' (probably could support 'libspf' and version varients if
> there's interest). Avoids having a seperate milter to achieve  this. One
> example where this is useful is prior to 'spamassassin'  as it will use
> the header rather than duplicating the lookups etc.

milter-greylist already has a addheader ACLclause that let you do things
like this:
racl continue spf pass addheader "Received-SPF: pass"

Therefore a more usefull contribution IMO would be a format string
substitution for SPF status. Something like this:
%is -> SPF status
%id -> DKIM status 

-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu@...

On 10 September 2015 00:13:59 BST, "manu@... [milter-greylist]" <milter-greylist@yahoogroups.com> wrote:
>Steven Hiscocks steven@... [milter-greylist]
><milter-greylist@yahoogroups.com> wrote:
>
>> I've created a patch which optionally adds a 'Received-SPF' header
>when
>> using 'libspf2' (probably could support 'libspf' and version varients
>if
>> there's interest). Avoids having a seperate milter to achieve  this.
>One
>> example where this is useful is prior to 'spamassassin'  as it will
>use
>> the header rather than duplicating the lookups etc.
>
>milter-greylist already has a addheader ACLclause that let you do
>things
>like this:
>racl continue spf pass addheader "Received-SPF: pass"
>
>Therefore a more usefull contribution IMO would be a format string
>substitution for SPF status. Something like this:
>%is -> SPF status
>%id -> DKIM status 
>
>-- 
>Emmanuel Dreyfus
>http://hcpnet.free.fr/pubz
>manu@...

Sure thing. The only issue is that a SPF header needs to be inserted before the top most "received" header to be valid. How about adding a 'insheader' ACL clause where the first argument is the header index (should be 0 in the case of SPF, and typically 1 for a DKIM auth result header) and second the header string?

-- 
Steven Hiscocks

10 \u0441\u0435\u043d\u0442\u044f\u0431\u0440\u044f 2015�\u0433. 8:48:43 CEST, "Steven Hiscocks steven@... [milter-greylist]" <milter-greylist@yahoogroups.com> \u043f\u0438\u0448\u0435\u0442:
>On 10 September 2015 00:13:59 BST, "manu@... [milter-greylist]"
><milter-greylist@yahoogroups.com> wrote:
>>Steven Hiscocks steven@... [milter-greylist]
>><milter-greylist@yahoogroups.com> wrote:
>>
>>> I've created a patch which optionally adds a 'Received-SPF' header
>>when
>>> using 'libspf2' (probably could support 'libspf' and version
>varients
>>if
>>> there's interest). Avoids having a seperate milter to achieve  this.
>>One
>>> example where this is useful is prior to 'spamassassin'  as it will
>>use
>>> the header rather than duplicating the lookups etc.
>>
>>milter-greylist already has a addheader ACLclause that let you do
>>things
>>like this:
>>racl continue spf pass addheader "Received-SPF: pass"
>>
>>Therefore a more usefull contribution IMO would be a format string
>>substitution for SPF status. Something like this:
>>%is -> SPF status
>>%id -> DKIM status 
>>
>>-- 
>>Emmanuel Dreyfus
>>http://hcpnet.free.fr/pubz
>>manu@...
>
>Sure thing. The only issue is that a SPF header needs to be inserted
>before the top most "received" header to be valid. How about adding a
>'insheader' ACL clause where the first argument is the header index
>(should be 0 in the case of SPF, and typically 1 for a DKIM auth result
>header) and second the header string?

Does SMTP or MIME dictate the ordering of header fields? IIRC only that headers are key:value pairs, with hanging lines indented. Anyhow, SPF at least only verifies the last hop - whether we give any extra trust to the relay or other client which contacted 'us' to submit the message. And keep in mind this trust should not be blind - spammers do set up 'proper-looking' domains including SPF, and botnets do capture accounts or whole relays from legit networks. Captured my users twice in 15 years ;)

So SPF, just like DNS naming consistency and hostname patterns (aka 'looks like dialup'), p0f and several other factors, is just a piece of score to stick longer or shorter greylists in my rulesets. If we do not yet trust the host (manual or auto white), it gets stuck for at least a couple of minutes and at most several hours. Might never retry (bot), might show up in RBL by then (spam).

Users got used to this - after some initial whitelisting and woes, this causes little disruption in casual business comms, and relatively little noise passes through (some still does, and is mostly tagged by SpamAssassin which only happens after the cheap MGL tests all succeed).

HTH, Jim
--
Typos courtesy of K-9 Mail on my Samsung Android

On Thu, Sep 10, 2015 at 07:48:43AM +0100, Steven Hiscocks steven@... [milter-greylist] wrote:
> Sure thing. The only issue is that a SPF header needs to be inserted before the top most "received" header to be valid. How about adding a 'insheader' ACL clause where the first argument is the header index (should be 0 in the case of SPF, and typically 1 for a DKIM auth result header) and second the header string?

Or a more compact syntax with optional index?
addheader 0:"Name:Value"

-- 
Emmanuel Dreyfus
manu@...

On 10 September 2015 09:22:21 BST, "Jim Klimov jimklimov@... [milter-greylist]" <milter-greylist@yahoogroups.com> wrote:
>10 \u0441\u0435\u043d\u0442\u044f\u0431\u0440\u044f 2015�\u0433. 8:48:43 CEST, "Steven Hiscocks
>steven@... [milter-greylist]"
><milter-greylist@yahoogroups.com> \u043f\u0438\u0448\u0435\u0442:
>>On 10 September 2015 00:13:59 BST, "manu@... [milter-greylist]"
>><milter-greylist@yahoogroups.com> wrote:
>>>Steven Hiscocks steven@... [milter-greylist]
>>><milter-greylist@yahoogroups.com> wrote:
>>>
>>>> I've created a patch which optionally adds a 'Received-SPF' header
>>>when
>>>> using 'libspf2' (probably could support 'libspf' and version
>>varients
>>>if
>>>> there's interest). Avoids having a seperate milter to achieve 
>this.
>>>One
>>>> example where this is useful is prior to 'spamassassin'  as it will
>>>use
>>>> the header rather than duplicating the lookups etc.
>>>
>>>milter-greylist already has a addheader ACLclause that let you do
>>>things
>>>like this:
>>>racl continue spf pass addheader "Received-SPF: pass"
>>>
>>>Therefore a more usefull contribution IMO would be a format string
>>>substitution for SPF status. Something like this:
>>>%is -> SPF status
>>>%id -> DKIM status 
>>>
>>>-- 
>>>Emmanuel Dreyfus
>>>http://hcpnet.free.fr/pubz
>>>manu@...
>>
>>Sure thing. The only issue is that a SPF header needs to be inserted
>>before the top most "received" header to be valid. How about adding a
>>'insheader' ACL clause where the first argument is the header index
>>(should be 0 in the case of SPF, and typically 1 for a DKIM auth
>result
>>header) and second the header string?
>
>Does SMTP or MIME dictate the ordering of header fields? IIRC only that
>headers are key:value pairs, with hanging lines indented. Anyhow, SPF
>at least only verifies the last hop - whether we give any extra trust
>to the relay or other client which contacted 'us' to submit the
>message. And keep in mind this trust should not be blind - spammers do
>set up 'proper-looking' domains including SPF, and botnets do capture
>accounts or whole relays from legit networks. Captured my users twice
>in 15 years ;)
>
SPF standard states that the header must be above the Received header. http://www.openspf.org/SPF_Received_Header

As an example, SpamAssassin will only look for SPF headers above the last "internal" host 'Received' header, as any headers after this can't be trusted.


>So SPF, just like DNS naming consistency and hostname patterns (aka
>'looks like dialup'), p0f and several other factors, is just a piece of
>score to stick longer or shorter greylists in my rulesets. If we do not
>yet trust the host (manual or auto white), it gets stuck for at least a
>couple of minutes and at most several hours. Might never retry (bot),
>might show up in RBL by then (spam).
>
>Users got used to this - after some initial whitelisting and woes, this
>causes little disruption in casual business comms, and relatively
>little noise passes through (some still does, and is mostly tagged by
>SpamAssassin which only happens after the cheap MGL tests all succeed).
>
>HTH, Jim
>--
>Typos courtesy of K-9 Mail on my Samsung Android


-- 
Steven Hiscocks

On 10/09/15 10:24, Emmanuel Dreyfus manu@... [milter-greylist] wrote:
>  
> 
> On Thu, Sep 10, 2015 at 07:48:43AM +0100, Steven Hiscocks
> steven@... [milter-greylist] wrote:
>> Sure thing. The only issue is that a SPF header needs to be inserted
> before the top most "received" header to be valid. How about adding a
> 'insheader' ACL clause where the first argument is the header index
> (should be 0 in the case of SPF, and typically 1 for a DKIM auth result
> header) and second the header string?
> 
> Or a more compact syntax with optional index?
> addheader 0:"Name:Value"
> 

I've attached a new patch which allows setting of optional integer after
the addheader ACL clause for inserting (as appose to appending), and
includes string substitution for SPF header. Example:

    racl whitelist spf pass addheader "Received-SPF: %Qs" 0

-- 
Steven Hiscocks