Milter-greylist

I've been happily using milter-greylist for a few years, with pretty
great results -- spam level is barely noticeable, and I'm only using
greylist by itself at the moment.

Recently I've noticed again I'm occasionally missing messages, or at
least hitting multiple iterations of delays; when I dig into the logs
it appears to result from the sender using a mail relay server "pool".

E.g. the 1st delivery attempt comes from relay1.example.com, which then
gets greylisted; the 2nd attempt comes from relay2.example.com, the 3rd
from relay3, and so on.

In some cases a previous relay will get the message to try again, and
the message will eventually be delivered normally.

In other situations, possibly with very large relay server pools (?),
the autowhite timer for the relay(s) expires, and the sender eventually
gives up presumably due to their own retry policies.

I've noticed this happening a few times with domains hosted by office365
outlook.com, though I'd expect there are others.

When I notice these sort of issues, because it's a person or domain I
want to associate with, I simply add them to my white list as needed.

But I'm wondering if anyone has found a more automatic / elegant way
of handling situations like this.

Cheers,
sr.

On Wed, 24 Apr 2019, Steve Rikli sr@... [milter-greylist] wrote:
>
> But I'm wondering if anyone has found a more automatic / elegant way
> of handling situations like this.

Using whitelisting DNS validation and/or SPF validation and some 
measure of trust for hosts which pass these tests might be acceptable 
when greylisting is too hard.

Bob
-- 
Bob Friesenhahn
bfriesen@..., http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/
Public Key,     http://www.simplesystems.org/users/bfriesen/public-key.txt

Provided that the members of the relay pool reside in the same subnet this statement is your savior:

subnetmatch /24

Best

John

Le mercredi 24 avril 2019 � 22:22 +0200, John Damm S�rensen
john@... [milter-greylist] a �crit :
>   
> Provided that the members of the relay pool reside in the same subnet
> this statement is your savior:
> 
> subnetmatch /24

subnetmatch /16  or less /12 is better.

For example gmail has four MX CIDR:74.125.0.0/16
and one in CIDR:108.177.0.0/17

# dig +short MX gmail.com
20 alt2.gmail-smtp-in.l.google.com.
30 alt3.gmail-smtp-in.l.google.com.
40 alt4.gmail-smtp-in.l.google.com.
10 alt1.gmail-smtp-in.l.google.com.
5 gmail-smtp-in.l.google.com.
# dig +short alt1.gmail-smtp-in.l.google.com.
74.125.205.26
# dig +short alt2.gmail-smtp-in.l.google.com.
74.125.68.26
# dig +short alt3.gmail-smtp-in.l.google.com.
108.177.125.26
# dig +short alt4.gmail-smtp-in.l.google.com.
74.125.195.26
# dig +short gmail-smtp-in.l.google.com.
74.125.206.27

Another example

nasa.gov CIDR:198.120.0.0/14, 198.116.0.0/14
the 8 MX are inside  198.117.0.0/16

> 
> Best
> 
> John
> 
> 
> Den 24-04-2019 kl. 15:45 skrev Steve Rikli sr@...
> [milter-greylist]:
> 
> >   
> > I've been happily using milter-greylist for a few years, with pretty
> > great results -- spam level is barely noticeable, and I'm only using
> > greylist by itself at the moment.
> > 
> > Recently I've noticed again I'm occasionally missing messages, or at
> > least hitting multiple iterations of delays; when I dig into the
> > logs
> > it appears to result from the sender using a mail relay server
> > "pool".
> > 
> > E.g. the 1st delivery attempt comes from relay1.example.com, which
> > then
> > gets greylisted; the 2nd attempt comes from relay2.example.com, the
> > 3rd
> > from relay3, and so on.
> > 
> > In some cases a previous relay will get the message to try again,
> > and
> > the message will eventually be delivered normally.
> > 
> > In other situations, possibly with very large relay server pools
> > (?),
> > the autowhite timer for the relay(s) expires, and the sender
> > eventually
> > gives up presumably due to their own retry policies.
> > 
> > I've noticed this happening a few times with domains hosted by
> > office365
> > outlook.com, though I'd expect there are others.
> > 
> > When I notice these sort of issues, because it's a person or domain
> > I
> > want to associate with, I simply add them to my white list as
> > needed.
> > 
> > But I'm wondering if anyone has found a more automatic / elegant way
> > of handling situations like this.
> > 
> > Cheers,
> > sr.
> > 
> > 
> > 
> 
> 
> Virusfri. www.avast.com 
> 
> 
> 

-- 
Christian P�lissier
\ue40aONERA DSI/ISR BP72 92322 Chatillon CEDEX\ue409
\ue201 34419

On Thu, Apr 25, 2019 at 09:24:27AM +0200, Christian P\ufffdlissier Christian.Pelissier@... [milter-greylist] wrote:
> For example gmail has four MX CIDR:74.125.0.0/16
> and one in CIDR:108.177.0.0/17
> 
> # dig +short MX gmail.com
> 20 alt2.gmail-smtp-in.l.google.com.
> 30 alt3.gmail-smtp-in.l.google.com.
> 40 alt4.gmail-smtp-in.l.google.com.
> 10 alt1.gmail-smtp-in.l.google.com.
> 5 gmail-smtp-in.l.google.com.
> # dig +short alt1.gmail-smtp-in.l.google.com.
> 74.125.205.26
> # dig +short alt2.gmail-smtp-in.l.google.com.
> 74.125.68.26
> # dig +short alt3.gmail-smtp-in.l.google.com.
> 108.177.125.26
> # dig +short alt4.gmail-smtp-in.l.google.com.
> 74.125.195.26
> # dig +short gmail-smtp-in.l.google.com.
> 74.125.206.27

But if you look at the SPF records for gmail.com, which I think are more
relevant than the MX records for this purpose, you see more subnets:

dig +short -t txt gmail.com
"v=spf1 redirect=_spf.google.com"

This expands to:

ip4:35.190.247.0/24
ip4:64.233.160.0/19
ip4:66.102.0.0/20
ip4:66.249.80.0/20
ip4:72.14.192.0/18
ip4:74.125.0.0/16
ip4:108.177.8.0/21
ip4:173.194.0.0/16
ip4:209.85.128.0/17
ip4:216.58.192.0/19
ip4:216.239.32.0/19
ip6:2001:4860:4000::/36
ip6:2404:6800:4000::/36
ip6:2607:f8b0:4000::/36
ip6:2800:3f0:4000::/36
ip6:2a00:1450:4000::/36
ip6:2c0f:fb50:4000::/36
ip4:172.217.0.0/19
ip4:172.217.32.0/20
ip4:172.217.128.0/19
ip4:172.217.160.0/20
ip4:172.217.192.0/19
ip4:108.177.96.0/19
ip4:35.191.0.0/16
ip4:130.211.0.0/22

Regards,
Andy

I think if you go through the logs you will see that the servers handling a retries of a specific mail are all on the same subnet.

That's what I observed.

/john

Le jeudi 25 avril 2019 � 08:12 -0400, 'Andrew J. Schorr'
aschorr@... [milter-greylist] a �crit :
>   
> On Thu, Apr 25, 2019 at 09:24:27AM +0200, Christian P�lissier
> Christian.Pelissier@... [milter-greylist] wrote:
> > For example gmail has four MX CIDR:74.125.0.0/16
> > and one in CIDR:108.177.0.0/17
> > 
> > # dig +short MX gmail.com
> > 20 alt2.gmail-smtp-in.l.google.com.
> > 30 alt3.gmail-smtp-in.l.google.com.
> > 40 alt4.gmail-smtp-in.l.google.com.
> > 10 alt1.gmail-smtp-in.l.google.com.
> > 5 gmail-smtp-in.l.google.com.
> > # dig +short alt1.gmail-smtp-in.l.google.com.
> > 74.125.205.26
> > # dig +short alt2.gmail-smtp-in.l.google.com.
> > 74.125.68.26
> > # dig +short alt3.gmail-smtp-in.l.google.com.
> > 108.177.125.26
> > # dig +short alt4.gmail-smtp-in.l.google.com.
> > 74.125.195.26
> > # dig +short gmail-smtp-in.l.google.com.
> > 74.125.206.27
> 
> But if you look at the SPF records for gmail.com, which I think are
> more
> relevant than the MX records for this purpose, you see more subnets:
> 
> dig +short -t txt gmail.com
> "v=spf1 redirect=_spf.google.com"
> 
> This expands to:
> 
> ip4:35.190.247.0/24
> ip4:64.233.160.0/19
> ip4:66.102.0.0/20
> ip4:66.249.80.0/20
> ip4:72.14.192.0/18
> ip4:74.125.0.0/16
> ip4:108.177.8.0/21
> ip4:173.194.0.0/16
> ip4:209.85.128.0/17
> ip4:216.58.192.0/19
> ip4:216.239.32.0/19
> ip6:2001:4860:4000::/36
> ip6:2404:6800:4000::/36
> ip6:2607:f8b0:4000::/36
> ip6:2800:3f0:4000::/36
> ip6:2a00:1450:4000::/36
> ip6:2c0f:fb50:4000::/36
> ip4:172.217.0.0/19
> ip4:172.217.32.0/20
> ip4:172.217.128.0/19
> ip4:172.217.160.0/20
> ip4:172.217.192.0/19
> ip4:108.177.96.0/19
> ip4:35.191.0.0/16
> ip4:130.211.0.0/22
> 
> Regards,
> Andy

Sorry gmail.com and nasa.gov are not good examples because MX receive
mail and SPF listed TXT record send mail. However if you look at all the
SPF ipv4, /16 solve the problem if the pool sending again is in the same
CIDR class.

With SPF sender you can also use different delay to have a retry
accepted quickly (it's not a solution to the pool problem) and if the
sender retry period is short.

...
racl greylist spf fail delay 45m
racl greylist spf none delay 15m
racl greylist spf pass delay 1m
...

> 
> 

-- 
Christian P�lissier
\ue40aONERA DSI/ISR BP72 92322 Chatillon CEDEX\ue409
\ue201 34419

On Wed, Apr 24, 2019 at 02:30:32PM -0500, Bob Friesenhahn bfriesen@... [milter-greylist] wrote:
> On Wed, 24 Apr 2019, Steve Rikli sr@... [milter-greylist] wrote:
> >
> > But I'm wondering if anyone has found a more automatic / elegant way
> > of handling situations like this.
> 
> Using whitelisting DNS validation and/or SPF validation and some 
> measure of trust for hosts which pass these tests might be acceptable 
> when greylisting is too hard.

After checking the milter-greylist src README for ideas & references,
I found the section "dealing with mail farms" which mentions SPF as one
(partial?) solution.  I should have used the src earlier.  :-)

I'd originally turned off spf in my greylist.conf because I found many
spammers were passing that check, and conversations on this list and
elsewhere seemed to echo that reality.

But that seems like it may be a bigger hammer than I really want
nowdays; some helpful folks have posted a few snippet examples of using
spf with more precision -- is there a reference for that sort of usage,
and more greylist.conf rules examples?

The example greylist.conf (and annotated version) has the comment block
about "nospf" behavior, but nothing further I could see.

Thanks,
sr.