Milter-greylist

I have just noticed that recently gmail.com mails also come from the 
addresses in the range from 74.125.46.24 to 74.125.46.63. However, these are 
not included in the suite of whitelisted IPs in greylist.conf.

Andrzej

-- 
-----------------------------------------------------------------------------  
Andrzej Marecki                | 
Torun Centre for Astronomy     |   e-mail: amr@...
N. Copernicus University       |   WWW:    http://www.astro.uni.torun.pl
ul. Gagarina 11                |   tel: +48 56 6113032
PL-87-100 Torun, POLAND        |   fax: +48 56 6113009
-----------------------------------------------------------------------------

Andrzej Marecki wrote:
> I have just noticed that recently gmail.com mails also come from
> the addresses in the range from 74.125.46.24 to 74.125.46.63.
> However, these are not included in the suite of whitelisted IPs in
> greylist.conf.

Google's SMTP server farm is too big to pass greylisting.  They have
139,000+ servers listed as legitimate in their SPF record, so if the
odds of *ever* sending from the same server with the same sender and
recipient email addresses are all but zero.  While I doubt they've
actually enlisted all 139k servers for SMTP, they certainly have
enough to send users delivery warnings, and I've seen mail bounce
thanks to not doubling back on the same server within the 4 day period.

My solution was to include ALL of gmail/google's SPF records in the
milter-greylist "broken mta" whitelist.  It looks like this
(whitespace shortened to fit in 72-column email):

list "broken mta" addr {   \
  [...]
  216.239.32.0/19 64.233.160.0/19 66.249.80.0/20 72.14.192.0/18 \
  209.85.128.0/17 66.102.0.0/20 74.125.0.0/16 \ # google/gmail
}

I didn't remove redundant entries above it so that a quick diff on the
next released sample config would still pass (rather than having to
review each "new" google entry).  Milter-greylist does not complain
about the duplicates.

This is (obviously) not needed unless you've enabled "nospf"
I got this idea from:
http://bsdly.blogspot.com/2007_07_01_archive.html#spf-potentially-useful

See also our previous thread on this topic from March, archived at
http://tech.groups.yahoo.com/group/milter-greylist/message/4228

-Adam

--- In milter-greylist@yahoogroups.com, Adam Katz <yegsa-yahoo@...> wrote:

> My solution was to include ALL of gmail/google's SPF records in the
> milter-greylist "broken mta" whitelist.  It looks like this
> (whitespace shortened to fit in 72-column email):
> 
> list "broken mta" addr {   \
>   [...]
>   216.239.32.0/19 64.233.160.0/19 66.249.80.0/20 72.14.192.0/18 \
>   209.85.128.0/17 66.102.0.0/20 74.125.0.0/16 \ # google/gmail
> }

Except the last item - 74.125.0.0/16 - all else have been included in
the template greylist.conf bundled with milter-greylist (ver. 4.0)
sources. Thus, I'd suggest to add 74.125.0.0/16, too. (I did manually
and it helped!)

--
Andrzej

Adam Katz <yegsa-yahoo@...> wrote:

> My solution was to include ALL of gmail/google's SPF records in the
> milter-greylist "broken mta" whitelist.  It looks like this
> (whitespace shortened to fit in 72-column email):

The following ACL should work with 4.0:
racl whitelist from /.*@gmail\.com/ spf pass

Nice, isn't it?

-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu@...

> The following ACL should work with 4.0:
> racl whitelist from /.*@gmail\.com/ spf pass
>
> Nice, isn't it?

That does ~80% of the job, yes.
Unfortunately, a good number of people out there send as

  From: myname@...
  Sender: mygoogleaccount@...
thus sending mail from google servers but not gmail.com accounts.

I'd prefer something like 'racl whitelist spf gmail.com'
or 'list "safe by spf" { gmail.com otherplace.net }'

-Adam

manu@... a \ufffdcrit :
> Adam Katz <yegsa-yahoo@...> wrote:
> 
>> My solution was to include ALL of gmail/google's SPF records in the
>> milter-greylist "broken mta" whitelist.  It looks like this
>> (whitespace shortened to fit in 72-column email):
> 
> The following ACL should work with 4.0:

No, only in 4.1.1.
SPF status code check has been added after 4.0 was released.

But this example is a classical "pass" test, so in 4.0 it could be written:

racl whitelist from /.*@gmail\.com/ spf

.. And to be sure to avoid "subdomain match abuse" ( ex. 
mail@...), the regex should better be written like this:

racl whitelist from /@gmail\.com>/ spf


> racl whitelist from /.*@gmail\.com/ spf pass
> 
> Nice, isn't it?
> 


-- 
Ce message a ete verifie par MailScanner
pour des virus ou des polluriels et rien de
suspect n'a ete trouve.

Adam Katz a \ufffdcrit :
>> The following ACL should work with 4.0:
>> racl whitelist from /.*@gmail\.com/ spf pass
>>
>> Nice, isn't it?
> 
> That does ~80% of the job, yes.
> Unfortunately, a good number of people out there send as
>   From: myname@...
>   Sender: mygoogleaccount@...
> thus sending mail from google servers but not gmail.com accounts.

So the envelope sender is mygoogleaccount@..., right ? Then it works.

-- 
Ce message a ete verifie par MailScanner
pour des virus ou des polluriels et rien de
suspect n'a ete trouve.

Benoit Branciard <benoit.branciard@...> wrote:

> No, only in 4.1.1.
> SPF status code check has been added after 4.0 was released.

Right. Perhaps I should resync the 4.0 branch with 4.1.x, as the new
features seems quite stable.

-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu@...

Hi Benoit,

> manu@... a \ufffdcrit :
> > Adam Katz <yegsa-yahoo@...> wrote:
> > 
> >> My solution was to include ALL of gmail/google's SPF records in the
> >> milter-greylist "broken mta" whitelist.  It looks like this
> >> (whitespace shortened to fit in 72-column email):
> > 
> > The following ACL should work with 4.0:
> 
> No, only in 4.1.1.
> SPF status code check has been added after 4.0 was released.
> 
> But this example is a classical "pass" test, so in 4.0 it could be written:
> 
> racl whitelist from /.*@gmail\.com/ spf
> 
> .. And to be sure to avoid "subdomain match abuse" ( ex. 
> mail@...), the regex should better be written like this:
> 
> racl whitelist from /@gmail\.com>/ spf

I've been following this thread and would also like to whitelist gmail with
valid spf.

I run 4.1.1

Do I need to just do:

racl whitelist from /@gmail\.com>/ spf

?

or do I also need to add all the IP's that was previously posted also?

Is there any compile option needed for milter-greylist 4.1.1 that I need to
have the spf checks there?

Thanks.

Michael.

> > racl whitelist from /.*@gmail\.com/ spf pass
> > 
> > Nice, isn't it?
> >
> 
> -- 
> Ce message a ete verifie par MailScanner
> pour des virus ou des polluriels et rien de
> suspect n'a ete trouve.
> 
> ------------------------------------
> 
> Yahoo! Groups Links
> 
> 
>

Michael Mansour <mic@...> wrote:

> racl whitelist from /@gmail\.com>/ spf

Or better (don't start with deprecated syntax):
racl whitelist from /@gmail\.com>/ spf pass

-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu@...

Hi Emmanuel,

> Michael Mansour <mic@...> wrote:
> 
> > racl whitelist from /@gmail\.com>/ spf
> 
> Or better (don't start with deprecated syntax):
> racl whitelist from /@gmail\.com>/ spf pass

Ok, this is nice. This is equivalent to:

racl whitelist from /.*@gmail\.com$/ spf pass

yes?

Michael.

> -- 
> Emmanuel Dreyfus
> http://hcpnet.free.fr/pubz
> manu@...
> 
> ------------------------------------
> 
> Yahoo! Groups Links
> 
> 
>

On Thu, May 15, 2008 at 05:08:08PM +1100, Michael Mansour wrote:
> > racl whitelist from /@gmail\.com>/ spf pass
> Ok, this is nice. This is equivalent to:
> racl whitelist from /.*@gmail\.com$/ spf pass

Yes and no: there was a typo. I meant 
racl whitelist from /@gmail\.com$/ spf pass

This is equivalent.

-- 
Emmanuel Dreyfus
manu@...

Emmanuel Dreyfus a \ufffdcrit :
> On Thu, May 15, 2008 at 05:08:08PM +1100, Michael Mansour wrote:
>>> racl whitelist from /@gmail\.com>/ spf pass
>> Ok, this is nice. This is equivalent to:
>> racl whitelist from /.*@gmail\.com$/ spf pass
> 
> Yes and no: there was a typo. I meant 
> racl whitelist from /@gmail\.com$/ spf pass
> 
> This is equivalent.
> 

None of these are totally equivalents.

- /.*@gmail\.com/ and /@gmail\.com/ *are* equivalents, and mean "any 
address that contains the string "@...". In this case the leading 
".*" is unsignificant.

- /@gmail\.com>/ means "any address ending with "@...>". In 
milter-greylist context, this means any address whose domain *is* 
"gmail.com" (the enclosing envelope <> are always included in the match).

- /@gmail\.com$/ means "any address ending with "@...". In 
milter-greylist this would never match, since all envelope FROMs always 
end with ">".


-- 
Ce message a ete verifie par MailScanner
pour des virus ou des polluriels et rien de
suspect n'a ete trouve.

Michael Mansour a \ufffdcrit :
> 
> Is there any compile option needed for milter-greylist 4.1.1 that I need to
> have the spf checks there?
> 

You need some version of libspf installed on your system (with devel 
headers) and the appropriate "--with-libspfXX=YY" option to your 
./configure.

For example to compile it on Debian Sarge you need the packages 
libspf2-0 and libspf2-dev, and the configure option 
"--with-libspf2_10=/usr/lib".

Also take care to increase the milter timeout in your MTA config (see 
the README), since SPF checks may be time-consuming and default Sendmail 
timeout is ways too small.


-- 
Ce message a ete verifie par MailScanner
pour des virus ou des polluriels et rien de
suspect n'a ete trouve.

> 
> - /@gmail\.com$/ means "any address ending with "@...". In 
> milter-greylist this would never match, since all envelope 
> FROMs always 
> end with ">".
> 

Do the spammers know that they must comply to the standard?
I already seen e-mail addresses in the envelop without opening "<" and closing ">".
Unfortunately the sendmail happily accepts it.
Since that time I always use like:
/@gmail\.com[ 	>]*$/
(Please note, there are a tab and a space in addition to the ">" in the brackets.)

Bests,
Attila

attila.bruncsak@... a \ufffdcrit :
>> - /@gmail\.com$/ means "any address ending with "@...". In 
>> milter-greylist this would never match, since all envelope 
>> FROMs always 
>> end with ">".
>>
> 
> Do the spammers know that they must comply to the standard?
> I already seen e-mail addresses in the envelop without opening "<" and closing ">".
> Unfortunately the sendmail happily accepts it.
> Since that time I always use like:
> /@gmail\.com[ 	>]*$/
> (Please note, there are a tab and a space in addition to the ">" in the brackets.)
> 

You're right, I just verified Sendmail accepts envelope FROM's and 
RCPT's without angle brackets, and they are transmitted as-is to 
milter-greylist. So your regexp seems the right way to go...

-- 
Ce message a ete verifie par MailScanner
pour des virus ou des polluriels et rien de
suspect n'a ete trouve.

Hi,

> > Is there any compile option needed for milter-greylist 4.1.1 that I need to
> > have the spf checks there?
> 
> You need some version of libspf installed on your system (with devel 
> headers) and the appropriate "--with-libspfXX=YY" option to your 
> ./configure.

I have the libspf2 and libspf2-devel RPM's installed on my servers (RHEL4
based). I use the milter-greylist.spec and add a section in there as:

%define libspf2 1
%{?build_libspf2:%define libspf2 1}

and then further down:

%if %{libspf2}
        --with-libspf2=/usr/lib
%endif

which allows me to use:

rpmbuild -bb milter-greylist.spec

to build the RPM with dnsrbl and spf support.

> For example to compile it on Debian Sarge you need the packages 
> libspf2-0 and libspf2-dev, and the configure option 
> "--with-libspf2_10=/usr/lib".

When building the RPM, I see the:

--enable-dnsrbl
--with-libspf2=/usr/lib

which is part of the line:

+ ./configure --build=i686-redhat-linux-gnu --host=i686-redhat-linux-gnu
--target=i386-redhat-linux-gnu --program-prefix= --prefix=/usr
--exec-prefix=/usr --bindir=/usr/bin --sbindir=/usr/sbin --sysconfdir=/etc
--datadir=/usr/share --includedir=/usr/include --libdir=/usr/lib
--libexecdir=/usr/libexec --localstatedir=/var --sharedstatedir=/usr/com
--mandir=/usr/share/man --infodir=/usr/share/info --with-user=root
--enable-dnsrbl --with-libspf2=/usr/lib

Once I install the RPM and run milter-greylist, the very first message it
processes bombs it out ie. milter-greylist crashes.

When I disable libspf2 support, milter-greylist works fine.

Have no idea why.

Am I correct in defining /usr/lib as the source?

This is what the RPM's have:

# rpm -ql libspf2
/usr/lib/libspf2.so.2
/usr/lib/libspf2.so.2.0.0
/usr/share/doc/libspf2-1.2.5
/usr/share/doc/libspf2-1.2.5/Changelog
/usr/share/doc/libspf2-1.2.5/INSTALL
/usr/share/doc/libspf2-1.2.5/LICENSES
/usr/share/doc/libspf2-1.2.5/README
/usr/share/doc/libspf2-1.2.5/TODO
/usr/share/doc/libspf2-1.2.5/announce_v0.1.txt
/usr/share/doc/libspf2-1.2.5/announce_v0.2.txt
/usr/share/doc/libspf2-1.2.5/announce_v0.3.txt
/usr/share/doc/libspf2-1.2.5/announce_v0.4.txt
/usr/share/doc/libspf2-1.2.5/draft-mengwong-spf-00.txt

# rpm -ql libspf2-devel
/usr/include/spf2/spf.h
/usr/include/spf2/spf_dns.h
/usr/include/spf2/spf_dns_cache.h
/usr/include/spf2/spf_dns_null.h
/usr/include/spf2/spf_dns_resolv.h
/usr/include/spf2/spf_dns_rr.h
/usr/include/spf2/spf_dns_test.h
/usr/include/spf2/spf_dns_zone.h
/usr/include/spf2/spf_lib_version.h
/usr/include/spf2/spf_log.h
/usr/include/spf2/spf_record.h
/usr/include/spf2/spf_request.h
/usr/include/spf2/spf_response.h
/usr/include/spf2/spf_server.h
/usr/lib/libspf2.a
/usr/lib/libspf2.so
/usr/share/doc/libspf2-devel-1.2.5
/usr/share/doc/libspf2-devel-1.2.5/API

Any advice?

> Also take care to increase the milter timeout in your MTA config 
> (see the README), since SPF checks may be time-consuming and default 
> Sendmail timeout is ways too small.

Hmm.. thanks for this tip, if I do end up getting the spf support working,
I'll make sure I do this too.

Michael.

> -- 
> Ce message a ete verifie par MailScanner
> pour des virus ou des polluriels et rien de
> suspect n'a ete trouve.
> 
> ------------------------------------
> 
> Yahoo! Groups Links
> 
> 
>

Michael Mansour a \ufffdcrit :
> 
> I have the libspf2 and libspf2-devel RPM's installed on my servers (RHEL4
> based). I use the milter-greylist.spec and add a section in there as:
> [...]
> 
> Once I install the RPM and run milter-greylist, the very first message it
> processes bombs it out ie. milter-greylist crashes.
> 
> When I disable libspf2 support, milter-greylist works fine.
> 

Which version of SPF2 is contained in the package libspf2 ?

The API of libspf2 1.0 is incompatible with the one of later versions; 
you have to be sure your "--with-libspfXX=YY" matches your libspf2 API.

Or it may be that recent changes in Milter-greylist SPF code (SPF status 
  matching) still contains bugs under some libSPF APIs; I personally 
only tested libspf2 1.0...


-- 
Ce message a ete verifie par MailScanner
pour des virus ou des polluriels et rien de
suspect n'a ete trouve.

Hi Benoit,

> Michael Mansour a \ufffdcrit :
> > 
> > I have the libspf2 and libspf2-devel RPM's installed on my servers (RHEL4
> > based). I use the milter-greylist.spec and add a section in there as:
> > [...]
> > 
> > Once I install the RPM and run milter-greylist, the very first message it
> > processes bombs it out ie. milter-greylist crashes.
> > 
> > When I disable libspf2 support, milter-greylist works fine.
> >
> 
> Which version of SPF2 is contained in the package libspf2 ?

# rpm -qi libspf2
Name        : libspf2                      Relocations: (not relocatable)
Version     : 1.2.5                             Vendor: city-fan.org repo
http://www.city-fan.org/ftp/contrib/
Release     : 3                             Build Date: Mon 12 Feb 2007
19:30:22 EST
Install Date: Mon 02 Apr 2007 20:20:58 EST      Build Host:
metropolis.intra.city-fan.org
Group       : System Environment/Libraries   Source RPM: libspf2-1.2.5-3.src.rpm
Size        : 206169                           License: BSD & LGPL
Signature   : DSA/SHA1, Mon 12 Feb 2007 19:30:59 EST, Key ID e0be69c9b56a8bac
Packager    : Paul Howarth <paul@...>
URL         : http://www.libspf2.org/
Summary     : An implementation of the SPF specification
Description :
libspf2 is an implementation of the SPF (Sender Policy Framework)
specification as found at:
http://www.ietf.org/internet-drafts/draft-mengwong-spf-00.txt
SPF allows email systems to check SPF DNS records and make sure that
an email is authorized by the administrator of the domain name that
it is coming from. This prevents email forgery, commonly used by
spammers, scammers, and email viruses/worms.

A lot of effort has been put into making it secure by design, and a
great deal of effort has been put into the regression tests.

> The API of libspf2 1.0 is incompatible with the one of later 
> versions; you have to be sure your "--with-libspfXX=YY" matches your 
> libspf2 API.

Yes, I made sure of this. libspf2 1.2.5 is what I'm using so I used:

--with-libspf2=/usr/lib

> Or it may be that recent changes in Milter-greylist SPF code (SPF 
> status   matching) still contains bugs under some libSPF APIs; I 
> personally only tested libspf2 1.0...

Hmmm.. I'm using milter-greylist 4.1.1 and as the rpm -qi shows above for the
libspf2 (RPM obtained from http://www.libspf2.org) it should be a version
that's ok.

Is there anything I can do to help trouble-shoot this for Emmanuel to see if
it is a bug in milter-greylist?

Michael.

Am Donnerstag Mai 15 2008 16:00 schrieb Benoit Branciard:
> attila.bruncsak@... a écrit :
> >> - /@gmail\.com$/ means "any address ending with "@...". In
> >> milter-greylist this would never match, since all envelope
> >> FROMs always
> >> end with ">".
> >
> > Do the spammers know that they must comply to the standard?
> > I already seen e-mail addresses in the envelop without opening "<" and
> > closing ">". Unfortunately the sendmail happily accepts it.
> > Since that time I always use like:
> > /@gmail\.com[ 	>]*$/
> > (Please note, there are a tab and a space in addition to the ">" in the
> > brackets.)
>
> You're right, I just verified Sendmail accepts envelope FROM's and
> RCPT's without angle brackets, and they are transmitted as-is to
> milter-greylist. So your regexp seems the right way to go...

may be I am missing something, but why would you want to whitelist spammers 
that don't comply to the standard? 
Isn't it that legitimate(!) mails form gmail.com always would 
have "gmail.com>" in their envelope-from?

Juergen

> Am Donnerstag Mai 15 2008 16:00 schrieb Benoit Branciard:
> > attila.bruncsak@... a écrit :
> > >> - /@gmail\.com$/ means "any address ending with "@...". In
> > >> milter-greylist this would never match, since all envelope
> > >> FROMs always
> > >> end with ">".
> > >
> > > Do the spammers know that they must comply to the standard?
> > > I already seen e-mail addresses in the envelop without 
> opening "<" and
> > > closing ">". Unfortunately the sendmail happily accepts it.
> > > Since that time I always use like:
> > > /@gmail\.com[ 	>]*$/
> > > (Please note, there are a tab and a space in addition to 
> the ">" in the
> > > brackets.)
> >
> > You're right, I just verified Sendmail accepts envelope FROM's and
> > RCPT's without angle brackets, and they are transmitted as-is to
> > milter-greylist. So your regexp seems the right way to go...
> 
> may be I am missing something, but why would you want to 
> whitelist spammers 
> that don't comply to the standard? 
> Isn't it that legitimate(!) mails form gmail.com always would 
> have "gmail.com>" in their envelope-from?
> 

My note wasn't specific to gmail.com in whitelising their servers if SPF matches.
I used the term "like" gmail.com just for an example.
There are other valid ACL context where it really counts the precise match, for example:

racl greylist rcpt /@mycompany\.com>/

and

racl greylist rcpt /@mycompany\.com[ 	>]*$/

would give different result in the greylisting if the spammer use or
does not use surrounding "<" and ">" for the address myuser@... .

I hope this clarifies the point.

Bests,
Attila