Milter-greylist

I posted about this a couple of weeks ago.  We are a large ISP and  
receive approx. 8+ million emails a day.  We maintain a list of white- 
listed that is around 6k entries at present.  The question was raised  
as to why we have so many entries.  We have this many entries as we  
have found that there are a lot of broken MTA's out there, and our  
customers need to receive mail from legitimate hosts regardless of  
the limitations of of the MTA (old Novell groupwise, misconfigured  
Postfix, etc).


We are seeing addresses being greylisted despite being whitelisted.

rb:/etc/mail# grep 216.139.107.5 greylist.conf
acl whitelist addr 216.139.107.5



Sep 14 06:55:47 k.eac milter-greylist: k8EBt3YY018760: addr  
ipserv5.lamoni.k12.ia.us[216.139.107.5] from <user@...>  
to <user@...> delayed for 00:02:00


any suggestions?

--
Michael Osten

Michael Osten wrote:
> I posted about this a couple of weeks ago.  We are a large ISP and  
> receive approx. 8+ million emails a day.  We maintain a list of white- 
> listed that is around 6k entries at present.  The question was raised  
> as to why we have so many entries.  We have this many entries as we  
> have found that there are a lot of broken MTA's out there, and our  
> customers need to receive mail from legitimate hosts regardless of  
> the limitations of of the MTA (old Novell groupwise, misconfigured  
> Postfix, etc).
> 
> 
> We are seeing addresses being greylisted despite being whitelisted.
> 
> rb:/etc/mail# grep 216.139.107.5 greylist.conf
> acl whitelist addr 216.139.107.5
> 

Are you sure that whitelist statement occurs before your greylist statement(s)?

Order matters, as milter-greylist will scan from the start of the list and act
on the first ACL that matches.

Michael Osten <mosten@...> wrote:

> I posted about this a couple of weeks ago.  We are a large ISP and  
> receive approx. 8+ million emails a day.  We maintain a list of white-
> listed that is around 6k entries at present.  
(snip) 
> We are seeing addresses being greylisted despite being whitelisted.

What version?

-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu@...

On Sep 14, 2006, at 12:44 PM, Matt Kettler wrote:

> Michael Osten wrote:
>> I posted about this a couple of weeks ago.  We are a large ISP and
>> receive approx. 8+ million emails a day.  We maintain a list of  
>> white-
>> listed that is around 6k entries at present.  The question was raised
>> as to why we have so many entries.  We have this many entries as we
>> have found that there are a lot of broken MTA's out there, and our
>> customers need to receive mail from legitimate hosts regardless of
>> the limitations of of the MTA (old Novell groupwise, misconfigured
>> Postfix, etc).
>>
>>
>> We are seeing addresses being greylisted despite being whitelisted.
>>
>> rb:/etc/mail# grep 216.139.107.5 greylist.conf
>> acl whitelist addr 216.139.107.5
>>
>
> Are you sure that whitelist statement occurs before your greylist  
> statement(s)?
>
> Order matters, as milter-greylist will scan from the start of the  
> list and act
> on the first ACL that matches.


I was just reading that, but in the default greylist.conf file the  
"acl greylist default" was not at the bottom of the file.
--
Michael Osten

Michael Osten wrote:
> On Sep 14, 2006, at 12:44 PM, Matt Kettler wrote:
> 
>> Michael Osten wrote:
>>> I posted about this a couple of weeks ago.  We are a large ISP and
>>> receive approx. 8+ million emails a day.  We maintain a list of  
>>> white-
>>> listed that is around 6k entries at present.  The question was raised
>>> as to why we have so many entries.  We have this many entries as we
>>> have found that there are a lot of broken MTA's out there, and our
>>> customers need to receive mail from legitimate hosts regardless of
>>> the limitations of of the MTA (old Novell groupwise, misconfigured
>>> Postfix, etc).
>>>
>>>
>>> We are seeing addresses being greylisted despite being whitelisted.
>>>
>>> rb:/etc/mail# grep 216.139.107.5 greylist.conf
>>> acl whitelist addr 216.139.107.5
>>>
>> Are you sure that whitelist statement occurs before your greylist  
>> statement(s)?
>>
>> Order matters, as milter-greylist will scan from the start of the  
>> list and act
>> on the first ACL that matches.
> 
> 
> I was just reading that, but in the default greylist.conf file the  
> "acl greylist default" was not at the bottom of the file.

Sounds like a bug in the default config for your release. Do you have 3.0a1?
That's the only version that I have that suffers from this bug, but my
collection is not comprehensive.

3.0a2 and higher have correctly moved it to the bottom of the file.

2.1.2-5 and 2.0.2 don't even have the word "default" in the file at all.

>
> 3.0a2 and higher have correctly moved it to the bottom of the file.
>
> 2.1.2-5 and 2.0.2 don't even have the word "default" in the file at  
> all.
>


I'm running 2.1.2-5, and you are right, the "default" acl was not  
present.  I've moved it to the bottom of the file (I have a perl  
script that builds the whitelist acl's from a database.  Our support  
org can whitelist an ip via a web interface).
--
Michael Osten

On Thu, Sep 14, 2006 at 01:44:09PM -0400, Matt Kettler (mkettler@...) wrote:
> Michael Osten wrote:

{SNIP}

> Are you sure that whitelist statement occurs before your greylist statement(s)?
> 
> Order matters, as milter-greylist will scan from the start of the list and act
> on the first ACL that matches.

I am "hi-jacking" this thread from a previous thread "Changing return msg, code and ecode".

I have "milter-greylist-3.0a1" which has the greylist default
entry somewhere in the middle and changing the "return msg, code and ecode"
in the middle before the whitelist started greylisted even my INTERNAL
network!

I read the manual as in "man greylist.conf" but this did not ring any bells,
your reply did.

You just solved a problem for me!
Thank you!


jobst




-- 
We're from the government, we're here to help you...

             __, Jobst Schmalenbach, Technical Director
   _ _.--'-n_/   Barrett Consulting Group P/L & The Meditation Room P/L      
 -(_)------(_)=  +61 3 9532 7677, POBox 277, Caulfield South, 3162, Australia

Hi,

>>>>> On Thu, 14 Sep 2006 14:20:38 -0400
>>>>> Matt Kettler <mkettler@...> said:

> I was just reading that, but in the default greylist.conf file the  
> "acl greylist default" was not at the bottom of the file.

mkettler> Sounds like a bug in the default config for your release. Do you have 3.0a1?
mkettler> That's the only version that I have that suffers from this bug, but my
mkettler> collection is not comprehensive.

mkettler> 3.0a2 and higher have correctly moved it to the bottom of the file.

mkettler> 2.1.2-5 and 2.0.2 don't even have the word "default" in the file at all.

I feel that `default' is not a good keyword then.  It sounds to me
that the location in greylist.conf doesn't matter.  Something like
`any' is better, IMHO.

Sincerely,

--
Hajimu UMEMOTO @ Internet Mutual Aid Society Yokohama, Japan
ume@...  ume@{,jp.}FreeBSD.org
http://www.imasy.org/~ume/

Hajimu UMEMOTO <ume@...> wrote:

> I feel that `default' is not a good keyword then.  It sounds to me
> that the location in greylist.conf doesn't matter.  Something like
> `any' is better, IMHO.

Order of acl lines does matters: milter-gresylist stops reading the ACL
on first match.

'acl {grey|white|black}list default' will match anything, so if you have
more acl lines after such a line, they will be always ignored.

Perhaps milter-greylist should just issue a warning if it finds more acl
rules after a default rule.

-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu@...

I fully agree with Umes point. For a default rule like this, its position in 
the conf file should not matter. Ideally, all whitelist information should be 
appended to the end of the file, rather than middle or elsewhere, as this is 
most likely to change. In our environment, we append our whitelist acls to 
the end of a template and rebuild the conf several times a day through 
automated means. Migrating to 3.0 where this rule exists (it doesnt live in 
202) would be impossible for us without significantly changing the way we do 
automatic updates to our conf file. I agree for anything else, black or grey, 
order is fine. But this default rule that has to go at the end? Just a bad 
idea imho. I can work on patching this out if others are interested, but I 
have no idea how quickly I could get it done due to my current work load.

On Thursday 14 September 2006 09:45 pm, Hajimu UMEMOTO wrote:
> Hi,
>
> >>>>> On Thu, 14 Sep 2006 14:20:38 -0400
> >>>>> Matt Kettler <mkettler@...> said:
> >
> > I was just reading that, but in the default greylist.conf file the
> > "acl greylist default" was not at the bottom of the file.
>
> mkettler> Sounds like a bug in the default config for your release. Do you
> have 3.0a1? mkettler> That's the only version that I have that suffers from
> this bug, but my mkettler> collection is not comprehensive.
>
> mkettler> 3.0a2 and higher have correctly moved it to the bottom of the
> file.
>
> mkettler> 2.1.2-5 and 2.0.2 don't even have the word "default" in the file
> at all.
>
> I feel that `default' is not a good keyword then.  It sounds to me
> that the location in greylist.conf doesn't matter.  Something like
> `any' is better, IMHO.
>
> Sincerely,
>
> --
> Hajimu UMEMOTO @ Internet Mutual Aid Society Yokohama, Japan
> ume@...  ume@{,jp.}FreeBSD.org
> http://www.imasy.org/~ume/
>
>
>
> Yahoo! Groups Links
>
>
>
>
>
>

>>>
>>> I was just reading that, but in the default greylist.conf file the
>>> "acl greylist default" was not at the bottom of the file.
>>
>> mkettler> Sounds like a bug in the default config for your  
>> release. Do you
>> have 3.0a1? mkettler> That's the only version that I have that  
>> suffers from
>> this bug, but my mkettler> collection is not comprehensive.
>>
>> mkettler> 3.0a2 and higher have correctly moved it to the bottom  
>> of the
>> file.
>>
>> mkettler> 2.1.2-5 and 2.0.2 don't even have the word "default" in  
>> the file
>> at all.
>>
>> I feel that `default' is not a good keyword then.  It sounds to me
>> that the location in greylist.conf doesn't matter.  Something like
>> `any' is better, IMHO.


Ok, so then I'm confused.  Does 2.1.2-5 require that the "acl  
greylist default" be at the bottom of the file? Or is "default" not a  
valid option for this version?
--
Michael Osten

At 08:26 AM 9.15.2006 -0500, Michael Osten wrote:
>
>>>>
>>>> I was just reading that, but in the default greylist.conf file the
>>>> "acl greylist default" was not at the bottom of the file.
>>>
>>> mkettler> Sounds like a bug in the default config for your  
>>> release. Do you
>>> have 3.0a1? mkettler> That's the only version that I have that  
>>> suffers from
>>> this bug, but my mkettler> collection is not comprehensive.
>>>
>>> mkettler> 3.0a2 and higher have correctly moved it to the bottom  
>>> of the
>>> file.
>>>
>>> mkettler> 2.1.2-5 and 2.0.2 don't even have the word "default" in  
>>> the file
>>> at all.
>>>
>>> I feel that `default' is not a good keyword then.  It sounds to me
>>> that the location in greylist.conf doesn't matter.  Something like
>>> `any' is better, IMHO.
>
>
>Ok, so then I'm confused.  Does 2.1.2-5 require that the "acl  
>greylist default" be at the bottom of the file? Or is "default" not a  
>valid option for this version?
>--
>Michael Osten
>

That's where I have always kept it through the many past versions ever
since we had more and more custom specific "rules" to apply to certain
targets -- then everything else thereafter naturally applied to the normal
greylisting rule. It *needs* to be last -- as long as you allow recipients
in you didn't expect emails from.

I really like the configurable things that have been added over the past
couple of years. MGL has really matured into a powerful program and much
praise to Emmanuel and those that are capable of contributing those great
patches!

IMHO from just a dumb user....

(^_^)
Happy trails,
Jack L. Stone

System Admin
Sage-american

Michael Osten wrote:
> 
> 
> Ok, so then I'm confused.  Does 2.1.2-5 require that the "acl  
> greylist default" be at the bottom of the file? Or is "default" not a  
> valid option for this version?
> --

*IF* you include an "acl xxx default", it must be at the bottom, even in 2.0b2.

However, AFAIK the 2.x series assumes greylist for anything not matched by any
ACLs, so there's an implicit "acl greylist default" anyway.

So, you aren't required to have this statement, but if you do, it must be the last.