Milter-greylist

Jon, please refer to Matthias' previous email, regarding use of rbls to do 
greylisting, not blacklisting. Specifically these bits:

dnsrbl "SORBS DUN" dnsbl.sorbs.net 127.0.0.10
acl greylist dnsrbl "SORBS DUN" delay 12h

We get about 110 million emails a day inbound to our network at peak, with 
averages around 90 both out and in. We greylist everything and are pretty 
much not having any issues with our million+ mail addresses. The occasional 
domain may need to be whitelisted. While I am sure the BBC is a vast 
organization with many millions of messages being handled daily, the issue at 
hand is more one of how you decide to persue a resolution. I see that you 
handle about 1 million emails a day inbound. To be completely and totally 
frank with you, this is peanuts. In my humble opinion, you really should be 
running a much more draconian policy that utilizes third party databases of 
dynamic ips and overall broad greylisting, and only except that policy for 
particularly vociferous clients. Its far easier to greylist everything under 
the sun with varied durations and whitelist one problem user, than white list 
everyone and try to force a handful regular expressions to compensate for 
your overly lenient policy. In short, I would again suggest the following:

greylist everything for some minute time period (1-5 minutesish)
use dnsbl.sorbs to extend greylist durations as appropriate (2-8hours)
whitelist per complainy client

While Matt's regex hack he supplied does indeed cut memory costs, it is still 
by far the absolute most ineffecient way of doing things, simply because it 
will have to come before your default whitelist, causing everything thats not 
on a shortened greylist to be matched against it. The greylist all + 1min 
delay on non-dynamic ips (we greylist those in sorbs for 6 hours) has slashed 
our network bandwidth costs by 30%, and has knocked out a clean 78-85% of our 
inbound mail traffic. The principle behind the extremely short duration 
greylist is to obliterate botnets. Our broke mta whitelist numbers a meager 
84 entries; the combination of these settings allow us to handle the mail as 
previously mentioned with very little effect on performance (actually, we 
have seen a performance _gain_, due to the effect of not having to deal with 
mail delivery overhead), and run additional mail filtering in the way of 
spamassassin. 

Again, these are all just suggestions, but I very strongly feel your overly 
broad and resource intensive regex approach to the issue will ultimately bite 
you in the ass.


On Thursday 02 November 2006 11:48 am, Jonathan Perkin wrote:
> * On 2006-11-02 at 16:45 GMT, eclark wrote:
> > You might want to try either rewriting your rule to be more
> > reasonable
>
> ..which would look like..?
>
> > or use one of the varied rbl servers which specifically handle
> > dynamic ips.
>
> Possibly, but won't that end up doing the same thing?  I don't want to
> block dynamic IPs, just greylist them.
>
> > Even better, just greylist _everything_, and set exclusions as
> > appropriate.
>
> That is an inappropriate policy for the BBC, I would end up
> whitelisting millions of domains.