Milter-greylist

All,

I just spent an hour trying to explain greylisting to one of my end-users 
(I greylist against APEWS)

I am registering the above domain name for the usage as follows:

1) Placement of a page to document what greylisting is.

2) Placement of a page to document known MTA's and organizations which may 
have troubles sending to greylisted servers -- to include things like 
misconfigured setups, etc.

3) Hosting of a DNSWL, that people can (with manual review) opt themselves 
into, such that their mailserver will be bypassed by milter-greylist's 
DNSBL support.

Questions on the submission form will include:

1) Reason your server does not retry

2) Mail server versions (optional).

3) IP addresses.

4) Other strategies in play (SPF/DomainKeys/DKIM)

5) Outbound envelope domains.

6) Expected fix date for the issue (i.e. I'll accept an answer of "Never" 
but it's nice to know that this is going to continue to be a problem).

I would of course also provide links to the milter-greylist official site, 
links to RFC2821 and RFC821, as well as to other greylist implementations 
(Anthony Howe's Milter-Gris, DCC, etc).

Is there anyone who would be interested in contributing some content to 
this site -- FAQ's, etc?  I'd also welcome some testing of the zonefile 
format -- I'm hosting this with standard BIND, not any special RBLDNSd, so 
the goal is to write some sort of script to handle the conversion from 
database, to text-based zonefile and standard text list (which can be 
fetched into a normal greylist config without the RBL support and pulled 
in via INCLUDES).  Needless to say the parsing of something like, say, a 
/29's worth of whitelist into a zonefile is an interesting trick.

I also plan to release all the code as public.

It should be a day or so before the domain itself is up, but I'm going to 
initially-populate the zonefile (white.lightgreylist.org in a few minutes) 
-- you should be able to poll it from ns.gushi.org and ns2.gushi.org.

Any feedback on this thought?  Everything from "It's a terrible idea, you 
suck, here's why" (and it's that last part that's important)...to "here's 
how I can help" would be appreciated.

-Dan

--

"If you aren't going to try something, then we might as well just be
friends."

"We can't have that now, can we?"

-SK & Dan Mahoney,  December 9, 1998

--------Dan Mahoney--------
Techie,  Sysadmin,  WebGeek
Gushi on efnet/undernet IRC
ICQ: 13735144   AIM: LarpGM
Site:  http://www.gushi.org
---------------------------

On 10/24/07, Dan Mahoney, System Admin <danm@...> wrote:
>  1) Placement of a page to document what greylisting is.
>
>  2) Placement of a page to document known MTA's and organizations which may
>  have troubles sending to greylisted servers -- to include things like
>  misconfigured setups, etc.
>
>  3) Hosting of a DNSWL, that people can (with manual review) opt themselves
>  into, such that their mailserver will be bypassed by milter-greylist's
>  DNSBL support.

You may want to bring all the above together in something that you
feel is better than what is available today, I just want to make sure
you know that it IS available today.

1. http://en.wikipedia.org/wiki/Greylisting,
http://www.greylisting.org, and many more of course.

2. http://cvs.puremagic.com/viewcvs/greylisting/schema/whitelist_ip.txt

3. http://www.dnswl.org/

-- 
/peter

> From: milter-greylist@yahoogroups.com On Behalf Of Dan Mahoney, System
Admin
> 
> All,
> 
> I just spent an hour trying to explain greylisting to one of my end-users

  Been there, done that. :)  When the smoke cleared I ended up creating a
daily
report that users could opt into that would tell them what had been filtered
from
their mailbox during the previous 24 hours, and why.  The explination for
greylisting went something like this:

  Any mail server attempting to send you email, that cares whether you
actually
receive that email, will attempt to deliver it more than once.  This is
because
there are a large number of reasons why the first attempt might not succeed,
the
Internet is far from a perfect system.  Greylisting is where we reject the
first
attempt at delivering the message to you, knowing that they will try again a
little later.  This works because spammers using hijacked PC's don't retry
delivery while properly configured mail servers do.

  However, there are always mail servers run by companies who you'd think
should
know what they are doing, that are actually run by idiots, or even
deliberatly
configured to throw away the email to you if it can't be delivered the first
time
they try.  You'd be surprised at the names of some of these companies, we
were
the first time we saw it happen.  So, if you really care to receive email
from
companies that have made it clear they don't care whether email they send to
you actually gets to you, you may check the following box to disable
Greylisting
on your account.

  [ ] Disable greylisting on my email account.

  Warning: this will potentially expose your email account to a significant
increase in spam.  By reading your daily filtering report for a week or so,
and watching how much filtering is done by Greylisting, you will be better
equiped to understand the risk.

  All this was done via a daily email to the user, if they'd actually had
any
email filtered, and it directed them to an internal website where they could
access a filtering control form that contained the above text and checkbox.
Users who called the support desk and asked "could you just let this one
email
through" were told to access the webpage and disable Greylisting in general,
or we'd disable it for them if they didn't want to use the form.  After a
few
months of use we found that most customers quickly turned greylisting back
on
once they saw what they'd have to wade through.  We also encouraged them to
write to the sender who's email didn't get delivered because of a defective
or mis-configured mail server and have them bitch at their ISP and demand to
know why their outgoing email wasn't being sent with proper retries.

On Wed, Oct 24, 2007 at 02:10:17PM -0400, Dan Mahoney, System Admin wrote:
> Is there anyone who would be interested in contributing some content to 
> this site -- FAQ's, etc?  I'd also welcome some testing of the zonefile 
> format -- I'm hosting this with standard BIND, not any special RBLDNSd, so 
> the goal is to write some sort of script to handle the conversion from 
> database, to text-based zonefile and standard text list (which can be 
> fetched into a normal greylist config without the RBL support and pulled 
> in via INCLUDES).  Needless to say the parsing of something like, say, a 
> /29's worth of whitelist into a zonefile is an interesting trick.

You can also use DNS dynamic updates to manage your zone file from your 
forms.

> Any feedback on this thought?  Everything from "It's a terrible idea, you 
> suck, here's why" (and it's that last part that's important)...to "here's 
> how I can help" would be appreciated.

I can provide you a secondary DNS if you need one.

-- 
Emmanuel Dreyfus
manu@...

On Wed, 24 Oct 2007, shuttlebox wrote:

> On 10/24/07, Dan Mahoney, System Admin <danm@...> wrote:
>>  1) Placement of a page to document what greylisting is.
>>
>>  2) Placement of a page to document known MTA's and organizations which may
>>  have troubles sending to greylisted servers -- to include things like
>>  misconfigured setups, etc.
>>
>>  3) Hosting of a DNSWL, that people can (with manual review) opt themselves
>>  into, such that their mailserver will be bypassed by milter-greylist's
>>  DNSBL support.
>
> You may want to bring all the above together in something that you
> feel is better than what is available today, I just want to make sure
> you know that it IS available today.
>
> 1. http://en.wikipedia.org/wiki/Greylisting,
> http://www.greylisting.org, and many more of course.

Part of my desire here would be to actually put the code for this site 
right in the 451 return code.

> 2. http://cvs.puremagic.com/viewcvs/greylisting/schema/whitelist_ip.txt

Last updated 19 months ago, crashes at the end with a python traceback.

> 3. http://www.dnswl.org/

I've got my own issues with them, and we wouldn't be serving the same 
purpose.  My whitelist would be PURELY a dynamic list of "people not to 
greylist", not "people to give additional kudos in SpamAssassin, etc." 
DNSWL has added a bunch of mail relays (like LiveJournal) which relay mail 
for a given forwarder account but there's no logic in my spam filters to 
know to look "past" the livejournal servers for the actual spam source. 
(Short of adding livejournal's current MX ip to trusted_networks).  I 
digress.

Greylisting.org suggests something called "peer to peer whitelisting" -- 
which seems odd to me because, well, why should I install support for 
another protocol if it's inbuilt to (and possible with) the DNSBL 
protocol, on a nearly universal level?

-Dan

--

Pika Pika Pika!

-Pikachu, of Pokemon fame.

--------Dan Mahoney--------
Techie,  Sysadmin,  WebGeek
Gushi on efnet/undernet IRC
ICQ: 13735144   AIM: LarpGM
Site:  http://www.gushi.org
---------------------------

Dan Mahoney, System Admin <danm@...> wrote:

> Greylisting.org suggests something called "peer to peer whitelisting" --
> which seems odd to me because, well, why should I install support for
> another protocol if it's inbuilt to (and possible with) the DNSBL 
> protocol, on a nearly universal level?

I guess the goal is to address a possible DDoS attack of DNSRBL by
spammers.

-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu@...

On Thu, 25 Oct 2007, manu@... wrote:

> Dan Mahoney, System Admin <danm@...> wrote:
>
>> Greylisting.org suggests something called "peer to peer whitelisting" --
>> which seems odd to me because, well, why should I install support for
>> another protocol if it's inbuilt to (and possible with) the DNSBL
>> protocol, on a nearly universal level?
>
> I guess the goal is to address a possible DDoS attack of DNSRBL by
> spammers.

Sure, because spammers will DDOS a whitelist, so then everyone's mail is 
as delayed as theirs is?

-Dan

--

"this is too stupid even for irc"

-mtreal, EFnet #macintosh, 09/15/2K, 12:33 AM

--------Dan Mahoney--------
Techie,  Sysadmin,  WebGeek
Gushi on efnet/undernet IRC
ICQ: 13735144   AIM: LarpGM
Site:  http://www.gushi.org
---------------------------

Dan Mahoney, System Admin a \ufffdcrit :
> 
>> 3. http://www.dnswl.org/
> 
> I've got my own issues with them, and we wouldn't be serving the same 
> purpose.  My whitelist would be PURELY a dynamic list of "people not to 
> greylist", not "people to give additional kudos in SpamAssassin, etc." 
> DNSWL has added a bunch of mail relays (like LiveJournal) which relay mail 
> for a given forwarder account but there's no logic in my spam filters to 
> know to look "past" the livejournal servers for the actual spam source. 
> (Short of adding livejournal's current MX ip to trusted_networks).  I 
> digress.
> 

DNSWL aims to inventory all "known legitimate email servers", that are 
servers which are powered by real MTAs (not spambots).

Greylisting sole goal is to block fake MTAs (spambot-like), which are 
usually illegitimate and do little retries, if any.

Is is pointless to greylist DNSWL-listed servers, because if properly 
configured they WILL retry. If some of them are relaying SPAM, it will 
pass through greylisting; delaying is not blocking.

So it makes sense to use DNSWL as a whitelist to bypass greylisting. 
Primary benefit is to avoid greylisting delay on legitimate email, and 
secondary is to let through legitimate MTAs which aren't 
greylisting-aware by design, such as mail farms.

Remaining SPAM should be treated by other means, such as content 
analysis, embedded URL RBL-check, and DNSWL "Trustworthiness" score.


-- 
Ce message a ete verifie par MailScanner
pour des virus ou des polluriels et rien de
suspect n'a ete trouve.

On Thu, 25 Oct 2007, Benoit Branciard wrote:

> Dan Mahoney, System Admin a ?crit :
>>
>>> 3. http://www.dnswl.org/
>>
>> I've got my own issues with them, and we wouldn't be serving the same
>> purpose.  My whitelist would be PURELY a dynamic list of "people not to
>> greylist", not "people to give additional kudos in SpamAssassin, etc."
>> DNSWL has added a bunch of mail relays (like LiveJournal) which relay mail
>> for a given forwarder account but there's no logic in my spam filters to
>> know to look "past" the livejournal servers for the actual spam source.
>> (Short of adding livejournal's current MX ip to trusted_networks).  I
>> digress.
>>
>
> DNSWL aims to inventory all "known legitimate email servers", that are
> servers which are powered by real MTAs (not spambots).
>
> Greylisting sole goal is to block fake MTAs (spambot-like), which are
> usually illegitimate and do little retries, if any.
>
> Is is pointless to greylist DNSWL-listed servers, because if properly
> configured they WILL retry. If some of them are relaying SPAM, it will
> pass through greylisting; delaying is not blocking.

I'm aware of that -- my goal is different from yours, though.  Yours is to 
manage known senders of "good" mail.  Mine is simply to maintain a list of 
"broken" servers, mail farms, as well as to collect info on other setups 
and reasons, specifically, why greylisting might not work.  I've seen a 
significant reduction in spam since I started using it (right now I'm 
using it against a heavy-casualty blacklist), but I'd like to totally 
avoid the possibility of it causing any other issues for non-compliant 
senders.

Are you listing all the known cases of "breakage" with regard to those 
servers that don't work with greylisting (the one from CVS?)

> So it makes sense to use DNSWL as a whitelist to bypass greylisting.
> Primary benefit is to avoid greylisting delay on legitimate email, and
> secondary is to let through legitimate MTAs which aren't
> greylisting-aware by design, such as mail farms.
>
> Remaining SPAM should be treated by other means, such as content
> analysis, embedded URL RBL-check, and DNSWL "Trustworthiness" score.

Among other problems that are off-topic for this list.

-Dan

--

"Oh, and we just recently got an invoice..."
"Congratulations!"

-JC and DM, regarding Unpredictable Billing, 8/18/2001

--------Dan Mahoney--------
Techie,  Sysadmin,  WebGeek
Gushi on efnet/undernet IRC
ICQ: 13735144   AIM: LarpGM
Site:  http://www.gushi.org
---------------------------