Milter-greylist

Hi,

Quite a few times I get greylist delays from senders that use their ISP's mail
servers and thus come out looking as a sender from:

Received: from termserv.example.ad (xxx-xxx-xx-xx.static.example.com.au
[xxx.xxx.xx.xx])

and such a message (although valid) is delayed by 16hrs.

I only added the words "example" above and "xxx", the rest is as is so the .ad
and the .com.au don't match).

How would people recommend I deal with these?

Thanks.

Michael.

-------- Original Message  --------

Subject: [milter-greylist] Recommendation for static ADSL IP's
From: Michael Mansour <mic@...>
To: milter-greylist <milter-greylist@yahoogroups.com>
Date: Fri 29 Feb 2008 05:13:27 PM EST

> Hi,
> 
> Quite a few times I get greylist delays from senders that use their ISP's mail
> servers and thus come out looking as a sender from:
> 
> Received: from termserv.example.ad (xxx-xxx-xx-xx.static.example.com.au
> [xxx.xxx.xx.xx])
> 
> and such a message (although valid) is delayed by 16hrs.
> 
> I only added the words "example" above and "xxx", the rest is as is so the .ad
> and the .com.au don't match).
> 
> How would people recommend I deal with these?
> 

I use the following (on one line):

acl whitelist domain /[0-9]{1,3}\-[0-9]{1,3}\-[0-9]{1,3}\-[0-9]{1,3}.*\.static/


Ryan Moore
----------
Perigee.net Corporation
704-849-8355 (sales)
704-849-8017 (tech)
www.perigee.net

Michael Mansour wrote:
> Hi,
> 
> Quite a few times I get greylist delays from senders that use their ISP's mail
> servers and thus come out looking as a sender from:
> 
> Received: from termserv.example.ad (xxx-xxx-xx-xx.static.example.com.au
> [xxx.xxx.xx.xx])
> 
> and such a message (although valid) is delayed by 16hrs.
> 
> I only added the words "example" above and "xxx", the rest is as is so the .ad
> and the .com.au don't match).
> 
> How would people recommend I deal with these?

   Why would you want to whitelist a generic rDNS like that?  My own server logs
contain plenty of entries from infected PC's sitting behind entries like that and
a real business would have gone to the trouble of making the ISP change that
generic entry to something that reflected their company domain name.

Ryan Moore a \ufffdcrit :
> 
> I use the following (on one line):
> 
> acl whitelist domain /[0-9]{1,3}\-[0-9]{1,3}\-[0-9]{1,3}\-[0-9]{1,3}.*\.static/
> 

That's exactly the kind of client I would rather blacklist or "heavy" 
greylist. More than 90% of spam comes from infected customer ADSL 
clients, the ones which have such generic rDNS entries (either static or 
dynamic).

If someone wants to operate his own mail server on a private ADSL line, 
he must prove its legitimity by some reputation mechanisms:

- publish an SPF record for his domain
- get his server IP listed in list.dnswl.org whitelist
- get his server IP unlisted from pbl.spamhaus.org
- be sure to have his server RFC-compliant, for example by reliably 
retrying after 4.x tempfail until 5 days
- professionnally maintain his server, keeping it out of intrusions and 
spam relaying

All these mechanisms may be used in milter-greylist 4.0+ and help 
building an reputation for inbound clients.

-- 
Ce message a ete verifie par MailScanner
pour des virus ou des polluriels et rien de
suspect n'a ete trouve.

Hi Ryan,

> Ryan Moore a \ufffdcrit :
> > 
> > I use the following (on one line):
> > 
> > acl whitelist domain
/[0-9]{1,3}\-[0-9]{1,3}\-[0-9]{1,3}\-[0-9]{1,3}.*\.static/
> >
> 
> That's exactly the kind of client I would rather blacklist or 
> "heavy" greylist. More than 90% of spam comes from infected customer 
> ADSL clients, the ones which have such generic rDNS entries (either 
> static or dynamic).
> 
> If someone wants to operate his own mail server on a private ADSL 
> line, he must prove its legitimity by some reputation mechanisms:
> 
> - publish an SPF record for his domain
> - get his server IP listed in list.dnswl.org whitelist
> - get his server IP unlisted from pbl.spamhaus.org
> - be sure to have his server RFC-compliant, for example by reliably 
> retrying after 4.x tempfail until 5 days
> - professionnally maintain his server, keeping it out of intrusions 
> and spam relaying
> 
> All these mechanisms may be used in milter-greylist 4.0+ and help 
> building an reputation for inbound clients.

Although I totally agree with what you say above, it's difficult to explain
proper email practice to your clients that don't care to understand why their
message was delayed by 16 hours.

All they know is that their contacts email couldn't get through until 16 hours
past, and when they were on their previous provider, that never used to happen.

This is the only real problem I have with greylisting, in that you don't know
and can't "fix" an issue until after it has happened (either recognising the
delay in your daily log reports or waiting until the customer complains). At
either point, it's too late.

For me, I have lost one client because of this since I started using
greylisting. Even while I turned it off for their domain when it first
happened, they lost "confidence" that they weren't losing email as a result.

That's one of things all our technical skills at managing spam misses, that
clients only see A and Z, and don't care about the path from A to Z.

Regards,

Michael.

> -- 
> Ce message a ete verifie par MailScanner
> pour des virus ou des polluriels et rien de
> suspect n'a ete trouve.
------- End of Original Message -------

Michael Mansour <mic@...> wrote:

> Although I totally agree with what you say above, it's difficult to explain
> proper email practice to your clients that don't care to understand why their
> message was delayed by 16 hours.
> 
> All they know is that their contacts email couldn't get through until 16
> hours past, and when they were on their previous provider, that never used
> to happen.

I must have misunderstood something: you are an ISP, and you have to
greylist your own clients? How does it works with casual users that have
a mail software that sends to your SMTP server?

-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu@...

Hi Emmanuel,

> Michael Mansour <mic@...> wrote:
> 
> > Although I totally agree with what you say above, it's difficult to explain
> > proper email practice to your clients that don't care to understand why their
> > message was delayed by 16 hours.
> > 
> > All they know is that their contacts email couldn't get through until 16
> > hours past, and when they were on their previous provider, that never used
> > to happen.
> 
> I must have misunderstood something: you are an ISP, and you have to
> greylist your own clients? How does it works with casual users that have
> a mail software that sends to your SMTP server?

I'm a hosting provider yes, but I don't greylist my own clients, just greylist
emails sent to their domains.

There's plenty of spam each day that gets delayed by many hours, and by the
time that spam does get through greylisting, the other filters pick them up
and rate them for deletion (so the clients domain never gets them anyway). 

The problem is when a valid email gets delayed for an extended period of time,
that's the thing which has caused me pain.

It seems to be part of life for mail servers on the internet to be broken, not
setup correctly, not follow RFC's, etc, which is why we manually whitelist,
but from the clients point of view they just want their email to work and not
be delayed (and they don't care about the fact that the sending mail server is
broken), so such delays that cause them pain, cause me pain. This is why I've
had to stop greylisting for some domains to keep the client happy.

Ideally, I wish I could find an automated solution to have greylisting on
while still not delaying valid emails, and having no administrative overhead.

Regards,

Michael.

> -- 
> Emmanuel Dreyfus
> http://hcpnet.free.fr/pubz
> manu@...
> 
> Yahoo! Groups Links
> 
> 
>

Hi Ryan,

> -------- Original Message  --------
> Subject: [milter-greylist] Recommendation for static ADSL IP's
> From: Michael Mansour <mic@...>
> To: milter-greylist <milter-greylist@yahoogroups.com>
> Date: Fri 29 Feb 2008 05:13:27 PM EST
> 
> > Hi,
> > 
> > Quite a few times I get greylist delays from senders that use their ISP's mail
> > servers and thus come out looking as a sender from:
> > 
> > Received: from termserv.example.ad (xxx-xxx-xx-xx.static.example.com.au
> > [xxx.xxx.xx.xx])
> > 
> > and such a message (although valid) is delayed by 16hrs.
> > 
> > I only added the words "example" above and "xxx", the rest is as is so the .ad
> > and the .com.au don't match).
> > 
> > How would people recommend I deal with these?
> >
> 
> I use the following (on one line):
> 
> acl whitelist domain /[0-9]{1,3}\-[0-9]{1,3}\-[0-9]{1,3}\-[0-9]{1,3}.*\.static/

Hmm.. so basically I'd be allowing everything whitelisted from that ISP?

It makes sense I think if I know that the ISP is a trustworthy ISP (as this
one is but they provide many broadband links to many people in Australia). I
was just hoping there was another way other than that.

Thanks.

Michael.

> Ryan Moore
> ----------
> Perigee.net Corporation
> 704-849-8355 (sales)
> 704-849-8017 (tech)
> www.perigee.net
------- End of Original Message -------

On Sun, 2 Mar 2008, Michael Mansour wrote:

MM> 
MM> For me, I have lost one client because of this since I started using
MM> greylisting. Even while I turned it off for their domain when it first
MM> happened, they lost "confidence" that they weren't losing email as a result.
MM> 

And quite right too.  You shouldn't interfere with email in any way unless 
the receipient asks you to do so.


-- 
Alan

( Please do not email me AS WELL as replying to the list.  Please
  address personal email to alan+1@ as lists@ is not read. )

Exactly,
I agree with Benoit....

Benoit Branciard wrote:

>
> Ryan Moore a �crit :
> >
> > I use the following (on one line):
> >
> > acl whitelist domain
> /[0-9]{1,3}\-[0-9]{1,3}\-[0-9]{1,3}\-[0-9]{1,3}.*\.static/
> >
>
> That's exactly the kind of client I would rather blacklist or "heavy"
> greylist. More than 90% of spam comes from infected customer ADSL
> clients, the ones which have such generic rDNS entries (either static or
> dynamic).
>
> If someone wants to operate his own mail server on a private ADSL line,
> he must prove its legitimity by some reputation mechanisms:
>
> - publish an SPF record for his domain
> - get his server IP listed in list.dnswl.org whitelist
> - get his server IP unlisted from pbl.spamhaus.org
> - be sure to have his server RFC-compliant, for example by reliably
> retrying after 4.x tempfail until 5 days
> - professionnally maintain his server, keeping it out of intrusions and
> spam relaying
>
> All these mechanisms may be used in milter-greylist 4.0+ and help
> building an reputation for inbound clients.
>
> -- 
> Ce message a ete verifie par MailScanner
> pour des virus ou des polluriels et rien de
> suspect n'a ete trouve.
>
>

Automated solution - that the target we are all aiming for.
Unfortunately you will never find one, it is impossible - unless you
willing to pay for some outsourced commercial solution.
Ondrej

>
> Ideally, I wish I could find an automated solution to have greylisting on
> while still not delaying valid emails, and having no administrative
> overhead.
>
> Regards,
>


> Michael.
>
> > --
> > Emmanuel Dreyfus
> > http://hcpnet.free.fr/pubz <http://hcpnet.free.fr/pubz>
> > manu@... <mailto:manu%40netbsd.org>
> >
> > Yahoo! Groups Links
> >
> >
> >
>
>

On Mon, Mar 03, 2008 at 09:47:38AM +0100, Ondrej Valousek wrote:
> Automated solution - that the target we are all aiming for.
> Unfortunately you will never find one, it is impossible - unless you
> willing to pay for some outsourced commercial solution.

Well, there are still cool stuff that can be done with free tools: At
mine, milter-greylist looks up per-recipient settings in an LDAP 
directory. Settings include whether greylisting is enabled, for what
delays, what DNSRBL to use, blacklist and whitelist. 

Users have access to a web application which enable them to access
and modify their personnel filtering settings. The application also
shows them all milter-greylist activity collected through the stat 
feature: they can see what delivery attempt was accepted, temporary 
rejected, or permanently rejected, and for what reason. 

No filtering is enabled by default. Users that complain about spam are
directed to this tool, and they do use it to perform filtering. As the
only filtering done is what they have configured, there is no room
for lost mail complaints, and there is no administrative overhead.

-- 
Emmanuel Dreyfus
manu@...

Well,
That would be a fairly cool thing!
What I would be a bit concerned a bit is that in our case the
milter-greylist is running in de-militarized zone on the MTA. On our LAN
we have Microsoft AD domain which could be potentially used to store
greylisting data, but MTA could be potentially compromised and thus
expose the whole Active Directory to the attacker...
Ondrej

Emmanuel Dreyfus wrote:

>
> On Mon, Mar 03, 2008 at 09:47:38AM +0100, Ondrej Valousek wrote:
> > Automated solution - that the target we are all aiming for.
> > Unfortunately you will never find one, it is impossible - unless you
> > willing to pay for some outsourced commercial solution.
>
> Well, there are still cool stuff that can be done with free tools: At
> mine, milter-greylist looks up per-recipient settings in an LDAP
> directory. Settings include whether greylisting is enabled, for what
> delays, what DNSRBL to use, blacklist and whitelist.
>
> Users have access to a web application which enable them to access
> and modify their personnel filtering settings. The application also
> shows them all milter-greylist activity collected through the stat
> feature: they can see what delivery attempt was accepted, temporary
> rejected, or permanently rejected, and for what reason.
>
> No filtering is enabled by default. Users that complain about spam are
> directed to this tool, and they do use it to perform filtering. As the
> only filtering done is what they have configured, there is no room
> for lost mail complaints, and there is no administrative overhead.
>
> -- 
> Emmanuel Dreyfus
> manu@... <mailto:manu%40netbsd.org>
>
>

On Mon, Mar 03, 2008 at 10:20:27AM +0100, Ondrej Valousek wrote:
> That would be a fairly cool thing!
> What I would be a bit concerned a bit is that in our case the
> milter-greylist is running in de-militarized zone on the MTA. On our LAN
> we have Microsoft AD domain which could be potentially used to store
> greylisting data, but MTA could be potentially compromised and thus
> expose the whole Active Directory to the attacker...

Perhaps you can add another LDAP server on the DMZ that would hold the mail
address config branch, and setup AD so that it talks to it for that branch?

Alternatively, you can setup LDAP replicas on your MTA. Your AD will push 
there the information, filtering out anything you consider sensitive. I
am not sure AD knows how to do that, but at least it's possible with
openLDAP: I have local LDAP replicas on each MX, and thoses do not get 
userPassword attributes from the master, for instance.

-- 
Emmanuel Dreyfus
manu@...

Well, yes,
The replicas would solve the problem, indeed - but it is another layer
that needs to be managed.
Anyway - just thinking about the whole LDAP thing - would it be really
any good?
As someone already mentioned, most users actually do not care about the
finesses of greylisting - they just assume you would give them a working
setup  they do not have to touch.
Moreover, it would be actually more maintenance - you always risk the
some dummy user would misconfigure it and then come back to you crying
or complaining.

I would say - the ideal antispam solution is the one end user do not
have to touch or know about (i.e. no configuration, no training,...).

Ondrej

Emmanuel Dreyfus wrote:

>
> On Mon, Mar 03, 2008 at 10:20:27AM +0100, Ondrej Valousek wrote:
> > That would be a fairly cool thing!
> > What I would be a bit concerned a bit is that in our case the
> > milter-greylist is running in de-militarized zone on the MTA. On our LAN
> > we have Microsoft AD domain which could be potentially used to store
> > greylisting data, but MTA could be potentially compromised and thus
> > expose the whole Active Directory to the attacker...
>
> Perhaps you can add another LDAP server on the DMZ that would hold the
> mail
> address config branch, and setup AD so that it talks to it for that
> branch?
>
> Alternatively, you can setup LDAP replicas on your MTA. Your AD will push
> there the information, filtering out anything you consider sensitive. I
> am not sure AD knows how to do that, but at least it's possible with
> openLDAP: I have local LDAP replicas on each MX, and thoses do not get
> userPassword attributes from the master, for instance.
>
> -- 
> Emmanuel Dreyfus
> manu@... <mailto:manu%40netbsd.org>
>
>

-------- Original Message  --------

Subject: Re: [milter-greylist] Recommendation for static ADSL IP's
From: Michael Mansour <mic@...>
To: milter-greylist@yahoogroups.com
Date: Sat 01 Mar 2008 04:34:07 PM EST

> Hmm.. so basically I'd be allowing everything whitelisted from that ISP?
> 
> It makes sense I think if I know that the ISP is a trustworthy ISP (as this
> one is but they provide many broadband links to many people in Australia). I
> was just hoping there was another way other than that.
> 

Well the line I pasted whitelists any hostname that contains the IP followed 
by 'static'. I'm sure there are people that disagree with doing so, but 
milter-greylist is but just one of several tools we use to filter spam, and a 
vast majority of our clients that use our spam filtering are small businesses 
that have large volumes of email communications with other small businesses.

If you only want to whitelist the static IP customers on that ISP (you don't 
want to whitelist the dynamic ones most likely, that is where most the junk 
comes from), just change the regex to have the domain at the end somewhere, 
such as:

/[0-9]{1,3}\-[0-9]{1,3}\-[0-9]{1,3}\-[0-9]{1,3}.*\.static\.example\.com\.au/



Ryan Moore
----------
Perigee.net Corporation
704-849-8355 (sales)
704-849-8017 (tech)
www.perigee.net