Yahoo Groups archive

Milter-greylist

Index last updated: 2026-04-28 23:32 UTC

Thread

Backup MX: verifying addresses

Backup MX: verifying addresses

2006-12-08 by Oliver Fromme

Hi,

I'm sorry this is somewhat off-topic because it's not
directly related to milter-greylist.  But it has to do
with anti-spam and milters in general, and there seem
to be a lot of knowledgeable people about those topics
in this list ...

There's a mailserver (primary MX) with a bunch of users.
It's using milter-greylist, greet_pause, some custom-made
filtering with maildrop and some other things, and it's
working quite well.

Now the problem:  There's a backup MX (secondary) whose
sole purpose is to accept and spool mails if the primary
MX is down for some reason, and then send the queued stuff
to the primary when it's up again.  The secondary doesn't
know about valis users on the primary, so it accepts _all_
mails.  Most of them is spam or spam-probes to non-existing
addresses.  (Spammers seem to prefer secondaries even if the
primary MX is available ...  Maybe they think that on backup
mail servers there are less anti-spam measures.  Or maybe
they simply don't care about the MX priorities and send
their crap to a random MX.)

Of course, what happens is this:  As soon as the secondary
tries to relay the mails to the primary, it gets a "user
unknow" reply, which leads to a bounce message which is
either sent back to innocent people (because of forged
sender addresses) or stays in the mailq on the secondary
for a long time (because the bounce cannot be delivered to
the sender for various reasons).

So the reuslt of the situation is that the secondary gets
a huge mail queue which is full of bounces that never will
be delivered (and expire after five days, generating post-
master mails).  And furthermore, the seondary is sending
lots of bogus delivery errors to innocent people.

Is there a _simple_ solution to solve the problem?

Of course, one possibility would be to duplicate all users,
aliases, virtusertable etc. from the primary to the secon-
dary's virtusertable, so it knows all valid addresses.  But
I don't want to do that, because the secondary shouldn't
know about users on the primary (both are operated by
different people that don't need to know each other).
It would also make administration much more difficult,
because whenever an alias or user is added or changed,
I would need to arrange for the same change on the
secondary.  (In case of multiple backup MX servers it
gets even worse.)

My idea is to have a milter that tells the secondary's
sendmail which addresses are valid on the primary.  For
that purpose, it would connect to the primary and perform
a VRFY on the address.  The milter could cache the result
for a few minutes, so the load on the primary would be
reduced.  If the primary is down, the milter would have
to accept all addresses, of course (unless they're cached
to be invalid).

Does such a milter exist?  I searched Google up and down,
but couldn't find anything.  Or are there different
solutions for the problem that I overlooked so far?

Best regards
   Oliver

PS:  VRFY is usually disabled via PrivacyOptions in send-
mail, but I think it's possible to enable it for certain
hosts only via Srv_Features in access.db.  I haven't tried
that myself, though.

-- 
Oliver Fromme,  secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing
Dienstleistungen mit Schwerpunkt FreeBSD: http://www.secnetix.de/bsd
Any opinions expressed in this message may be personal to the author
and may not necessarily reflect the opinions of secnetix in any way.

"Life is short (You need Python)"
        -- Bruce Eckel, ANSI C++ Comitee member, author
           of "Thinking in C++" and "Thinking in Java"

Re: [milter-greylist] Backup MX: verifying addresses

2006-12-08 by Jacques Beigbeder

Hello,

>> Does such a milter exist?

I think this is 'milter-ahead'.

--
Jacques Beigbeder                    |  Jacques.Beigbeder@...
Service de Prestations Informatiques |     http://www.spi.ens.fr
Ecole normale sup\ufffdrieure             |
45 rue d'Ulm                         |Tel : (+33 1)1 44 32 37 96
F75230 Paris cedex 05                |Fax : (+33 1)1 44 32 20 75

Re: [milter-greylist] Backup MX: verifying addresses

2006-12-08 by manu@netbsd.org

Oliver Fromme <olli@...> wrote:

> Is there a _simple_ solution to solve the problem?

Get rid of secondary MX? :-)

In my situation, I have multiple primary MX to take care of failures,
but network outages are rare enough to make secondary MX useless (and
even harmful, for the reasons you explain). Your situation may be
different, of course.

-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu@...

Re: [milter-greylist] Backup MX: verifying addresses

2006-12-08 by Oliver Fromme

Jacques Beigbeder wrote:
 > 
 > > > Does such a milter exist?
 > 
 > I think this is 'milter-ahead'.

Yes, it seems to be able to do that, but its license does
not allow me to use it (it's not Open Source, among other
problems).  Apart from that, it seems to do a lot of other
things that I don't need, and I don't want that overhead.

Maybe when I have some time I'll try writing a milter for
that purpose myself.

Anyway, thanks for the hint.

Best regards
   Oliver

-- 
Oliver Fromme,  secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing
Dienstleistungen mit Schwerpunkt FreeBSD: http://www.secnetix.de/bsd
Any opinions expressed in this message may be personal to the author
and may not necessarily reflect the opinions of secnetix in any way.

"Clear perl code is better than unclear awk code; but NOTHING
comes close to unclear perl code"  (taken from comp.lang.awk FAQ)

Re: [milter-greylist] Backup MX: verifying addresses

2006-12-08 by Oliver Fromme

manu@... wrote:
 > Oliver Fromme wrote:
 > 
 > > Is there a _simple_ solution to solve the problem?
 > 
 > Get rid of secondary MX? :-)

I cannot do that.

 > In my situation, I have multiple primary MX to take care of failures,
 > but network outages are rare enough to make secondary MX useless (and
 > even harmful, for the reasons you explain). Your situation may be
 > different, of course.

I definitely want one (or more) backup MX servers for
redundancy.  Network outages do happen sometimes, and
so do hardware failures or human mistakes (e.g. the
admin inserts an error in some configuration file).

Only one of the MX can be primary (in my case), because
only that primary has all user accounts and knows which
addresses are valid.  The secondaries should only queue
messages and forward them to the primary.

In another situation i would like to offer a backup MX
service to a friend of mine who only has one (primary)
mailserver.  I don't care about what users he has and
what addresses are valid for him, and I certainly don't
want to change anything on my server every time he adds
or removes a user on his box.  I just want to setup the
backup MX service and be done with it, similar to a
secondary name server.

Maybe that's just wishful thinking.  :-)

Best regards
   Oliver


-- 
Oliver Fromme,  secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing
Dienstleistungen mit Schwerpunkt FreeBSD: http://www.secnetix.de/bsd
Any opinions expressed in this message may be personal to the author
and may not necessarily reflect the opinions of secnetix in any way.

"Clear perl code is better than unclear awk code; but NOTHING
comes close to unclear perl code"  (taken from comp.lang.awk FAQ)

Re: [milter-greylist] Backup MX: verifying addresses

2006-12-08 by Emmanuel Dreyfus

On Fri, Dec 08, 2006 at 11:16:42AM +0100, Oliver Fromme wrote:
> Maybe when I have some time I'll try writing a milter for
> that purpose myself.

You can also hijack milter-greylist for doing it, with the new
URL check feature: you'd need this config in milter-greylist on backup MX:

urlcheck "usrok" "http://mx1.example.net/usrok.cgi?rcpt=%r" 10

acl whitelist rcpt /@example\.net$/ urlcheck "usrok"
acl blacklist rcpt /@example\.net$/ msg "invalid user"

Then in http://mx1.example.net/usrok.cgi, implement the following logic:

if (valid($rcpt))
	printf("milterGreylistStatus: Ok\n");
else
	printf("\n");
	
The code for that is in CVS and has not made it into a snapshot release. 
syntax and interface is therefore subject to change.

-- 
Emmanuel Dreyfus
manu@...

Re: [milter-greylist] Backup MX: verifying addresses

2006-12-08 by Matthias Scheler

On Fri, Dec 08, 2006 at 08:39:22AM +0100, Oliver Fromme wrote:
> Is there a _simple_ solution to solve the problem?

Simple: yes
Comfortable: no

> My idea is to have a milter that tells the secondary's
> sendmail which addresses are valid on the primary.

Sendmail supports that out of the box:

1.) Add the domain to "/etc/mail/relay-domains" on the secondary MX and
    reload sendmail.

2.) Add entries to "/etc/mail/access" on the secondary MX which look
    like this:

To:valid-address-1@...			RELAY
To:valid-address-2@...			RELAY
To:valid-address-3@...			RELAY
[...]
To:do.main					error:5.1.1:550 User unknown

That will do the job. The problem is keeping "/etc/mail/access" uptodate
on the secondary MX.

	Kind regards

-- 
Matthias Scheler                                  http://zhadum.org.uk/

Re: [milter-greylist] Backup MX: verifying addresses

2006-12-08 by Oliver Fromme

Matthias Scheler wrote:
 > Oliver Fromme wrote:
 > > My idea is to have a milter that tells the secondary's
 > > sendmail which addresses are valid on the primary.
 > 
 > Sendmail supports that out of the box:
 > 
 > 1.) Add the domain to "/etc/mail/relay-domains" on the secondary MX and
 >     reload sendmail.
 > 
 > 2.) Add entries to "/etc/mail/access" on the secondary MX which look
 >     like this:
 > 
 > To:valid-address-1@...			RELAY
 > To:valid-address-2@...			RELAY
 > To:valid-address-3@...			RELAY
 > [...]
 > To:do.main					error:5.1.1:550 User unknown

Thanks, but I wrote in my first mail that I cannot do that.
Quoting myself:
 > Of course, one possibility would be to duplicate all users,
 > aliases, virtusertable etc. from the primary to the secon-
 > dary's virtusertable, so it knows all valid addresses.  But
 > I don't want to do that, because the secondary shouldn't
 > know about users on the primary (both are operated by
 > different people that don't need to know each other).
 > It would also make administration much more difficult,
 > because whenever an alias or user is added or changed,
 > I would need to arrange for the same change on the
 > secondary.  (In case of multiple backup MX servers it
 > gets even worse.)

Best regards
   Oliver

PS:  I've just read Emmanuel's email how to do it with
the CVS version of milter-greylist using the "urlcheck"
feature.  That sounds like it's worth trying.

(The only problem is that the primary MX isn't running
a web server, and it really shouldn't.  But maybe I can
hack something up for that purpose, maybe a simple app
that receives HTTP GET requests ...  I'll look into that.)

-- 
Oliver Fromme,  secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing
Dienstleistungen mit Schwerpunkt FreeBSD: http://www.secnetix.de/bsd
Any opinions expressed in this message may be personal to the author
and may not necessarily reflect the opinions of secnetix in any way.

One Unix to rule them all, One Resolver to find them,
One IP to bring them all and in the zone to bind them.

Re: [milter-greylist] Backup MX: verifying addresses

2006-12-08 by manu@netbsd.org

Oliver Fromme <olli@...> wrote:

> (The only problem is that the primary MX isn't running
> a web server, and it really shouldn't.  But maybe I can
> hack something up for that purpose, maybe a simple app
> that receives HTTP GET requests ...  I'll look into that.)

You can query another machine that will do a SMTP VRFY on your primary
MX within a CGI. That's a simple script that can be hacked within a few
minutes.

-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu@...

Re: [milter-greylist] Backup MX: verifying addresses

2006-12-08 by manu@netbsd.org

Emmanuel Dreyfus <manu@...> wrote:

> > (The only problem is that the primary MX isn't running
> > a web server, and it really shouldn't.  But maybe I can
> > hack something up for that purpose, maybe a simple app
> > that receives HTTP GET requests ...  I'll look into that.)
> 
> You can query another machine that will do a SMTP VRFY on your primary
> MX within a CGI. That's a simple script that can be hacked within a few
> minutes.

Or your CGI can check a list of valid address that you push from your
primary MX...

-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu@...

Re: [milter-greylist] Backup MX: verifying addresses

2006-12-08 by Oliver Fromme

manu@... wrote:
 > Oliver Fromme wrote:
 > 
 > > (The only problem is that the primary MX isn't running
 > > a web server, and it really shouldn't.  But maybe I can
 > > hack something up for that purpose, maybe a simple app
 > > that receives HTTP GET requests ...  I'll look into that.)
 > 
 > You can query another machine that will do a SMTP VRFY on your primary
 > MX within a CGI.

Yes, I could do that, but I don't want to tie yet another
machine into the equation.  It increases the number of
potential "points of failure".

 > That's a simple script that can be hacked within a few minutes.

Yes, in fact I have just written a small script in Python
that performs the SMTP VRFY part on a given number of
addresses.  It's only 5 lines.

I think I'm going to add a HTTP server part to that script.
which is also very easy in Python.  So I don't need to
install a fully-fledged web server like Apache.  It might
even has better performance.  I'll let it run on a non-
standard port on the primary MX.

Thanks for all your help!

Best regards
   Oliver

-- 
Oliver Fromme,  secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing
Dienstleistungen mit Schwerpunkt FreeBSD: http://www.secnetix.de/bsd
Any opinions expressed in this message may be personal to the author
and may not necessarily reflect the opinions of secnetix in any way.

Passwords are like underwear.  You don't share them,
you don't hang them on your monitor or under your keyboard,
you don't email them, or put them on a web site,
and you must change them very often.

Re: [milter-greylist] Backup MX: verifying addresses

2006-12-08 by Emmanuel Dreyfus

On Fri, Dec 08, 2006 at 01:45:54PM +0100, Oliver Fromme wrote:
> Yes, in fact I have just written a small script in Python
> that performs the SMTP VRFY part on a given number of
> addresses.  It's only 5 lines.

I don't know where we are going with URL checks, but I suspect at one 
point we will want to have a url check plugins folder in milter-greylist
distribution, containing various helpful hacks like this one...

-- 
Emmanuel Dreyfus
manu@...

Re: [milter-greylist] Backup MX: verifying addresses

2006-12-08 by Guy Feltin

Dans son message, Oliver Fromme ecrivait :
----------------------------------------------


 > Does such a milter exist?  

With sendmail you can use mimedefang (http://www.mimdefang.org) with the function : 
md_check_against_smtp_server:

md_check_against_smtp_server($sender, $recip, $helo, $server, $port)
              This function  connects  to  the  SMTP server $server and pretends to send mail from 
$sender to $recip.
              The return value is always a two-element array.  If the RCPT TO: command succeeds, 
the return value
              is ("CONTINUE", "OK").  If the RCPT fails with a permanent failure, the return value 
is ("REJECT", $msg),
              where $msg is the message from the SMTP server.  Any temporary failures, connection 
errors, etc. result
              in a return value of ("TEMPFAIL", $msg).
              The optional argument $port specifies the TCP port to connect to.  If it is not 
supplied, then
              the default SMTP port of 25 is used.




-- 
  Guy  Feltin   -   03 83 68 24 30

Re: [milter-greylist] Backup MX: verifying addresses

2006-12-09 by John Villalovos

On 12/7/06, Oliver Fromme <olli@...> wrote:
> Now the problem:  There's a backup MX (secondary) whose
> sole purpose is to accept and spool mails if the primary
> MX is down for some reason, and then send the queued stuff
> to the primary when it's up again.  The secondary doesn't
> know about valis users on the primary, so it accepts _all_
> mails.  Most of them is spam or spam-probes to non-existing
> addresses.  (Spammers seem to prefer secondaries even if the
> primary MX is available ...  Maybe they think that on backup
> mail servers there are less anti-spam measures.  Or maybe
> they simply don't care about the MX priorities and send
> their crap to a random MX.)
>
> Of course, what happens is this:  As soon as the secondary
> tries to relay the mails to the primary, it gets a "user
> unknow" reply, which leads to a bounce message which is
> either sent back to innocent people (because of forged
> sender addresses) or stays in the mailq on the secondary
> for a long time (because the bounce cannot be delivered to
> the sender for various reasons).
>
> So the reuslt of the situation is that the secondary gets
> a huge mail queue which is full of bounces that never will
> be delivered (and expire after five days, generating post-
> master mails).  And furthermore, the seondary is sending
> lots of bogus delivery errors to innocent people.
>
> Is there a _simple_ solution to solve the problem?

Have you heard about the mailfromd sendmail milter?  I haven't used it
but it looks like it would do what you want.
http://puszcza.gnu.org.ua/projects/mailfromd/

Documentation:
http://gray.gnu.org.ua/software/mailfromd/manual/

It looks like it does what you want.  It seems to do the same thing as
milter-ahead but it is free software.

John

Move to quarantaine

This moves the raw source file on disk only. The archive index is not changed automatically, so you still need to run a manual refresh afterward.