Yahoo Groups archive

Milter-greylist

Index last updated: 2026-04-28 23:32 UTC

Thread

Null Sender

Null Sender

2005-01-02 by Moritz Both

Hi,

Just having read the greylist whitepaper at
http://projects.puremagic.com/greylisting/whitepaper.html and was
excited, so I searched the correct implementation for me and it seemd
thet milter-greylist was it.

I have a question and a few comments.

1. the question:

The whitepaper describes a need for delaying the tempfail message until
the end of the DATA phase if the sender address is null. It appears to
me that milter-greylist does not do that. Giving a increasing number of
servers doing a "callback" to verify the sender address, doesn't this
delay sending mail in an unneccessary way?

2. The implementation seems to be well done. Thanks for the package.

3. On debian woody, which is what we use, you have to do the following
in order for milter-greylist to build:

  apt-get install libmilter-dev
  ln -s . /usr/lib/libmilter/lib
  ./configure --with-libmilter=/usr/lib/libmilter

This has been reported before on this list. Maybe the information could
be included in the README file.

4. The whitepaper also suggests a record expire time of 36 days. Which
expirery times do folks use? The defaults configuration file has three
days only. Probably this is due to the fear of too much memory consumption?

5. The mailing list robot somehow sucks... The archive is not very well
searchable. And I still don't want to "Complete my Yahoo! Groups
account" "by connecting my email address to Yahoo account". So please
excuse me if my posting is partly redundant.

Greetings,
Moritz

Re: [milter-greylist] Null Sender

2005-01-03 by manu@netbsd.org

Moritz Both <mb@...> wrote:

> so I searched the correct implementation for me and it seemd
> thet milter-greylist was it.

Yes, that's probably right :)

> The whitepaper describes a need for delaying the tempfail message until
> the end of the DATA phase if the sender address is null. It appears to
> me that milter-greylist does not do that. Giving a increasing number of
> servers doing a "callback" to verify the sender address, doesn't this
> delay sending mail in an unneccessary way?

That's right. The motivation behind filtering at the RCPT stage is that
we can perform greylisting for some users and not for others. If you
filter at the DATA stage, every recipient ge the same behavior.

A good fix would be to filter at RCPT stage except if the sender is <>
 
> 3. On debian woody, which is what we use, you have to do the following
> in order for milter-greylist to build:
> 
>   apt-get install libmilter-dev
>   ln -s . /usr/lib/libmilter/lib
>   ./configure --with-libmilter=/usr/lib/libmilter
> 
> This has been reported before on this list. Maybe the information could
> be included in the README file.

A much better way of handling that issue would be to fix the Debian
package. Who did it?
 
> 4. The whitepaper also suggests a record expire time of 36 days. Which
> expirery times do folks use? The defaults configuration file has three
> days only. Probably this is due to the fear of too much memory consumption?

Also because most mail servers will give up after retrying for 3 or 5
days.

-- 
Emmanuel Dreyfus
Publicité subliminale: achetez ce livre!
http://www.eyrolles.com/Informatique/Livre/9782212114638/livre-bsd.php
manu@...

Re: [milter-greylist] Null Sender

2005-01-03 by Matthieu Herrb

manu@... wrote:

>  
> 
>>4. The whitepaper also suggests a record expire time of 36 days. Which
>>expirery times do folks use? The defaults configuration file has three
>>days only. Probably this is due to the fear of too much memory consumption?
> 
> 
> Also because most mail servers will give up after retrying for 3 or 5
> days.
> 
I think there is some confusion here. 36 days is the recommended 
expiration for the auto-whitelist.
Here at LAAS, we use 3 days for the greylist timeout and 5 weeks for the 
white list. This leads us to a ~100k lines database with approximatly
50k auto-whitelist entries and 50k greylist entries.
-- 
					Matthieu

Patch: Delay reject until after DATA phase if sender is null (was: Re: Null Sender)

2005-01-03 by Moritz Both

So nobody said delaying until after the DATA phase if the sender is null
is a bad thing. Happy to be able to contribute easily, here is the patch
against 1.6 (I havn't tried the development versions because production
is my goal and of course this patch is bug free). Do with it whatevery
you want...

It's only slightly tested so far.

Greetings,
Moritz

diff -u -r milter-greylist-1.6/ChangeLog milter-greylist-1.6-mo/ChangeLog
--- milter-greylist-1.6/ChangeLog	Thu Dec  9 00:24:56 2004
+++ milter-greylist-1.6-mo/ChangeLog	Mon Jan  3 16:28:26 2005
@@ -1,3 +1,6 @@
+
+	Delay tempfail until end of DATA phase if envelope sender is <>
+	(Moritz Both)
  1.6:
  	Updated default whitelist
  1.6rc1:
diff -u -r milter-greylist-1.6/milter-greylist.c milter-greylist-1.6-mo/milter-greylist.c
--- milter-greylist-1.6/milter-greylist.c	Sat Nov 27 15:43:17 2004
+++ milter-greylist-1.6-mo/milter-greylist.c	Mon Jan  3 16:32:01 2005
@@ -78,6 +78,7 @@
  static char *strncpy_rmsp(char *, char *, size_t);
  static char *gmtoffset(time_t *, char *, size_t);
  static void writepid(char *);
+static void log_and_report_greylisting(SMFICTX *ctx, struct mlfi_priv *priv, char *rcpt);

  struct smfiDesc smfilter =
  {
@@ -291,10 +292,8 @@
  {
  	struct mlfi_priv *priv;
  	time_t remaining;
-	char hdr[HDRLEN + 1];
  	char addrstr[IPADDRSTRLEN];
  	char rcpt[ADDRLEN + 1];
-	int h, mn, s;

  	priv = (struct mlfi_priv *) smfi_getpriv(ctx);

@@ -384,29 +383,26 @@
  	    priv->priv_queueid) != 0)
  		return SMFIS_CONTINUE;

+	priv->priv_remaining = remaining;
+
  	/*
  	 * The message has been added to the greylist and will be delayed.
-	 * Log this and report to the client.
+	 * If the sender address is null, this will be done after the DATA
+	 * phase, otherwise immediately.
  	 */
-	h = remaining / 3600;
-	remaining = remaining % 3600;
-	mn = (remaining / 60);
-	remaining = remaining % 60;
-	s = remaining;
-
-	syslog(LOG_INFO, "%s: addr %s from %s to %s delayed for %02d:%02d:%02d",
-	    priv->priv_queueid, addrstr, priv->priv_from, *envrcpt, h, mn, s);
-
-	if (conf.c_quiet) {
-		(void)smfi_setreply(ctx, "451", "4.7.1",
-		    "Greylisting in action, please come back later");
-	} else {
-		snprintf(hdr, HDRLEN,
-		    "Greylisting in action, please come "
-		    "back in %02d:%02d:%02d", h, mn, s);
-		(void)smfi_setreply(ctx, "451", "4.7.1", hdr);
+	if (strcmp(priv->priv_from, "<>") == 0) {
+		priv->priv_delayed_reject = 1;
+		if (*priv->priv_rcpt == 0)
+			strcpy(priv->priv_rcpt, rcpt);
+		else
+			strcpy(priv->priv_rcpt, "(multiple recipients)");
+		return SMFIS_CONTINUE;
  	}

+	/*
+	 * Log temporary failure and report to the client.
+	 */
+	log_and_report_greylisting(ctx, priv, *envrcpt);
  	return SMFIS_TEMPFAIL;
  }

@@ -429,6 +425,11 @@

  	priv = (struct mlfi_priv *) smfi_getpriv(ctx);

+	if (priv->priv_delayed_reject) {
+		log_and_report_greylisting(ctx, priv, priv->priv_rcpt);
+		return SMFIS_TEMPFAIL;
+	}
+
  	if ((fqdn = smfi_getsymval(ctx, "{j}")) == NULL) {
  		syslog(LOG_DEBUG, "smfi_getsymval failed for {j}");
  		gethostname(host, ADDRLEN);
@@ -1134,4 +1135,50 @@

  	syslog(LOG_INFO, "Exiting");
  	return;
+}
+
+void
+log_and_report_greylisting(ctx, priv, rcpt)
+	SMFICTX *ctx;
+	struct mlfi_priv *priv;
+	char *rcpt;
+{
+	int h, mn, s;
+	char hdr[HDRLEN + 1];
+	char addrstr[IPADDRSTRLEN];
+	time_t remaining;
+	char *delayed_rj;
+
+	/*
+	 * The message has been added to the greylist and will be delayed.
+	 * Log this and report to the client.
+	 */
+	iptostring(SA(&priv->priv_addr), priv->priv_addrlen, addrstr,
+	    sizeof(addrstr));
+
+	remaining = priv->priv_remaining;
+	h = remaining / 3600;
+	remaining = remaining % 3600;
+	mn = (remaining / 60);
+	remaining = remaining % 60;
+	s = remaining;
+
+	if (priv->priv_delayed_reject)
+		delayed_rj = " after DATA phase";
+	else
+		delayed_rj = "";
+
+	syslog(LOG_INFO, "%s: addr %s from %s to %s delayed%s for %02d:%02d:%02d",
+	    priv->priv_queueid, addrstr, priv->priv_from, rcpt, delayed_rj,
+	    h, mn, s);
+
+	if (conf.c_quiet) {
+		(void)smfi_setreply(ctx, "451", "4.7.1",
+		    "Greylisting in action, please come back later");
+	} else {
+		snprintf(hdr, HDRLEN,
+		    "Greylisting in action, please come "
+		    "back in %02d:%02d:%02d", h, mn, s);
+		(void)smfi_setreply(ctx, "451", "4.7.1", hdr);
+	}
  }
diff -u -r milter-greylist-1.6/milter-greylist.h milter-greylist-1.6-mo/milter-greylist.h
--- milter-greylist-1.6/milter-greylist.h	Mon Aug  2 14:11:48 2004
+++ milter-greylist-1.6-mo/milter-greylist.h	Mon Jan  3 16:15:38 2005
@@ -85,9 +85,12 @@
  	char priv_hostname[ADDRLEN + 1];
  	char priv_helo[ADDRLEN + 1];
  	char priv_from[ADDRLEN + 1];
+	char priv_rcpt[ADDRLEN + 1];
  	time_t priv_elapsed;
  	int priv_whitelist;
  	char *priv_queueid;
+	int priv_delayed_reject;
+	time_t priv_remaining;
  };

  sfsistat mlfi_connect(SMFICTX *, char *, _SOCK_ADDR *);

Re: [milter-greylist] Patch: Delay reject until after DATA phase if sender is null (was: Re: Null Sender)

2005-01-04 by manu@netbsd.org

Moritz Both <mb@...> wrote:

> So nobody said delaying until after the DATA phase if the sender is null
> is a bad thing. Happy to be able to contribute easily, here is the patch
> against 1.6 (I havn't tried the development versions because production
> is my goal and of course this patch is bug free). 

Well, developping against non latest code is not a good idea. I'll try
if it applies on 1.7.x.

-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu@...

Re: [milter-greylist] Patch: Delay reject until after DATA phase if sender is null (was: Re: Null Sender)

2005-01-04 by manu@netbsd.org

Moritz Both <mb@...> wrote:

> So nobody said delaying until after the DATA phase if the sender is null
> is a bad thing. Happy to be able to contribute easily, here is the patch
> against 1.6 (I havn't tried the development versions because production
> is my goal and of course this patch is bug free). Do with it whatevery
> you want...

Reading your code, I remembered why this was a difficult problem. Think
about this kind of scenario:

greylist.conf contains this:
acl whitelist rcpt john.doe@...
acl greylist default

Here comes a message:
addr 192.0.2.13, from <>, rcpt someone@...,john.doe@...

When you see someone@... you tag it for rejection after the DATA
stage, so you answer OK.

When you see john.doe@..., you just accept the message, you
answer OK.

Then you reach DATA. What should we do? 
1) Sending TEMPFAIL now will cause the message to john.doe@...
to be delayed whereas john.doe@... were not to have greylisted
mail.  
2) Sending OK will cause the mail to someone@... to pass through
whereas we wanted to have it greylisted.

I'm not sure what your code will do (can you tell it? :-), but anyway it
is bound to badly address this scenario as none of the two alternative
are satisfying.

How should milter-greylist behave in such a situation? As we have to
choose between two broken behaviors, I wonder if there shouldn't be a
config knob to select between the three behaviors when receiving a mail
from <>:
- don't wait DATA stage, do the TEMPFAIL or OK at RCPT stage
- wait DATA stage but fallback to TEMPFAIL if RCPT stage lead to
incompatible results.
- wait DATA stage but fallback to OK if RCPT stages lead to incompatible
results.

Something such as:
delayed_reject (no|fallback greylist|fallback whitelist)

Opinions on this topic?
 
-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu@...

Re: [milter-greylist] Patch: Delay reject until after DATA phase if sender is null (was: Re: Null Sender)

2005-01-04 by Scot L. Harris

On Tue, 2005-01-04 at 17:46, manu@... wrote:
> Moritz Both <mb@...> wrote:
> 
> > So nobody said delaying until after the DATA phase if the sender is null
> > is a bad thing. Happy to be able to contribute easily, here is the patch
> > against 1.6 (I havn't tried the development versions because production
> > is my goal and of course this patch is bug free). Do with it whatevery
> > you want...
> 
> Reading your code, I remembered why this was a difficult problem. Think
> about this kind of scenario:

<good example snipped>


> 
> Opinions on this topic?
>  

Personally I think keeping it simple is the best option, if it has a
null sender delay it by default.

Trying to get to tricky with it will result in problems which will
result in more complex schemes to resolve those problems etc.


-- 
Scot L. Harris
webid@...

Are you having fun yet?

Re: [milter-greylist] Patch: Delay reject until after DATA phase if sender is null (was: Re: Null Sender)

2005-01-06 by Moritz Both

Scot L. Harris schrieb:

> Personally I think keeping it simple is the best option, if it has a
> null sender delay it by default.
> 
> Trying to get to tricky with it will result in problems which will
> result in more complex schemes to resolve those problems etc.

I agree. We talk about delaying a message for a couple of hours, not 
about dropping it. Probably I'm not aware of all situations in which a 
null sender message is sent, but frankly I cannot think of a situation 
where a well-behaved MTA would send one to multiple recipients?!

Wouldn't a legetimate null sender message almost always have one 
recipient only since it is an error message to a single sender of 
another message?

The patch will make milter-greylist to tempfail if any of the recipients 
is not white listed.

Moritz

Re: [milter-greylist] Patch: Delay reject until after DATA phase if sender is null (was: Re: Null Sender)

2005-01-06 by manu@netbsd.org

Moritz Both <mb@...> wrote:

> I agree. We talk about delaying a message for a couple of hours, not 
> about dropping it. Probably I'm not aware of all situations in which a
> null sender message is sent, but frankly I cannot think of a situation
> where a well-behaved MTA would send one to multiple recipients?!

It's true I have trouble to imagine a legal situation where we would
have a null sender and multiple recipients.

Ideas, anyone?

-- 
Emmanuel Dreyfus
Publicité subliminale: achetez ce livre!
http://www.eyrolles.com/Informatique/Livre/9782212114638/livre-bsd.php
manu@netbsd.org

Re: [milter-greylist] Patch: Delay reject until after DATA phase if sender is null (was: Re: Null Sender)

2005-01-06 by Stephane Lentz

On Thu, Jan 06, 2005 at 09:27:34PM +0100, manu@... wrote:
> 
> Moritz Both <mb@...> wrote:
> 
> > I agree. We talk about delaying a message for a couple of hours, not 
> > about dropping it. Probably I'm not aware of all situations in which a
> > null sender message is sent, but frankly I cannot think of a situation
> > where a well-behaved MTA would send one to multiple recipients?!
> 
> It's true I have trouble to imagine a legal situation where we would
> have a null sender and multiple recipients.
> 
> Ideas, anyone?
> 
> -- 
Alias expansion case is maybe a case :
http://www.exim.org/pipermail/exim-users/Week-of-Mon-20020923/044111.html

mentioning RFC2505 2.6.1 ( http://www.faqs.org/rfcs/rfc2505.html )

But Mailshield offers such a blocking :
http://www.lyris.com/mshelp/RejectemptyMAILFROMwithmultiplerecipients.html

SL/

Re: [milter-greylist] Patch: Delay reject until after DATA phase if sender is null (was: Re: Null Sender)

2005-01-07 by Matthias Scheler

On Thu, Jan 06, 2005 at 09:27:34PM +0100, Emmanuel Dreyfus wrote:
> > I agree. We talk about delaying a message for a couple of hours, not 
> > about dropping it. Probably I'm not aware of all situations in which a
> > null sender message is sent, but frankly I cannot think of a situation
> > where a well-behaved MTA would send one to multiple recipients?!
> It's true I have trouble to imagine a legal situation where we would
> have a null sender and multiple recipients.

Maybe like this?

1.) Host "mail.a.com" hast an alias "foobar@..." which contains "1@..."
    and "2@...".

2.) Host "mail.a.com" send an e-mail to "non@..." which bounces
    with "foobar@..." as the sender.

3.) Host "mail.a.com" delivers the bounce to "foobar@...", expands it
    to "1@..." and "2@..." and tries to deliver that to "mail.b.com".

	Kind regards

-- 
Matthias Scheler                                  http://scheler.de/~matthias/

Re: [milter-greylist] Patch: Delay reject until after DATA phase if sender is null (was: Re: Null Sender)

2005-01-08 by manu@netbsd.org

Matthias Scheler <tron@...> wrote:

> 1.) Host "mail.a.com" hast an alias "foobar@..." which contains "1@..."
>     and "2@...".
> 
> 2.) Host "mail.a.com" send an e-mail to "non@..." which bounces
>     with "foobar@..." as the sender.
> 
> 3.) Host "mail.a.com" delivers the bounce to "foobar@...", expands it
>     to "1@..." and "2@..." and tries to deliver that to "mail.b.com".

So, would it be a problem to greylist the DSN if 1@... asked for
greylisting and not 2@...? I'd say nobody expects instant DSN, so
greylisting it will not be an issue...



-- 
Emmanuel Dreyfus
Publicité subliminale: achetez ce livre!
http://www.eyrolles.com/Informatique/Livre/9782212114638/livre-bsd.php
manu@...

Re: [milter-greylist] Patch: Delay reject until after DATA phase if sender is null (was: Re: Null Sender)

2005-01-11 by Matthias Scheler

On Sat, Jan 08, 2005 at 09:05:57AM +0100, Emmanuel Dreyfus wrote:
> > 1.) Host "mail.a.com" hast an alias "foobar@..." which contains "1@..."
> >     and "2@...".
> > 
> > 2.) Host "mail.a.com" send an e-mail to "non@..." which bounces
> >     with "foobar@..." as the sender.
> > 
> > 3.) Host "mail.a.com" delivers the bounce to "foobar@...", expands it
> >     to "1@..." and "2@..." and tries to deliver that to "mail.b.com".
> So, would it be a problem to greylist the DSN if 1@... asked for
> greylisting and not 2@...?

No. In step 3 the mailer on "mail.a.com" will deliver an e-mail with a null
sender address to two people on "mail.b.com" which IIUC is what the
discussion was about.

	Kind regards

-- 
Matthias Scheler                                  http://scheler.de/~matthias/

Move to quarantaine

This moves the raw source file on disk only. The archive index is not changed automatically, so you still need to run a manual refresh afterward.