Yahoo Groups archive

Milter-greylist

Index last updated: 2026-04-28 23:32 UTC

Thread

spamd checks hang when message contains

spamd checks hang when message contains

2010-06-21 by Enrico Scholz

Hi,

I see from time to time messages like

| milter-greylist: SPAMD/1.0 79 Timeout: (300 second timeout while trying to CHECK) 

in my logfiles.  After analyzing the related network traffic, all these
messages have in common that they contain an ASCII \0 in their bodies.
This character is allowed by RFC 822 (but not in 2822 + 5322).

In milter-greylist code, there is written

|     "CHECK SPAMC/1.2\r\nContent-length: %d\r\n\r\n",
|       (unsigned int)(priv->priv_msgcount + strlen(rcvhdr)));
| ...
|     spamd_write(sock, b->b_lines, strlen(b->b_lines))

--> spamd expects a content length which includes the ASCII \0, but
there are copied only ASCIIZ strings and 'spamd' does not get enough
data hence.

A trivial fix might be 'shutdown(sock, SHUT_WR)' but there are required
more invasive changes (make b_lines a buffer instead of ASCIIZ strings)
to do it right.


Enrico

Re: [milter-greylist] spamd checks hang when message contains

2010-06-21 by Petar Bogdanovic

On Mon, Jun 21, 2010 at 07:46:17PM +0200, Enrico Scholz wrote:
> 
> in my logfiles.  After analyzing the related network traffic, all these
> messages have in common that they contain an ASCII \0 in their bodies.
> This character is allowed by RFC 822 (but not in 2822 + 5322).

While it's a bit unclean that the spamd part of milter-greylist can't
handle a null character, it's also not likely that I'll ever think a
null character belongs to an SMTP conversation.

If you're running Postfix, you could add '\0' to

	message_reject_characters

		Petar Bogdanovic

Re: spamd checks hang when message contains

2010-06-21 by Enrico Scholz

Petar Bogdanovic <petar-+Dgt6vZh/JqsTnJN9+BGXg@...> writes:

>> in my logfiles.  After analyzing the related network traffic, all
>> these messages have in common that they contain an ASCII \0 in their
>> bodies.  This character is allowed by RFC 822 (but not in 2822 +
>> 5322).
>
> While it's a bit unclean that the spamd part of milter-greylist can't
> handle a null character, it's also not likely that I'll ever think a
> null character belongs to an SMTP conversation.

I have seen the error on automated generated messages only (e.g. newsletters)
where the sender has probably an off-by-one error.  But as I said, legacy
SMTP allows it and would not explicitly block it hence.


Enrico

[PATCH] Re: spamd checks hang when message contains

2010-06-21 by Enrico Scholz

Enrico Scholz
<enrico.scholz-jNDFPZUTrfQ+B2oLq8eQJv4efur1V5z/s0AfqQuZ5sE@...>
writes:

> | milter-greylist: SPAMD/1.0 79 Timeout: (300 second timeout while trying to CHECK) 

attached patch against 4.2.5 seems to fix the timeout.  But 'spamd'
itself seems to get confused by such messages and responds immediately
with

| spamd: bad protocol: header error: (Content-Length mismatch: Expected 14121 bytes, got 2448 bytes)

where the reported count is the number of chars before the \0.


Enrico

Index: milter-greylist-4.2.5/milter-greylist.c
===================================================================
--- milter-greylist-4.2.5.orig/milter-greylist.c
+++ milter-greylist-4.2.5/milter-greylist.c
@@ -735,6 +735,7 @@ real_header(ctx, name, value)
 	strcat(h->h_line, sep);
 	strcat(h->h_line, value);
 	strcat(h->h_line, crlf);
+	h->h_len = len;
 
 	TAILQ_INSERT_TAIL(&priv->priv_header, h, h_list);
 
@@ -814,6 +815,7 @@ real_body(ctx, chunk, size)
 			exit(EX_OSERR);
 		}
 
+		b->b_len = strlen(crlf);
 		TAILQ_INSERT_TAIL(&priv->priv_body, b, b_list);
 
 		priv->priv_msgcount += strlen(crlf);
@@ -847,6 +849,7 @@ real_body(ctx, chunk, size)
 
 		memcpy(b->b_lines + priv->priv_buflen, chunk, i);
 		b->b_lines[linelen] = '\0';
+		b->b_len = linelen;
 		priv->priv_buflen = 0;
 
 		TAILQ_INSERT_TAIL(&priv->priv_body, b, b_list);
@@ -904,6 +907,7 @@ real_eom(ctx)
 		}
 
 		b->b_lines = priv->priv_buf;
+		b->b_len = priv->priv_buflen - 1;
 		b->b_lines[priv->priv_buflen - 1] = '\0';
 
 		priv->priv_buf = NULL;
Index: milter-greylist-4.2.5/milter-greylist.h
===================================================================
--- milter-greylist-4.2.5.orig/milter-greylist.h
+++ milter-greylist-4.2.5/milter-greylist.h
@@ -173,11 +173,13 @@ struct rcpt {
 
 struct header {
 	char *h_line;
+	size_t h_len;
 	TAILQ_ENTRY(header) h_list;
 };
 
 struct body {
 	char *b_lines;
+	size_t b_len;
 	TAILQ_ENTRY(body) b_list;
 };
 
Index: milter-greylist-4.2.5/spamd.c
===================================================================
--- milter-greylist-4.2.5.orig/spamd.c
+++ milter-greylist-4.2.5/spamd.c
@@ -186,11 +186,11 @@ spamd_check(ad, stage, ap, priv)
 			return -1;
 
 	TAILQ_FOREACH(h, &priv->priv_header, h_list)
-		if (spamd_write(sock, h->h_line, strlen(h->h_line)) == -1)
+		if (spamd_write(sock, h->h_line, h->h_len) == -1)
 			return -1;
 			
 	TAILQ_FOREACH(b, &priv->priv_body, b_list)
-		if (spamd_write(sock, b->b_lines, strlen(b->b_lines)) == -1)
+		if (spamd_write(sock, b->b_lines, b->b_len) == -1)
 			return -1;
 
 	if (spamd_read(sock, buffer, SPAMD_BUFLEN) == -1)

Re: [milter-greylist] [PATCH] Re: spamd checks hang when message contains

2010-06-21 by Petar Bogdanovic

On Mon, Jun 21, 2010 at 09:25:04PM +0200, Enrico Scholz wrote:
> Enrico Scholz
> <enrico.scholz-jNDFPZUTrfQ+B2oLq8eQJv4efur1V5z/s0AfqQuZ5sE@...>
> writes:
> 
> > | milter-greylist: SPAMD/1.0 79 Timeout: (300 second timeout while trying to CHECK) 
> 
> attached patch against 4.2.5 seems to fix the timeout.  But 'spamd'
> itself seems to get confused by such messages and responds immediately
> with
> 
> | spamd: bad protocol: header error: (Content-Length mismatch: Expected 14121 bytes, got 2448 bytes)

If you really want to fix that, please do it properly.

		Petar Bogdanovic

Re: [PATCH] Re: spamd checks hang when message contains

2010-06-21 by Enrico Scholz

Petar Bogdanovic <petar-+Dgt6vZh/JqsTnJN9+BGXg@...> writes:

>> | spamd: bad protocol: header error: (Content-Length mismatch: Expected 14121 bytes, got 2448 bytes)
>
> If you really want to fix that, please do it properly.

I did it properly.  The message above is caused by a bug in 'spamd'.


Enrico

Re: [milter-greylist] Re: [PATCH] Re: spamd checks hang when message contains

2010-06-21 by Petar Bogdanovic

On Mon, Jun 21, 2010 at 10:15:59PM +0200, Enrico Scholz wrote:
> 
> 
> Petar Bogdanovic <petar-+Dgt6vZh/JqsTnJN9+BGXg@...> writes:
> 
> >> | spamd: bad protocol: header error: (Content-Length mismatch: Expected 14121 bytes, got 2448 bytes)
> >
> > If you really want to fix that, please do it properly.
> 
> I did it properly.  The message above is caused by a bug in 'spamd'.

No, you didn't.  What your change does is nothing more than passing the
buck to spamassassin which is way worse given how many lines of perl SA
pulls in its default configuration.

If you want to do it properly, sanitize the input.  Otherwise, leave it
because you don't want to trigger unknown SA bugs.  Especially not then
when the gain is nonexistent besides your maillog containing less noise.

		Petar Bogdanovic

Re: [milter-greylist] Re: [PATCH] Re: spamd checks hang when message contains

2010-06-22 by manu@netbsd.org

Petar Bogdanovic <petar@...> wrote:

> No, you didn't.  What your change does is nothing more than passing the
> buck to spamassassin which is way worse given how many lines of perl SA
> pulls in its default configuration.

RIght, I backed out the change, until we get a better idea of how to
deal with that.

-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu@...

Re: [PATCH] Re: spamd checks hang when message contains

2010-06-22 by Enrico Scholz

Enrico Scholz
<enrico.scholz-jNDFPZUTrfQ+B2oLq8eQJv4efur1V5z/s0AfqQuZ5sE@...>
writes:

> attached patch against 4.2.5 seems to fix the timeout.  But 'spamd'
> itself seems to get confused by such messages and responds immediately
> with
>
> | spamd: bad protocol: header error: (Content-Length mismatch: Expected 14121 bytes, got 2448 bytes)

my fault... I applied the patch on the wrong machine and restarted
'milter-greylist' on the right one.  At restart, the connection was cut
and spamd created the message above.

Patch works fine.


Enrico

Re: spamd checks hang when message contains

2010-07-09 by Enrico Scholz

Enrico Scholz
<enrico.scholz-jNDFPZUTrfQ+B2oLq8eQJv4efur1V5z/s0AfqQuZ5sE@...>
writes:

> | milter-greylist: SPAMD/1.0 79 Timeout: (300 second timeout while trying to CHECK) 
>
> in my logfiles.  After analyzing the related network traffic, all these
> messages have in common that they contain an ASCII \0 in their bodies.
> This character is allowed by RFC 822 (but not in 2822 + 5322).

I have to correct this; <NULL> is completely valid e.g. in a
'Content-Transfer-Encoding: binary' MIME attachment submitted by using
the 8BITMIME (RFC 1652) extension.


Enrico

Re: [milter-greylist] Re: spamd checks hang when message contains

2010-07-09 by Petar Bogdanovic

On Fri, Jul 09, 2010 at 10:06:48AM +0200, Enrico Scholz wrote:
> Enrico Scholz
> <enrico.scholz-jNDFPZUTrfQ+B2oLq8eQJv4efur1V5z/s0AfqQuZ5sE@...>
> writes:
> 
> > | milter-greylist: SPAMD/1.0 79 Timeout: (300 second timeout while trying to CHECK) 
> >
> > in my logfiles.  After analyzing the related network traffic, all these
> > messages have in common that they contain an ASCII \0 in their bodies.
> > This character is allowed by RFC 822 (but not in 2822 + 5322).
> 
> I have to correct this; <NULL> is completely valid e.g. in a
> 'Content-Transfer-Encoding: binary' MIME attachment submitted by using
> the 8BITMIME (RFC 1652) extension.

I just looked up a couple of things and it seems that various MUAs as
well as the Cyrus IMAP server won't handle a NULL properly or will
reject it altogether.

That said, while I never felt the need for accepting messages with NULL
characters, I also think that it should not be the job of milt.-greylist
to reject or sanitize those messages.  Thats what message_reject_char.
and friends are for.  Also there is a NULL_IN_BODY check in SpamAssassin
which tells me that they probably know how to deal with it.

So after re-reading your original patch, it now seems ok to me.

		Petar Bogdanovic

Re: [milter-greylist] Re: spamd checks hang when message contains

2010-07-09 by Emmanuel Dreyfus

On Fri, Jul 09, 2010 at 04:05:11PM +0200, Petar Bogdanovic wrote:
> So after re-reading your original patch, it now seems ok to me.

Ok, so if there is a consensus, then I will commit it when I will
be back home (sunday).

-- 
Emmanuel Dreyfus
manu@...

spamd protocol version error

2010-07-13 by Michael V. David

Hi all,

I compiled milter-greylist 4.2.6 and 4.3.8 with the same
configuration, then tested, and observed this difference:

under 4.2.6, spamd ACLs match as expected.

under 4.3.8, spamd ACLs never match, and these error messages are 
logged:

spamd protocol version mismatch
spamd returned non-ok

mvd

p.s. configure had:
--enable-p0f --enable-spamassassin --enable-dnsrbl --with-openldap 
--with-libcurl --with-libGeoIP --with-libspf2
Linux version 2.6.34.1
gcc version 4.4.3 20100127 (Red Hat 4.4.3-4) (GCC)



-- 
Michael V. David - http://mvdavid.com

Re: [milter-greylist] spamd protocol version error

2010-07-13 by manu@netbsd.org

Michael V. David <michael@...> wrote:

> under 4.2.6, spamd ACLs match as expected.
> under 4.3.8, spamd ACLs never match, and these error messages are 
> logged:

I guess this is why 4.2.x is called stable, and 4.3.x is called
developemetn snapshot :-)

-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu@...

Re: [milter-greylist] spamd protocol version error

2010-07-13 by Petar Bogdanovic

------- Original message -------
> From: Michael V. David <michael@...>
> To: milter-greylist@yahoogroups.com
> Sent: 13.7.'10,  6:16
>
> under 4.2.6, spamd ACLs match as expected.
>
> under 4.3.8, spamd ACLs never match, and these error messages are
> logged:
>
> spamd protocol version mismatch
> spamd returned non-ok

Make spamd use an inet socket, try to reproduce the problem,
listen to the chatter with tcpdump -A -s0 and post the output
somewhere.

Petar

Move to quarantaine

This moves the raw source file on disk only. The archive index is not changed automatically, so you still need to run a manual refresh afterward.