Yahoo Groups archive

Milter-greylist

Index last updated: 2026-04-28 23:32 UTC

Thread

"peer queue overflow error" and patch proposal

"peer queue overflow error" and patch proposal

2009-01-16 by Laurence MOINDROT

Hi everyone,

We are running milter-greylist 4.0.1 on FreeBSD 7.0.
We have 8 servers, each handling about 500,000 messages daily.
Servers stop about 90% of unsolicited messages and they
continuously synchronize themself with each other.

Sometimes one server stops synchronizing with the others.
The sync queue grows and the error "peer queue overflow"
appears.

The only solution to solve the problem is to kill the milter
process and restart it. We've tried to increase the SYNC_MAXQLEN
value to 16384, but the problem still occurs.

We found a potentially infinite loop in the code in sync_send() function,
label "get_more:".
Enclosed is the patch we wrote. We added a counter to limit the time
in the "get_more:" loop in which the program get stuck when a peer
doesn't respond. We tried to find out why a peer stops responding,
but we didn't figure it out yet. The TCP connection remains established
and we don't see any connectivity problem.

We've been using it since October 2008.
We still see a bit of  "peer queue overflow" errors during spam's flood,
but the patch has really improved the situation.

We also added a SYNC_MAXQLEN parameter in the greylist.conf
so it's easier to dynamically change the value.

Hope it will help.

Laurence MOINDROT and Jean BENOIT
--
University of Strasbourg
Direction Informatique

Re: [milter-greylist] "peer queue overflow error" and patch proposal

2009-01-16 by manu@netbsd.org

Laurence MOINDROT <Laurence.Moindrot@crc.u-strasbg.fr> wrote:

> We've been using it since October 2008.
> We still see a bit of  "peer queue overflow" errors during spam's flood,
> but the patch has really improved the situation.

Congratulations for tracking this down.
If nobody objects, I will integrate this patch in the next release.

-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu@...

milter-greylist-4.1.9 is available

2009-01-16 by manu@netbsd.org

Emmanuel Dreyfus <manu@...> wrote:

> If nobody objects, I will integrate this patch in the next release.

And here is a unstable snapshot with the patch in it:

http://ftp.espci.fr/pub/milter-greylist/milter-greylist-4.1.9.tgz
MD5 (milter-greylist-4.1.9.tgz) = 8676b80498e2ac14bf3280ea8c025d6d


-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu@...

Re: [milter-greylist] milter-greylist-4.1.9 is available

2009-01-17 by Jack L. Stone

At 10:28 PM 1.16.2009 +0100, you wrote: 
>>>>
  

Emmanuel Dreyfus <<mailto:manu%40netbsd.org>manu@...> wrote:

> If nobody objects, I will integrate this patch in the next release.

And here is a unstable snapshot with the patch in it:

<http://ftp.espci.fr/pub/milter-greylist/milter-greylist-4.1.9.tgz>http://ft
p.espci.fr/pub/milter-greylist/milter-greylist-4.1.9.tgz
MD5 (milter-greylist-4.1.9.tgz) = 8676b80498e2ac14bf3280ea8c025d6d

-- 
Emmanuel Dreyfus
<http://hcpnet.free.fr/pubz>http://hcpnet.free.fr/pubz
<mailto:manu%40netbsd.org>manu@...

Just tried to build this new version but ran into a snag I have yet to
figute out.

What caveat did I forget to read? ver-4.1.6 built okay.

configure: creating ./config.status
config.status: creating Makefile
config.status: creating milter-greylist.spec
config.status: creating config.h
===>  Building for milter-greylist-4.1.9
cc -O2 -fno-strict-aliasing -pipe -march=pentium4  -I/usr/local/include
-Wall -I/usr/local/include -DUSE_DNSRBL -D_BSD_SOURCE -I. -I. -c
milter-greylist.c
cc -O2 -fno-strict-aliasing -pipe -march=pentium4  -I/usr/local/include
-Wall -I/usr/local/include -DUSE_DNSRBL -D_BSD_SOURCE -I. -I. -c pending.c
cc -O2 -fno-strict-aliasing -pipe -march=pentium4  -I/usr/local/include
-Wall -I/usr/local/include -DUSE_DNSRBL -D_BSD_SOURCE -I. -I. -c sync.c
cc -O2 -fno-strict-aliasing -pipe -march=pentium4  -I/usr/local/include
-Wall -I/usr/local/include -DUSE_DNSRBL -D_BSD_SOURCE -I. -I. -c dnsrbl.c
cc -O2 -fno-strict-aliasing -pipe -march=pentium4  -I/usr/local/include
-Wall -I/usr/local/include -DUSE_DNSRBL -D_BSD_SOURCE -I. -I. -c list.c
cc -O2 -fno-strict-aliasing -pipe -march=pentium4  -I/usr/local/include
-Wall -I/usr/local/include -DUSE_DNSRBL -D_BSD_SOURCE -I. -I. -c macro.c
bison -y -p`echo conf_yacc.c|sed 's/^\([^_]\{1,\}_\).*$/\1/'` conf_yacc.y
conf_yacc.y:443.24-34: symbol LOGFAC_USER is used, but is not defined as a
token and has no rules
conf_yacc.y:457.24-36: symbol LOGFAC_LOCAL3 is used, but is not defined as
a token and has no rules
conf_yacc.y:444.24-34: symbol LOGFAC_MAIL is used, but is not defined as a
token and has no rules
conf_yacc.y:458.24-36: symbol LOGFAC_LOCAL4 is used, but is not defined as
a token and has no rules
conf_yacc.y:459.24-36: symbol LOGFAC_LOCAL5 is used, but is not defined as
a token and has no rules
conf_yacc.y:445.24-36: symbol LOGFAC_DAEMON is used, but is not defined as
a token and has no rules
conf_yacc.y:460.24-36: symbol LOGFAC_LOCAL6 is used, but is not defined as
a token and has no rules
conf_yacc.y:769.25-27: symbol P0F is used, but is not defined as a token
and has no rules
conf_yacc.y:446.24-34: symbol LOGFAC_AUTH is used, but is not defined as a
token and has no rules
conf_yacc.y:461.24-36: symbol LOGFAC_LOCAL7 is used, but is not defined as
a token and has no rules
conf_yacc.y:447.24-36: symbol LOGFAC_SYSLOG is used, but is not defined as
a token and has no rules
conf_yacc.y:794.25-29: symbol SPAMD is used, but is not defined as a token
and has no rules
conf_yacc.y:448.24-33: symbol LOGFAC_LPR is used, but is not defined as a
token and has no rules
conf_yacc.y:971.25-33: symbol DKIMCHECK is used, but is not defined as a
token and has no rules
conf_yacc.y:449.24-34: symbol LOGFAC_NEWS is used, but is not defined as a
token and has no rules
conf_yacc.y:1281.17-24: symbol LDAPCONF is used, but is not defined as a
token and has no rules
conf_yacc.y:450.24-34: symbol LOGFAC_UUCP is used, but is not defined as a
token and has no rules
conf_yacc.y:998.25-33: symbol LDAPCHECK is used, but is not defined as a
token and has no rules
conf_yacc.y:451.24-34: symbol LOGFAC_CRON is used, but is not defined as a
token and has no rules
conf_yacc.y:325.17-27: symbol DOMAINEXACT is used, but is not defined as a
token and has no rules
conf_yacc.y:452.24-38: symbol LOGFAC_AUTHPRIV is used, but is not defined
as a token and has no rules
conf_yacc.y:453.24-33: symbol LOGFAC_FTP is used, but is not defined as a
token and has no rules
conf_yacc.y:395.17-23: symbol P0FSOCK is used, but is not defined as a
token and has no rules
conf_yacc.y:94.20-29: symbol SPF_STATUS is used, but is not defined as a
token and has no rules
conf_yacc.y:408.17-25: symbol SPAMDSOCK is used, but is not defined as a
token and has no rules
conf_yacc.y:730.25-29: symbol NOLOG is used, but is not defined as a token
and has no rules
conf_yacc.y:454.24-36: symbol LOGFAC_LOCAL0 is used, but is not defined as
a token and has no rules
conf_yacc.y:441.9-14: symbol LOGFAC is used, but is not defined as a token
and has no rules
conf_yacc.y:756.25-33: symbol ADDHEADER is used, but is not defined as a
token and has no rules
conf_yacc.y:455.24-36: symbol LOGFAC_LOCAL1 is used, but is not defined as
a token and has no rules
conf_yacc.y:95.20-29: symbol SPAMDSOCKT is used, but is not defined as a
token and has no rules
conf_yacc.y:442.24-34: symbol LOGFAC_KERN is used, but is not defined as a
token and has no rules
conf_yacc.y:456.24-36: symbol LOGFAC_LOCAL2 is used, but is not defined as
a token and has no rules
*** Error code 1



(^_^)
Happy trails,
Jack L. Stone

System Admin
Sage-american

Re: [milter-greylist] milter-greylist-4.1.9 is available

2009-01-18 by manu@netbsd.org

Jack L. Stone <jacks@...> wrote:

> What caveat did I forget to read? ver-4.1.6 built okay.

Um, yes, there are important bits missing here. Stay tuned, 4.1.10 is
coming soon.

-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu@...

milter-greylist-4.1.10 is available

2009-01-18 by manu@netbsd.org

Here is milter-greylist-4.1.10, which is just 4.1.9 with all the changes
really included. That should help it building.

http://ftp.espci.fr/pub/milter-greylist/milter-greylist-4.1.10.tgz
MD5 (milter-greylist-4.1.10.tgz) = 1ffdd0ab6176e900a5f4b0136f89eb98

-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu@...

Re: [milter-greylist] milter-greylist-4.1.10 is available

2009-01-18 by Jack L. Stone

At 06:48 AM 1.18.2009 +0100, you wrote: 

>>>>

  


Here is milter-greylist-4.1.10, which is just 4.1.9 with all the 
changes

really included. That should help it building.


<<http://ftp.espci.fr/pub/milter-greylist/milter-greylist-4.1.10.tgz>http://ftp.espci.fr/pub/milter-greylist/milter-greylist-4.1.10.tgz

MD5 (milter-greylist-4.1.10.tgz) = 1ffdd0ab6176e900a5f4b0136f89eb98


-- 

Emmanuel Dreyfus

<<http://hcpnet.free.fr/pubz>http://hcpnet.free.fr/pubz

<<mailto:manu%40netbsd.org>manu@...


<<<<<<<<

All is well again. Thanks!


Jack

>>>>





(^_^)

Happy trails,

Jack L. Stone


System Admin

Sage-american

Re: "peer queue overflow error" and patch proposal

2009-02-12 by reschauzier

> doesn't respond. We tried to find out why a peer stops responding,
> but we didn't figure it out yet. The TCP connection remains established
> and we don't see any connectivity problem.

I am wondering if this could be a dead lock situation between the
autowhite and pending queues. Probably not, but to be sure, is there a
chance you can try the no_autowhite patch to see if that helps your
situation? The patch combines the queues into one, so it avoids
potentially locking conflicts between the two.

The patch is available at
http://files.eschauzier.org/milter-greylist/no_autowhite-4.1.10-1.patch

It patches against v4.1.10 thru v4.1.12.

Thanks,
Rudy.

Re: [milter-greylist] Re: "peer queue overflow error" and patch proposal

2009-02-12 by Emmanuel Dreyfus

On Thu, Feb 12, 2009 at 09:25:22AM -0000, reschauzier wrote:
> The patch is available at
> http://files.eschauzier.org/milter-greylist/no_autowhite-4.1.10-1.patch

Note on the integration timeline: I will be away from keyboards
next week, and I ma not sure I will have time to handle it before leaving.
But don't worry, I keep in mind I have to do it.

-- 
Emmanuel Dreyfus
manu@...

Move to quarantaine

This moves the raw source file on disk only. The archive index is not changed automatically, so you still need to run a manual refresh afterward.