Milter-greylist

Hi,

a few hours ago milter-greylist died on one of our servers:

	milter-greylist: spamd socket failed: Too many open files
	milter-greylist: ACL evaluation failure
	milter-greylist: mkstemp("/var/milter/mgrey.db-000098ss") failed: Too many open files

I raised the limits but I'd still like to know if milter-greylist is
really supposed to die in such a case?


Also, I saw this after starting milter-greylist again:

	milter-greylist: unknown token ":" line 19202, skipping 
	milter-greylist: dump error at line 19202: syntax error 
	milter-greylist: unknown token "@" line 24283, skipping 
	milter-greylist: dump error at line 24283: syntax error 
	milter-greylist: unknown token "@" line 89828, skipping 
	milter-greylist: dump error at line 89828: syntax error 
	milter-greylist: unknown token ":" line 110011, skipping 
	milter-greylist: dump error at line 110011: syntax error 
	milter-greylist: unknown token "@" line 110011, skipping 
	milter-greylist: unknown token "@" line 208033, skipping 
	milter-greylist: dump error at line 208033: syntax error 

The problem were some sender-addresses like `ab@cd@...'.


Any ideas?


Thanks,

Petar

Petar Bogdanovic <petar@...> wrote:

> a few hours ago milter-greylist died on one of our servers:
> 
>       milter-greylist: spamd socket failed: Too many open files
>       milter-greylist: ACL evaluation failure
>       milter-greylist: mkstemp("/var/milter/mgrey.db-000098ss") failed:
Too many open files
> 
> I raised the limits but I'd still like to know if milter-greylist is
> really supposed to die in such a case?

milter-greylist is in the situation where the database cannot be dumped.
Shall we ignore that?
 
> Also, I saw this after starting milter-greylist again: 
>       milter-greylist: unknown token ":" line 19202, skipping 
>       milter-greylist: dump error at line 19202: syntax error 
(...)
> The problem were some sender-addresses like `ab@cd@...'.

The dump file parser is more strict that the code that stores tuples, so
some tuple are rejected on reload. This is harmless.

-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu@...

On Sat, Nov 08, 2008 at 07:03:14PM +0100, manu@... wrote:
> Petar Bogdanovic <petar@...> wrote:
> 
> > a few hours ago milter-greylist died on one of our servers:
> > 
> >       milter-greylist: spamd socket failed: Too many open files
> >       milter-greylist: ACL evaluation failure
> >       milter-greylist: mkstemp("/var/milter/mgrey.db-000098ss") failed:
> Too many open files
> > 
> > I raised the limits but I'd still like to know if milter-greylist is
> > really supposed to die in such a case?
> 
> milter-greylist is in the situation where the database cannot be dumped.
> Shall we ignore that?

If you let the daemon live he could recover as soon as some fds are
available again (the situation described above happened during a pretty
busy hour). It's a suboptimal configuration but the ability to dump the
database does not seem critical to me.


> > Also, I saw this after starting milter-greylist again: 
> >       milter-greylist: unknown token ":" line 19202, skipping 
> >       milter-greylist: dump error at line 19202: syntax error 
> (...)
> > The problem were some sender-addresses like `ab@cd@...'.
> 
> The dump file parser is more strict that the code that stores tuples, so
> some tuple are rejected on reload. This is harmless.

Ok, thanks.

Petar Bogdanovic <petar@...> wrote:

> > milter-greylist is in the situation where the database cannot be dumped.
> > Shall we ignore that?
> 
> If you let the daemon live he could recover as soon as some fds are
> available again (the situation described above happened during a pretty
> busy hour). It's a suboptimal configuration but the ability to dump the
> database does not seem critical to me.

This is the optimist scenario, but on the other hand, the system could
be running amok and never have more free file descriptors. This is what
will happen if a library has a file descriptor leak, for instance.

-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu@...

On Sat, Nov 08, 2008 at 09:24:47PM +0100, manu@... wrote:
> This is the optimist scenario, but on the other hand, the system could
> be running amok and never have more free file descriptors. This is what
> will happen if a library has a file descriptor leak, for instance.

I think this issue is not only about "too many open files". There are
other reasons why the database dump might fail (e.g. filesystem full).
The question is, if dumping the database is a vital part of mg and if
shutting down has any benefits?

Conditions, where the database can't be written might only be temporary
and do not or just partly affect the operation of milter-greylist. Hence
the stabilty of a mail system would profit if mg kept running.

Manuel