Milter-greylist

Hello,

after ten days of operation milter-greylist uses 1.8 GB virtual
memory with some estimated 50 to 100 MB more per day, but greylist.db
has only 42 MB.
If this is expected behaviour, we would plan an automatic re-start.
When to restart best?

Using 4.0a1, x86_64 SuSe Linux 10.1, dnsrbl and extended regexp configuration.

Michael

Michael Fromme <fromme@...> wrote:

> after ten days of operation milter-greylist uses 1.8 GB virtual
> memory with some estimated 50 to 100 MB more per day, but greylist.db
> has only 42 MB.

I'd say there is a leak.

-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu@...

Hello,

searching for a reason I found the dnsrbl_list_cleanup function
of dnsrbl.c which is not called anywhere (?)

A place would be real_close, just before free(priv) like
#ifdef USE_DNSRBL
		dnsrbl_list_cleanup(priv);
#endif

Is the dnsrbl list bound to a single email - or am I
missing something here? DNSBL lookups may take time, and
it's a good idea to keep knowledge for let's say an hour.

Michael

On Fri, May 04, 2007 at 01:57:46PM +0200, Michael Fromme wrote:
> searching for a reason I found the dnsrbl_list_cleanup function
> of dnsrbl.c which is not called anywhere (?)
> 
> A place would be real_close, just before free(priv) like
> #ifdef USE_DNSRBL
> 		dnsrbl_list_cleanup(priv);
> #endif

Indeed you found a memory leak.

> Is the dnsrbl list bound to a single email - or am I
> missing something here? 

Yes, the goal is to avoid multiple DNS requests when the same DNSRBL
is used in several ACL lines.

> DNSBL lookups may take time, and
> it's a good idea to keep knowledge for let's say an hour.

I'm not sure it's interesting to build a cache system: you can 
have a local DNS on your machine to make the lookups faster, and that
local DNS will respect the TTL set by the DNSRBL publisher.

-- 
Emmanuel Dreyfus
manu@...

Sorry to say, but 4.0a4 still leaks memory with our configuration.

My question to other users: Do they observe this, too?

If so, this would be worth to track down.
Tools like valgrind impose load and risk for a production system.
Another idea are wrapper functions (e.g. safe_malloc for malloc ...)
and have some statistics with these, but this would imply global
code changes, and thread locking would be an issue.

Michael

Michael Fromme wrote:
 > Sorry to say, but 4.0a4 still leaks memory with our configuration.

How do you determine that?

It is quite normal that the milter-greylist process grows.
The more SMTP connections you get, the more tuples it has
to store in memory.  The old tuples are not always
immediately freed when they expire, but they're only
garbage-collected when they are encountered during
traversing the list.  Emmanuel has explained that several
times in the past.

How much memory (or rather VM pages, to be exact) are used
also depends a lot on the malloc() implementation of the
libc of your operating system (for those who are not
familiar with C programming:  malloc() is the standard
C library call used to allocate memory for a process).

milter-greylist uses a lot of malloc() and free(), which
can lead to fragmentation, and some operating systems can
handle that better than others.  Often it is a trade-off
between memory efficiency and speed.  There are even malloc
implementations that never return unused VM pages back to
the kernel (except on process termination, of course).

Just as an example, on FreeBSD you can configure various
options for malloc() via /etc/malloc.conf.  For example,
by enabling the "R" flag, fragmentation can be reduced for
better memory efficiency, at the cost of speed, and there
are other flags to influence the behaviour.  Other
operating systems might have different ways to configure
malloc() behaviour, or possibly use a different libc with
a different implementation.

 > My question to other users: Do they observe this, too?

I don't observe any unusual behaviour that could only be
explained by a leak in milter-greylist.  Of course, the
process grows with time, but I think that's normal.  It
also starts to shrink somehwat when there are days with
lower number of SMTP connections.

Note that I'm not saying that there is no leak.  I'm just
saying that I don't have any hard evidence for that.

By the way, I see an interesting behaviour of the local
DNS server process (BIND) which seems to "follow" the
milter-greylist process.  When there is high activity
and milter-greylist grows, the named process also grows
somewhat.  I guess that's because sendmail and/or milter-
greylist perform DNS lookups for all the hosts which
connect (most of them are trying to deliver spam, of
course), and BIND caches them in memory for some time.
I'm not worried about that either, and I don't think
that BIND has a serious leak.

Best regards
   Oliver

-- 
Oliver Fromme, secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing b. M.
Handelsregister: Registergericht Muenchen, HRA 74606,  Gesch\ufffdftsfuehrung:
secnetix Verwaltungsgesellsch. mbH, Handelsregister: Registergericht M\ufffdn-
chen, HRB 125758,  Gesch\ufffdftsf\ufffdhrer: Maik Bachmann, Olaf Erb, Ralf Gebhart

FreeBSD-Dienstleistungen, -Produkte und mehr:  http://www.secnetix.de/bsd

"When your hammer is C++, everything begins to look like a thumb."
        -- Steve Haflich, in comp.lang.c++

Oliver Fromme schrieb:
>  > Sorry to say, but 4.0a4 still leaks memory with our configuration.
> 
> How do you determine that?

By observation : found one memory leak by strict definition,
and found a steady increase of virtual memory size.
I'd expect asymtotic memory usage of a properly tuned system.
If e.g. less than 50 MB raw data are stored then 500 MB
should be the maximum.

Finally I wrote some statistic routines with minimal
code intervention (e.g. malloc directed to call
safe_malloc by macro), fought with debug-crashes and
got some numbers.
These indicate no further leak by strict defintion,
but fragmentation for our configuration.

Still only a hypothesis: blocks with diffent sizes ~ 65536
can't be reclaimed. I will test if it helps to allocate these
with sizes rounded to 4K boundaries.
Probably they are allocated in real_body. I did not expect that,
but a rule
racl greylist from ...
fires acl_add_clause, which sets conf.c_maxpeek to -1

My tool files available on request:
mtool.c
mtool.h  to be included in conf.h, conf_lex.l, conf_yacc.l

Best regards,

Michael