Milter-greylist

We've noticed that over time (and in our configuration), the memory
consumption of milter-greylist has grown to over 400 MB per server instance.
It would seem from a quick code scan, that this is mainly the result of the
sizeof(struct rcpt) (defined in milter-greylist.h).

struct rcpt {
        char r_addr[ADDRLEN + 1];
        LIST_ENTRY(rcpt) r_list;
};

The ADDRLEN macro is defined in dump.h and based on RFC-2821, it is set to a
value of 324.   Our database is 2.2 million entries, which is why I believe
we're consuming in the area of 400 MB.

I would like to reduce the memory footprint of milter-greylist.

In the long run, perhaps the best approach is to refactor the code to use a
char* for r_addr, and malloc() new entries based on the actual length of the
address.

But as a quick hack, I'm considering just dropping ADDRLEN to 100. 

Does anyone know whether this would break anything too severely?

Thanks
Jim C.

Jim Carroll <jim@...> wrote:

> In the long run, perhaps the best approach is to refactor the code to use a
> char* for r_addr, and malloc() new entries based on the actual length of the
> address.

Yes, that should improve a lot the situation. 

Another trick would be to use the fact that the recipients are often the
same for the same server. We could have a dynamically allocated table of
recipients seen so far, and we could have pointers to the list,
therefore sharing memory for each message going to the same recipient.

Splitting recipient name and domain would help too, as the domains are
even more scarce for a given server.
 
> But as a quick hack, I'm considering just dropping ADDRLEN to 100. 
> Does anyone know whether this would break anything too severely?

You may burn yourself hot with unique addresses generated by some
software, otherwise it looks okay as an interm hack.

-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu@...

Jim Carroll a \ufffdcrit :
> We've noticed that over time (and in our configuration), the memory
> consumption of milter-greylist has grown to over 400 MB per server instance.
> [...]
> In the long run, perhaps the best approach is to refactor the code to use a
> char* for r_addr, and malloc() new entries based on the actual length of the
> address.

Our milter-greylist currently eats up to 2GB. This is not a problem for 
now since our server has plenty of RAM (16GB), but since we compiled it 
in 32-bit, it may some day hit the 4GB-limit of per-process address 
space. So optimization would not hurt...

By the way, our database holds only 35000 entries; could there be some 
memory leak ? hmmm..


-- 
Ce message a ete verifie par MailScanner
pour des virus ou des polluriels et rien de
suspect n'a ete trouve.

On Dec 13, 2007 5:31 PM, Benoit Branciard
<benoit.branciard@...> wrote:
>  Our milter-greylist currently eats up to 2GB. This is not a problem for
>  now since our server has plenty of RAM (16GB), but since we compiled it
>  in 32-bit, it may some day hit the 4GB-limit of per-process address
>  space. So optimization would not hurt...
>
>  By the way, our database holds only 35000 entries; could there be some
>  memory leak ? hmmm..

My db holds almost 200.000 entries and uses around 600 MB.

-- 
/peter

shuttlebox wrote:
> On Dec 13, 2007 5:31 PM, Benoit Branciard
> <benoit.branciard@...> wrote:
>>  Our milter-greylist currently eats up to 2GB. This is not a problem for
>>  now since our server has plenty of RAM (16GB), but since we compiled it
>>  in 32-bit, it may some day hit the 4GB-limit of per-process address
>>  space. So optimization would not hurt...
>>
>>  By the way, our database holds only 35000 entries; could there be some
>>  memory leak ? hmmm..
> 
> My db holds almost 200.000 entries and uses around 600 MB.

400,000 588MB

I too suspect a leak, as ours grows over time and dramatically shrinks 
on a restart.

Phil

Phil Randal a \ufffdcrit :
> shuttlebox wrote:
>> On Dec 13, 2007 5:31 PM, Benoit Branciard
>> <benoit.branciard@...> wrote:
>>>  Our milter-greylist currently eats up to 2GB. This is not a problem for [...]
>>>
>>>  By the way, our database holds only 35000 entries; could there be some
>>>  memory leak ? hmmm..
>> My db holds almost 200.000 entries and uses around 600 MB.
> 
> 400,000 588MB
> 
> I too suspect a leak, as ours grows over time and dramatically shrinks 
> on a restart.
> 

Restart: 2GB -> 100MB (still ~36000 entries)
It was running for about one month.

We are running a 2007/11/08 CVS snapshot (v4.0 plus SPF status report), 
with SPF, DNSRBL, URLCHECK and IPv6 enabled.


-- 
Ce message a ete verifie par MailScanner
pour des virus ou des polluriels et rien de
suspect n'a ete trouve.

Hello,

> By the way, our database holds only 35000 entries; could there be some
> memory leak ? hmmm..

It could, I found some for our environment with an unknown origin outside
the reach of milter-greylist or libmilter, deep down at thread creation or
libbind.
We monitor milter-greylist usage, which is a good thing anyway. If it grows
beyond a threshold of real memory usage, we stop it, and reduce greylist.db by
- all entries with no valid recipient (LDAP knowledge, address form)
- all greylist entries of sending IPs with too many recipients
which eliminates 80%, and start milter-greylist afterwards.
This cycle takes ~ 5 seconds and hurts nobody, sendmail defaults to temporary rejection.

M. Fromme

Michael Fromme a \ufffdcrit :
> Hello,
> 
>> By the way, our database holds only 35000 entries; could there be some
>> memory leak ? hmmm..
> 
> It could, I found some for our environment with an unknown origin outside
> the reach of milter-greylist or libmilter, deep down at thread creation or
> libbind.

It would be great if some people running milter-greylist publish on this 
list the usual runtime values for their setups : number of entries in 
db, memory footprint (real and virtual if applicable), together with the 
milter-greylist version, system architecture, compiled-in options, and 
whether they observe memory footprint growth over time or not. This 
would help tracking the problem...


-- 
Ce message a ete verifie par MailScanner
pour des virus ou des polluriels et rien de
suspect n'a ete trouve.

Benoit Branciard <benoit.branciard@...> wrote:

> It would be great if some people running milter-greylist publish on this
> list the usual runtime values for their setups : number of entries in
> db, memory footprint (real and virtual if applicable), together with the
> milter-greylist version, system architecture, compiled-in options, and
> whether they observe memory footprint growth over time or not. This 
> would help tracking the problem...

Sure, what about setting up a wiki?

-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu@...

Michael Fromme a \ufffdcrit :
> Hello,
> 
>> By the way, our database holds only 35000 entries; could there be some
>> memory leak ? hmmm..
> 
> It could, I found some for our environment with an unknown origin outside
> the reach of milter-greylist or libmilter, deep down at thread creation or
> libbind.

Some news:
we used to compile our milter-greylist against libbind9. This was not 
needed, so last week I recompiled it without libbind, and restarted.

Stability was not affected, but memory usage still grows over time: from 
100MB/30MB (virtual/real) on startup to 380MB/200MB three days later.

So libbind is not the cause of leak in our case.

-- 
Ce message a ete verifie par MailScanner
pour des virus ou des polluriels et rien de
suspect n'a ete trouve.