Yahoo Groups archive

Milter-greylist

Index last updated: 2026-04-28 23:32 UTC

Message

Re: [milter-greylist] memory consumption

2008-10-23 by Patrick Domack

I use milter-greylist on my personal email, and it's been great, and  
the 2.x code, and the current development code both don't leak memory  
on my system, holds steady around 12megs.

Now another system I admin, we have used several different database  
methods, mainly cause I knew I was going have a memory issue with  
using milter-greylist, and the systems when I got them hardly had  
enough memory to run as they where.

I used gpsd, some other g thing, and now policyd. The biggest issue I  
have with them, is anytime the database is slow, locked, or otherwise  
busy, everything comes to a dead halt, as it waits for a response from  
the database, and this causes more and more threads to build up. It  
just gets to be a issue back and forth. I finally have managed to keep  
the greylisting table smaller than 6gigs now, using 1.4million  
blacklisted ip's based on the failure of passing greylisting tests,  
and this keeps it around 1gig in size, and fits in memory good, so  
speed is better.

Now I suppose all these issues could be overcome, using an intergrated  
db, and making it so unlimited number of threads can read from it at  
will, and do limited updating. This could be achieved, but it would  
take a great amount of time, design, ...

And in the end, I believe, the more widely used, and better  
greylisting gets, the more likely it is for the spammers to work  
around it. It shouldn't be hard for them to add a retry logic into the  
virus's, or make more compliant smtp servers.

I'm just a milter-greylist user, and love it. But everything will have  
design limitations.

Quoting Mark Walker <furface@...m>:

> Is the decision to keep data in memory as opposed to some sort of
> database a speed issue?  It seems that there would be a lot of benefit
> to keeping the active data in an sql database accessible from other apps.
>
>
> Oliver Fromme wrote:
>>
>>
>> Petar Bogdanovic wrote:
>> > I just did some stress-testing on milter-greylist in order to test its
>> > memory consumption and it seems that there are some leaks to be found.
>>
>> milter-greylist keeps all data (all triples) in memory.
>> So it is expected that it will grow if new triples come
>> in faster than old triples are expired, which was certainly
>> the case in your 30-minutes test when the timeout was set
>> to 1d.
>>
>> Best regards
>> Oliver
>>
>> --
>> Oliver Fromme, secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing b. M.
>> Handelsregister: Registergericht Muenchen, HRA 74606, Geschäftsfuehrung:
>> secnetix Verwaltungsgesellsch. mbH, Handelsregister: Registergericht Mün-
>> chen, HRB 125758, Geschäftsführer: Maik Bachmann, Olaf Erb, Ralf Gebhart
>>
>> FreeBSD-Dienstleistungen, -Produkte und mehr:
>> http://www.secnetix.de/bsd <http://www.secnetix.de/bsd>
>>
>> "To this day, many C programmers believe that 'strong typing'
>> just means pounding extra hard on the keyboard."
>> -- Peter van der Linden
>>
>>
>
>
> ------------------------------------
>
> Yahoo! Groups Links
>
>
>
>

Attachments

Move to quarantaine

This moves the raw source file on disk only. The archive index is not changed automatically, so you still need to run a manual refresh afterward.