Yahoo Groups archive

Milter-greylist

Index last updated: 2026-04-28 23:32 UTC

Message

Re: [milter-greylist] slow reloading of the database: the problem explained

2007-01-10 by AIDA Shinra

At Tue, 9 Jan 2007 21:32:51 +0000,
Emmanuel Dreyfus wrote:
> 
> Hello
> 
> I tracked down the problem of slow database reloading in 3.1.x series.
> 
> The problem was introduced with the pending_insert_to_queue() and
> autowhite_insert_to_queue() functions, that sort the in-memory database
> by date. For each tuple to be inserted, the whole database is searched, 
> which results in a major slow down when thousands of tuples are reloaded,
> at startup time.
>
> Are these functions useful at all? I don't see the point of sorting 
> the tuples, since the code in charge of retreiving tuples does not use
> their sorted property to win a shortcut.
> 
> Attached is a patch that remove the sorting of tuples. It restores the
> instant loading time we had before 3.1 series. Feedback is welcome: does
> it harm in any way to nuke that code?

The whole database need to be sorted in order for pending_timeout()
and autowhite_timeout() to work.

I think pending_insert_to_queue() and autowhite_insert_to_queue()
finish quickly as long as entries are inserted in old-to-new order. Am
I wrong? Profiler will clarify where is the problem.

Attachments

Move to quarantaine

This moves the raw source file on disk only. The archive index is not changed automatically, so you still need to run a manual refresh afterward.