Yahoo Groups archive

Milter-greylist

Index last updated: 2026-04-28 23:32 UTC

Message

Re: [milter-greylist] slow reloading of the database: the problem explained

2007-01-13 by AIDA Shinra

At Wed, 10 Jan 2007 08:58:10 +0000,
Emmanuel Dreyfus wrote:
> 
> On Wed, Jan 10, 2007 at 04:40:33PM +0900, AIDA Shinra wrote:
> > > Attached is a patch that remove the sorting of tuples. It restores the
> > > instant loading time we had before 3.1 series. Feedback is welcome: does
> > > it harm in any way to nuke that code?
> > 
> > The whole database need to be sorted in order for pending_timeout()
> > and autowhite_timeout() to work.
> > 
> > I think pending_insert_to_queue() and autowhite_insert_to_queue()
> > finish quickly as long as entries are inserted in old-to-new order. Am
> > I wrong? Profiler will clarify where is the problem.
> 
> That functions causes the reloading operation to switch from O(n) to O(n^2),
> since we have to walk all previously inserted tuples before inserting a new
> tuple. The cost is huge for a big databases.

No. These functions are O(n) as long as the dump is sorted in old to
new order because they search insertion position from latest to
earliest order.

I believe the real source of our problem lies in another
code. Profiling will reveal it.

Attachments

Move to quarantaine

This moves the raw source file on disk only. The archive index is not changed automatically, so you still need to run a manual refresh afterward.