At Wed, 10 Jan 2007 08:58:10 +0000, Emmanuel Dreyfus wrote: > > On Wed, Jan 10, 2007 at 04:40:33PM +0900, AIDA Shinra wrote: > > > Attached is a patch that remove the sorting of tuples. It restores the > > > instant loading time we had before 3.1 series. Feedback is welcome: does > > > it harm in any way to nuke that code? > > > > The whole database need to be sorted in order for pending_timeout() > > and autowhite_timeout() to work. > > > > I think pending_insert_to_queue() and autowhite_insert_to_queue() > > finish quickly as long as entries are inserted in old-to-new order. Am > > I wrong? Profiler will clarify where is the problem. > > That functions causes the reloading operation to switch from O(n) to O(n^2), > since we have to walk all previously inserted tuples before inserting a new > tuple. The cost is huge for a big databases. No. These functions are O(n) as long as the dump is sorted in old to new order because they search insertion position from latest to earliest order. I believe the real source of our problem lies in another code. Profiling will reveal it.
Message
Re: [milter-greylist] slow reloading of the database: the problem explained
2007-01-13 by AIDA Shinra
Attachments
- No local attachments were found for this message.