Milter-greylist

Hello,

It was discussed last October, starting October 17.

I also get messages like this:

  >> Oct 17 09:38:42 mailscanner6 sendmail[31656]: k9HGcLRM031656: Milter (greylist): timeout before data read
  >> Oct 17 09:38:42 mailscanner6 sendmail[31624]: k9HGcL2F031624: Milter (greylist): timeout before data read

But my problem is related to the size of my tables.

My OS: FreeBSD 4.8, 1 Gb memory, SCSI disks.
My mail-server gets 700.000 hits per day, that is 10 connections per second.
With a limited set of rules, I now have:
	-rw-------  1 root  wheel  128422056 Nov 20 14:35 /var/milter-greylist/greylist.db
	root   57002 11.2 23.9 249312 248876  ??  Ss   Tue10PM 381:08.14 /usr/sbin/milter-greylist...
/var/milter-greylist/greylist.db has now 1.263.835 lines.

The message 'timeout before data read' occurs every 30m (= dumpfreq),
for 14 seconds,
when milter-greylist writes /var/milter-greylist/greylist.db.

If all my email address get protection with milter-greylist,
the file will be 300 Mb, the process will be 600 Mb,
the delay to write the file will be 40 seconds,
and within 40s, 10 sendmail/second give 400 sendmail process
waiting for an extended timeout:
	Xgreylist, S=local:/var/milter-greylist/milter-greylist.sock, T=C:1m;S:30s;R:2m;E:2m
                                                                      ^^^^^^^^^^^^^^^^^^^^^^
Splitting in 2 MX doesn't change the delay for writing greylist.db.

How can I solve this problem?

Some ideas:
. splitting /var/milter-greylist/greylist.db in several files,
  for instance 10 times 30 Mb. A file will be every 30mn,
  and milter-greylist writes a file every 3 mn.
  ( 3 mn = dumpfreq / #files )
. splitting milter-greylist in several process, may be
  on several computers. NB: this approach also solves
  the problem of CPU and memory: a set of MX talking
  to a set of milter-greylist daemons.

Thanks,

--
Jacques Beigbeder                    |  Jacques.Beigbeder@...
Service de Prestations Informatiques |     http://www.spi.ens.fr
Ecole normale sup\ufffdrieure             |
45 rue d'Ulm                         |Tel : (+33 1)1 44 32 37 96
F75230 Paris cedex 05                |Fax : (+33 1)1 44 32 20 75