Milter-greylist

Milter-greylist is working:
Oct 17 09:38:32 mailscanner6 milter-greylist: k9HGcLRM031656: addr 
wmflb12na02.ezweb.ne.jp[222.15.69.197] from <> to 
<xvafeyaoef@...> delayed for 00:04:14 (ACL 111)

Then all of a sudden:
Oct 17 09:38:42 mailscanner6 sendmail[31656]: k9HGcLRM031656: Milter 
(greylist): timeout before data read
Oct 17 09:38:42 mailscanner6 sendmail[31624]: k9HGcL2F031624: Milter 
(greylist): timeout before data read
Oct 17 09:38:44 mailscanner6 sendmail[31679]: k9HGcOwh031679: Milter 
(greylist): timeout before data read
Oct 17 09:38:44 mailscanner6 sendmail[31679]: k9HGcOwh031679: Milter 
(greylist): to error state
Oct 17 09:38:42 mailscanner6 sendmail[31653]: k9HGcMfq031653: Milter 
(greylist): timeout before data read
Oct 17 09:38:43 mailscanner6 sendmail[31647]: k9HGcLYm031647: Milter 
(greylist): timeout before data read
Oct 17 09:38:43 mailscanner6 sendmail[31660]: k9HGcMZq031660: Milter 
(greylist): timeout before data read
Oct 17 09:38:46 mailscanner6 sendmail[31663]: k9HGcNI5031663: Milter 
(greylist): to error state
Oct 17 09:38:46 mailscanner6 sendmail[31684]: k9HGcOle031684: Milter 
(greylist): timeout before data read
Oct 17 09:38:46 mailscanner6 sendmail[31708]: k9HGcP3Y031708: Milter 
(greylist): timeout before data read


[root@mailscanner6 log]# free
             total       used       free     shared    buffers     
cached
Mem:        499388     410436      88952          0      26160      
58844
-/+ buffers/cache:     325432     173956
Swap:      2097136       9720    2087416


If I issue a : service miltergreylist restart
It starts working again for a few minutes, then bombs again.

Not using LIBSPF, results happen on both Fedora Core 6 Test 3 and 
Fedora Core 2 servers
Fedora Core 6 Test 3 running 8.13.7 Sendmail with Milter-Greylist 
3.0rc5 with dnsrbl enabled

In sendmail.mc I have
FEATURE(`milter-greylist')
INPUT_MAIL_FILTER(`greylist',`S=local:/var/milter-greylist/milter-
greylist.sock')
define(`confMILTER_MACROS_CONNECT', `j, {if_addr}')
define(`confMILTER_MACROS_HELO', `{verify}, {cert_subject}')
define(`confMILTER_MACROS_ENVFROM', `i, {auth_authen}')
define(`confMILTER_MACROS_ENVRCPT', `{greylist}')

In greylist.conf I have
acl whitelist list "my network"
acl whitelist list "broken mta"
#acl greylist list "grey users" dnsrbl "SORBS DUN" delay 24h 
autowhite 3d
acl greylist list "grey users" delay 10m autowhite 3d
dnsrbl "SORBS DUN" dnsbl.sorbs.net 127.0.0.10
dnsrbl "SPAMCOP BL" bl.spamcop.net 127.0.0.2
acl greylist dnsrbl "SORBS DUN" delay 12h
acl greylist dnsrbl "SPAMCOP BL" delay 12h
acl greylist default delay 5m autowhite 3d


PEER's temporarily disabled because of 1024 queue error.


Anything I should try?