Milter-greylist

Hello!

I am trying to implement milter-greylist at a quite high mail volume site. In peak we have about 12 mails a second and a total of about 500.000 mails / day.

The greylist daemon seem to last for about 10 minutes or so before the following errors pop up on the log..

Aug 31 16:24:38 7800005 sendmail[14741]: [ID 801593 mail.error] l7VEOS0D014741: Milter (greylist): timeout before data read, where=negotiate
Aug 31 16:24:38 7800005 sendmail[14741]: [ID 801593 mail.info] l7VEOS0D014741: Milter (greylist): to error state
Aug 31 16:24:38 7800005 sendmail[14741]: [ID 801593 mail.error] l7VEOS0D014741: Milter (greylist): init failed to open
Aug 31 16:24:38 7800005 sendmail[14741]: [ID 801593 mail.info] l7VEOS0D014741: Milter (greylist): to error state

Does this mean the milter daemon simply cant cope ?

In busy hours we have about 400 sendmail children running.

The Machine is a Quad Cpu AMD-64 (extra all) with 4 GB memory running Solars 10, and yes I have compiled 64 bit binarys. 

Is there anyone out there with experiences using milter-greylist in a  site with similar volumes ? 

Tia.

// Jonas Israelsson


-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.

Jonas Israelsson wrote:
> Hello!
>
> I am trying to implement milter-greylist at a quite high mail volume site. In peak we have about 12 mails a second and a total of about 500.000 mails / day.
>
> The greylist daemon seem to last for about 10 minutes or so before the following errors pop up on the log..
>
> Aug 31 16:24:38 7800005 sendmail[14741]: [ID 801593 mail.error] l7VEOS0D014741: Milter (greylist): timeout before data read, where=negotiate
> Aug 31 16:24:38 7800005 sendmail[14741]: [ID 801593 mail.info] l7VEOS0D014741: Milter (greylist): to error state
> Aug 31 16:24:38 7800005 sendmail[14741]: [ID 801593 mail.error] l7VEOS0D014741: Milter (greylist): init failed to open
> Aug 31 16:24:38 7800005 sendmail[14741]: [ID 801593 mail.info] l7VEOS0D014741: Milter (greylist): to error state
>
> Does this mean the milter daemon simply cant cope ?
>
> In busy hours we have about 400 sendmail children running.
>
> The Machine is a Quad Cpu AMD-64 (extra all) with 4 GB memory running Solars 10, and yes I have compiled 64 bit binarys. 
>
> Is there anyone out there with experiences using milter-greylist in a  site with similar volumes ? 
>   

Just a comment on the 400 sendmail children running . . .

We were seeing increasing numbers of sendmail children, until the limit
was hit and mail could not be accepted. The log files showed repeated
entries where it stopped accepting connections and then resumed. But the
system load shown by uptime was low, and top didn't show sendmail,
mimedefang or any related stuff eating the cpu. Finally, netstat showed
that the sendmails had established connections and weren't doing
anything. It seems the bots are not closing the connection now when they
are rejected. The default in sendmail.cf is to wait an hour before
closing the connection, which seems archaic in this day and age. So, we
started diddling parameters in sendmail. Took the number of children way
down and greatly increased performance quality for our users. I believe
the most pertinent parameter for that particular issue was
Timeout.command. We took our number of children from 200 down to 20-50
almost instantly, by changing that from 1hr to 5m. Your volume is higher
than ours, so numbers will vary.



---------------

Chris Hoogendyk

-
   O__  ---- Systems Administrator
  c/ /'_ --- Biology & Geology Departments
 (*) \(*) -- 140 Morrill Science Center
~~~~~~~~~~ - University of Massachusetts, Amherst 

<hoogendyk@...>

--------------- 

Erd\ufffds 4

On Wed, 5 Sep 2007, Jonas Israelsson wrote:

> Date: 5 Sep 2007 19:08:00 +0200
> From: Jonas Israelsson <jonas@...>
> Reply-To: milter-greylist@yahoogroups.com
> To: milter-greylist@yahoogroups.com
> Subject: [milter-greylist] Milter Greylist on a rather busy site
> 
> Hello!
>
> I am trying to implement milter-greylist at a quite high mail volume site. In peak we have about 12 mails a second and a total of about 500.000 mails / day.
>
> The greylist daemon seem to last for about 10 minutes or so before the following errors pop up on the log..
>
> Aug 31 16:24:38 7800005 sendmail[14741]: [ID 801593 mail.error] l7VEOS0D014741: Milter (greylist): timeout before data read, where=negotiate
> Aug 31 16:24:38 7800005 sendmail[14741]: [ID 801593 mail.info] l7VEOS0D014741: Milter (greylist): to error state
> Aug 31 16:24:38 7800005 sendmail[14741]: [ID 801593 mail.error] l7VEOS0D014741: Milter (greylist): init failed to open
> Aug 31 16:24:38 7800005 sendmail[14741]: [ID 801593 mail.info] l7VEOS0D014741: Milter (greylist): to error state
>
> Does this mean the milter daemon simply cant cope ?
>
> In busy hours we have about 400 sendmail children running.
>
> The Machine is a Quad Cpu AMD-64 (extra all) with 4 GB memory running Solars 10, and yes I have compiled 64 bit binarys.
>
> Is there anyone out there with experiences using milter-greylist in a  site with similar volumes ?

What version of milter-greylist?  Which compiler did you use?  What is the
syslogging from milter-greylist itself look like?

Jeff Earickson
Colby College

I can answer only two.. 


>What version of milter-greylist?  

Latest stable ...3.0 ? Downloaded last week.

Which compiler did you use? 

Suns Compiler (Sun Studio 11)

 What is the
>syslogging from milter-greylist itself look like?

Good question, I have not managed to get milter-greylist to log anything (I think) , this is on my todo, to read and enable/set more logging.

Brgds Jonas




::::::::::::::::::::::::::::::::
Jonas Israelsson

Mail: jonas@...
................................
WEHAY AB
Mailbox 366
Regeringsgatan 88
111 73 Stockholm

http://www.wehay.com

::::::::::::::::::::::::::::::::  


-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.

Forgot, are there any stress tool written for milter-greylist ? If not, think socat could be used todo that ? 


-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.

Jonas Israelsson wrote:
> Hello!
>
> I am trying to implement milter-greylist at a quite high mail volume site. In peak we have about 12 mails a second and a total of about 500.000 mails / day.
>
> The greylist daemon seem to last for about 10 minutes or so before the following errors pop up on the log..
>
> Aug 31 16:24:38 7800005 sendmail[14741]: [ID 801593 mail.error] l7VEOS0D014741: Milter (greylist): timeout before data read, where=negotiate
> Aug 31 16:24:38 7800005 sendmail[14741]: [ID 801593 mail.info] l7VEOS0D014741: Milter (greylist): to error state
> Aug 31 16:24:38 7800005 sendmail[14741]: [ID 801593 mail.error] l7VEOS0D014741: Milter (greylist): init failed to open
> Aug 31 16:24:38 7800005 sendmail[14741]: [ID 801593 mail.info] l7VEOS0D014741: Milter (greylist): to error state
>
> Does this mean the milter daemon simply cant cope ?
>
> In busy hours we have about 400 sendmail children running.
>
> The Machine is a Quad Cpu AMD-64 (extra all) with 4 GB memory running Solars 10, and yes I have compiled 64 bit binarys. 
>
> Is there anyone out there with experiences using milter-greylist in a  site with similar volumes ? 
>
> Tia.
>
> // Jonas Israelsson
>
>
>   

We're not as busy as you, we only see about 300 sendmail processes 
running at a time. I did run into issues with it dumping out to file way 
too often. I increased that time and have not seen any issues since.

Jonas Israelsson <jonas@...> wrote:

> The greylist daemon seem to last for about 10 minutes or so before the
> following errors pop up on the log..
(snip)
> Aug 31 16:24:38 7800005 sendmail[14741]: [ID 801593 mail.info]
> l7VEOS0D014741: Milter (greylist): to error state

That's a generic error that sendmail fires when milter-greylist stops.
Check the logs to see if you have a message from milter-greylist telling
why it exited. If you have not, it means it crashed.

On Solaris the biggest problem is the file descriptor limit, but since
you used a 64 bit binary, that should not happen.

The next usual culprit is memory limit. Check ulimit -a output and
monitor milter-greylist memory usage. If it stops when reaching the
limit, then you need a higher memory limit.

Given the setup you have, milter-greylist should eat a lot of memory.
Are you applying greylisting to all your users?

-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu@...

Hello!

I'm working at the University of Örebro and we are running milter greylist and we have been so for a couple of years.
Our site is not so busy as yours but we have experiensed a problem this summer resulting in zombie sendmail demons.

A rather big amount of connections from around the world just hanging and dont do anything.
Default configuered sendmail leave them there for one houre before diconnecting them. This resulting in Milter Greylist crashing when sendmail try to open more
then 255 connections to the socket for Milter Greylist. 255 is the max amount of open files in solaris 2.8.

When we realised this we tuned Sendmail to disconnect zombies after 2 minutes. (RFC says min is 5 min) Resulting in a much more stabel mail service.
At the peek we did have 32000 zombies disconnected in one houre.

We have also turned of the dumping of the Greylist.db because Milter Greylist did miss mails when it was writing the db.

/Tomas Liljebergh

----- Original Message -----

<tomas.liljebergh@...> wrote:

> <P><B>Hello!</B></P>

Please post plain text.

-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu@...

>(snip)
>> Aug 31 16:24:38 7800005 sendmail[14741]: [ID 801593 mail.info]
>> l7VEOS0D014741: Milter (greylist): to error state
>
>That's a generic error that sendmail fires when milter-greylist stops.
>Check the logs to see if you have a message from milter-greylist telling
>why it exited. If you have not, it means it crashed.

When this happens, what happens to the mail-delivery ? Is the milter-plugin just disabled and mail dilivered as usual, or will delivery be rejected ? Sorry for asking this but I have not yet had time to dig into this.

TIA 

Rgds Jonas
   


-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.

Jonas Israelsson <jonas@...> wrote:

> When this happens, what happens to the mail-delivery ? Is the
> milter-plugin just disabled and mail dilivered as usual, or will
> delivery be rejected ? Sorry for asking this but I have not yet had time
> to dig into this.

It depends how you configured the milter in sendmail.cf. 

-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu@...

manu@... (09/09/2007 06:04):
>Jonas Israelsson <jonas@...> wrote:
>
>> When this happens, what happens to the mail-delivery ? Is the
>> milter-plugin just disabled and mail dilivered as usual, or will
>> delivery be rejected ? Sorry for asking this but I have not yet had time
>> to dig into this.
>
>It depends how you configured the milter in sendmail.cf. 

Well I haven't yet really configured anything, I just used the M4-file that comes with the source. I believe however you are referring to F=T, right ?

Xgreylist, S=local:/usr/local/milter-greylist/var/milter-greylist.sock, F=T

Brgds Jonas


-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.

Jonas Israelsson wrote:
>> (snip)
>>     
>>> Aug 31 16:24:38 7800005 sendmail[14741]: [ID 801593 mail.info]
>>> l7VEOS0D014741: Milter (greylist): to error state
>>>       
>> That's a generic error that sendmail fires when milter-greylist stops.
>> Check the logs to see if you have a message from milter-greylist telling
>> why it exited. If you have not, it means it crashed.
>>     
>
> When this happens, what happens to the mail-delivery ? Is the milter-plugin just disabled and mail dilivered as usual, or will delivery be rejected ? Sorry for asking this but I have not yet had time to dig into this.
>   


on our system, sendmail continues processing "normally", but more stuff
slips through because milter-greylist is not running. We have a regular
cron job that checks to see if milter-greylist is running and restarts
it if not. Since I've just implemented mon (from kernel.org), I'm
considering redoing that cron job as a mon monitor with an action script.


---------------

Chris Hoogendyk

-
   O__  ---- Systems Administrator
  c/ /'_ --- Biology & Geology Departments
 (*) \(*) -- 140 Morrill Science Center
~~~~~~~~~~ - University of Massachusetts, Amherst 

<hoogendyk@...>

--------------- 

Erd\ufffds 4

>Just a comment on the 400 sendmail children running . . .
>
>We were seeing increasing numbers of sendmail children, until the limit
>was hit and mail could not be accepted. The log files showed repeated
>entries where it stopped accepting connections and then resumed. But the
>system load shown by uptime was low, and top didn't show sendmail,
>mimedefang or any related stuff eating the cpu. Finally, netstat showed
>that the sendmails had established connections and weren't doing
>anything. It seems the bots are not closing the connection now when they
>are rejected. The default in sendmail.cf is to wait an hour before
>closing the connection, which seems archaic in this day and age. So, we
>started diddling parameters in sendmail. Took the number of children way
>down and greatly increased performance quality for our users. I believe
>the most pertinent parameter for that particular issue was
>Timeout.command. We took our number of children from 200 down to 20-50
>almost instantly, by changing that from 1hr to 5m. Your volume is higher
>than ours, so numbers will vary.

Thanks Chris, I'll have a look at the timeout options. 

Rgds Jonas


-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.

Jonas Israelsson <jonas@...> wrote:

[behavior on failure]
> I believe however you are referring to F=T, right ?
> Xgreylist, S=local:/usr/local/milter-greylist/var/milter-greylist.sock, F=T

Yes, that's it.

-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu@...