Milter-greylist

The site http://mxtoolbox.com/ claims that my inbound mail is 
particularly slow.  The specific test page is at 
http://mxtoolbox.com/domain/www.simplesystems.org/.  The claim is that 
it takes 8.5 seconds to deliver a mail to my domain and results are 
consistent.

Yesterday I switched from early 1994 SPARC hardware (still with lots 
of memory so almost everything is in RAM) to modern 64-bit Intel 
hardware with fast SSDs and a ton of RAM.  The test page for this site 
says that the timing between the new and old hardware is almost 
identical.

I am using milter-greylist as the Milter.  Unless the test page is 
faulty, the only part of the chain which could take any time is 
milter-greylist since it does DNS queries.  However, I am not seeing 
any indicated slowness in the logs.  Only every once in many emails do 
I see a long SPF lookup (have seen 15 seconds before).

Is this test page faulty or is there something in my mail delivery 
chain (e.g. milter-greylist) which is adding time?

Bob
-- 
Bob Friesenhahn
bfriesen@..., http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/

8 \u0444\u0435\u0432\u0440\u0430\u043b\u044f 2016�\u0433. 5:23:51 CET, "Bob Friesenhahn bfriesen@... [milter-greylist]" <milter-greylist@yahoogroups.com> \u043f\u0438\u0448\u0435\u0442:
>The site http://mxtoolbox.com/ claims that my inbound mail is 
>particularly slow.  The specific test page is at 
>http://mxtoolbox.com/domain/www.simplesystems.org/.  The claim is that 
>it takes 8.5 seconds to deliver a mail to my domain and results are 
>consistent.
>
>Yesterday I switched from early 1994 SPARC hardware (still with lots 
>of memory so almost everything is in RAM) to modern 64-bit Intel 
>hardware with fast SSDs and a ton of RAM.  The test page for this site 
>says that the timing between the new and old hardware is almost 
>identical.
>
>I am using milter-greylist as the Milter.  Unless the test page is 
>faulty, the only part of the chain which could take any time is 
>milter-greylist since it does DNS queries.  However, I am not seeing 
>any indicated slowness in the logs.  Only every once in many emails do 
>I see a long SPF lookup (have seen 15 seconds before).
>
>Is this test page faulty or is there something in my mail delivery 
>chain (e.g. milter-greylist) which is adding time?
>
>Bob

Hi Bob, do you by chance have stuff like GreetingDelay or somesuch in sendmail and/or some of the milters, maybe newly introduced in yiur remade setup?

This is one way to add consistent delays - waiting for some time and then emitting an SMTP server banner and beginning to process incoming bits at this point. This approach helps rule out spambots which connect and spew their monologue rightnaway.

An effectively similar option might be tarpitting (may be even in milter-greylist), especially against yet-unacquainted hosts.

And DNS can lag too, yes :-)

On another hand, why does a consistent and relatively short delay bother you? Humans do not normally connect (on 25/tcp unauthenticated) to your smtp so likely won't notice the lag? And on 587/tcp you can require auth and do away with most milters and other protections.

Anyhow, with greylisting you incur a larger delay on transfer of messages (at least first ones before autowhiting), which is what gets some people annoyed.

Jim
--
Typos courtesy of K-9 Mail on my Samsung Android

On Sun, Feb 07, 2016 at 10:23:51PM -0600, Bob Friesenhahn bfriesen@... [milter-greylist] wrote:
> The site http://mxtoolbox.com/ claims that my inbound mail is 
> particularly slow.  The specific test page is at 
> http://mxtoolbox.com/domain/www.simplesystems.org/.  The claim is that 
> it takes 8.5 seconds to deliver a mail to my domain and results are 
> consistent.
[..]
> 
> Is this test page faulty or is there something in my mail delivery 
> chain (e.g. milter-greylist) which is adding time?

Maybe this is the cause:
Sendmail has usually IDENT queries enabled which timeouts in 10 or 15 secs
(if the TCP connection back to clients port 113 does not lead to an
immediate connection refused). 

I have following line included in my .mc config:

define(`confTO_IDENT',`0s')dnl

which prevents your server to do any IDENT queries.


Have a nice day,

Johann

On Mon, 8 Feb 2016, Jim Klimov jimklimov@... [milter-greylist] wrote:
>
> Hi Bob, do you by chance have stuff like GreetingDelay or somesuch 
> in sendmail and/or some of the milters, maybe newly introduced in 
> yiur remade setup?

I am not finding any delay statements.  Manual telnet to port 25 
produces an immediate response from a local host.

> And DNS can lag too, yes :-)

It is likely that sendmail does reverse DNS for each connection.  If 
reverse DNS times out, that would likely produce a large delay.

> On another hand, why does a consistent and relatively short delay 
> bother you? Humans do not normally connect (on 25/tcp 
> unauthenticated) to your smtp so likely won't notice the lag? And on 
> 587/tcp you can require auth and do away with most milters and other 
> protections.

I am not sure if I should be concerned about delays.  There is already 
port 587/tcp support in my setup.

Bob
-- 
Bob Friesenhahn
bfriesen@..., http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/

On Mon, 8 Feb 2016, Johann Klasek johann@... [milter-greylist] wrote:

> On Sun, Feb 07, 2016 at 10:23:51PM -0600, Bob Friesenhahn bfriesen@... [milter-greylist] wrote:
>> The site http://mxtoolbox.com/ claims that my inbound mail is
>> particularly slow.  The specific test page is at
>> http://mxtoolbox.com/domain/www.simplesystems.org/.  The claim is that
>> it takes 8.5 seconds to deliver a mail to my domain and results are
>> consistent.
> [..]
>>
>> Is this test page faulty or is there something in my mail delivery
>> chain (e.g. milter-greylist) which is adding time?
>
> Maybe this is the cause:
> Sendmail has usually IDENT queries enabled which timeouts in 10 or 15 secs
> (if the TCP connection back to clients port 113 does not lead to an
> immediate connection refused).
>
> I have following line included in my .mc config:
>
> define(`confTO_IDENT',`0s')dnl
>
> which prevents your server to do any IDENT queries.

Thanks for the heads-up.  Hardly anyone runs IDENT servers any more.
However, local telnet produces a quick response.

I have a local caching DNS BIND server here which should make 
short-work of any repeated queries.

Bob
-- 
Bob Friesenhahn
bfriesen@..., http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/

On Mon, Feb 08, 2016 at 08:30:06AM -0600, Bob Friesenhahn bfriesen@... [milter-greylist] wrote:
> On Mon, 8 Feb 2016, Johann Klasek johann@... [milter-greylist] wrote:
> 
> > On Sun, Feb 07, 2016 at 10:23:51PM -0600, Bob Friesenhahn bfriesen@... [milter-greylist] wrote:
> >> The site http://mxtoolbox.com/ claims that my inbound mail is
> >> particularly slow.  The specific test page is at
> >> http://mxtoolbox.com/domain/www.simplesystems.org/.  The claim is that
> >> it takes 8.5 seconds to deliver a mail to my domain and results are
> >> consistent.
> > [..]
> >>
> >> Is this test page faulty or is there something in my mail delivery
> >> chain (e.g. milter-greylist) which is adding time?
> >
> > Maybe this is the cause:
> > Sendmail has usually IDENT queries enabled which timeouts in 10 or 15 secs
> > (if the TCP connection back to clients port 113 does not lead to an
> > immediate connection refused).
> >
> > I have following line included in my .mc config:
> >
> > define(`confTO_IDENT',`0s')dnl
> >
> > which prevents your server to do any IDENT queries.
> 
> Thanks for the heads-up.  Hardly anyone runs IDENT servers any more.

That's not the point, even if no one runs this service, if your sendmail
does a lookup normaly a firewall on client-side (!) drops incoming
113/TCP and your sendmail has to wait the timeout interval ...

> However, local telnet produces a quick response.

This would ask local port 113 which is likly refusing connection
quick ...

I did a test from my site to yours, tracing the traffic via port 113 and
what we can see is this:

16:33:19.087663 IP (tos 0x0, ttl  54, id 2478, offset 0, flags [DF],
length: 60) smtp.simplesystems.org.62068 > x.x.x.x.ident: S [tcp sum ok] 1786079858:1786079858(0) win 64240 <mss 1460,sackOK,timestamp 420629205 0,nop,wscale 1>

So, your server explicitly *does* IDENT queries. Simply turn them off. ;)


> I have a local caching DNS BIND server here which should make 
> short-work of any repeated queries.

These are IDENT (RFC1413) queries, which are not related to DNS ...
(maybe I did't get the point of your statement ;) )



Johann

On Mon, 8 Feb 2016, Johann Klasek johann@... [milter-greylist] wrote:
>
> This would ask local port 113 which is likly refusing connection
> quick ...
>
> I did a test from my site to yours, tracing the traffic via port 113 and
> what we can see is this:
>
> 16:33:19.087663 IP (tos 0x0, ttl  54, id 2478, offset 0, flags [DF],
> length: 60) smtp.simplesystems.org.62068 > x.x.x.x.ident: S [tcp sum ok] 1786079858:1786079858(0) win 64240 <mss 1460,sackOK,timestamp 420629205 0,nop,wscale 1>
>
> So, your server explicitly *does* IDENT queries. Simply turn them off. ;)

Done now.  The response time issue is gone.  Thank you very much!

>> I have a local caching DNS BIND server here which should make
>> short-work of any repeated queries.
>
> These are IDENT (RFC1413) queries, which are not related to DNS ...
> (maybe I did't get the point of your statement ;) )

This was in response to Jim Klimov's comments.

Bob
-- 
Bob Friesenhahn
bfriesen@..., http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/