Milter-greylist

Hello Manu et al,

   Today I've caught a crash of milter-greylist, here are some details.

   The last entries in log were:

Aug 12 16:16:23 ucs milter-greylist: [ID 471652 mail.debug] Incoming 
connection from host 'tcp-daemon.imap.ucs.domain.ru'
Aug 12 16:16:23 ucs milter-greylist: [ID 447650 mail.error] unexpected AF

   The "host name" is a Messaging Server queue name with the trailing
name of the local MTA host. This string is not resolvable to an IP
address via DNS or local /etc/hosts (as of yet, at least).

   The "unexpected AF" leads me (in source) to 380:p0f.c function
p0f_lookup() - here it tries to interpret priv->priv_addr, gets to
default (not AF_INET nor AF_INET6) and falls out.

   I am not sure if these two clues are hard-linked, or if the queue
handler does not set some macros that are present during a live SMTP
dialog with a remote host. Still, I wonder if "localaddr" should come
into play as a fallback here as well?

   My workaround for now would be to disable p0f in config, and later
to disable milter-greylist for internal server processing - can't
find where it gets called here, and have more urgent quests now...

   Ideas welcome, though...
Thanks,
//Jim Klimov

Hello all,

   This p0f bug does need more attention, from myself at least, to
produce some workarounds. It hit my setup a few more times, with
such queue reprocessing "sources"; luckily, Solaris SMF put the
service back up, so there were only seconds of downtime. I have
a few questions about the milter-greylist side for you, though.

   The main question is: if the "local address" is wrong in p0f.c:380,
should we really abort the whole program, or just not do further p0f 
processing?

   Also, I did not find a (documented) toggle to disable p0f if it was
enabled during compilation. There are currently no configuration lines
with "p0f" in my config, but the codepath is taken and in certain cases
leads to program abortion...

   Second question... Here's the latest log snippet from the crash, as
seen in syslog:

Aug 13 10:07:05 ucs milter-greylist: [ID 471652 mail.debug] Incoming 
connection from host 'process-daemon.imap.ucs.domain.com'

Aug 13 10:07:05 ucs milter-greylist: [ID 447650 mail.error] unexpected AF

   Here's one more line I missed yesterday (similar was there) - I am
not sure what to make of it. Seems like some syslog messages were
buffered by the greylist, and the buffer was flushed during the crash
"as is", as one big chunk of text? Should anything be done about this
(i.e. in terms of security, memory leaks, etc.), or is it just the way
things are and I shouldn't worry nor bother?

Aug 13 10:07:05 ucs root: [ID 702911 mail.info] 2013/08/12 21:16:42 
host18-56-dynamic.8-87-r.retail.telecomitalia.it [87.8.56.18] 
tlnikitenko@... -> qqp@... tempfail (ACL 1474) 451 
4.7.1 SPF verification soft failure: sender host 
'host18-56-dynamic.8-87-r.retail.telecomitalia.it'[87.8.56.18] not among 
explicitly allowed origin hosts for domain 'nicepostmail.ru', but misses 
are permitted; delay by 02:00:00 2013/08/12 21:17:59 
hosted-by.leaseweb.com [95.211.222.157] ivdiakonova@... -> 
tavria@... accept (ACL 1480)    Sender passed SPF test, not 
delayed by milter-greylist-4.4.3f-COS (imap.ucs.domain.com 
[10.0.16.60]); Mon, 12 Aug 2013 21:17:59 +0400 (MSK)2013/08/12 21:20:15 
80-121-99-166.adsl.highway.telekom.at [80.121.99.166] 
kvsafronova@... -> mwx@... tempfail (ACL 1474) 451 
4.7.1 SPF verification soft failure: sender host 
'80-121-99-166.adsl.highway.telekom.at'[80.121.99.166] not among 
explicitly allowed origin hosts for domain 'nicepostmail.ru', but mis


The line just aborts mid-word... ("domain.com" are obfuscations of
my customer's domain).

Thanks for ideas,
//Jim

On 2013-08-12 14:48, Jim Klimov wrote:
> Hello Manu et al,
>
> Today I've caught a crash of milter-greylist, here are some details.
>
> The last entries in log were:
>
> Aug 12 16:16:23 ucs milter-greylist: [ID 471652 mail.debug] Incoming
> connection from host 'tcp-daemon.imap.ucs.domain.ru'
> Aug 12 16:16:23 ucs milter-greylist: [ID 447650 mail.error] unexpected AF
>
> The "host name" is a Messaging Server queue name with the trailing
> name of the local MTA host. This string is not resolvable to an IP
> address via DNS or local /etc/hosts (as of yet, at least).
>
> The "unexpected AF" leads me (in source) to 380:p0f.c function
> p0f_lookup() - here it tries to interpret priv->priv_addr, gets to
> default (not AF_INET nor AF_INET6) and falls out.
>
> I am not sure if these two clues are hard-linked, or if the queue
> handler does not set some macros that are present during a live SMTP
> dialog with a remote host. Still, I wonder if "localaddr" should come
> into play as a fallback here as well?
>
> My workaround for now would be to disable p0f in config, and later
> to disable milter-greylist for internal server processing - can't
> find where it gets called here, and have more urgent quests now...

On 2013-08-13 12:07, Jim Klimov wrote:
> The main question is: if the "local address" is wrong in p0f.c:380,
> should we really abort the whole program, or just not do further p0f
> processing?
>
> Also, I did not find a (documented) toggle to disable p0f if it was
> enabled during compilation. There are currently no configuration lines
> with "p0f" in my config, but the codepath is taken and in certain cases
> leads to program abortion...


So here's a patch which I hope would help me :)

This includes two fixes:
1) Do not abort the program on bad address family, but just exit the
routine - same as we do later for bad sockets, etc.

2) Report inability to connect to a socket, once in a row. Reset the
flag which blocks these reports upon a successful connection (i.e.
for cases that p0fd was restarting and was temporarily unavailable).
In my case with p0f commented away from greylist.conf, this prints

Aug 13 15:08:50 ucs milter-greylist: [ID 291231 mail.error] can't 
connect to p0f socket "", skipping p0f

and thus prompts the admin (me) to fix the config and add a socket
name, but does not do so upon every message, either :)

HTH,
//Jim Klimov

On Tue, Aug 13, 2013 at 12:07:33PM +0200, Jim Klimov wrote:
>   The main question is: if the "local address" is wrong in p0f.c:380,
> should we really abort the whole program, or just not do further p0f
> processing?

Please send code snippet? I do not see what part you talk about.

> Aug 13 10:07:05 ucs milter-greylist: [ID 447650 mail.error] unexpected AF

milter-greylist was built without Ipv6 support?

-- 
Emmanuel Dreyfus
manu@...

On 2013-08-13 13:26, Jim Klimov wrote:
> So here's a patch which I hope would help me :)
>
> This includes two fixes:
> 1) Do not abort the program on bad address family, but just exit the
> routine - same as we do later for bad sockets, etc.

Just wanted to report that since I made and used this patch (to return
instead of exit), the milter's uptime is uninterrupted for a couple of
days now. Occasionally these things are logged, but are no longer fatal:

Aug 15 17:39:19 ucs milter-greylist: [ID 471652 mail.debug] Incoming 
connection from host 'tcp-daemon.imap.ucs.domain.com'
Aug 15 17:39:19 ucs milter-greylist: [ID 144841 mail.error] unexpected 
AF in priv->priv_addr, skipping p0f

So far so good :)

//Jim