Milter-greylist

On Feb 15, 2012, I posted a problem with p0f v3 and subsequently a new milter-greylist 4.41a1 was built and that fixed the problem.
 
I had p0f 3.03b installed and working with Milter-Greylist: milter-greylist-4.4a1.  I just downloaded the latest version of p0f 3.06b, compiled it and now I am getting errors.
 
Jan 23 19:58:50 mscan milter-greylist: p0f rejected query
Jan 23 19:59:33 mscan milter-greylist: p0f rejected query
Jan 23 19:59:54 mscan milter-greylist: p0f rejected query
Jan 23 20:00:58 mscan milter-greylist: p0f rejected query
Jan 23 20:01:29 mscan milter-greylist: p0f rejected query
Jan 23 20:01:53 mscan milter-greylist: p0f rejected query
Jan 23 20:02:03 mscan milter-greylist: p0f rejected query
 
and in the /var/log/p0f.log.error:
 
[!] WARNING: Query with bad magic (0x1000000).
[!] WARNING: Query with bad magic (0x1000000).
[!] WARNING: Query with bad magic (0x1000000).
[!] WARNING: Query with bad magic (0x1000000).
[!] WARNING: Query with bad magic (0x1000000).
[!] WARNING: Query with bad magic (0x1000000).
[!] WARNING: Query with bad magic (0x1000000).
[!] WARNING: Query with bad magic (0x1000000).
[!] WARNING: Query with bad magic (0x1000000).

If I run 3.03b, it works.  I thought maybe that milter-greylist 4.4.1 might fix the problem so I updated to that version but still have the same issue.  
 
What could be the problem?
 
Thanks,
 
Gary Faith

As a followup, I e-mailed the author of p0f and this is what he sent back: Version 3.06 fixed a query structure alignment issue present in earlier versions of

I don't know if this is exactly the same problem, but when I run milter-greylist with p0f, p0f dies with the following message, when a mail message comes through:

[!] WARNING: Query with bad magic (0xdefaced).
[-] SYSTEM ERROR : read() on API socket fails despite POLLIN.
        Location : live_event_loop(), p0f.c:916
      OS message : Connection reset by peer

I'm running p0f from the command line with:
/usr/sbin/p0f -i br0 -f /etc/p0f.fp -s /var/run/p0f.socket -u smmsp

I have tried the latest stable and unstable, and cvs versions of milter-greylist and p0f 3.03 and 3.06.  I see that there have been discussions about this in the past.

So far I'm only able to run milter-greylist-2.4.6 with p0f 2.0.8

I run 64-bit gentoo Linux.

--- In milter-greylist@yahoogroups.com, "Gary Faith"  wrote:

>
> As a followup, I e-mailed the author of p0f and this is what he sent back:
> 
> Version 3.06 fixed a query structure alignment issue present in
> earlier versions of p0f v3. That may be causing problems. I'd ping the
> author of the filter. It should be a trivial change.
> 
> If you want to temporarily "fix" your version, edit api.h for p0f 3.06
> and remove the two mentions of __attribute__((packed)). This will
> restore the old behavior.
> 
> /mz
> 
> I removed the two mentions of the __attribute__((packed)) and it works now but going forward can milter-greylist be updated to work with the new p0f without the change?
> 
> Thanks,
> 
> Gary
> 
> >>> "Gary Faith"  1/23/2013 8:34 PM >>>
>   
> On Feb 15, 2012, I posted a problem with p0f v3 and subsequently a new milter-greylist 4.41a1 was built and that fixed the problem.
> 
> I had p0f 3.03b installed and working with Milter-Greylist: milter-greylist-4.4a1.  I just downloaded the latest version of p0f 3.06b, compiled it and now I am getting errors.
> 
> Jan 23 19:58:50 mscan milter-greylist: p0f rejected query
> Jan 23 19:59:33 mscan milter-greylist: p0f rejected query
> Jan 23 19:59:54 mscan milter-greylist: p0f rejected query
> Jan 23 20:00:58 mscan milter-greylist: p0f rejected query
> Jan 23 20:01:29 mscan milter-greylist: p0f rejected query
> Jan 23 20:01:53 mscan milter-greylist: p0f rejected query
> Jan 23 20:02:03 mscan milter-greylist: p0f rejected query
> 
> 
> and in the /var/log/p0f.log.error:
> 
> [!] WARNING: Query with bad magic (0x1000000).
> [!] WARNING: Query with bad magic (0x1000000).
> [!] WARNING: Query with bad magic (0x1000000).
> [!] WARNING: Query with bad magic (0x1000000).
> [!] WARNING: Query with bad magic (0x1000000).
> [!] WARNING: Query with bad magic (0x1000000).
> [!] WARNING: Query with bad magic (0x1000000).
> [!] WARNING: Query with bad magic (0x1000000).
> [!] WARNING: Query with bad magic (0x1000000).
> 
> If I run 3.03b, it works.  I thought maybe that milter-greylist 4.4.1 might fix the problem so I updated to that version but still have the same issue.  
> 
> What could be the problem?
> 
> Thanks,
> 
> Gary Faith
>

What version of milter-greylist are you trying with p0f 3.03 and 3.06? Do a Google search for: Query with bad magic (0xdefaced). and you will find my post to

Anyone?

>>> "Gary Faith" <gafaith@...> 1/23/2013 9:07 PM >>>
As a followup, I e-mailed the author of p0f and this is what he sent back:
 
Version 3.06 fixed a query structure alignment issue present in
earlier versions of p0f v3. That may be causing problems. I'd ping the
author of the filter. It should be a trivial change.
 
If you want to temporarily "fix" your version, edit api.h for p0f 3.06
and remove the two mentions of __attribute__((packed)). This will
restore the old behavior.
 
/mz

I removed the two mentions of the __attribute__((packed)) and it works now but going forward can milter-greylist be updated to work with the new p0f without the change?
 
Thanks,
 
Gary

>>> "Gary Faith" <gafaith@...> 1/23/2013 8:34 PM >>>
  
On Feb 15, 2012, I posted a problem with p0f v3 and subsequently a new milter-greylist 4.41a1 was built and that fixed the problem.

 
I had p0f 3.03b installed and working with Milter-Greylist: milter-greylist-4.4a1.  I just downloaded the latest version of p0f 3.06b, compiled it and now I am getting errors.
 
Jan 23 19:58:50 mscan milter-greylist: p0f rejected query
Jan 23 19:59:33 mscan milter-greylist: p0f rejected query
Jan 23 19:59:54 mscan milter-greylist: p0f rejected query
Jan 23 20:00:58 mscan milter-greylist: p0f rejected query
Jan 23 20:01:29 mscan milter-greylist: p0f rejected query
Jan 23 20:01:53 mscan milter-greylist: p0f rejected query
Jan 23 20:02:03 mscan milter-greylist: p0f rejected query
 
and in the /var/log/p0f.log.error:
 
[!] WARNING: Query with bad magic (0x1000000).
[!] WARNING: Query with bad magic (0x1000000).
[!] WARNING: Query with bad magic (0x1000000).
[!] WARNING: Query with bad magic (0x1000000).
[!] WARNING: Query with bad magic (0x1000000).
[!] WARNING: Query with bad magic (0x1000000).
[!] WARNING: Query with bad magic (0x1000000).
[!] WARNING: Query with bad magic (0x1000000).
[!] WARNING: Query with bad magic (0x1000000).

If I run 3.03b, it works.  I thought maybe that milter-greylist 4.4.1 might fix the problem so I updated to that version but still have the same issue.  
 
What could be the problem?
 
Thanks,
 
Gary Faith

>
> What version of milter-greylist are you trying with p0f 3.03 and 3.06?
> The solution is install milter-greylist 4.4.1 and use p0f 3.03b or greater with the exception of 3.06b.  You can use 3.06b if you modify the api.h before you build it.
> 
> I don't know if this is exactly the same problem, but when I run milter-greylist with p0f, p0f dies with the following message, when a mail message comes through:
> 
> [!] WARNING: Query with bad magic (0xdefaced).
> [-] SYSTEM ERROR : read() on API socket fails despite POLLIN.
> Location : live_event_loop(), p0f.c:916
> OS message : Connection reset by peer
> 
> I'm running p0f from the command line with:
> /usr/sbin/p0f -i br0 -f /etc/p0f.fp -s /var/run/p0f.socket -u smmsp
> 
> > As a followup, I e-mailed the author of p0f and this is what he sent back:
> > 
> > Version 3.06 fixed a query structure alignment issue present in
> > earlier versions of p0f v3. That may be causing problems. I'd ping the
> > author of the filter. It should be a trivial change.
> > 
> > If you want to temporarily "fix" your version, edit api.h for p0f 3.06
> > and remove the two mentions of __attribute__((packed)). This will
> > restore the old behavior.
> > 

Hi.

After many months I've finally fixed this.  I was using

# ./configure --enable-p0f

instead of 

# ./configure --enable-p0f3

I have used version 3.06b with __attribute__((packed)) removed and it works now, with milter-greylist 4.4.1

One problem that I now have is that my logs don't always pick up the OS of the sender - this didn't happen with earlier versions of milter-greylist and p0f 2 - eg - see "()" in:

2013/01/27 14:43:21 smtpna.posta.tim.it [217.200.184.87] () tsole@... -> me@... accept (ACL 339)    Delayed for 05:18:28 by milter-greylist-4.4.1 (mydomain.com.au [192.168.0.40]); Sun, 27 Jan 2013 14:43:21 +1100 (EST)

This is the relevant line in my config:

stat ">>/var/log/milter-greylist.log" "%T{%Y/%m/%d %T} %d [%i] (%Fx) %f -> %r %S (ACL %A) %Xc %Xe %Xm %Xh\n"

Maybe this is not important - it still seems to delay messages appropriately.

I start p0f with:
# /usr/sbin/p0f -i br0 -f /etc/p0f.fp -s /var/run/p0f.socket -u smmsp -o /var/log/p0f-audit.log 'tcp and tcp[13] & 2 = 2'

Another question - I see the new version of p0f has "Windows 7 or 8".   Should this be greylisted?  Currently it is not:

## See http://milter-greylist.wikidot.com/using-p0f
# safe Windows hosts
racl whitelist p0f "Windows 2003"       addheader "X-Greylist-OS: %Fx"
racl whitelist p0f "Windows 2008"       addheader "X-Greylist-OS: %Fx"
racl whitelist p0f "Windows 2000 SP4"   addheader "X-Greylist-OS: %Fx"

# unsafe Windows hosts -- put this line below ALL racl whitelist lines
racl greylist  p0f "Windows" addheader "X-Greylist-OS: %Fx"

super_1337_2010 <robert.spam.me.senseless@...> wrote:

> After many months I've finally fixed this.  I was using
> 
> # ./configure --enable-p0f
> 
> instead of 
> 
> # ./configure --enable-p0f3

I was just going to look at the issue. Glad to see it is already fixed
:-)

-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu@...

Did I miss something?  I started this topic on 1-23-2013 and no one has responded to my problem. As a followup, I e-mailed the author of p0f and this is what

Gary Faith <gafaith@...> wrote:

> I removed the two mentions of the __attribute__((packed)) and it works now
> but going forward can milter-greylist be updated to work with the new
> p0f without the change?

Did you use configure --enable-p0f or configure --enable-p0f3 ? The
latter should work without patching anything.

-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu@...

Emmanuel, I have used the following for configuring Milter-Greylist. ./configure --with-libspf2=/usr/local --enable-p0f3 --sysconfdir=/etc --localstatedir=/var

Gary Faith <gafaith@...> wrote:

> So the issues isn't adding the --enable-p0f3  to the ./configure line
> because I already had it.  The issue is the change is structure in the
> p0f_api_query and p0f_api_response.

If you use --enable-p0f3, we get the 0f_api_query and p0f_api_response
through #include <api.h>. Therefore if you install a fixed p0f-3.x and
rebuild milter-greylist, you are done, and no change to milter-greylist
is required.

-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu@...

Manu, I think you are missing the point.  The point is that the author of p0f changed the structure of p0f_api_query and p0f_api_response in version 3.06  to

Gary Faith <gafaith@...> wrote:

> So is your solution to remove the "fix for the query structure alignment
> issue" in the new version of p0f?

As I understand there is only an issue if you upgrade p0f without
rebuilding milter-greylist (or without installing a version that has
been built for the newer p0f). You think it is not reasonable?

-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu@...

I upgraded p0f to version 3.06b and installed milter-greylist 4.4.1.  They don t talk as I have explained 3 times now. If you don t want to fix it, fine.  I

>without installing a version that has
>been built for the newer p0f
Do you have a version built for the newer p0f?  That is what I have been asking for all along.

>>> <manu@...> 1/29/13 10:05 AM >>>
without installing a version that has
been built for the newer p0f

Gary Faith <gafaith@...> wrote:

> I upgraded p0f to version 3.06b and installed milter-greylist 4.4.1.  They
> don't talk as I have explained 3 times now.
> 
> If you don't want to fix it, fine.  I don't care.

I am willing to fix it, it's just that I failed to understand where
there is a thing to fix in milter-greylist. If --enabled-p0f3 is used,
we do not define the p0f structures in milter-greylist code, therefore I
have trouble to grasp how we could fix them. 

But I am able to undertstand there is something broken with latest p0f,
I will give it a a try.

-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu@...