Milter-greylist

Hello everyone

I just integrated Hajimu Umemoto's patches to bring complete
IPv6 support to milter-greylist. Version 1.5.4 is therefore a big change
from version 1.5.3. I hope it will not introduce too much problems.
Please test it and report anything wrong you encounter. 

Now I have to integrate Cyril Guibourg's patch for MX sync bind address
selection.

-- 
Emmanuel Dreyfus
Il y a 10 sortes de personnes dans le monde: ceux qui comprennent 
le binaire et ceux qui ne le comprennent pas.
manu@...

On Sun, 1 Aug 2004 manu@... wrote:
> I just integrated Hajimu Umemoto's patches to bring complete
> IPv6 support to milter-greylist. Version 1.5.4 is therefore a big change
> from version 1.5.3. I hope it will not introduce too much problems.
> Please test it and report anything wrong you encounter. 
> Now I have to integrate Cyril Guibourg's patch for MX sync bind address
> selection.

I hope you don't forget me... :-(

-Dan

Dan Hollis <goemon@...> wrote:

> I hope you don't forget me... :-(

Yes, I don't forget you.

-- 
Emmanuel Dreyfus
Il y a 10 sortes de personnes dans le monde: ceux qui comprennent 
le binaire et ceux qui ne le comprennent pas.
manu@...

At 11:26 AM 8.1.2004 +0200, you wrote:
>Hello everyone
>
>I just integrated Hajimu Umemoto's patches to bring complete
>IPv6 support to milter-greylist. Version 1.5.4 is therefore a big change
>from version 1.5.3. I hope it will not introduce too much problems.
>Please test it and report anything wrong you encounter. 
>
>Now I have to integrate Cyril Guibourg's patch for MX sync bind address
>selection.
>
>-- 
>Emmanuel Dreyfus

Hi, Emmanuel. Thanks for the new version. Is IPv6 support the only change?
(don't need it myself - yet)

Also, when will this new one be in FBSD ports as I typically update/upgrade
from there?

Again, thanks for a great spam tool! By adding milter-greylist to the MTA
level of spam nets, I have cut spam that gets past the MTA to 2% from
previous 30% -- I was catching the 30% with Spamassassin & Procmail. Now, I
only need to invoke the use of heavier resources for the 2%.

Other skeptics have said "...well, just wait until the messages return from
retries...." Been using GL for about a month and don't see any rise of spam
getting past MTA in my stats.

That's why I was so interested in your new rcptfilter. BTW, it still gets
invoked, but doesn't snag anything, nor does it show any report of "abort"
as do the 3 other milters used.

I am reluctant to bother you as you did say "as is" ... but, what "Input
Filter" line are you using in sendmail.mc/cf...." The config file loads
okay so it sees that -- just ignores it.

Best regards,
Jack L. Stone,
Administrator

Sage American
http://www.sage-american.com
jacks@...

Jack L. Stone <jacks@...> wrote:

> Hi, Emmanuel. Thanks for the new version. Is IPv6 support the only change?
> (don't need it myself - yet)

1.5.4:
        Avoid race conditions when reloading the config (Attila
Bruncsak)
        Full blown IPv6 support, from Hajimu Umemoto
        rc-debian.sh script, from Joel Bertrand

So if you don't hit the bug Attila experienced, you don't really need to
upgrade. Testers are welcom, though.
 
> Also, when will this new one be in FBSD ports as I typically update/upgrade
> from there?

I thought it was already available:  
http://www.freebsd.org/cgi/query-pr.cgi?pr=ports/67252
 
> Other skeptics have said "...well, just wait until the messages return from
> retries...." Been using GL for about a month and don't see any rise of spam
> getting past MTA in my stats.

In fact a lot of spammers do retry now, with an interval of a few
seconds. This defeats low-end greylisting implementation, but
milter-greylist is still safe.
 
> That's why I was so interested in your new rcptfilter. BTW, it still gets
> invoked, but doesn't snag anything, nor does it show any report of "abort"
> as do the 3 other milters used.

Have you tried poking a few printf at key places to check what happens?
 
> I am reluctant to bother you as you did say "as is" ... but, what "Input
> Filter" line are you using in sendmail.mc/cf...." The config file loads
> okay so it sees that -- just ignores it.

Here is the revelant part of my sendmail.cf (I don't use .mc files to
build it):

# Enable milter
O InputMailFilters=rcptfilter,greylist

Xgreylist, S=/var/run/milte-greylist/greylist.sock
Xrcptfilter, S=/var/run/milter-rcptfilter/sock

O Milter.macros.connect=j, _, {daemon_name}, {if_name}, {if_addr},
{client_addr}
O Milter.macros.helo={tls_version}, {cipher}, {cipher_bits},
{cert_subject}, {ce
rt_issuer}
O Milter.macros.envfrom=i, {auth_type}, {auth_authen}, {auth_ssf},
{auth_author}
, {mail_mailer}, {mail_host}, {mail_addr}
O Milter.macros.envrcpt={rcpt_mailer}, {rcpt_host}, {rcpt_addr}

-- 
Emmanuel Dreyfus
Il y a 10 sortes de personnes dans le monde: ceux qui comprennent 
le binaire et ceux qui ne le comprennent pas.
manu@...

On Sun, 2004-08-01 at 09:43, Jack L. Stone wrote:
> Other skeptics have said "...well, just wait until the messages return from
> retries...." Been using GL for about a month and don't see any rise of spam
> getting past MTA in my stats.

If/when that starts to become a problem just combine milter-greylist
with a real time block list.  The idea being that milter-greylist delays
delivery of the spam long enough for that site to show up in a spamtrap
which is fed into a real time block list.  The next time the message
comes in after the delay has expired it gets blocked by the RBL check.

I have to say the results of using milter-greylist far exceeded my
expectations.  Went from 3000-6000 spam a day to 5-10 a day which get
caught by spamassassin.

-- 
Scot L. Harris
webid@...

If God didn't mean for us to juggle, tennis balls wouldn't come three to a can.

Scot L. Harris <webid@...> wrote:

> If/when that starts to become a problem just combine milter-greylist
> with a real time block list.  The idea being that milter-greylist delays
> delivery of the spam long enough for that site to show up in a spamtrap
> which is fed into a real time block list.  The next time the message
> comes in after the delay has expired it gets blocked by the RBL check.

Yep. I've written some code for building a good spamtrap network, but
I'm stopped on that front because for now it's quite useless. 
ftp://ftp.espci.fr/pub/dst

I should probably be pushing it a bit further so that we get ready the
day spammers will defeat milter-greylist.  

-- 
Emmanuel Dreyfus
Il y a 10 sortes de personnes dans le monde: ceux qui comprennent 
le binaire et ceux qui ne le comprennent pas.
manu@...

"Jack L. Stone" <jacks@...> writes:

> Also, when will this new one be in FBSD ports as I typically update/upgrade
> from there?

Hello Jack,

As maintainer I plan to play a little with 1.5.4 to check that at least
nothing is broken with IPv4 before updating the port.

If you don't need IPv6 you can use the actual port which is version 1.5.3
of milter-greylist.

Cheers.

At 11:50 PM 8.1.2004 +0200, Cyril Guibourg wrote:
>"Jack L. Stone" <jacks@...> writes:
>
>> Also, when will this new one be in FBSD ports as I typically update/upgrade
>> from there?
>
>Hello Jack,
>
>As maintainer I plan to play a little with 1.5.4 to check that at least
>nothing is broken with IPv4 before updating the port.
>
>If you don't need IPv6 you can use the actual port which is version 1.5.3
>of milter-greylist.
>
>Cheers.
>

Hi, Cyril,

Thanks for that clarification. I'm already runnning the v-1.5.3 from the
ports and what you said about checking it further is exactly why I prefer
to use the ports, figuring the prorgams are fairly-well matured by work
like yours.

I'll just wait since nothing essential affects the IPv4 side of things.

Best regards,
Jack L. Stone,
Administrator

Sage American
http://www.sage-american.com
jacks@...

Jack L. Stone <jacks@...> wrote:

> Thanks for that clarification. I'm already runnning the v-1.5.3 from the
> ports and what you said about checking it further is exactly why I prefer
> to use the ports, figuring the prorgams are fairly-well matured by work
> like yours.

On the other hand if everyone relies on Cyril to test it, you'll end up
with a program only tested by one person :)

This version 1.5.4 has possible issues because a lot of code has
changed. It would be nice if it could be largely tested.

-- 
Emmanuel Dreyfus
Il y a 10 sortes de personnes dans le monde: ceux qui comprennent 
le binaire et ceux qui ne le comprennent pas.
manu@...

manu@... writes:

> Please test it and report anything wrong you encounter. 

No problems so far, IPv6 seems to work fine. I do not get much mail
via IPv6 but it worked fine for all I have received since upgrading.

I have just a small observation which leads to a concern (which is
probably because I do not understand the mechanism). With 1.5.3, the
dump file contained the actual IP addresses of the connecting
systems. Now that I have upgraded to 1.5.4 it lists a short 'handle'
instead of the IP address. For example the mail from sourceforge
mailing lists now shows as 'lists'.

manu@... writes:

> On the other hand if everyone relies on Cyril to test it, you'll end up
> with a program only tested by one person :)

Very good point, and I must admit there are much more better testers
than me ;-)

> This version 1.5.4 has possible issues because a lot of code has
> changed. It would be nice if it could be largely tested.

Update to 1.5.4 should not last for a long time.

On Mon, Aug 02, 2004 at 08:39:07AM +0100, Graham Murray wrote:
> I have just a small observation which leads to a concern (which is
> probably because I do not understand the mechanism). With 1.5.3, the
> dump file contained the actual IP addresses of the connecting
> systems. Now that I have upgraded to 1.5.4 it lists a short 'handle'
> instead of the IP address. For example the mail from sourceforge
> mailing lists now shows as 'lists'. 

How does it look?

-- 
Emmanuel Dreyfus
manu@...

Emmanuel Dreyfus <manu@...> writes:

> How does it look?

Here are the first 2 fields from the end of my current dump:-

out006      <Mrq3JAEJATBCOgUACAAAAAAAKbkwHg
out008      <Mrq3JAEJATBCOtAACAAAAAAAKbkwHg
out008      <Mrq3JAEJATBCOtQACAAAAAAAKbkwHg
livejournal             <hlwscizmsy@...>
livejournal      <magic@mitigatingcircumstances.
livejournal            <d_odomwo@...>
cicero                                   <>
out006      <Mrq3JAEJATBCO+wACAAAAAAAKbkwHg
bm0-1      <IYMN1N-A7DCB-IORRM-H@... 
iad-fw-global         <auto-shipping@...>

On Mon, Aug 02, 2004 at 01:10:40PM +0100, Graham Murray wrote:
> out006      <Mrq3JAEJATBCOgUACAAAAAAAKbkwHg
> out008      <Mrq3JAEJATBCOtAACAAAAAAAKbkwHg
> out008      <Mrq3JAEJATBCOtQACAAAAAAAKbkwHg

That's bad because it means you won't be able to reload this dump.
What OS?
Anyone else has this problem?

-- 
Emmanuel Dreyfus
manu@...

Emmanuel Dreyfus <manu@...> writes:

> That's bad because it means you won't be able to reload this dump.
> What OS?

Linux

I have identified the problem, milter-greylist was picking up the
getnameinfo from libbind (as libspf was linked against this) rather
than glibc and this seemed to be ignoring the NI_NUMERICHOST flag. 

I have now ensured that libbind is not linked, and it now works
correctly with the IP address in the first column (but as you said, it
has lost most of the entried which were previously there so is
greylisting mail which would previously have not been - but it should
soon relearn them)

On Mon, Aug 02, 2004 at 02:57:33PM +0100, Graham Murray wrote:
> I have identified the problem, milter-greylist was picking up the
> getnameinfo from libbind (as libspf was linked against this) rather
> than glibc and this seemed to be ignoring the NI_NUMERICHOST flag. 

What version of BIND do you link against?

-- 
Emmanuel Dreyfus
manu@...

Emmanuel Dreyfus <manu@...> writes:

> What version of BIND do you link against?

9.3.0rc2