Milter-greylist

Hi 

milter-greylist-1.5.6 is now available. It introduces the long awaited
patch from Cyril Guibourg that enable MX sync local address selection.

Next versions roadmap:
1.5.7: maintenance patch from Cyril Guibourg to clean up some mess.
1.5.8: per recipient whitelist patch from Dan Hollis
1.5.9: better handling of real time callbacks, from me
and then we'll move to a new stable release.

While I'm there, I released milter-rcptfilter-0.9, which fixes some 
build problems on Linux and uses less memory.

-- 
Emmanuel Dreyfus
manu@...

Hi,

>>>>> On Tue, 10 Aug 2004 10:20:40 +0000
>>>>> Emmanuel Dreyfus <manu@...> said:

manu> milter-greylist-1.5.6 is now available.

It seems there is a problem in composing.  There is unwanted
milter-greylist-1.5.6/milter-greylist directory.

manu> It introduces the long awaited patch from Cyril Guibourg that
manu> enable MX sync local address selection.

In current implementation, IPv4 and IPv6 is exclusive.  In
sync_listen() in sync.c, we can see following lines:

	if (conf.c_syncaddr != NULL) {
		if (strchr(conf.c_syncaddr, ':'))
		    sync_listen(conf.c_syncaddr, conf.c_syncport,
				&sync_master6);
		else
		    sync_listen(conf.c_syncaddr, conf.c_syncport,
				&sync_master4);
	} else {

It is better to have conf.c_syncaddr and conf.c_syncaddr6, and call
sync_listen() for both IPv4 and IPv6.

Sincerely,

--
Hajimu UMEMOTO @ Internet Mutual Aid Society Yokohama, Japan
ume@...  ume@{,jp.}FreeBSD.org
http://www.imasy.org/~ume/

On Tue, Aug 10, 2004 at 08:00:07PM +0900, Hajimu UMEMOTO wrote:
> It seems there is a problem in composing.  There is unwanted
> milter-greylist-1.5.6/milter-greylist directory.

Oh, yes, I screwed up things when packaging. 
I just updated the tarball, the new MD5 sum is
MD5 (milter-greylist-1.5.6.tgz) = 69e3a19412a61b220c06a82f2bf80858

-- 
Emmanuel Dreyfus
manu@...

On Tue, Aug 10, 2004 at 08:00:07PM +0900, Hajimu UMEMOTO wrote:
> It is better to have conf.c_syncaddr and conf.c_syncaddr6, and call
> sync_listen() for both IPv4 and IPv6.

It would be even better to enable multiple use of syncaddr so that one
can bind to several addresses, either IPv4 or IPv6. This means 
introducing a chained list of addresses/ports and to bind to all of them.

-- 
Emmanuel Dreyfus
manu@...

At 10:20 AM 8.10.2004 +0000, Emmanuel Dreyfus wrote:
>Hi 
>
>milter-greylist-1.5.6 is now available. It introduces the long awaited
>patch from Cyril Guibourg that enable MX sync local address selection.
>
>Next versions roadmap:
>1.5.7: maintenance patch from Cyril Guibourg to clean up some mess.
>1.5.8: per recipient whitelist patch from Dan Hollis
>1.5.9: better handling of real time callbacks, from me
>and then we'll move to a new stable release.
>
>While I'm there, I released milter-rcptfilter-0.9, which fixes some 
>build problems on Linux and uses less memory.
>
>-- 
>Emmanuel Dreyfus
>manu@...
>

I got these warnings when compiling milter-rcptfilter-0.9. Is is fatal?:

(running FBSD-4.10Rp2/Sendmail-8.12.11)

gcc -o milter-rcptfilter conf.o except.o milter-rcptfilter.o conf_yacc.o
-lc_r -lmilter
/usr/lib/libc.so: WARNING!  setkey(3) not present in the system!
/usr/lib/libc.so: warning: this program uses gets(), which is unsafe.
/usr/lib/libc.so: warning: mktemp() possibly used unsafely; consider using
mkstemp()
/usr/lib/libc.so: WARNING!  des_setkey(3) not present in the system!
/usr/lib/libc.so: WARNING!  encrypt(3) not present in the system!
/usr/lib/libc.so: warning: tmpnam() possibly used unsafely; consider using
mkstemp()
/usr/lib/libc.so: warning: this program uses f_prealloc(), which is not
recommended.
/usr/lib/libc.so: WARNING!  des_cipher(3) not present in the system!
/usr/lib/libc.so: warning: tempnam() possibly used unsafely; consider using
mkstemp()
sed "s|@BINDIR[@]|/usr/local/bin|g; s|@USER[@]|smmsp|g"  rc-bsd.sh.in >
rc-bsd.sh

Best regards,
Jack L. Stone,
Administrator

Sage American
http://www.sage-american.com
jacks@...

....update: actually I DO have those things not found on the system.

Noted the milter binary is still dated Aug 3.....

>
>I got these warnings when compiling milter-rcptfilter-0.9. Is is fatal?:
>
>(running FBSD-4.10Rp2/Sendmail-8.12.11)
>
>gcc -o milter-rcptfilter conf.o except.o milter-rcptfilter.o conf_yacc.o
-lc_r -lmilter
>/usr/lib/libc.so: WARNING!  setkey(3) not present in the system!
>/usr/lib/libc.so: warning: this program uses gets(), which is unsafe.
>/usr/lib/libc.so: warning: mktemp() possibly used unsafely; consider using
mkstemp()
>/usr/lib/libc.so: WARNING!  des_setkey(3) not present in the system!
>/usr/lib/libc.so: WARNING!  encrypt(3) not present in the system!
>/usr/lib/libc.so: warning: tmpnam() possibly used unsafely; consider using
mkstemp()
>/usr/lib/libc.so: warning: this program uses f_prealloc(), which is not
recommended.
>/usr/lib/libc.so: WARNING!  des_cipher(3) not present in the system!
>/usr/lib/libc.so: warning: tempnam() possibly used unsafely; consider
using mkstemp()
>sed "s|@BINDIR[@]|/usr/local/bin|g; s|@USER[@]|smmsp|g"  rc-bsd.sh.in >
rc-bsd.sh

Best regards,
Jack L. Stone,
Administrator

Sage American
http://www.sage-american.com
jacks@...

Jack L. Stone <jacks@...> wrote:

> I got these warnings when compiling milter-rcptfilter-0.9. Is is fatal?:
> (running FBSD-4.10Rp2/Sendmail-8.12.11)

Proably not, especially since it does not use any of the functions
listed below. Do you still have the warnings if you replace -lc_r by
-pthread?
 
> gcc -o milter-rcptfilter conf.o except.o milter-rcptfilter.o conf_yacc.o
> -lc_r -lmilter
> /usr/lib/libc.so: WARNING!  setkey(3) not present in the system!
> /usr/lib/libc.so: warning: this program uses gets(), which is unsafe.
> /usr/lib/libc.so: warning: mktemp() possibly used unsafely; consider using
> mkstemp()
> /usr/lib/libc.so: WARNING!  des_setkey(3) not present in the system!
> /usr/lib/libc.so: WARNING!  encrypt(3) not present in the system!
> /usr/lib/libc.so: warning: tmpnam() possibly used unsafely; consider using
> mkstemp()
> /usr/lib/libc.so: warning: this program uses f_prealloc(), which is not
> recommended.
> /usr/lib/libc.so: WARNING!  des_cipher(3) not present in the system!
> /usr/lib/libc.so: warning: tempnam() possibly used unsafely; consider using
> mkstemp()

-- 
Emmanuel Dreyfus
Il y a 10 sortes de personnes dans le monde: ceux qui comprennent 
le binaire et ceux qui ne le comprennent pas.
manu@...

At 08:00 AM 8.11.2004 +0200, manu@... wrote:
>Jack L. Stone <jacks@...> wrote:
>

No more emails for a week+.... have I been unsubscribed...???

Best regards,
Jack L. Stone,
Administrator

Sage American
http://www.sage-american.com
jacks@...

Hi Jack,

Quoting "Jack L. Stone" <jacks@...>:

> At 08:00 AM 8.11.2004 +0200, manu@... wrote:
> >Jack L. Stone <jacks@...> wrote:
> >
>
> No more emails for a week+.... have I been unsubscribed...???
>
> Best regards,
> Jack L. Stone,
> Administrator

August means vacations ...
I guess that Manu and the other milter-greylist users enjoy "real-life"
a bit....

I really long for a new stable release for milter-greylist
(I havent' been bold enough to use 1.5.6).
Been using 1.4 for some test sub-domains and it really rocks :
I just received 3 spam in one week instead of 70-100 ones in my
personal mailbox ....

Feedback :
- syslog(LOG_INFO, "%s: testmode: skipping greylist " in except.c
generates too many syslog lines for people who restrict greylisting to
some addresses/domains (-T). I feel it should be turned off by
default.
- I had to tweak a little the linux rc script for SUSE... When polished
I will submit it back.
- some (perl, shell) contrib statistics scripts in the tarball  would be
 nice so we can state clearly how many spam messages were not accepted ...
- the addresses are shortened in the logs but I guess 1.5.x versions
do not have this limitations

Bravo to Manu and all who helped devised this nice software,

SL/

At 08:04 PM 8.21.2004 +0200, milter@... wrote:
>Hi Jack,
>
>Quoting "Jack L. Stone" <jacks@...>:
>
>> At 08:00 AM 8.11.2004 +0200, manu@... wrote:
>> >Jack L. Stone <jacks@...> wrote:
>> >
>>
>> No more emails for a week+.... have I been unsubscribed...???
>>
>> Best regards,
>> Jack L. Stone,
>> Administrator
>
>August means vacations ...
>I guess that Manu and the other milter-greylist users enjoy "real-life"
>a bit....
>
>I really long for a new stable release for milter-greylist
>(I havent' been bold enough to use 1.5.6).
>Been using 1.4 for some test sub-domains and it really rocks :
>I just received 3 spam in one week instead of 70-100 ones in my
>personal mailbox ....
>
>Feedback :
>- syslog(LOG_INFO, "%s: testmode: skipping greylist " in except.c
>generates too many syslog lines for people who restrict greylisting to
>some addresses/domains (-T). I feel it should be turned off by
>default.
>- I had to tweak a little the linux rc script for SUSE... When polished
>I will submit it back.
>- some (perl, shell) contrib statistics scripts in the tarball  would be
> nice so we can state clearly how many spam messages were not accepted ...
>- the addresses are shortened in the logs but I guess 1.5.x versions
>do not have this limitations
>
>Bravo to Manu and all who helped devised this nice software,
>
>SL/
>

Hi: Thanks for the reply! I was worried because I do enjoy this list and
this particular milter (plus milter-regex) has really relieved the load on
resources needed to run the other layers of spam tools. Now, I hardly ever
see spamassassin needed now and it soaks up the power!

I have been running v1.5.6 since the day it came out and no problems at all.

My platform is FBSD-4.10-p2

It is clear that the closer to the MTA "front door" we can get to catching
stuff, the better. I am really enjoying the "milters".

I too agree with your "bravo"....!!

Best regards,
Jack L. Stone,
Administrator

Sage American
http://www.sage-american.com
jacks@...

<milter@...> wrote:

> August means vacations ...
> I guess that Manu and the other milter-greylist users enjoy "real-life"
> a bit....

No, I'm still there (though I'll go offline for a week soon). It's just
that developpement of milter-greylist has quite setteled since it does
everything I need. Now I'm just incorporating patches from people that
need more features.

In fact it's not true it does all I need: I'd like it to defer the
tempfail to the end of DATA stage when message is from <> so that it
works more nicely with sender callbacks. I'll do that once Dan Hollis
will have submitted his patch. And after this we'll make a new stable
release.

And after that we'll work to support more SPF-like whitelisting
directory methods. I haven't looked at the others, but the more we have
the cleaner it is. So developpement has settled, but not halted. 

> - syslog(LOG_INFO, "%s: testmode: skipping greylist " in except.c
> generates too many syslog lines for people who restrict greylisting to
> some addresses/domains (-T). I feel it should be turned off by
> default.

I like this one when I track down mail delivery problems. 

> - some (perl, shell) contrib statistics scripts in the tarball  would be
>  nice so we can state clearly how many spam messages were not accepted ...

Sure, go ahead and contribute. 
I'd prefer a C programm, though. You already have a parser for the dump
file in milter-greylist...

> - the addresses are shortened in the logs but I guess 1.5.x versions
> do not have this limitations

It's just that milter-greylist only store the 32 first bytes of each
address. We should fix it by using malloc insteead of a fixed size
string. But we must think about a DoS protection. 

-- 
Emmanuel Dreyfus
Il y a 10 sortes de personnes dans le monde: ceux qui comprennent 
le binaire et ceux qui ne le comprennent pas.
manu@...

Jack L. Stone <jacks@...> wrote:

> Hi: Thanks for the reply! I was worried because I do enjoy this list and
> this particular milter (plus milter-regex) has really relieved the load on
> resources needed to run the other layers of spam tools. Now, I hardly ever
> see spamassassin needed now and it soaks up the power!

That makes me think: anyone uses a sender callback system? I tried
milter-sender but it quickly crashed on my system, so I was considering
rewriting one from scratch, but if something works fine, I'd prefer not
reinventing the wheel.

-- 
Emmanuel Dreyfus
Il y a 10 sortes de personnes dans le monde: ceux qui comprennent 
le binaire et ceux qui ne le comprennent pas.
manu@...

At 08:42 PM 8.21.2004 +0200, manu@... wrote:
>Jack L. Stone <jacks@...> wrote:
>
>> Hi: Thanks for the reply! I was worried because I do enjoy this list and
>> this particular milter (plus milter-regex) has really relieved the load on
>> resources needed to run the other layers of spam tools. Now, I hardly ever
>> see spamassassin needed now and it soaks up the power!
>
>That makes me think: anyone uses a sender callback system? I tried
>milter-sender but it quickly crashed on my system, so I was considering
>rewriting one from scratch, but if something works fine, I'd prefer not
>reinventing the wheel.
>
>-- 
>Emmanuel Dreyfus

Hi, Emmanuel:

Interesting! Milter-sender core dumps on me too every time & I've tried it
numerous times over the past 6 months or so -- more than one version too, I
think.

Wish you would write it, because it looked like a very, very good idea.

Again, my platform is:
FBSD-4.10-p2/Sendmail-8.12.11

Best regards,
Jack L. Stone,
Administrator

Sage American
http://www.sage-american.com
jacks@...

Quoting "Jack L. Stone" <jacks@...>:

>
> At 08:42 PM 8.21.2004 +0200, manu@... wrote:
> >Jack L. Stone <jacks@...> wrote:
> >
> >> Hi: Thanks for the reply! I was worried because I do enjoy this list and
> >> this particular milter (plus milter-regex) has really relieved the load on
> >> resources needed to run the other layers of spam tools. Now, I hardly ever
> >> see spamassassin needed now and it soaks up the power!
> >
> >That makes me think: anyone uses a sender callback system? I tried
> >milter-sender but it quickly crashed on my system, so I was considering
> >rewriting one from scratch, but if something works fine, I'd prefer not
> >reinventing the wheel.
> >
> >--
> >Emmanuel Dreyfus
>
> Hi, Emmanuel:
>
> Interesting! Milter-sender core dumps on me too every time & I've tried it
> numerous times over the past 6 months or so -- more than one version too, I
> think.
>
> Wish you would write it, because it looked like a very, very good idea.
>
> Again, my platform is:
> FBSD-4.10-p2/Sendmail-8.12.11
>
> Best regards,
> Jack L. Stone,
> Administrator
>

Emmanuel, Jack,

Which versions did you use ?
Did you report it to Anthony (milter-sender's author) ?
I CC him as I know he's pretty active and might help on these Netbsd/FreeBSD
issues.
Strangely the Freebsd port is not yet updated to version 0.60 (still only 0.58)
As for Netbsd port It seems really old too...

Regards,

SL/

At 10:02 PM 8.21.2004 +0200, milter@... wrote:
>Quoting "Jack L. Stone" <jacks@...>:
>
>>
>> At 08:42 PM 8.21.2004 +0200, manu@... wrote:
>> >Jack L. Stone <jacks@...> wrote:
>> >
>> >> Hi: Thanks for the reply! I was worried because I do enjoy this list and
>> >> this particular milter (plus milter-regex) has really relieved the
load on
>> >> resources needed to run the other layers of spam tools. Now, I hardly
ever
>> >> see spamassassin needed now and it soaks up the power!
>> >
>> >That makes me think: anyone uses a sender callback system? I tried
>> >milter-sender but it quickly crashed on my system, so I was considering
>> >rewriting one from scratch, but if something works fine, I'd prefer not
>> >reinventing the wheel.
>> >
>> >--
>> >Emmanuel Dreyfus
>>
>> Hi, Emmanuel:
>>
>> Interesting! Milter-sender core dumps on me too every time & I've tried it
>> numerous times over the past 6 months or so -- more than one version too, I
>> think.
>>
>> Wish you would write it, because it looked like a very, very good idea.
>>
>> Again, my platform is:
>> FBSD-4.10-p2/Sendmail-8.12.11
>>
>> Best regards,
>> Jack L. Stone,
>> Administrator
>>
>
>Emmanuel, Jack,
>
>Which versions did you use ?
>Did you report it to Anthony (milter-sender's author) ?
>I CC him as I know he's pretty active and might help on these Netbsd/FreeBSD
>issues.
>Strangely the Freebsd port is not yet updated to version 0.60 (still only
0.58)
>As for Netbsd port It seems really old too...
>
>Regards,
>
>SL/
>

I've tried both 0.58 and 0.60 without luck....

Best regards,
Jack L. Stone,
Administrator

Sage American
http://www.sage-american.com
jacks@...

<milter@...> wrote:

> Which versions did you use ?
> Did you report it to Anthony (milter-sender's author) ?
> I CC him as I know he's pretty active and might help on these Netbsd/FreeBSD
> issues.
> Strangely the Freebsd port is not yet updated to version 0.60 (still only
> 0.58) As for Netbsd port It seems really old too...

Yes, I did the NetBSD port once, and it was so badly broken I lose
interest and did not upgrade it. So you think I should give it another
try?

I suspect it has been killed by DNS resolution and thread safety. That's
the big pitfall in milter developement.

-- 
Emmanuel Dreyfus
Il y a 10 sortes de personnes dans le monde: ceux qui comprennent 
le binaire et ceux qui ne le comprennent pas.
manu@...

Jack L. Stone <jacks@...> wrote:

[milter-sender]
> I've tried both 0.58 and 0.60 without luck....

Did you linked against a thread-safe resolver (ie: BIND 9.x libbind.a)?
I don't know about milter-sender code, but it cannot work otherwise. 


-- 
Emmanuel Dreyfus
Il y a 10 sortes de personnes dans le monde: ceux qui comprennent 
le binaire et ceux qui ne le comprennent pas.
manu@...

Quoting manu@...:

> <milter@...> wrote:
>
> > Which versions did you use ?
> > Did you report it to Anthony (milter-sender's author) ?
> > I CC him as I know he's pretty active and might help on these
> Netbsd/FreeBSD
> > issues.
> > Strangely the Freebsd port is not yet updated to version 0.60 (still only
> > 0.58) As for Netbsd port It seems really old too...
>
> Yes, I did the NetBSD port once, and it was so badly broken I lose
> interest and did not upgrade it. So you think I should give it another
> try?

=> Anthony has given some answers on this topic.

> I suspect it has been killed by DNS resolution and thread safety. That's
> the big pitfall in milter developement.
>

=> Right. Everybody using Milter applications on BSD distros faced such
problems one day. I know  many Milter programs which worked out the box on
Solaris and Linux systems but crashed on BSD systems ....
I will try to add  a programming section with such info on my updated Milter
Introduction & Products List.

SL/

<milter@...> wrote:

> => Right. Everybody using Milter applications on BSD distros faced such
> problems one day. I know  many Milter programs which worked out the box on
> Solaris and Linux systems but crashed on BSD systems ....

The rule is simple: do DNS resolutions with a thread-unsafe resolver in
a milter and you crash. Now you just have to make a list of
thread-unsafe resolvers in libc. Here is the first entry of your list:
NetBSD<2.0

The workaround: link with BIND 9's libbind.a. The milter (and all the
libraries it links with) should also use nres_* DNS functions instead of
res_* functions. That way it won't build at all with a thread-unsafe
resolver.   

-- 
Emmanuel Dreyfus
Il y a 10 sortes de personnes dans le monde: ceux qui comprennent 
le binaire et ceux qui ne le comprennent pas.
manu@...

On Sun, Aug 22, 2004 at 10:40:18AM +0200, milter@... wrote:
> => Right. Everybody using Milter applications on BSD distros faced such
> problems one day. I know  many Milter programs which worked out the box on
> Solaris and Linux systems but crashed on BSD systems ....

The good news is that at least NetBSD-current uses the thread safe
BIND 9 resolver out of the box.

	Kind regards

-- 
Matthias Scheler                                  http://scheler.de/~matthias/

At 11:33 AM 8.22.2004 +0200, manu@... wrote:
><milter@...> wrote:
>
>> => Right. Everybody using Milter applications on BSD distros faced such
>> problems one day. I know  many Milter programs which worked out the box on
>> Solaris and Linux systems but crashed on BSD systems ....
>
>The rule is simple: do DNS resolutions with a thread-unsafe resolver in
>a milter and you crash. Now you just have to make a list of
>thread-unsafe resolvers in libc. Here is the first entry of your list:
>NetBSD<2.0
>
>The workaround: link with BIND 9's libbind.a. The milter (and all the
>libraries it links with) should also use nres_* DNS functions instead of
>res_* functions. That way it won't build at all with a thread-unsafe
>resolver.   
>
>-- 
>Emmanuel Dreyfus

FYI.... here is my milter-sendmail build/install on
FBSD-4.10-p2/Sendmail-8.12.11
http://www.sage-american.com/tmp/milter-sender.txt

First, I had to update the port from 0.58 to 0.60

It has all of the dependencies required by the port I believe. Dunno about
bind9.

Appreciate feedback.

Best regards,
Jack L. Stone,
Administrator

Sage American
http://www.sage-american.com
jacks@...