Milter-greylist

Here is 4.0rc1

http://ftp.espci.fr/pub/milter-greylist/milter-greylist-4.0rc1.tgz
MD5 (milter-greylist-4.0rc1.tgz) = afacbfedb0d0b29e895cd776133eef2c

Contributors: if you made a contribution that is not acknowledged in the man
page (AUTHOR(S) section), please complain before the 4.0 final release.


Changes since 3.0:
4.0rc1
        Accept --disable-dnsrbl without a bug
        Fix message size when multiple messages are sent in one session
        Do not build fd_pool.c if it is not used
4.0b4
        Add missing bits for Solaris 256 stream limitation workaround
        Fix build problem on Solaris
        Documentation fix (Tim Mooney)
4.0b3
        Warn in README that bison may be required for buidling
        Fix configure error on Solaris (Tim Mooney)
        Workaround for Solaris 256 stream limitation (Johann E. Klasek)
        Fix spelling errors (Nerijus Baliunas)
        Restore build for systems like Tru64 where ld does not acccept -R
        Cleanup temporary file after DB dump failure (Johann E. Klasek)
        Handle libc that fails stdio without setting errno (Johann E. Klasek)
        Fixes the usage of the thread-proof resolver library (Johann E. Klasek)
        Do not quit on non fatal errors (Johann E. Klasek)
        Display ACL when matching whitelist ACL (Benoit Branciard)
        Add troubleshooting in README (Rogier Maas)
4.0b2
        Treat protocol errors in urlcheck clauses as temporary failures
        Report missing SPF reasons in X-Greylist (Benoit Branciard)
        Allow building objects outside of source directory (Mattheu Herrb)
        Fix configure LDFLAG generation, -R was missing (Mattheu Herrb)
        Fix MX sync on Solaris (Mattheu Herrb)
4.0b1
        Add checks against libmilter giving us NULL private structure
        Properly reset message filters when handling MAIL FROM after DATA
        Allow rcpt clause in dacl statement, as documented
        Add google pools to default greylist.conf
        Add hooks for libdmalloc
        Improve debug message (Yaroslav Boychuk)
        Improve GeoIP status report in X-Greylist (Hideki ONO)
4.0a6
        Avoid GeoIP reporting "--" for broken domain (Bernhard Schneider )
        Fix run-away string (AIDA Shinra)
        Check for libbind9 if libbind cannot be found (Fabien Tassin)
        Document ACL on TLS DN (Fabien Tassin)
        Avoid spurious exit on DATA stage ACL match for multiple recipients
4.0a5
        Don't use YY_FLUSH_BUFFER, use flush_buffer() instead (AIDA Shinra)
4.0a4
        Fix upgrade pitfal, where write access to the PID file is missing
        Back out a Debian build fix that broke other systems
4.0a3
        Add an urlcheck reply which is ignored: milterGreylistIgnore
        Build fixes for Debian (Bernhard Schneider)
        Add a configure flag to not use --rpath (Bernhard Schneider)
        Cleanup stale PID file on startup (Bernhard Schneider)
        Fix memory leak with DNSRBL (Michael Fromme)
4.0a2
        For urlcheck answer, cope with a trailing line not terminated by CR
        Fix X-Greylist lossage (AIDA Shinra)
        Fix garbled log (Bernhard Schneider)
        Fix build on Tru64 (Attila Bruncsak)
4.0a1
        Fix again an accept-all-bug when built with SPF (AIDA Shinra)
        Fix macro handling in ACL (AIDA Shinra)
        %% syntax in format strings (AIDA Shinra)
        Clarified "%Xm" and "%Xh" semantics (AIDA Shinra)

        Fixed a memory leak at mlfi_eom (AIDA Shinra)
        Fix a bug where doing RCPT twice when blacklised succeed (AIDA Shinra)
3.1.8
        Allow filtering on the HELO string
        Back out previous SPF fix, it caused accept-all with SPF-less builds
3.1.7
        Fix an accept-all-bug when built with SPF (AIDA Shinra)
        Option to perform urlchecks in forked instance to avoid thread-unsafety
        Allow escaping of " in strings and / in regex
        Fix whitelisting using access.db (Georg Horn)
        Fix NULL pointer referencing in urlcheck code
        Fix body storage so that locators (^ and $) work for body regex
        Fix spurious warning about unknown whitelisting conditions
        Fix memory leaks when using urlchecks
        Makes urlcheck properties and values case insensitives
        Do not retain urlcheck properties if the urlcheck clause did not match
        Allow loading regex in lists of body clauses
        Option to clear urlcheck prop before handling a new recipient
        %D format string for getting the list of matching DNSRBL
        Avoid performing multiple DNSRBL checks for the same IP
        Allow reusing in the ACL of properties gathered from urlcheck
        Fix wrong display of %Xc %Xe and %Xh substitutions
        Fix documentation: %Xh instead of %Xr
        stat should not report X-Greylist header is message was rejected
        Fix a documentation bug: sender e-mail is %f, not %s
3.1.6
        More format strings, for SMTP code, extended code, message, header
        Fix a crash when using regex without grouping ()
        Document the CVS location in README
3.1.5
        Fix double free when using content filtering
        ACL clauses can now be negated
        msg clauses in ACL can use format strings substitution
        Add an ACL clause to customize X-Greylist header
        Fix serious bugs in DNSRBL code (Jacques Beigbeder)
        Fix a display bug for netblocks
        Add a time clause to match against time sets
        GeoIP support, through the geoip clause
        Fix warning for Postfix build (Nerijus Baliunas)
        %g for substitution by regex back references
        %I for susbtitution by sender IP masked by a CIDR
        Allow specifying socket mode in config file
        Set default user to root for .spec file
        Do not drop root privs if we do not run as root
3.1.4
        Add support for switching to a given group (Ralf S. Engelschall)
        Add substitutions for sendmail macro and strftime in URL checks
        Add custom logs of milter-greylist actions, see stat in greylist.conf
        Fixes for running with Postfix (Nerijus Baliunas)
        Fix configure problem with DNSRBL on Linux (Andrew McGill)
        Document how to use milter-greylist with Postfix (Nerijus Baliunas)
        Update .spec for Postfix (Nerijus Baliunas)
        Optionnaly post the message body to an URL check at DATA stage
        Integrate SPF, SMTP AUTH and STARTTLS in ACL
        Add msgsize and rcptcount clauses to ACL
        Allow rcpt clause at DATA stage ACL
        Fix crashes when running with -D on some systems (John Thiltges)
        Do not sort the databases, it makes startup slow and buys nothing
3.1.3
        Fix various bugs (regex searches, lists matches, uninitialized memory)
3.1.2
        Fix timespamp on sparc64 (Gert Doering)
        Add the ability to query extrnal sources in ACL using URL
        Add connexion pools for URL queries to enable persistent connexions
        Fix dependency in rc-bsd.sh: it's mail, not sendmail
        Add DATA-stage ACL
        Update to .spec file (Rudy Eschauzier)
        Allow header and body searches in DATA-stage ACL
        Allow CIDR match for DNSRBL
        Allow multiple macro, dnsrbl, urlcheck, body and header clauses in ACL
3.1.1
        Fix crashes during dump reloads (AIDA Shinra)
        Fix DoS in MX sync protocol (AIDA Shinra)
        Check for -lc_r before others, for FreeBSD (AIDA Shinra)
        Fix configure for libspf2 (AIDA Shinra)
        Fix FreeBSD build (AIDA Shinra)
        Avoid buffer overflow in DNSRBL code (AIDA Shinra)
        Fix build problem with newer BerkeleyDB (AIDA Shinra)
        Check if -lbind requires -lpthread, for Linux
        Correctly enable non blocking I/O fox MX sync (Attila Bruncsak)
        Pointer to DRAC documentation (Matthias Scheler)
        FreeBSD build fixes (Hajimu UMEMOTO)
        Remove the /tmp/access-list.debug for security sake (AIDA Shinra)
        Fix display bug in log messages (AIDA Shinra)
        Updated the list of broken MTA
        Fix MX sync between Tru64 and Linux (Attila Bruncsak)
        Silly build fix for platforms that lack vsyslog()
        Honour LINE_MAX for syslog (Attila Bruncsak)
        Warn about ignored ACL lines after acl default rule
        Fix various race conditions (AIDA Shinra)
        Fix big bugs in macro support (AIDA Shinra)
        Fix build warning on Tru64
        Build fix on Solaris
        Documentation fix

-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu@...

Tere.

I haven't read this list for a while and using right now version 3.1.8, 
I'd like to give try to latest version, but is there some upgrade guide, 
or should I just compile it like before:

./configure --with-user=smmsp --without-db --without-drac-db 
\                                                    
    --with-conffile=/etc/mail/greylist.conf 
--with-dumpfile=/etc/mail/greylist.db \                               
    --bindir=/usr/local/sbin --enable-dnsrbl --with-libbind=/usr/lib

And using same greylist.conf (or some syntax did change?) everything 
should run well?

-- 
Mart

Mart Pirita <sysadmin@...> wrote:

> I haven't read this list for a while and using right now version 3.1.8,
> I'd like to give try to latest version, but is there some upgrade guide,
> or should I just compile it like before:
> 
> ./configure --with-user=smmsp --without-db --without-drac-db 
> \                                                    
>     --with-conffile=/etc/mail/greylist.conf 
> --with-dumpfile=/etc/mail/greylist.db \
>     --bindir=/usr/local/sbin --enable-dnsrbl --with-libbind=/usr/lib

No configure flag has been removed or altered in its semantics, so it
should work.
 
> And using same greylist.conf (or some syntax did change?) everything 
> should run well?

greylist.conf has evolved, but backward compatibility has been
maintained, so you can just drop your old file and it will work. Check
the man page for the new features. 

The biggest change is acl being replaced by racl (RCPT-stage ACL) and
dacl (DATA-stage ACL). acl is now a synonym for racl so that older
config keep working as they did before.

-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu@...

Tere.
>
> No configure flag has been removed or altered in its semantics, so it
> should work.
>   

Ok.
>  
>   
> greylist.conf has evolved, but backward compatibility has been
> maintained, so you can just drop your old file and it will work. Check
> the man page for the new features. 
>
> The biggest change is acl being replaced by racl (RCPT-stage ACL) and
> dacl (DATA-stage ACL). acl is now a synonym for racl so that older
> config keep working as they did before.
>
>   
Great, compiled and it's running with old setup well so far. Btw I 
didn't find from conf files any new options, but there was talk about 
blocking pdf -s, helo etc.

-- 
Mart

Mart Pirita <sysadmin@...> wrote:

> Great, compiled and it's running with old setup well so far. Btw I 
> didn't find from conf files any new options, but there was talk about
> blocking pdf -s, helo etc.

Everything is in the man page. Indeed we need new examples.

-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu@...

manu@... writes:

> Here is 4.0rc1

two issues:

* using '%E*' in 'stat' returns bogus (unitialized?)  values for
  'tempfail' actions

* the ./configure checks for e.g. libbind and curl are setting RPATHs
  (-Wl,r...) and are testing for these libs in hardcoded <path>/lib
  directories.  At least under Linux, RPATHs for standard directories
  (e.g. /usr/lib) are heavily deprecated and <path>/lib will not work
  for mulitlib archs where libs are located in <prefix>/lib64.

  I suggest to use pkgconfig for libbind and curl detection? E.g.

  | PKG_CHECK_MODULES(CURL,libcurl,
  |                   [CFLAGS="$CPPFLAGS -DUSE_CURL $CURL_CFLAGS"],
  |                   [... current checks ...])



Then:

would it be possible to add a new format specifier which prints out the
current line? This can be useful for 'stat' to calculate the real ACL on
dynamic lists. E.g. when adding entries to the list in

| list "...." {
|    ...
| }
| 
| stat "..."
| 
| acl ...

both 'stat' and 'acl' will move by the same number of lines and the
matching acl can be identified by the difference of these numbers.

Alternatively, acls could be given an unique id to identify them in
'stat' messages.



Enrico

On Mon, Oct 15, 2007 at 01:23:22PM +0200, Enrico Scholz wrote:
> * using '%E*' in 'stat' returns bogus (unitialized?)  values for
>   'tempfail' actions

I'll look into it later this evening.

> * the ./configure checks for e.g. libbind and curl are setting RPATHs
>   (-Wl,r...) and are testing for these libs in hardcoded <path>/lib
>   directories.  At least under Linux, RPATHs for standard directories
>   (e.g. /usr/lib) are heavily deprecated and <path>/lib will not work
>   for mulitlib archs where libs are located in <prefix>/lib64.
> 
>   I suggest to use pkgconfig for libbind and curl detection? E.g.
> 
>   | PKG_CHECK_MODULES(CURL,libcurl,
>   |                   [CFLAGS="$CPPFLAGS -DUSE_CURL $CURL_CFLAGS"],
>   |                   [... current checks ...])

What happens if pkgconfig is not availabled? It's not on my systems?
Whould you contribute a patch for integrating that check?

> Alternatively, acls could be given an unique id to identify them in
> 'stat' messages.

Somethign such as this?
racl blacklist dnsrbl "foo" msg "go away" aclid "foo-dnsrbl" 

And you'd get a format string for getting the id?

-- 
Emmanuel Dreyfus
manu@...

Thanks for the hard work, Emmanuel, on getting this out.  I have
one feedback.

I installed the rc1 and it worked just fine (I was running 3.1.6 before)
but there was one issue with creating the socket:
           /var/milter-greylist/milter-greylist.sock

For some reason, /var/milter-greylist was owned by root and in the
group smmsp.  I had to change the owner of the /var/milter-greylist
to smmsp and then it was all OK.  The version I was running before
the rc1 obviously was OK with the owner being root (I have not
verified).

Regards.
-- 
Harish Pillay h.pillay@... gpg id: 74609E3
fingerprint: F7F5 5CCD 25B9 FC25 303E 3DA2 0F80 27DB 7468 09E3

On Mon, Oct 15, 2007 at 10:19:42PM +0800, Harish Pillay wrote:
> For some reason, /var/milter-greylist was owned by root and in the
> group smmsp.  I had to change the owner of the /var/milter-greylist
> to smmsp and then it was all OK.  The version I was running before
> the rc1 obviously was OK with the owner being root (I have not
> verified).

I guess you used configure --with-user with your previous install and
you did not use it for 4.0rc1. Right?

-- 
Emmanuel Dreyfus
manu@...

> > For some reason, /var/milter-greylist was owned by root and in the
> > group smmsp.  I had to change the owner of the /var/milter-greylist
> > to smmsp and then it was all OK.  The version I was running before
> > the rc1 obviously was OK with the owner being root (I have not
> > verified).
>
> I guess you used configure --with-user with your previous install and
> you did not use it for 4.0rc1. Right?

It was done earlier this year and my notes do not suggest using any other
options when I ran ./configure.  In general, I try to keep the default options
and document (WhatIDid<date>.txt) when I do use something out of the
ordinary. But, it might as well have been and it would point to a lapse in
my record keeping.  So, you could be right.

Thanks.
-- 
Harish Pillay h.pillay@... gpg id: 74609E3
fingerprint: F7F5 5CCD 25B9 FC25 303E 3DA2 0F80 27DB 7468 09E3

On Mon, Oct 15, 2007 at 01:23:22PM +0200, Enrico Scholz wrote:
> * using '%E*' in 'stat' returns bogus (unitialized?)  values for
>   'tempfail' actions

Can you send me a trimmed down version of the setup that shows that
behavior?

-- 
Emmanuel Dreyfus
manu@...

Enrico Scholz <greylist-milter@...> wrote:

> would it be possible to add a new format specifier which prints out the
> current line? This can be useful for 'stat' to calculate the real ACL on
> dynamic lists. E.g. when adding entries to the list in

I've written a patch for adding optionnal acl id strings, like this:
racl "foo" greylist default

The "foo" keyword can be substituted from %a in format strings.

Would you give it a try?
-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu@...