Yahoo Groups archive

Milter-greylist

Index last updated: 2026-04-28 23:32 UTC

Thread

Enhancement / feature requests

Enhancement / feature requests

2007-09-30 by Michael Mansour

Hi,

I have two feature requests:

1. perform a consistency check on the greylist.conf file before reloading it.

I made a rule mistake today when modifying the greylist.conf file, and found
that when it auto-reloaded, the greylist was erroring in my maillog as not
functioning any more.

It would be good for milter-greylist to first check that the greylist.conf
file makes sense to it before a reload, and if it doesn't (contains errors)
then not auto-reload it but notifying about the error in the maillog (or email
the admin).

The way it is now is, it's easy to break with any admin keying the wrong
sequence of acl's or even typo's.

2. have the ability to support an external whitelist file.

I've recently started looking at www.dnswl.org which provides a whitelist of
sites that are "trusted" and should not be greylisted.

Obviously it wouldn't make much sense to use their full IP list and import it
into the greylist.conf file, but it would make sense if I could have in the
greylist.conf file something like:

include dnswl.whitelist

so milter-greylist could then load the whitelist whenever it's updated (or
have milter-greylist have the ability to query the dnswl site to check if an
IP is whitelisted and not greylist if it is).

3. have a web interface to the whitelist system.

It would be good to have a web interface for users to be able to access their
own whitelist system so they can enter "from" and "to" etc rules to manage
their greylist. Maybe even keep these in a MySQL database for each management?

4. tools to pull out relevant information from the /var/log/maillog file.

I'm going to start creating some tools myself for this, but the first tool I'd
like is to get emailed periodically for any "Delayed for" greater than
01:00:00 so that I can look into the problem or simply whitelist the sender.

Other tools like summaries of statistics etc, to pull the information out of
/var/log/maillog and summarise any information available.

Any comments are welcome.

Michael.

Re: [milter-greylist] Enhancement / feature requests

2007-09-30 by Hokan

On Mon, Oct 01, 2007 at 02:56:35AM +1000, Michael Mansour wrote:
> 
> 2. have the ability to support an external whitelist file.
> 
> I've recently started looking at www.dnswl.org which provides a whitelist of
> sites that are "trusted" and should not be greylisted.
> 
> Obviously it wouldn't make much sense to use their full IP list and import it
> into the greylist.conf file, but it would make sense if I could have in the
> greylist.conf file something like:
> 
> include dnswl.whitelist
> 
> so milter-greylist could then load the whitelist whenever it's updated (or
> have milter-greylist have the ability to query the dnswl site to check if an
> IP is whitelisted and not greylist if it is).

If your list is DNS based, then you can use the "dnsrbl" feature.  I
haven't done this yet, but will soon.  I think this would work:

dnsrbl "DNSWL" list.dnswl.org 127.0.0.2
acl whitelist dnsrbl "DNSWL"

Re: [milter-greylist] Enhancement / feature requests

2007-09-30 by manu@netbsd.org

Michael Mansour <mic@...> wrote:

> 1. perform a consistency check on the greylist.conf file before reloading it.

I use to run milter-greylist -cf on the new file before moving it to
/etc/mail.

I agree that testing the config before reloading could be useful, but it
can also be quite time and memory consuming for sites that have very
large config file. There should probably be an option for that.
 
> 2. have the ability to support an external whitelist file.
> 
> I've recently started looking at www.dnswl.org which provides a whitelist of
> sites that are "trusted" and should not be greylisted.

Configure it as a DNSRBL and use it in a whitelist rule, you'll have
real time DNS lookups on this DNSWL.

Perhaps the dnsrbl keyword needs to be changed into "dnsl" so that
people understand it can be used for any purpose (blacklist or
whitelist)?

> 3. have a web interface to the whitelist system.
> 
> It would be good to have a web interface for users to be able to access their
> own whitelist system so they can enter "from" and "to" etc rules to manage
> their greylist. Maybe even keep these in a MySQL database for each management?

At work, I store per-user settings in a LDAP directory. milter-greylist
lookus up the directory using an urlcheck clause with an ldap:// URI.

I've a few web forms for user to enter their personnal white list, black
list, the DNSRBL they want to use, and the delays they want for
greylisting. They can also enable some content filter implemented by
milter-greylist: no HTML with embedded GIF, no PDF files smaller than
30k, and so on.
 
> 4. tools to pull out relevant information from the /var/log/maillog file.

Did you had a look at what can be done with the stat keyword? I use it
to provide a web-based per-user log of what hapened to each e-mail:
rejected, delayed or accepted, with the sender and the reasons for the
decision.

-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu@...

Re: {Disarmed} Re: [milter-greylist] Enhancement / feature requests

2007-10-01 by Michael Mansour

Hi Manu,

> Michael Mansour <mic@...> wrote:
> 
> > 1. perform a consistency check on the greylist.conf file before reloading it.
> 
> I use to run milter-greylist -cf on the new file before moving it to
> /etc/mail.
> 
> I agree that testing the config before reloading could be useful,
>  but it can also be quite time and memory consuming for sites that 
> have very large config file. There should probably be an option for that.

Yes. Admittedly I didn't know about the -cf option and ran it and it ran
through pretty quickly.

I agree if this was included (which I still think it should be because it's
way too easy to break the setup with an incorrect rule), then it should also
include the option to turn it off for people with large config files.

> > 2. have the ability to support an external whitelist file.
> > 
> > I've recently started looking at www.dnswl.org which provides a whitelist of
> > sites that are "trusted" and should not be greylisted.
> 
> Configure it as a DNSRBL and use it in a whitelist rule, you'll have
> real time DNS lookups on this DNSWL.
>
> Perhaps the dnsrbl keyword needs to be changed into "dnsl" so that
> people understand it can be used for any purpose (blacklist or
> whitelist)?

Hmm.. good point. I'm still new to milter-greylist and didn't realise I could
configure this in the greylist.conf file (just read the man page). I'll make
sure I compiled with the dnsrbl enabled (I built milter-greylist to an RPM and
installed from there) and see if I can figure out how to use the feature.

> > 3. have a web interface to the whitelist system.
> > 
> > It would be good to have a web interface for users to be able to access their
> > own whitelist system so they can enter "from" and "to" etc rules to manage
> > their greylist. Maybe even keep these in a MySQL database for each management?
> 
> At work, I store per-user settings in a LDAP directory. milter-greylist
> lookus up the directory using an urlcheck clause with an ldap:// URI.
> 
> I've a few web forms for user to enter their personnal white list, black
> list, the DNSRBL they want to use, and the delays they want for
> greylisting. They can also enable some content filter implemented by
> milter-greylist: no HTML with embedded GIF, no PDF files smaller than
> 30k, and so on.

Ok, I can see there's alot I can do in this regard. None of these types of
things (instructions or scripts or forms) are available anywhere on the net
are they?

> > 4. tools to pull out relevant information from the /var/log/maillog file.
> 
> Did you had a look at what can be done with the stat keyword? I use 
> it to provide a web-based per-user log of what hapened to each e-
> mail: rejected, delayed or accepted, with the sender and the reasons 
> for the decision.

I've search through man pages for both greylist.conf and milter-greylist, I
can't find any references to the "stat" keyword?

I'm using milter-greylist 3.0

Thanks.

Michael.

> -- 
> Emmanuel Dreyfus
> http://hcpnet.free.fr/pubz
> manu@...
------- End of Original Message -------

Re: {Disarmed} Re: [milter-greylist] Enhancement / feature requests

2007-10-01 by manu@netbsd.org

Michael Mansour <mic@...> wrote:

> Ok, I can see there's alot I can do in this regard. None of these types of
> things (instructions or scripts or forms) are available anywhere on the net
> are they?

The man page is a good start, feel free to write an
introduction/tutorial/howto once you'll have experimented:

From greylist.conf(5):

URL checks
       milter-greylist(8)  is  able  to  query external sources of
       information through various URL, if it was built with
       --with-libcurl.  Here  is  an example:

                urlcheck "glusr" \
                          "http://www.example.net/mgl-config?rcpt=%r" 5

                racl greylist urlcheck "glusr" delay 15m
                racl whitelist  default

       The trailing 5 at the end of the urlcheck  definition  is  the
       maximum number  of  simultaneous  connexions we want to launch on
       this URL. For each message, the URL will be querried, with  %
       format tags being  sub- tituted. For instance, %r is substituted
       by the recipient. See the FORMAT STRINGS section for the
       complete list of substitutions.

       milter-greylist(8) expects an answer containing a list of \n
       terminated lines, with key: value pairs. The most basic answer to
       get a match is:

                milterGreylistStatus: Ok

       TRUE can be used as an alias for Ok here. The  answer  can  be
       more complex, with keys that will overload the ACL settings:

       milterGreylistDelay The greylisting delay to use (time unit
       suffix allowed).

       milterGreylistAutowhite The autowhite delay to use (time unit
       suffix allowed).

       milterGreylistFlushAddr The value is ignored. If  this  key  is
       present,  then  the  IP address for the sender machine will be
       flushed from greylist and autowhite databases.

       milterGreylistCode The SMTP code to return (e.g.: 551).

       milterGreylistECode The SMTP extended code to return (e.g.:
       5.7.1)

       milterGreylistMsg The string to return with SMTP codes.

       milterGreylistReport The string to display in the X-Greylist
       header.

       milterGreylistIgnore This line will be ignored, without warnings
       in the logs.

       milterGreylistAction This feature is nifty but use it with
       caution, as it  makes  the access  list  a  bit  difficult to
       understand. By specifying the values greylist, whitelist, or
       blacklist,  it  is  possible  to overload the ACL action itself.

       The ACL will match if any of the above key is returned:
       milterGreylistStatus is not mandatory.

       If you use an URL check in a DATA stage ACL, you can post  the
       message header  and body to the URL. This is done by appending
       the postmsg keyword to the urlcheck statement, like in the
       example above:

                urlcheck "extfilter"  \
                       "http://www.example.net/f.cgi" 5 postmsg

                dacl blacklist urlcheck "extfilter" whitelist default


       It is also possible to gather the properties returned by  the
       URL  and reuse  them in the ACL. This behavior is enabled by the
       getprop keyword at the end of urlcheck definition. If this option
       is enabled, the gath- ered  properties  can  be  accessed in the
       current and following ACL by prefixing them by a dollar ($).  If
       the clear keyword  is  added,  then properties  will  be  cleaned
       up before handling a new recipient. This avoids properties for
       several recipients to  mix.   Finally,  the  fork keyword
       instructs  milter-greylist(8)  to  fork a separate instance of
       itself for performing the queries. Use  it  if  you  encounter
       thread-safety problems.  fork is not compatible with postmsg.

       Here  is  an  example  that will use various DNSRBL depending on
       a per-recipient setting stored in the dnsrbl attribute of a LDAP
       directory.

                dnsrbl "RBL2" "rbl.example.net" "127.0.0.2" 
                dnsrbl "RBL3" "rbl.example.net" "127.0.0.3" 
                dnsrbl "RBL4" "rbl.example.net" "127.0.0.4" 
                urlcheck  "userconf" \
"ldap://localhost/dc=example,dc=net?milterGreylistStatus,dnsrbl?one?mail=%r" \
            5 getprop clear

                racl  blacklist  urlcheck  "userconf"  \
                     $dnsrbl  "RBL2" dnsrbl "RBL2" 
                racl blacklist $dnsrbl "RBL3" dnsrbl "RBL3"
                racl blacklist $dnsrbl "RBL4" dnsrbl "RBL4"

       Note that when matching gathered properties, format strings  and
       regex can be used.
 
> > > 4. tools to pull out relevant information from the /var/log/maillog file.
> > Did you had a look at what can be done with the stat keyword? I use 
> > it to provide a web-based per-user log of what hapened to each e-
> > mail: rejected, delayed or accepted, with the sender and the reasons 
> > for the decision.
> I've search through man pages for both greylist.conf and milter-greylist, I
> can't find any references to the "stat" keyword?
> 
> I'm using milter-greylist 3.0

Perhaps it's time to upgrade?

From 4.0b2's greylist.conf(5)

CUSTOM REPORTS
       The  stat  keyword  can  be used to specify a custom report for
       milter-greylist activity.  It should be supplied with an output
       (either  file or external command) and a format string. Here is
       an example:

                stat ">>/var/log/milter-greylist.log" \
                       "%T{%T},%i,%f,%r,%A\n"

       If  the output starts by >> or > then it is a file. Use >> to
       append to an existing file, and use > to overwrite it. If the
       output starts by  a | then the output is a shell command, like in
       the example below:

                stat "|logger -p local7.info" "%T{%T},%i,%f,%r,%A\n"

       The format string gets substituted as URL checks format string:
       %r gets substituted by the recipient, %f by the sender, and so
       on. See the FORMAT STRINGS section for a complete list of
       available substitutions.

-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu@...

Move to quarantaine

This moves the raw source file on disk only. The archive index is not changed automatically, so you still need to run a manual refresh afterward.