Milter-greylist

Hi, I am currently testing milter-greylist to reduce spam received by the university of Versailles.  The results of the tests are quite good but 

Remy Card <Remy.Card@...> wrote:

>       The patch against milter-greylist 1.5.8 is enclosed.  Comments are
> welcome!

Well we started talking about this by private e-mail...

It's being a long time I had the envy of replacing the testmode by
something more general. My idea was to implement access lists. Something
like this:

rcpt-acl bypass /.*@.../ 
rcpt-acl greylist /.*@.../
rcpt-acl greylist /webmaster@.*.example.com/
rcpt-acl bypass webmaster@...

And last match win. Default is to greylist anything, except if testmode
is on.

testmode and rcpt would still be available but would be deprecated in
favor of rcpt-acl. testmode would be equivalent to starting the config
by rcpt-acl bypass /.*/

For the future, we could add sub access-lists (a la head/group in
IPfilter), but that can be kept for later.

The problem I have with that approach is the relation to the addr,
domain and from keywords. addr-acl domain-acl and from-acl could be
added, but should we have a global ACL for all keywoards, or a
per-keyword ACL. Given that rcpt is used quite differently than addr,
domain and from, a per-keyword ACL could be better. I'd like opinions
about this.

The difference with your norcpt proposal is that you stop on the first
match, as I understand. I think it could be interesting to build ACLs.

Opinions?

-- 
Emmanuel Dreyfus
Il y a 10 sortes de personnes dans le monde: ceux qui comprennent 
le binaire et ceux qui ne le comprennent pas.
manu@...

On Wed, 29 Sep 2004, Remy Card wrote:
> 	The patch against milter-greylist 1.5.8 is enclosed.  Comments are
> welcome!

I've implemented a patch which is even more specific (and flexible, for 
ISP use):

You can regex on sender + recipient pair. For example a user who wants 
everything _except specific senders_ to be greylisted.

So you can regex either:

rcpt /user@.../

or

rcpt /grandma@...,user@.../

In the former example, user@... would not greylist anything, everyone 
would get through without grelisting. In the latter example, only 
grandma@... is whitelisted, everyone else sending mail to 
user@... would be greylisted.

Regex works on both sender or receiver in the rcpt clause.

This is the way we've implemented it on our ISP, to accomodate user's 
individual whitelists. In most cases users want greylisting except for 
specific senders.

-Dan

On Thu, Sep 30, 2004 at 01:54:34AM -0700, Dan Hollis wrote:
> rcpt /grandma@...,user@.../
> 
> In the former example, user@... would not greylist anything, everyone 
> would get through without grelisting. In the latter example, only 
> grandma@... is whitelisted, everyone else sending mail to 
> user@... would be greylisted.
> 
> Regex works on both sender or receiver in the rcpt clause.

The feature is nice and desirable but I have concerns with the syntax. 
rcpt means recipient, it sounds weird to add the sender. What about introducing
a new keyword? (it would probably make the parser simplier)

Another starnge thing is that I'd expect /grandma@...,user@.../
to be single regex, what about this? /grandma@.../ /user@.../

We could do something like this:
rcptfrom grandma@... user@...
rcptfrom grandma@... /.*/

I'm not happy with "rcptfrom", I think we can find better. What do you think?

Another problem is that it does not cope very well with the rcpt-acl idea.
Maybe we can come to a much more general acl syntax. Anyone has a proposition?

-- 
Emmanuel Dreyfus
manu@...

Emmanuel Dreyfus <manu@...> writes:

> Another problem is that it does not cope very well with the rcpt-acl
> idea.  Maybe we can come to a much more general acl syntax. Anyone has
> a proposition?

what about something like:

#general parameters

timeout 4d
quiet


some-recipien-domain.com {
    addr X.Y.Z.T    
    rcpt john.doe
    from friend@...
    domain spamfree.org   
}

?

On Thu, Sep 30, 2004 at 03:38:29PM +0200, Cyril Guibourg wrote:
> some-recipien-domain.com {
>     addr X.Y.Z.T    
>     rcpt john.doe
>     from friend@...
>     domain spamfree.org   
> }

That does not address the need Dan has, does it? He wants to greylist
specific sender/reciever pairs.

-- 
Emmanuel Dreyfus
manu@...

Emmanuel Dreyfus <manu@...> writes:

> That does not address the need Dan has, does it? He wants to greylist
> specific sender/reciever pairs.

What about fromto: <from> <rcpt> ?

On Thu, Sep 30, 2004 at 03:38:29PM +0200, Cyril Guibourg wrote:
> Emmanuel Dreyfus <manu@...> writes:
> 
> > Another problem is that it does not cope very well with the rcpt-acl
> > idea.  Maybe we can come to a much more general acl syntax. Anyone has
> > a proposition?

Squid has already invented this particular wheel, yes?

from squid.conf (with changes)...

#  TAG: acl
#       Defining an Access List
#
#       acl aclname acltype string1 ...
#       acl aclname acltype "file" ...
#
#       when using "file", the file should contain one item per line
#
#       acltype is one of the types described below
#
#       By default, regular expressions are CASE-SENSITIVE.  To make
#       them case-insensitive, use the -i option.
#
#       acl aclname src      ip-address/netmask ... (clients IP address)
#       acl aclname src      addr1-addr2/netmask ... (range of addresses)
#
#       acl aclname srcdomain   .foo.com ...    # sender domain match
#       acl aclname srcsender   grandma ...    # exact match on sender 
#       acl aclname dstdomain   .foo.com ...    # Destination server
#       acl aclname dstrecip    jimmy ...    # Exact match on recipient

#       acl aclname srcdom_regex [-i] xxx ...   # regex matching sender domain
#       acl aclname dstdomain_regex [-i] xxx ...   # regex matching recipient domain
#       acl aclname srcsender_regex [-i] xxx ...   # regex matching sender name
#       acl aclname dstrecip_regex [-i] xxx ...   # regex matching recipient

So then after defining your acls,

action whitelist aclname1
action whitelist aclname2
...
action greylist all

results in a "non-testing" mode for a "first match wins" scenario.

This lets you get away from the testing / non-testing mode, as you can
just alter the action for the "all" acl at the end to switch modes.

Of course, this makes your configuration hideously complex :]

Graham

Graham Dunn <gdunn@...> wrote:

> Squid has already invented this particular wheel, yes?

Good, I hate re-inventing the wheel.
(snip)

> Of course, this makes your configuration hideously complex :]

You don't *have* to use the complex features, but it's good to have them
available:

So we'd have a first match list of greylist/whitelist keywords, each
using an ACL using a last match basis. Here is a complete proposal, I'd
appreciate if people could comment:

First, a few examples...

# first config, Dan's setup:
acl grandma {
        rcpt grandma@...
        not from friend@...
        not from another.friend@...
}
greylist grandma
whitelist default



# Second config, Remy's setup:
acl foo {
        rcpt .*@...
        not rcpt postmaster@...
}
greylist foo 
whitelist default



# And finaly, Manu's setup:
acl myusers {
        rcpt user1@...
        rcpt user2@...
        rcpt user3@...
}
acl mywhitelist {
        addr 193.54.0.0/16
        domain friendly.com
}
whitelist mywhitelist
greylist myusers
whitelist default 





default is an internally defined ACL that match anything.

It seems to me that it matches all the needs that have been described
here so far. Now, let's try to be more formal:

For each {greylist|whitelist} keyword {
        for each entry in its ACL {
                if it's addr and the IP doesn't match
                        break out of the ACL loop
                if it's domain and the domain doesn't match
                        break out of  the ACL loop
                if it's from and the sender doesn't match
                        break out of  the ACL loop
                if it's rcpt and the recipient doesn't match
                        break out of  the ACL loop
        }
        if all entries in the ACL matched or if it was void {
                retain the {greylist|whitelist} decision 
                exit the keyword loop
        }
}



The last problem is to convert the older keyword to the new framework so
that we can retain backward compatibility in the config file.

"rcpt X" in test mode will be interpreted as
acl X {
        rcpt X
}
greylist X


"rcpt X" in "normal" mode will be interpreted as
acl X {
        rcpt X
}
whitelist X
 

"{addr|from|domain} X" turns into
acl X {
        {addr|from|domain} X
}
whitelist X


Test mode turns into adding "whitelist default" at the end of the config
Normal mode turns into adding "greylist default" at the end of the
config.


We would tag as obsolete all the old syntax, but we will keep it working
for the sake of backward compatibility (did I ever explained I hated
software that broke my config files? :-) 


Opinions? 

-- 
Emmanuel Dreyfus
Il y a 10 sortes de personnes dans le monde: ceux qui comprennent 
le binaire et ceux qui ne le comprennent pas.
manu@...

On Thu, Sep 30, 2004 at 10:53:16PM +0200, manu@... wrote:
> # And finaly, Manu's setup:
> acl myusers {
>         rcpt user1@...
>         rcpt user2@...
>         rcpt user3@...
> }
> acl mywhitelist {
>         addr 193.54.0.0/16
>         domain friendly.com
> }
> whitelist mywhitelist
> greylist myusers
> whitelist default 


That's wrong. Because an ACL has to match all rules to get selected, my setup 
would be:

acl user1 {
        rcpt user1@...
}
acl user2 (
        rcpt user2@...
}
acl user3 {
        rcpt user3@...
}
acl mywhitelist {
        addr 193.54.0.0/16
        domain friendly.com
}
whitelist mywhitelist
greylist user1
greylist user2
greylist user3
whitelist default 

Of course that makes my config file much more heavyweigth.
-- 
Emmanuel Dreyfus
manu@...