Milter-greylist

--- In milter-greylist@yahoogroups.com, manu@... wrote:
> It seems we have various tests we want to perform against a domain's
> MXes. Let's try to figure a configuration syntax general enough to avoid
> the thing getting too messy.
> 
> Two ideas that need some feedback and improvement:
> 
> First idea:
> Existing tests such as DNSRBL can be extended to work on various data:
> sender IP, sender DNS, sender MX IP, sender MX DNS, and so on.
> 
> We could use format strings to help DNSRBL cover a larger range of
> situations. We already have:
> %i      sender IP address
> %d      sender DNS address
> %mf     sender e-mail domain
> We could add:
> %xi     MX IP address
> %xd     MX DNS address
> %n{}    a reversed dot-separated item. eg: if sender IP address is
> 10.1.2.3, %n{%i} gives you 3.2.1.10.
>  
> Then, DNSRBL definitions can be improved like this:
> dnsrbl "foo" "%n{%i}.dnsrbl.example.com" 127.0.0.0/8
> 
> You'll note that this syntax accomodate the use of either DNSRBL or
> RHSRBL, based on whatever you want: sender machine, MX, sender domain,
> receiver domain.
> 
> Given that it would not work only for DNSRBL, perhaps it's time to
> rename it (it would make config parsing a bit easier), and make it even
> more general by allowing the DNS query type to be specified:
> dnslookup "foo" "%n{%i}.dnsrbl.example.com" A 127.0.0.0/8
> 
> Problem to solve: for a given domain, MX is usually not unique. The %x
> thing quickly tend to be difficult to define correctly. Suggestions
> welcome.
> 
> 
> Second idea:
> Clauses are about matching a condition. We have different kind of tests,
> with different syntaxes:
> rcpt foo@...    variable, value 
> rcptcount >= 2          variable, operator, value
> dnsrbl "foo"            reference to a condition defined earlier.
> 
> Moreover, we have variables that can be set by the urlcheck clause
> $foo >= 2
> 
> There is room for unifying a lot of things. instead of writing
> rcpt foo@...
> we could write
> $rcpt == "foo@..."
> 
> Of course that looks heavier, but that opens the way for adding more
> operators (== for exact match, =~ for regexp match, <= for substring
> match), and we can think about adding functions and logical operators.
> 
> Example:
> dns(dns(machine($rcpt), "MX"), "A") >= $addr
> 
> - $rcpt would be set up as the recipient e-mail by milter-greylist
> - machine($rcpt) would be the machine part of $rcpt (after the @), that
> is: the sender address domain
> - dns(machine($rcpt), "MX") would give us the comma separated list of MX
> for the sender address domain
> - dns(dns(machine($rcpt), "MX"), "A") would give us the comma separated
> list of IP addresses for the sender address domain MX
> - $addr would be set up as the sender machine IP by milter-greylist
> - and the >= operator would check if $addr is a substring of the list of
> IP addresses for the sender address domain MX.
> 
> 
> 
> That's just rough ideas, I'm not sure there is really something to do
> with it. What do you think?
> 

This sounds quite powerful, but at the expense of config file readability.

So what about :

- have a powerful and generic engine and syntax as you proposed;
- and ability to define macros (or functions), some of the being
predefined: DNSRL, RHSL, BADMX, SENDERISMX...


Some of the basic features which may be useful :

1) predefined variables for all directly usable parameters (some of
the already implemented): sender IP, sender DNS name, envelope from
address, sender HELO string...

2) generic DNS query :
    dns(query,type)
    ex: dns("domaine.com","MX").

this function :
- may return one or several records; in case of multiple responses,
they should be treated as such, for example in an array, and not
concatenated in one string, to ease further treatment.
- should accept one or more queries of the same type ("query" shoud
accept an array): in case of multiple queries, all the responses of
all queries should be merged in the resulting array.
- if type "A" is specified, should also report "AAAA" responses; or we
could define a special type "A+" which returns both, or accept an
array of types : dns("mydomain.com",["A","AAAA"]);

3) address domain extraction (as explained in your post):
    domain("toto@...") -> "domain.com"

As above, ability to handle multiple arguments may be useful:
    domain("toto@domain.com","zozo@...") ->
["domain.com","other.mail"]

This same function, or another, may be used to strip the host name of
an FQDN:
    domain("machine.domain.com") -> "domain.com"

or maybe more generic substring extraction fonctions should be used ?
(à la Perl, or bash, or sed "s/^.*@//"...)

3bis) domain concatenation:

    addom(["toto","titi"],"mydomain.com") ->
["toto.mydomain.com","titi.mydomain.com"]

4) IP reversing (not generic string manipulation, RFC-dependent for IPv6):

    iprev(192.168.1.3) -> 3.1.168.192
    iprev(2001:660:3305::12:2f) ->
f.2.0.0.2.1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.5.0.3.3.0.6.6.0.1.0.0.2

here also multiple arguments may be accepted.

5) various specific operators:

- regexp matching : $string =~ /abc.*de/
- string equality : $string == "abcde"
- substring matching: $string >= "abc"
- domain matching : $fqdn indomain domain.com
- IP range matching : $ip inrange 192.168.0.10/24 (without specifying
a CIDR mask, this would be an exact IP match, similar to string
equality, except it would accomodate for various IP syntaxes:
"2001:660:3305:0:0:0:0:22 inrange 2001:660:3305::22" should match)

In case arguments are multiple, these operators should match if *any*
of the arguments match. If useful, one could also add a modifier to
force matching *all* arguments: "$ips all inrange 192.168.0.10/24".

6) a syntax to define fonctions (macros) :

define badMX { dns(dns(domain(%f),"MX"),"A") inrange [192.168.0.0./16,
127.0.0.0/8, ::1/128 ...] }

define DNSL(%suffix,%range) { dns(addom(iprev(%i),%suffix),"A")
inrange %range }

define RHSL(%suffix,%range) { dns(addom(%d,%suffix),"A") inrange %range }


Of course above syntax is only an example.
But is all that worth doing in milter-greylist ? Aren't we targeting
too big ? Maybe implementing missing features (with simplified syntax
and hardcoded arguments) would suffice. Some features like badMX could
be done by other means (sendmail 8.14 handles this); the only (but
serious) advantage having it in milter-greylist is we can integrate it
in ACL system and do complex things with it (weighted greylisting,
multiple conditions...).