Milter-greylist

Hi 

Now that the scallability problem has been somewhat handled, I think about
pushing a new stable release. Of course this means testing latest 1.3.x
code a bit, and before that, implementing the most-wanted-but-easy-to-implement
features. Complicated stuff will be worked on after 1.4 will be out.

So, what's the most-wanted-but-easy-to-implement features? On the to do 
list, I have:
1) DNS support in greylist.conf
The idea is to whitelist an IP address or a domain. Example:
domain /.*netbsd\.org/

This could be implemented by doing a gethostbyaddr() on the sender IP 
address and checking the enaswers (primary address and aliases) against
the regex. That does not look very complicated, but I suspect we can run 
into DNS resolver thread-safety and performance slowdown because of DNS 
lookups. Maybe this should be dealt after 1.4

2) per-user configuration
Definitively an intrusive modification, for after 1.4

3) config file option for greylist entries timeout
Already done for upcoming 1.3.4

More ideas? Opinions? Thoughts? 

-- 
Emmanuel Dreyfus
manu@...

Emmanuel Dreyfus <manu@...> writes:

> Hi

Hello,

> So, what's the most-wanted-but-easy-to-implement features? On the to do 
> list, I have:
> 1) DNS support in greylist.conf
> The idea is to whitelist an IP address or a domain. Example:
> domain /.*netbsd\.org/
>
> This could be implemented by doing a gethostbyaddr() on the sender IP 
> address and checking the enaswers (primary address and aliases) against
> the regex. That does not look very complicated, but I suspect we can run 
> into DNS resolver thread-safety and performance slowdown because of DNS 
> lookups. Maybe this should be dealt after 1.4

Imho this could end into another issue if for some reason reverse resolution
is broken for a while (NXDOMAIN ?) how would whitelist code react then ?

> More ideas? Opinions? Thoughts? 

4) Per (virtual-)domain config ?

On Wed, May 26, 2004 at 11:46:10AM +0200, Cyril Guibourg wrote:
> > This could be implemented by doing a gethostbyaddr() on the sender IP 
> > address and checking the enaswers (primary address and aliases) against
> > the regex. That does not look very complicated, but I suspect we can run 
> > into DNS resolver thread-safety and performance slowdown because of DNS 
> > lookups. Maybe this should be dealt after 1.4
> Imho this could end into another issue if for some reason reverse resolution
> is broken for a while (NXDOMAIN ?) how would whitelist code react then ?

It would just not whitelist, and the message would go through greylisting.
Not something really horrible.

> > More ideas? Opinions? Thoughts? 
> 4) Per (virtual-)domain config ?

Oh, yes, good idea. In fact, per-user config could be made more general:
per recipient config, where the recipient can be given as a regex. That
way you can give a domain if you want.

That's something for releases beyond 1.4: too intrusive. But we can 
still discuss the user interface. What about this?

rcpt /.*example\.com$/ {
	lazyaw
	report none
	autowhite 3d
}

One problem to solve: how to handle multiple matches?

-- 
Emmanuel Dreyfus
manu@...

Emmanuel Dreyfus <manu@...> writes:

> It would just not whitelist, and the message would go through greylisting.
> Not something really horrible.

Oooops, that make sense. I was into another galaxy when I wrote it, I'm
going to drink another coffee ;^)

> Oh, yes, good idea. In fact, per-user config could be made more general:
> per recipient config, where the recipient can be given as a regex. That
> way you can give a domain if you want.

Yep, it's nice.

> That's something for releases beyond 1.4: too intrusive. But we can 
> still discuss the user interface. What about this?
>
> rcpt /.*example\.com$/ {
> 	lazyaw
> 	report none
> 	autowhite 3d
> }

Why not, I agree with the use of sections in the configuration file. 

> One problem to solve: how to handle multiple matches?

I think that, here, the choice is whether milter-greylist processes first
match of last match. Trying anything else would end into code complexity
and fuzzy behaviour under some circumstances.

On Wed, May 26, 2004 at 09:39:25AM +0000, Emmanuel Dreyfus wrote:
> More ideas? Opinions? Thoughts? 

What about auto MX whitelisting:
If "host.foo.bar" delivers an e-mail from "user@..." an "host.foo.bar"
is listed in the MX records of "do.main" don't delay the e-mail.

	Kind regards

-- 
Matthias Scheler                                  http://scheler.de/~matthias/

On Wed, May 26, 2004 at 03:45:13PM +0200, Matthias Scheler wrote:
> What about auto MX whitelisting:
> If "host.foo.bar" delivers an e-mail from "user@..." an "host.foo.bar"
> is listed in the MX records of "do.main" don't delay the e-mail.

Well this seems to be the kind of problem why SPF was created for, wasn't it?

-- 
Emmanuel Dreyfus
manu@...

Emmanuel Dreyfus wrote:
> Hi 
> 
> Now that the scallability problem has been somewhat handled, I think about
> pushing a new stable release. Of course this means testing latest 1.3.x
> code a bit, and before that, implementing the most-wanted-but-easy-to-implement
> features. Complicated stuff will be worked on after 1.4 will be out.

[...]
> 
> More ideas? Opinions? Thoughts? 

Make the config file parser a bit more robust. I got caught a couple of 
times after manually adding a whitelist entry with a (unwanted) syntax 
error (like ommitting the addr keyword): when milter-greylist reloads 
the configuration file it continues with some unpredictables behaviour 
(it looks like it considers everything as white listed, but I'm not sure).

I'd prefer if it could parse the file once for syntax, and then only 
load it if correct (and bail loudly in syslog if an error is found).

-- 
Matthieu Herrb

On Wed, May 26, 2004 at 04:29:03PM +0200, Matthieu Herrb wrote:
> Make the config file parser a bit more robust. I got caught a couple of 
> times after manually adding a whitelist entry with a (unwanted) syntax 
> error (like ommitting the addr keyword): when milter-greylist reloads 
> the configuration file it continues with some unpredictables behaviour 
> (it looks like it considers everything as white listed, but I'm not sure).

I can see only one explanation for that: it beileived x.x.x.x/0 was 
whitelisted. We could probably have a particular test to catch that, 
because nobody ever want to whitelist the whole IPv4 address space..
 
> I'd prefer if it could parse the file once for syntax, and then only 
> load it if correct (and bail loudly in syslog if an error is found).

Do you see how to implement this easily?

-- 
Emmanuel Dreyfus
manu@...

On Wed, May 26, 2004 at 01:59:47PM +0000, Emmanuel Dreyfus wrote:
> Well this seems to be the kind of problem why SPF was created for, wasn't it?

SPF records are not widely-used at moment. And my proposal would provide
the same functionality with requiring SPF records.

	Kind regards

-- 
Matthias Scheler                                  http://scheler.de/~matthias/

Quoting Matthias Scheler <tron@...>:

> On Wed, May 26, 2004 at 01:59:47PM +0000, Emmanuel Dreyfus wrote:
> > Well this seems to be the kind of problem why SPF was created for, wasn't
> it?
> 
> SPF records are not widely-used at moment. And my proposal would provide
> the same functionality with requiring SPF records.
> 

As the song goes I feel "a change's gonna come" (*) : 
http://news.com.com/2100-1032-5220253.html

There is a coming merge of SPF and M$ Caller Id .
At least some future standard in sight ... 

SL/

(*) a Sam Cooke famous tune