Milter-greylist

Hello,

Can milter-greylist be interfaced with hashcash?  (*)
I would like to whitelist (i.e. not delay) mails that
contain a valid hashcash header line.

Best regards
   Oliver

(*) http://www.hashcash.org/

-- 
Oliver Fromme, secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing b. M.
Handelsregister: Registergericht Muenchen, HRA 74606,  Gesch\ufffdftsfuehrung:
secnetix Verwaltungsgesellsch. mbH, Handelsregister: Registergericht M\ufffdn-
chen, HRB 125758,  Gesch\ufffdftsf\ufffdhrer: Maik Bachmann, Olaf Erb, Ralf Gebhart

FreeBSD-Dienstleistungen, -Produkte und mehr:  http://www.secnetix.de/bsd

"C++ is the only current language making COBOL look good."
        -- Bertrand Meyer

Oliver Fromme <olli@...> wrote:

> Can milter-greylist be interfaced with hashcash?  (*)
> I would like to whitelist (i.e. not delay) mails that
> contain a valid hashcash header line.

That could only fit in a DATA stage ACL, so you will not be able to
avoid greylisting decided at RCPT stage. 

-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu@...

manu@... wrote:
> Oliver Fromme <olli@...> wrote:
> 
>> Can milter-greylist be interfaced with hashcash?  (*)
>> I would like to whitelist (i.e. not delay) mails that
>> contain a valid hashcash header line.
> 
> That could only fit in a DATA stage ACL, so you will not be able to
> avoid greylisting decided at RCPT stage. 
> 

It's things like this that make you wish SMTP had a "HEADER" phase.. but it 
doesn't, and there's nothing that can be done now to change that. Oh well, 
hindsight on protocol design is always 20/20, right?


As a further clarification to Oliver, when it comes to greylisting you're pretty 
much limited to information that can be obtained from 4 sources: the connection 
itself (ie: source IP address), and the following commands: HELO, RCPT TO: and 
MAIL FROM:, and various DNS lookups or other operations based on the data 
obtained from those 4 sources.

Any other aspect of the message all comes as one big lump in the DATA phase, and 
  milter-greylist can only respond to such things after it has all been 
transfered. While that might not sound so bad, attempting to greylist at the end 
of the DATA stage is self-DoSing.

Every time a server tries to deliver a message, the whole message will have to 
be transfered before a 450 can be issued. Then when it retries, the whole 
message will be transfered before it can be 450'ed again.. Repeat as many times 
as it takes until the greylist timeout expires, and you could have just blown a 
lot of bandwidth transfering the same message hundreds or in case of a 
misbehaving server even tens of thousands of times before actually accepting it. 
Multiply that by a few hundred or thousand messages that are currently being 
greylisted and you'll raise your mailserver's bandwidth needs to really absurd 
levels.  (i.e.: could you handle a sustained 10000 fold increase in mailserver 
bandwidth usage?)

Matt Kettler wrote:
 > manu@... wrote:
 > > Oliver Fromme wrote:
 > > 
 > > > Can milter-greylist be interfaced with hashcash?  (*)
 > > > I would like to whitelist (i.e. not delay) mails that
 > > > contain a valid hashcash header line.
 > > 
 > > That could only fit in a DATA stage ACL, so you will not be able to
 > > avoid greylisting decided at RCPT stage. 
 > > 
 > 
 > It's things like this that make you wish SMTP had a "HEADER" phase.. but it 
 > doesn't, and there's nothing that can be done now to change that. Oh well, 
 > hindsight on protocol design is always 20/20, right?
 > 
 > As a further clarification to Oliver, when it comes to greylisting you're pretty 
 > much limited to information that can be obtained from 4 sources: the connection 
 > itself (ie: source IP address), and the following commands: HELO, RCPT TO: and 
 > MAIL FROM:, and various DNS lookups or other operations based on the data 
 > obtained from those 4 sources.
 > 
 > Any other aspect of the message all comes as one big lump in the DATA phase,

I'm already aware of all of that.

But isn't it possible that -- in theory -- the client side
closes the connection as soon as the headers are received
during the data phase?  A compliant mail server has to
treat that like a 4xx temporary error.  So you would have
to receive all the headers, but not the mail bodies.
Actually I thought that milter-greylist already supported
something like that.

Best regards
   Oliver

-- 
Oliver Fromme, secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing b. M.
Handelsregister: Registergericht Muenchen, HRA 74606,  Gesch\ufffdftsfuehrung:
secnetix Verwaltungsgesellsch. mbH, Handelsregister: Registergericht M\ufffdn-
chen, HRB 125758,  Gesch\ufffdftsf\ufffdhrer: Maik Bachmann, Olaf Erb, Ralf Gebhart

FreeBSD-Dienstleistungen, -Produkte und mehr:  http://www.secnetix.de/bsd

"The scanf() function is a large and complex beast that often does
something almost but not quite entirely unlike what you desired."
        -- Chris Torek

Oliver Fromme wrote:
> 
> I'm already aware of all of that.
> 
> But isn't it possible that -- in theory -- the client side
> closes the connection as soon as the headers are received
> during the data phase? 

Yes, in theory but milter-greylist isn't the one holding the connection. 
Sendmail is.

Milter-greylist is a milter under sendmail, and thus is also limited by the 
milter API, which dictates at what points Sendmail will check with it.

AFAIK, sendmail calls milters for error codes at the end of each command, but 
not in the middle of any of them.

> A compliant mail server has to
> treat that like a 4xx temporary error.

True, but I'd also expect that intentinally relying on this violates the RFCs, 
and would likely result in at least some server admins blacklisting you for it. 
I wont swear to it, but I'd be very shocked if this was considered RFC compliant 
behavior to engage in intentionally.

I'd also not be surprised to find that some popular MTA's don't handle this 
condition properly.

> So you would have
> to receive all the headers, but not the mail bodies.
> Actually I thought that milter-greylist already supported
> something like that.

Milter-greylist doesn't support anything like that, and can't because Sendmail 
doesn't support anything like that.

You'd have to write your own MTA to do that.

Oliver Fromme <olli@...> wrote:

> But isn't it possible that -- in theory -- the client side
> closes the connection as soon as the headers are received
> during the data phase? 

There is another problem: if you filter at the DATA stage, you loose the
ability to have per-recipient settings.

-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu@...

Matt Kettler <mkettler@...> wrote:

> AFAIK, sendmail calls milters for error codes at the end of each command, but
> not in the middle of any of them.

I suspect you could reject it early by returning SMFIS_TEMPFAIL at the
end of mlfi_eoh(). milter-greylist would not get the message body, but I
suspect sendmail would, as there is no way to interrupt the client at
this stage of the SMTP protocol.

-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu@...

On Wed, May 28, 2008 at 10:05:27PM +0200, manu@... wrote:
> Matt Kettler <mkettler@...> wrote:
> 
> > AFAIK, sendmail calls milters for error codes at the end of each command, but
> > not in the middle of any of them.
> 
> I suspect you could reject it early by returning SMFIS_TEMPFAIL at the
> end of mlfi_eoh(). milter-greylist would not get the message body, but I
> suspect sendmail would, as there is no way to interrupt the client at
> this stage of the SMTP protocol.

Indeed, as far I understand the sendmail source code the milter data
call back (also EOH) is called *after* the call to the collect routine
which stores everything received in the DATA phase into the queue or
some temporary "storage". The milter interaction in the DATA phase is
not synchronous a line-by-line manner.

Johann E. K.