Milter-greylist

Currently, the way milter-greylist processes data stage commands (dacl lines in greylist.conf) is less than optimal.

Data stage commands are commands that activate during the body stage of the milter and are useful for content scanning (e.g. Spamassasin) and message signing and verification (e.g. DomainKeys and Dkim).

Dacl commands are expensive, in that the entire message (up to maxpeek, if defined) is filtered through the milter. Racl commands, on the other hand, base their decisions on connection information only and are light on the CPU.

In the current setup, milter-greylist will process the dacl commands after the racl ones and send all messages that do not get rejected (blacklist) by a racl command through the dacl filter. Even messages that get whitelisted by a racl line, will have their contents inspected. On top of that, auto-white listed messages and messages that got globally whitelisted, such as messages from authenticated users, go through this CPU intensive task.

I have had some email exchange with Emmanuel, and we both agree there is room for improvement. It is not clear, however, how this improvement should be implemented. Here are a couple of options:

1. Introduce a new keyword, e.g 'skipdacl'. Placing this keyword in a racl line, will cause the dacl commands to be skipped if the racl line is matched.

2. Only have messages that get greylisted by a racl command pass through the dacl stage. All globally whitelisted messages (through auth or global spf for example) whill skip dacl, as will auto-white listed messages.

This is based on the premise that racl lines will only blacklist or whitelist a message if the matching rule guarantees nearly 100% certainty that the message is either ham or spam. Otherwise the result of the line will be either false positives or false negatives. Only in case of doubt, will a greylist line be matched. These are the cases that will be forwarded for further evaluation by the dacl lines.

3. Base the precedence of dacl over racl commands on the line number. Currently, the order of racl lines determines the precedence of one line over the other. The first matching line gets executed, all others skipped. By putting whitelist lines before greylist lines, we can make sure that messages of which it is certain they are ham get accepted directly, for example.

Dacl lines get processed after racl lines, even if they come first in the conf file. This is related to the order in which Sendmail invokes the milter subroutines. It is not very intuitive from a user point of view, however.
Why not have to order of the racl and dacl commands determine if a dacl stage command needs to be executed? In this case, the user can decide to put all racl lines before the dacl lines, causing the dacl stage to be skipped whenever is racl match if found. Or put racl whitelist lines first, than the dacl lines and then the racl greylist lines. This set up will make sure that messages that get whitelisted in the racl stage, do not get forwarded to the dacl stage, but racl greylisted messages do. You get the idea.

To me, option 3 seems like the most elegant one, extending the current philosophy of milter-greylist seamlessly to the dacl stage processing. Option 2 is not too bad either (and easier to implement). You may have a different opinion, though. Please let me know what you think. I a prepared to implement a solution, once we have achieved some consensus.