Milter-greylist

Hi,

I have created a php urlcheck page that checks if the sender IP address 
is in the same subnet of any of the real MX hosts.

This makes the greylisting a bit less intrusive. I have it running for a
bit now and it is actively whitelisting real mail servers without delay.

Needless to say, dynamic dial up or dsl or cable anything client users
fail these test so there is no harm there.

One less reason to use greylisting.

You can test this by putting the following urlcheck in yourgreylist.conf

urlcheck "mxhostcheck" 
"https://webmail.coltex.nl/spam/mxhostcheck.php?domain=%sf&ip=%i&fuzz=24&delay=180" 
5
acl whitelist urlcheck "mxhostcheck"

The arguments to this check are as follows.
domain, needs to be the sender domain.
ip, needs to be the sender ip.
fuzz, is a subnetmask to narrow down the match. Pick one. I like 22.
delay, a delay in seconds for greylist timeouts.

The source of the php page can be viewed here.
https://webmail.coltex.nl/spam/mxhostcheck.txt

One of the issues with the script currently is that the greylist 
response is not correctly parsed by milter-greylist-4.0a1 yet.
FIXED in milter-greylist-4.0a2!

If the match is correct you should see this message in the mail log as well.
Milter add: header: X-Greylist: URL check passed, not delayed by
milter-greylist-4.0a1

I need some feedback on this.

Test the page for your self in your milter-greylist or host the php page
localy and work from there.

I would very much like that any improvements be communicated back.
The PHP code is BSD licensed. I do expect fair use of the url and no 
more then 5 concurrent connections at the same time.

Note that if you want to host the page locally you require the "host" 
binary to be available on your system.

Kind regards,

Seth Mos

On Wed, Apr 18, 2007 at 09:48:48AM +0200, Seth Mos wrote:
> urlcheck "mxhostcheck" 
> "https://webmail.coltex.nl/spam/mxhostcheck.php?domain=%sf&ip=%i&fuzz=24&delay=180" 
> 5
> acl whitelist urlcheck "mxhostcheck"

Quick poll about urlchecks in milter-greylist
1) Do you know what it is? 
2) Would you be interested by a contrib directory in milter-greylist 
distribution, which would contain various urlchecks from contributors?
3) Would a web page or a wiki with list of contrbuted urlchecks be
a better alternative to the above?
-- 
Emmanuel Dreyfus
manu@...

Emmanuel Dreyfus wrote:
 > Seth Mos wrote:
 > > urlcheck "mxhostcheck" 
 > > "https://webmail.coltex.nl/spam/mxhostcheck.php?domain=%sf&ip=%i&fuzz=24&delay=180" 5
 > > acl whitelist urlcheck "mxhostcheck"
 > 
 > Quick poll about urlchecks in milter-greylist
 > 1) Do you know what it is? 
 > 2) Would you be interested by a contrib directory in milter-greylist 
 > distribution, which would contain various urlchecks from contributors?
 > 3) Would a web page or a wiki with list of contrbuted urlchecks be
 > a better alternative to the above?

Just a quick question (not related to the poll):  Does
milter-greylist support calling a local executable for
checking, as a more efficient replacement for urlcheck?

(I couldn't find something like that in my manual pages,
but I have to admit that I don't have the latest version
installed, and I'm reluctant to update because it's
working fine so far.  However, such an "execcheck"
feature would be a good reason to update.)

Best regards
   Oliver

-- 
Oliver Fromme, secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing b. M.
Handelsregister: Registergericht Muenchen, HRA 74606,  Gesch\ufffdftsfuehrung:
secnetix Verwaltungsgesellsch. mbH, Handelsregister: Registergericht M\ufffdn-
chen, HRB 125758,  Gesch\ufffdftsf\ufffdhrer: Maik Bachmann, Olaf Erb, Ralf Gebhart

FreeBSD-Dienstleistungen, -Produkte und mehr:  http://www.secnetix.de/bsd

'Instead of asking why a piece of software is using "1970s technology,"
start asking why software is ignoring 30 years of accumulated wisdom.'

Emmanuel Dreyfus wrote:
> 
> Quick poll about urlchecks in milter-greylist
> 1) Do you know what it is?
> 2) Would you be interested by a contrib directory in milter-greylist
> distribution, which would contain various urlchecks from contributors?
> 3) Would a web page or a wiki with list of contrbuted urlchecks be
> a better alternative to the above?

Not sure on 3. I have a number of scripts not related to urlchecks. But 
they are related to parsing mail log files for statistics.

These include some rrd statistics and graphs to show email distribution 
and delay times. And the urlcheck page I wrote ofcourse.

Cheers,

Seth

Oliver Fromme <olli@...> wrote:

> Just a quick question (not related to the poll):  Does
> milter-greylist support calling a local executable for
> checking, as a more efficient replacement for urlcheck?

If you call a local executable, you fork a process on each query. I'm
not sure it's more efficient.

urlcheck relies on curl to perform the query, so you can query any URL
supported by CURL. I guess you can use a telnet URL against a local
executable launched by inetd.

-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu@...

manu@... wrote:
 > Oliver Fromme wrote:
 > 
 > > Just a quick question (not related to the poll):  Does
 > > milter-greylist support calling a local executable for
 > > checking, as a more efficient replacement for urlcheck?
 > 
 > If you call a local executable, you fork a process on each query.

Not necessarily.  A single process could handle multiple
queries (i.e. read one query from stdin, write result to
stdout), and it only needs to be restarted if it terminates
for some reason.

 > I'm not sure it's more efficient.

Performing HTTP requests with all of the overhead of a
webserver (Apache or whatever) plus CGI or PHP script is
_definitely_ less efficient.

 > urlcheck relies on curl to perform the query, so you can query any URL
 > supported by CURL. I guess you can use a telnet URL against a local
 > executable launched by inetd.

Uhm, that wouldn't be very efficient either.  :-)

Best regards
   Oliver

-- 
Oliver Fromme, secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing b. M.
Handelsregister: Registergericht Muenchen, HRA 74606,  Gesch\ufffdftsfuehrung:
secnetix Verwaltungsgesellsch. mbH, Handelsregister: Registergericht M\ufffdn-
chen, HRB 125758,  Gesch\ufffdftsf\ufffdhrer: Maik Bachmann, Olaf Erb, Ralf Gebhart

FreeBSD-Dienstleistungen, -Produkte und mehr:  http://www.secnetix.de/bsd

One Unix to rule them all, One Resolver to find them,
One IP to bring them all and in the zone to bind them.

Oliver Fromme wrote:
> manu@... <mailto:manu%40netbsd.org> wrote:
>  > Oliver Fromme wrote:

>  > If you call a local executable, you fork a process on each query.
> 
> Not necessarily. A single process could handle multiple
> queries (i.e. read one query from stdin, write result to
> stdout), and it only needs to be restarted if it terminates
> for some reason.

There is another aspect you need to consider. Security.

When you call out the urlcheck against your "authentication" source 
there is no actual authentication information sent across the network.

With the urlcheck facility the authentication is processed on the local 
host which then decides to whitelist greylist or whatever.

And in the case of my urlcheck against a https site the end-to-end 
transport is secure as well. This against running a local binary which 
then authenticates against a remote source (ldap) without encryption.
Which then quite possibly communicates authentication information or 
user information unencrypted.

Think Big.

Just my 2 cents.

Cheers,

Seth

On Wed, Apr 18, 2007 at 04:03:32PM +0200, Seth Mos wrote:
> And in the case of my urlcheck against a https site the end-to-end 
> transport is secure as well. This against running a local binary which 
> then authenticates against a remote source (ldap) without encryption.
> Which then quite possibly communicates authentication information or 
> user information unencrypted.

FWIW, I use an urlcheck with a ldap://localhost URL. There is a replica of
the master LDAP on localhost, with changes being pushed through LDAP/SSL

-- 
Emmanuel Dreyfus
manu@...