Milter-greylist

Hello, all.

May be useful to expand option "quiet" for override 
hard-coded string: "Greylisting in action bla-bla-bla..." 
like this

...
quiet Server is busy now, come back later. Sorry for 
inconvenience
...

P.S. May be useful to enhance power of milter-greylist by 
adding real-time sender address callback verification? I 
can provide my own realization this callback.

Kind regards,
Eugene Kurmanin
---
Professional hosting for everyone - http://www.host.ru

On Wed, Nov 17, 2004 at 11:45:31AM +0300, ubr@... wrote:
> P.S. May be useful to enhance power of milter-greylist by 
> adding real-time sender address callback verification?

Is real-time sender address verification what Postfix does to find out
if a sender is valid? That trick and Milter Greylist don't like each
very much. Milter Greylist blocks the verification and Postfix blocks
the reception. It takes a while until they finally agree to exchange
e-mails.

	Kind regards

-- 
Matthias Scheler                                  http://scheler.de/~matthias/

Matthias Scheler <tron@...> wrote:

> Is real-time sender address verification what Postfix does to find out
> if a sender is valid? That trick and Milter Greylist don't like each
> very much. Milter Greylist blocks the verification and Postfix blocks
> the reception. It takes a while until they finally agree to exchange
> e-mails.

I have been wanting to implement a change to milter-greylist for a while
but never found the time to do so: for mail from <> we should wait the
end of the DATA stage to tempfail. That should help.

-- 
Emmanuel Dreyfus
Il y a 10 sortes de personnes dans le monde: ceux qui comprennent 
le binaire et ceux qui ne le comprennent pas.
manu@...

I speak about absolutly another thing :)
You speak about blocking by milter-greylist sender verification 
requests by Postfix or Exim or e.t.c.
This line in milter-greylist.conf can help:

...
from /<>/
...

Block bounce from <> is prohibited by many RFC & not help to war 
with spam, such messages is easy filter later...
Such spam is very little on volume, but block bounce may do big 
problems with succesful delivering process.

P.S. I speak about adding to milter-greylist technology like in Exim 
or Postfix to verify sender envelope. After successful verification 
do greylist in recipient envelope filter part as usually. Coz milter-
greylist don't protect against falsificated mail come from provider 
servers, that will try to deliver mail for 3-5 days :) Greylist is 
from 1 minute to 1 hour at sample. I use 1 minute, btw. It's enough 
for me.


--- In milter-greylist@yahoogroups.com, Matthias Scheler <tron@z...> 
wrote:
> On Wed, Nov 17, 2004 at 11:45:31AM +0300, ubr@o... wrote:
> > P.S. May be useful to enhance power of milter-greylist by 
> > adding real-time sender address callback verification?
> 
> Is real-time sender address verification what Postfix does to find 
out
> if a sender is valid? That trick and Milter Greylist don't like 
each
> very much. Milter Greylist blocks the verification and Postfix 
blocks
> the reception. It takes a while until they finally agree to 
exchange
> e-mails.
> 
> 	Kind regards
> 
> -- 
> Matthias Scheler                                  
http://scheler.de/~matthias/

On Thu, Nov 18, 2004 at 12:55:21PM -0000, eugene_kurmanin wrote:
> This line in milter-greylist.conf can help:
> 
> ...
> from /<>/
> ...

Some spam or virus SMPT engines use empty envelope addresses, too.

> Block bounce from <> is prohibited by many RFC ...

Milter Greylist isn't blocking it, it's delaying it.

> ... & not help to war with spam, ...

I disagree, see above.

	Kind regards

-- 
Matthias Scheler                                  http://scheler.de/~matthias/

> Some spam or virus SMPT engines use empty envelope addresses, too.

Sure. But for viruses is exist another technology to detect & delete 
(remove executable attach, filter mail by helo = your domain, 
antivirus plugins...)
Many viruses come from post servers & greylisting don't fight with 
such mails :)
He succesful stop mail only from "zombi" computer, dialup & cable 
users, that spread spam&viruses.

We receive is about 500,000 spam letters in month (kill it) + 35,000 
viruses (kill it)
Mail with sender = <> ~ 1000-1500, they filter later, if that needed.

> Milter Greylist isn't blocking it, it's delaying it.

I see :) But i don't want to block my outbound mail to system that 
used callback sender verification with NULL sender envelope.

Thank you.

eugene_kurmanin <ubr@...> wrote:

> Many viruses come from post servers & greylisting don't fight with 
> such mails :)

milter-greylist never claimed to be an anti-virus solution...

-- 
Emmanuel Dreyfus
Il y a 10 sortes de personnes dans le monde: ceux qui comprennent 
le binaire et ceux qui ne le comprennent pas.
manu@...

> milter-greylist never claimed to be an anti-virus solution...

Am i say that milter-greylist anti-virus solution???
The question on struggle against viruses was lifted not by me :)
I talk about another thing.

On Thu, Nov 18, 2004 at 04:30:15PM -0000, eugene_kurmanin wrote:
> > Some spam or virus SMPT engines use empty envelope addresses, too.
> Sure. But for viruses is exist another technology to detect & delete 
> (remove executable attach, filter mail by helo = your domain, 
> antivirus plugins...)

So what? Greylisting is very useful to keep that junk away from my system.

> > Milter Greylist isn't blocking it, it's delaying it.
> I see :) But i don't want to block my outbound mail to system that 
> used callback sender verification with NULL sender envelope.

Me neither. But white listing <> isn't a good option.

	Kind regards

-- 
Matthias Scheler                                  http://scheler.de/~matthias/

What about dispute?
I have simply offered to add technology of the checking the return 
address in analogy as is done in Exim, Postfix, milter-sender by 
Snert in milter-greylist.
If this not it is necessary, i do not :)

--- In milter-greylist@yahoogroups.com, Matthias Scheler <tron@z...> 
wrote:
> On Thu, Nov 18, 2004 at 04:30:15PM -0000, eugene_kurmanin wrote:
> > > Some spam or virus SMPT engines use empty envelope addresses, 
too.
> > Sure. But for viruses is exist another technology to detect & 
delete 
> > (remove executable attach, filter mail by helo = your domain, 
> > antivirus plugins...)
> 
> So what? Greylisting is very useful to keep that junk away from my 
system.
> 
> > > Milter Greylist isn't blocking it, it's delaying it.
> > I see :) But i don't want to block my outbound mail to system 
that 
> > used callback sender verification with NULL sender envelope.
> 
> Me neither. But white listing <> isn't a good option.
> 
> 	Kind regards
> 
> -- 
> Matthias Scheler                                  
http://scheler.de/~matthias/

eugene_kurmanin <ubr@...> wrote:

> What about dispute?
> I have simply offered to add technology of the checking the return 
> address in analogy as is done in Exim, Postfix, milter-sender by 
> Snert in milter-greylist.
> If this not it is necessary, i do not :)

My point of view is that if you can get the functionnality in another
milter, then you should do so. Adding unrelated functionnality to
milter-greylist will lead to code bloat, which leads to bugs and
security holes, themselves leading to the dark side of the force. 

What would you win by incorporating callbacks to milter-greylist,
compared to using milter-greylist and milter-sender on the same machine?

-- 
Emmanuel Dreyfus
Il y a 10 sortes de personnes dans le monde: ceux qui comprennent 
le binaire et ceux qui ne le comprennent pas.
manu@...

> What would you win by incorporating callbacks to milter-greylist,
> compared to using milter-greylist and milter-sender on the same 
machine?

You are right :)
Just nothing, only filter "all in one" ;)

I see, it's a not good idea.

On Tue, Nov 23, 2004 at 01:55:40PM -0000, eugene_kurmanin wrote:
> > What would you win by incorporating callbacks to milter-greylist,
> > compared to using milter-greylist and milter-sender on the same 
> machine?
> 
> You are right :)
> Just nothing, only filter "all in one" ;)

I see: "one filter to rule them all" :)

-- 
Emmanuel Dreyfus
manu@...

At 09:58 PM 11.22.2004 +0100, manu@... wrote:
>
>eugene_kurmanin <ubr@...> wrote:
>
>> What about dispute?
>> I have simply offered to add technology of the checking the return 
>> address in analogy as is done in Exim, Postfix, milter-sender by 
>> Snert in milter-greylist.
>> If this not it is necessary, i do not :)
>
>My point of view is that if you can get the functionnality in another
>milter, then you should do so. Adding unrelated functionnality to
>milter-greylist will lead to code bloat, which leads to bugs and
>security holes, themselves leading to the dark side of the force. 
>
>What would you win by incorporating callbacks to milter-greylist,
>compared to using milter-greylist and milter-sender on the same machine?
>
>-- 
>Emmanuel Dreyfus

I agree that it would not be good to "bloat" the code on milter-greylist,
but I sure would like to use the "callback" feature. I cannot do that by
using milter-sender because it requires a later version of DB than the one
already compiled in sendmail-8.12.11 -- which is already compiled in my
base system of FBSD-4.10.

It would be nice to be able to make use of existing DBs like the
"/etc/mailaccess" and aliases,etc.

Why does milter-sender need the later version of Berkley DB....??

Sorry if off-topic, but is a followup on this thread.


Happy trails,
Jack L. Stone

System Admin
Sage-american

On Tue, Nov 23, 2004 at 08:43:08AM -0600, Jack L. Stone wrote:
> I agree that it would not be good to "bloat" the code on milter-greylist,
> but I sure would like to use the "callback" feature. I cannot do that by
> using milter-sender because it requires a later version of DB than the one
> already compiled in sendmail-8.12.11 -- which is already compiled in my
> base system of FBSD-4.10.

You can't install two versions of DB at the same time?

But if you need the callback function and could not use milter-sender, 
then what about writing another milter?

> It would be nice to be able to make use of existing DBs like the
> "/etc/mailaccess" and aliases,etc.

Here I agree. It would be nice to get milter-greylist capable of getting
its whitelist from other sources than than the config file. 

Such a change should probably await for Remy Card's work on ACL, though. 
Remy, what's your opinion here?

-- 
Emmanuel Dreyfus
manu@...

At 02:51 PM 11.23.2004 +0000, you wrote:
>
>On Tue, Nov 23, 2004 at 08:43:08AM -0600, Jack L. Stone wrote:
>> I agree that it would not be good to "bloat" the code on milter-greylist,
>> but I sure would like to use the "callback" feature. I cannot do that by
>> using milter-sender because it requires a later version of DB than the one
>> already compiled in sendmail-8.12.11 -- which is already compiled in my
>> base system of FBSD-4.10.
>
>You can't install two versions of DB at the same time?
>

Sure, and I have tried that. However, the milter-sender instructs says:
[...}Note that Sendmail will probably have to be rebuilt to use Berkeley
DB, especially if the library was never installed and/or Sendmail was built
against an older version of Berkeley DB.[...]

So, if not compiled in, sendmail/milters won't use the new DB just merely
installed.

>But if you need the callback function and could not use milter-sender, 
>then what about writing another milter?
>
....alas, I don't write code -- not that I wouldn't want to, but it is just
not one of my skills.
Also, I suppose "bloat" can stem from too many milters running -- sort of
winds up at the same place as one bigger milter...??

>> It would be nice to be able to make use of existing DBs like the
>> "/etc/mailaccess" and aliases,etc.
>
>Here I agree. It would be nice to get milter-greylist capable of getting
>its whitelist from other sources than than the config file. 
>
....yes, this is what I have rooted for if that were possible.

>Such a change should probably await for Remy Card's work on ACL, though. 
>Remy, what's your opinion here?
>

Hope Remy is encouraged in this direction.

>-- 
>Emmanuel Dreyfus
>manu@...
>
>
>
> 
>Yahoo! Groups Links
>
>
>
> 
>
>
>
>


Happy trails,
Jack L. Stone

System Admin
Sage-american

At 02:51 PM 11.23.2004 +0000, Emmanuel Dreyfus wrote:
>
>On Tue, Nov 23, 2004 at 08:43:08AM -0600, Jack L. Stone wrote:
>> I agree that it would not be good to "bloat" the code on milter-greylist,
>> but I sure would like to use the "callback" feature. I cannot do that by
>> using milter-sender because it requires a later version of DB than the one
>> already compiled in sendmail-8.12.11 -- which is already compiled in my
>> base system of FBSD-4.10.
>
>You can't install two versions of DB at the same time?
>
>But if you need the callback function and could not use milter-sender, 
>then what about writing another milter?
>
>> It would be nice to be able to make use of existing DBs like the
>> "/etc/mailaccess" and aliases,etc.
>
>Here I agree. It would be nice to get milter-greylist capable of getting
>its whitelist from other sources than than the config file. 
>
>Such a change should probably await for Remy Card's work on ACL, though. 
>Remy, what's your opinion here?
>
>-- 
>Emmanuel Dreyfus
>manu@...
>

Let me reflect on this issue of ACL a bit more:

I'm presently using in the following order:
- Sendmail-8.12.11 with its own unique acl (DBs=access/aliases/virusertable)
- milter-regex with its own acl from a very large config
- milter-greylist with its own acl via config
- spamass-milter/spamassassin with MANY acls via many configs
- procmail with its various filters/recipes/rcs -- another form of acl

The above "layered" method, I catch 90+% with the first 3 layers and
another 9.999% with the rest.
They are VERY effective, but the downside is that they all require
different formats/syntax for their respective acl files.

I'm not complaining, just looking for some standardization and perhaps have
the "horses" all drink from the same "trough" to get their inique info.


Happy trails,
Jack L. Stone

System Admin
Sage-american

Jack L. Stone <jacks@...> wrote:

> I'm not complaining, just looking for some standardization and perhaps have
> the "horses" all drink from the same "trough" to get their inique info.

But what is the way to go? Using sendmail access file? Using a LDAP
schema?

-- 
Emmanuel Dreyfus
Il y a 10 sortes de personnes dans le monde: ceux qui comprennent 
le binaire et ceux qui ne le comprennent pas.
manu@...

> Let me reflect on this issue of ACL a bit more:
> 
> I'm presently using in the following order:
> - Sendmail-8.12.11 with its own unique acl 
(DBs=access/aliases/virusertable)
> - milter-regex with its own acl from a very large config
> - milter-greylist with its own acl via config
> - spamass-milter/spamassassin with MANY acls via many configs
> - procmail with its various filters/recipes/rcs -- another form of 
acl

Very interesting...
milter-regex from Daniel Hartmeier ( http://www.benzedrine.cx/ ) ?
Why a very large config?! I use a very small config (just only few 
lines) that catch is about 100-150K spam & viruses in month. It's a 
very powerful milter.

spamass is a buggy milter. milter-spamd from Daniel much more 
stable, i don't have any problem with it absolutly. I add only to 
this milter feature do not check messages larger than 64K.

i write my own milter like milter-sender with much less features 
than milter-sender, but for me it's enough. simple & efective 
callback verification + autowitelisting in Berkley DB format any 
versions with using DB 1.85 compatibility API.

i recommend you to use milter-dnsrbl from Richard Gooch. it's a 
rule :)

Kind Regards,
Eugene Kurmanin

At 07:05 AM 11.24.2004 +0100, manu@... wrote:
>
>Jack L. Stone <jacks@...> wrote:
>
>> I'm not complaining, just looking for some standardization and perhaps have
>> the "horses" all drink from the same "trough" to get their inique info.
>
>But what is the way to go? Using sendmail access file? Using a LDAP
>schema?
>
....for my preference, I would choose the sendmail access file.

>-- 
>Emmanuel Dreyfus
>Il y a 10 sortes de personnes dans le monde: ceux qui comprennent 
>le binaire et ceux qui ne le comprennent pas.
>manu@...
>
>
>
> 
>Yahoo! Groups Links
>
>
>
> 
>
>
>
>


Happy trails,
Jack L. Stone

System Admin
Sage-american

At 01:30 PM 11.24.2004 -0000, eugene_kurmanin wrote:
>
>
>> Let me reflect on this issue of ACL a bit more:
>> 
>> I'm presently using in the following order:
>> - Sendmail-8.12.11 with its own unique acl 
>(DBs=access/aliases/virusertable)
>> - milter-regex with its own acl from a very large config
>> - milter-greylist with its own acl via config
>> - spamass-milter/spamassassin with MANY acls via many configs
>> - procmail with its various filters/recipes/rcs -- another form of 
>acl
>
>Very interesting...
>milter-regex from Daniel Hartmeier ( http://www.benzedrine.cx/ ) ?
>Why a very large config?! I use a very small config (just only few 
>lines) that catch is about 100-150K spam & viruses in month. It's a 
>very powerful milter.
>

Indeed, it is very powerful -- my config is 1330 lines!
I'm probably not very good at regex.....

>spamass is a buggy milter. milter-spamd from Daniel much more 
>stable, i don't have any problem with it absolutly. I add only to 
>this milter feature do not check messages larger than 64K.
>
With spamass, I used to have an occasional abort, so used daemontools
to keep it going.... now with regex & greylist catching the bulk,
it has relieved the load on spamass & spamd which is a memory hog.

>i write my own milter like milter-sender with much less features 
>than milter-sender, but for me it's enough. simple & efective 
>callback verification + autowitelisting in Berkley DB format any 
>versions with using DB 1.85 compatibility API.
>
....great! Would you mind sharing that milter?

>i recommend you to use milter-dnsrbl from Richard Gooch. it's a 
>rule :)
>

Will have to look that one up. Do you have its URL handy?

Many thanks!

Happy trails,
Jack




Happy trails,
Jack L. Stone

System Admin
Sage-american

Hi,

On Tue, Nov 23, 2004 at 02:51:17PM +0000, Emmanuel Dreyfus wrote:
> Here I agree. It would be nice to get milter-greylist capable of getting
> its whitelist from other sources than than the config file. 
> 
> Such a change should probably await for Remy Card's work on ACL, though. 
> Remy, what's your opinion here?

	I am currently working on the ACL scheme that was discussed on this
list.  I am a bit late because I have spent lots of time fighting with yacc
when modifying the parser.  I still have things to debug (some data sometimes
become corrupted and I have to track this bug).  I plan to send an ACL patch
as soon as this problem is solved.

	BTW, I have implemented an ACL API, that can be used to add entries
to the access-list.  This API offers functions that initialize an ACL entry
and functions that add the entry to the list.

	I think that this API can be used to integrate whitelisted entries
from other sources, e.g. sendmail access file or others.  So, I think that,
once ACL are integrated into milter-greylist, it will be esay to extend
the access-list scheme by reading other sources and using the API to add
entries in the access list.  The ACL API will be extended, if needed.

	Cheers,

		R\ufffdmy

> Indeed, it is very powerful -- my config is 1330 lines!
> I'm probably not very good at regex.....

my config is about 20 lines, thats enough for all cases, imho :)
we can exchanging our configs, and if you want, i can optimize your 
config with success :)

if you want i can look at your sendmail.mc to advice for enhance 
antispam tricks.

> With spamass, I used to have an occasional abort, so used 
daemontools
> to keep it going.... now with regex & greylist catching the bulk,
> it has relieved the load on spamass & spamd which is a memory hog.

spamass --->>> /dev/null
try to use milter-spamd, it's rule :)

> ....great! Would you mind sharing that milter?
sure.
but i must to finish some work to optimize work with mx records & 
detect and remove source of small memory leaks =\
may be i remove using Berkley DB and change to more fast & reliable 
QDBM support for database manipulation. Coz Berkley DB 1.85 
compatibility realization may crash multithread application although 
using mutex locking... =(

now i test it on the production web-server with low mail traffic, 
then i want to test it on mail hub with 800K messages in month.

you can be a beta-tester ;)

> Will have to look that one up. Do you have its URL handy?
http://www.atnf.csiro.au/people/rgooch/email/index.html
ftp://ftp.atnf.csiro.au/pub/people/rgooch/email-
utilities/mailutils.tgz

If you want you can write me to:
Eugene dot Kurmanin at mvideo-corp dot ru
or
ubr at ok dot ru

At 06:03 PM 11.26.2004 -0000, eugene_kurmanin wrote:
>
>
>> Indeed, it is very powerful -- my config is 1330 lines!
>> I'm probably not very good at regex.....
>
>my config is about 20 lines, thats enough for all cases, imho :)
>we can exchanging our configs, and if you want, i can optimize your 
>config with success :)
>
I'd be happy to exchange our configs, but am meeting publishing deadlines
and will have to wait after Dec 1.

Quick question thought: How much memory is your milter-regex allocating?

>if you want i can look at your sendmail.mc to advice for enhance 
>antispam tricks.
>
...okay, will do that too.... thanks.

>> With spamass, I used to have an occasional abort, so used 
>daemontools
>> to keep it going.... now with regex & greylist catching the bulk,
>> it has relieved the load on spamass & spamd which is a memory hog.
>
>spamass --->>> /dev/null
>try to use milter-spamd, it's rule :)
>
>> ....great! Would you mind sharing that milter?
>sure.
>but i must to finish some work to optimize work with mx records & 
>detect and remove source of small memory leaks =\
>may be i remove using Berkley DB and change to more fast & reliable 
>QDBM support for database manipulation. Coz Berkley DB 1.85 
>compatibility realization may crash multithread application although 
>using mutex locking... =(
>
>now i test it on the production web-server with low mail traffic, 
>then i want to test it on mail hub with 800K messages in month.
>
>you can be a beta-tester ;)
>

...be glad to.

>> Will have to look that one up. Do you have its URL handy?
>http://www.atnf.csiro.au/people/rgooch/email/index.html
>ftp://ftp.atnf.csiro.au/pub/people/rgooch/email-
>utilities/mailutils.tgz
>

I found it..... again, will try it after Dec 1.

>If you want you can write me to:
>Eugene dot Kurmanin at mvideo-corp dot ru
>or
>ubr at ok dot ru
>
>
>
>
>
>
>
>
>
> 
>Yahoo! Groups Links
>
>
>
> 
>
>
>
>


Happy trails,
Jack L. Stone

System Admin
Sage-american