Milter-greylist

a nice feature might be to to do an fstat on this file
before the dump. for me i don't care if non-priveleged
users read this and the dump is creating a new file 0600
each time.

perhaps going a bit futher milter-greylist could detect
changes before the dump and if so instead of dumping
reinitialize the white/grey lists in memory. the idea of
course would be to manually (or even programatically)
manipulate the white/grey entries.

the risk of course is that some bonehead would somehow make
the file unparsable.

-george elgin

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com

George Elgin <gelgin@...> wrote:

> a nice feature might be to to do an fstat on this file
> before the dump. for me i don't care if non-priveleged
> users read this and the dump is creating a new file 0600
> each time.

We could have a config file option to choose the mode...

> perhaps going a bit futher milter-greylist could detect
> changes before the dump and if so instead of dumping
> reinitialize the white/grey lists in memory. the idea of
> course would be to manually (or even programatically)
> manipulate the white/grey entries.

The in-memory and on-disk databases would always have conflicts, and
there is no easy way to resolve them.

If you want to alter the database, IMO, the right way is a command line
tool that would talk to the daemon through a Unix socket 

-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu@...

>
> If you want to alter the database, IMO, the right way is a command line
> tool that would talk to the daemon through a Unix socket
>

This tool could also be used to dump the db and stop the milter.

-- 
--------------------------------------------------------------------------------
M.Menge                                 Tel.: (49) 7071/29-70316
Universitaet Tuebingen                  Fax.: (49) 7071/29-5912
Zentrum fuer Datenverarbeitung          mail: menge@...-tuebingen.de
Waechterstrasse 76
72074 Tuebingen

On Fri, 2006-06-23 at 11:39 +0200, Michael Menge wrote:
> >
> > If you want to alter the database, IMO, the right way is a command
> line
> > tool that would talk to the daemon through a Unix socket
> >
> 
> This tool could also be used to dump the db and stop the milter.

I have this already in progress, though not done yet...

It would be useful for me to know as to what other commands would be
useful to have.

Things I have in mind currently are:
      * Ping daemon to see if alive or stuck
      * Reconfigure
      * Force db dump
      * Clean shutdown of daemon
      * Increase/decrease debugging level
      * Stats, and these could be the following:
              * Current thread count
              * Total number of connections to daemon
              * Greylisted responses count
              * Whitelisted responses count
              * Greylisted entries
              * Whitelisted entries
              * Expired count
              * ...


R.

----- Original Message ----- 

From: "Ranko Zivojnovic" <ranko@...>
To: <milter-greylist@yahoogroups.com>
Sent: Friday, June 23, 2006 5:33 AM
Subject: Re: [milter-greylist] /var/milter-greylist/greylist.db


> On Fri, 2006-06-23 at 11:39 +0200, Michael Menge wrote:
> > >
> > > If you want to alter the database, IMO, the right way is a
command
> > line
> > > tool that would talk to the daemon through a Unix socket
> > >
> >
> > This tool could also be used to dump the db and stop the milter.
>
> I have this already in progress, though not done yet...
>
> It would be useful for me to know as to what other commands would be
> useful to have.
>
> Things I have in mind currently are:
>       * Ping daemon to see if alive or stuck
>       * Reconfigure
>       * Force db dump
>       * Clean shutdown of daemon
>       * Increase/decrease debugging level
>       * Stats, and these could be the following:
>               * Current thread count
>               * Total number of connections to daemon
>               * Greylisted responses count
>               * Whitelisted responses count
>               * Greylisted entries
>               * Whitelisted entries
>               * Expired count
>               * ...
>

Dunno if this is worthwhile to most, but if the logs, after rereading
a modified config file, would report when a bad syntax has been
entered would be helpful. Twice recently, unbeknownst to me, I made a
typo, but the milter kept on going but did not tell me about the bad
syntax. Apparently it reads down to the mistake, and stops and loads
the defaults instead. The first time, I noticed in the maillog that
the delay had changed to 30 mins instead of my setting of 8 mins.
Plus, I got some complaints from users that they were being delayed by
the greylist. I stopped & restarted the milter and then saw the syntax
error reported nicely or me on line #209. Corrected it anf all was
fine again.

Then, when the recent thread showed up and redirecting the milter GL
logs to a separate filter, I thought I would be more likely to spot
any other such mistake. Not so. I had just happened to make another
mistake with this new log setup and still didn't see the goof until I
completely stopped/restarted again and saw the error.

I know the best solution is not to make mistakes, but haste does make
waste at times. Am I missing the error somehow already by it's
re-reading of the config, or do I need to stop/restart to guarantee I
see any errors made?

A humble user & Sys Admin....

Ranko Zivojnovic wrote:
 > Michael Menge wrote:
 > > > 
 > > > If you want to alter the database, IMO, the right way is
 > > > a command line tool that would talk to the daemon through
 > > > a Unix socket
 > > 
 > > This tool could also be used to dump the db and stop the milter.
 > 
 > I have this already in progress, though not done yet...
 > 
 > It would be useful for me to know as to what other commands would be
 > useful to have.
 > 
 > Things I have in mind currently are:
 >       * Ping daemon to see if alive or stuck
 >       * Reconfigure
 >       * Force db dump
 >       * Clean shutdown of daemon
 >       * Increase/decrease debugging level
 >       * Stats [...]

My most important wish:
 * Manually remove an entry from the automatic whitelist.

Best regards
   Oliver

-- 
Oliver Fromme,  secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing
Dienstleistungen mit Schwerpunkt FreeBSD: http://www.secnetix.de/bsd
Any opinions expressed in this message may be personal to the author
and may not necessarily reflect the opinions of secnetix in any way.

"A language that doesn't have everything is actually easier
to program in than some that do."
        -- Dennis M. Ritchie

> > > This tool could also be used to dump the db and stop the milter.
> >
> > I have this already in progress, though not done yet...
> >
> > It would be useful for me to know as to what other commands would 
be
> > useful to have.
> >
> > Things I have in mind currently are:
> >       * Ping daemon to see if alive or stuck

[gelgin@server gelgin]$ ps -clu spfmilt
F S   UID   PID  PPID CLS PRI ADDR SZ WCHAN  TTY          TIME CMD
5 S   100  6006     1 TS   23 -  2568 -      ?        00:00:00 milter-
greylist
1 S   100  6007  6006 TS   23 -  2568 -      ?        00:00:00 milter-
greylist
1 S   100  6008  6007 TS   24 -  2568 -      ?        00:00:00 milter-
greylist
1 S   100  6009  6007 TS   20 -  2568 rt_sig ?        00:00:00 milter-
greylist
1 S   100  6011  6007 TS   20 -  2568 rt_sig ?        00:00:00 milter-
greylist


> >       * Reconfigure

again if it sees a .db file change (like it does for greylist.conf) 
OR SIGUSR[12]


> >       * Force db dump
could accept a signal. set dump time to 0 ?

> >       * Clean shutdown of daemon

i thought it does a dump on SIGTERM ?


> >       * Increase/decrease debugging level
> >       * Stats, and these could be the following:
> >               * Current thread count

see above rt_sig is a thread.

> >               * Total number of connections to daemon

is this a problem ?. what would i infer from that # ??

> >               * Greylisted responses count
> >               * Whitelisted responses count

i'd still like for the non-priveleged user to be able to read .db 
file. why can't grey milter preserve it's [u]mask on dump ?? [seems 
like a 2-3 line code change]

> >               * Greylisted entries
> >               * Whitelisted entries

alias ts='tail -2 /var/milter-greylist/greylist.db'

> >               * Expired count
> >               * ...
> >
>

personnally i think this 'tool' is overkill. just want to to perl awk 
grep my way to ? for example i thought about building a per/user 
barchart of greylisted entries [to see which users are getting 
hammered] see for example :

http://www.exgig.com/spam.cgi

sorry i still think rereading greylist.db if it's changed is a good 
idea. as someone pointed out on 'serious' parse error it reverts 
which is also a good idea.

-george

How 'bout a test function...

I seem to remember that we had to install tcpwrappers at one point  
(oh so long ago, it appears).
And there was a utility to check functionality.

---
suggested syntax changes:
  -t  - testing mode (requires -e or -d or -i)
  -e  - email address to check
  -d  - domain to check
  -i  - ip address to check
----

Say you run:
milter-greylist -t -e test@...

and the response is:
delayed

or: (note, there could be multiple entries)
not delayed from 66.163.187.78 till 2006-06-26 08:35:10
not delayed from 66.163.187.79 till 2006-06-29 12:35:10

or:
autowhitelisted



or for more beautific info...:
   Milter-greylist version: 1.84

   Email      : test@...
   From IP    : 66.163.187.78
   Valid till : 2006-06-26 08:30:10
   Whitelisted: auto


or:
   milter-greylist
       Version: 1.84

   Testing mode
       E-mail        : test@...
       From IP       : 66.163.187.78
       1st Connection: 2006-06-26 08:28:20

---
Say you run:
milter-greylist -t -d domain.com

and the response is:
all delayed

or: (note, there could be multiple entries)
webmster@... not delayed from 66.163.187.78 till 2006-06-26  
08:35:10
user@... not delayed from 66.163.187.79 till 2006-06-29 12:35:10

or:
autowhitelisted



or for more beautific info...:
   Milter-greylist version: 1.84

   Domain     : domain.com

   Email      : test@...
   From IP    : 66.163.187.78
   Valid till : 2006-06-26 08:30:10
   Whitelisted: auto

   Email      : webmaster@...
   From IP    : 66.163.187.78
   Valid till : 2006-06-26 08:30:10
   Whitelisted: auto

or:
   milter-greylist
       Version: 1.84

   Testing mode
       Domain        : domain.com

       E-mail        : test@...
       From IP       : 66.163.187.78
       1st Connection: 2006-06-26 08:28:20

       E-mail        : webmaster@...
       From IP       : 66.163.187.78
       1st Connection: 2006-06-26 08:28:20

---
Say you run:
milter-greylist -t -i 66.163.187.78

and the response is:
delayed

or: (note, there could be multiple entries)
webmaster@... not delayed from 66.163.187.78 till 2006-06-26  
08:35:10
user@... not delayed from 66.163.187.78 till 2006-06-29 12:35:10

or:
autowhitelisted



or for more beautific info...:
   Milter-greylist version: 1.84

   Email      : test@...
   From IP    : 66.163.187.78
   Valid till : 2006-06-26 08:30:10
   Whitelisted: auto

   Email      : user@...
   From IP    : 66.163.187.78
   Valid till : 2006-06-26 08:30:10
   Whitelisted: auto

or:
   milter-greylist
       Version: 1.84

   Testing mode
       E-mail        : test@...
       From IP       : 66.163.187.78
       1st Connection: 2006-06-26 08:28:20

... I guess I could just write a perl script to read the db and do  
the same thing.. but just a thought.


Bill

On Jun 23, 2006, at 3:33 AM, Ranko Zivojnovic wrote:

> On Fri, 2006-06-23 at 11:39 +0200, Michael Menge wrote:
>>>
>>> If you want to alter the database, IMO, the right way is a command
>> line
>>> tool that would talk to the daemon through a Unix socket
>>>
>>
>> This tool could also be used to dump the db and stop the milter.
>
> I have this already in progress, though not done yet...
>
> It would be useful for me to know as to what other commands would be
> useful to have.
>
> Things I have in mind currently are:
>       * Ping daemon to see if alive or stuck
>       * Reconfigure
>       * Force db dump
>       * Clean shutdown of daemon
>       * Increase/decrease debugging level
>       * Stats, and these could be the following:
>               * Current thread count
>               * Total number of connections to daemon
>               * Greylisted responses count
>               * Whitelisted responses count
>               * Greylisted entries
>               * Whitelisted entries
>               * Expired count
>               * ...
>
>
> R.
>
>
>
> ------------------------ Yahoo! Groups Sponsor -------------------- 
> ~-->
> Something is new at Yahoo! Groups.  Check out the enhanced email  
> design.
> http://us.click.yahoo.com/SISQkA/gOaOAA/yQLSAA/W4wwlB/TM
> -------------------------------------------------------------------- 
> ~->
>
>
> Yahoo! Groups Links
>
>
>
>
>
>

bill:

sounds good. one problem as you point out the list's are
based on tuples and it's concievable some users for a
domain (or ip) might be greylisted and others white.

and conversely for example i have seen 3 entries for the
same user/domain because their isp had 3 different outbound
smtp servers (causing me to get multiple copies of the same
-e).

the problem is of course simplified [a little] if one uses
-L or lazyaw modes.

as you say though it might better be a parsing problem for
perl (php)

-george

--- Bill Levering <idbill@...> wrote:

> 
> How 'bout a test function...
> 
> I seem to remember that we had to install tcpwrappers at
> one point  
> (oh so long ago, it appears).
> And there was a utility to check functionality.
> 
> ---
> suggested syntax changes:
>   -t  - testing mode (requires -e or -d or -i)
>   -e  - email address to check
>   -d  - domain to check
>   -i  - ip address to check
> ----
> 
> Say you run:
> milter-greylist -t -e test@...
> 
> and the response is:
> delayed
> 
> or: (note, there could be multiple entries)
> not delayed from 66.163.187.78 till 2006-06-26 08:35:10
> not delayed from 66.163.187.79 till 2006-06-29 12:35:10
> 
> or:
> autowhitelisted
> 
> 
> 
> or for more beautific info...:
>    Milter-greylist version: 1.84
> 
>    Email      : test@...
>    From IP    : 66.163.187.78
>    Valid till : 2006-06-26 08:30:10
>    Whitelisted: auto
> 
> 
> or:
>    milter-greylist
>        Version: 1.84
> 
>    Testing mode
>        E-mail        : test@...
>        From IP       : 66.163.187.78
>        1st Connection: 2006-06-26 08:28:20
> 
> ---
> Say you run:
> milter-greylist -t -d domain.com
> 
> and the response is:
> all delayed
> 
> or: (note, there could be multiple entries)
> webmster@... not delayed from 66.163.187.78 till
> 2006-06-26  
> 08:35:10
> user@... not delayed from 66.163.187.79 till
> 2006-06-29 12:35:10
> 
> or:
> autowhitelisted
> 
> 
> 
> or for more beautific info...:
>    Milter-greylist version: 1.84
> 
>    Domain     : domain.com
> 
>    Email      : test@...
>    From IP    : 66.163.187.78
>    Valid till : 2006-06-26 08:30:10
>    Whitelisted: auto
> 
>    Email      : webmaster@...
>    From IP    : 66.163.187.78
>    Valid till : 2006-06-26 08:30:10
>    Whitelisted: auto
> 
> or:
>    milter-greylist
>        Version: 1.84
> 
>    Testing mode
>        Domain        : domain.com
> 
>        E-mail        : test@...
>        From IP       : 66.163.187.78
>        1st Connection: 2006-06-26 08:28:20
> 
>        E-mail        : webmaster@...
>        From IP       : 66.163.187.78
>        1st Connection: 2006-06-26 08:28:20
> 
> ---
> Say you run:
> milter-greylist -t -i 66.163.187.78
> 
> and the response is:
> delayed
> 
> or: (note, there could be multiple entries)
> webmaster@... not delayed from 66.163.187.78 till
> 2006-06-26  
> 08:35:10
> user@... not delayed from 66.163.187.78 till
> 2006-06-29 12:35:10
> 
> or:
> autowhitelisted
> 
> 
> 
> or for more beautific info...:
>    Milter-greylist version: 1.84
> 
>    Email      : test@...
>    From IP    : 66.163.187.78
>    Valid till : 2006-06-26 08:30:10
>    Whitelisted: auto
> 
>    Email      : user@...
>    From IP    : 66.163.187.78
>    Valid till : 2006-06-26 08:30:10
>    Whitelisted: auto
> 
> or:
>    milter-greylist
>        Version: 1.84
> 
>    Testing mode
>        E-mail        : test@...
>        From IP       : 66.163.187.78
>        1st Connection: 2006-06-26 08:28:20
> 
> .... I guess I could just write a perl script to read the
> db and do  
> the same thing.. but just a thought.
> 


__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com

george <gelgin@...> wrote:

> could accept a signal. 

libmilter handles signals. The API does not allow a milter to catch
signals.

-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu@...

To manually remove an entry from the automatic whitelist (or rather
all entrys on a certain ipnumber) is also my most important wish.

As a matter of fact I joined this list a few days ago only to discuss
that feature, and my intention was to contribute with this myself, but
if it's already in progress, then there is no reason for me to do this.

How i would use it?
We have another milter created by me, that runs Spamassassin and
antivirus from F-Secure.
After installing milter-greylist the viruses stopped by F-secure
decreased from about 50-100 viruses a day to less than one a day.
Now after a few weeks it's up to about 5-10 a day.
About the same thing has happended with Spamassassin.
I think this is because something has happend with these ipnumbers, 
perhaps it was a dynamic ipnumber that someone else connected to, or
the computer on that ipnumber got infected by a virus or some
spamsending program.
To prevent this, I would like my own milter to call something in
milter-greylist to remove everything on that ipnumber from the
autowhitelist every time something bad is recognised of Spamassassin
or F-Secure.
After they are removed from the whitelist they can be greylisted again.

--- In milter-greylist@yahoogroups.com, Oliver Fromme <olli@...> wrote:

>
> 
> Ranko Zivojnovic wrote:
>  > Michael Menge wrote:
>  > > > 
>  > > > If you want to alter the database, IMO, the right way is
>  > > > a command line tool that would talk to the daemon through
>  > > > a Unix socket
>  > > 
>  > > This tool could also be used to dump the db and stop the milter.
>  > 
>  > I have this already in progress, though not done yet...
>  > 
>  > It would be useful for me to know as to what other commands would be
>  > useful to have.
>  > 
>  > Things I have in mind currently are:
>  >       * Ping daemon to see if alive or stuck
>  >       * Reconfigure
>  >       * Force db dump
>  >       * Clean shutdown of daemon
>  >       * Increase/decrease debugging level
>  >       * Stats [...]
> 
> My most important wish:
>  * Manually remove an entry from the automatic whitelist.
> 
> Best regards
>    Oliver
> 
> -- 
> Oliver Fromme,  secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing
> Dienstleistungen mit Schwerpunkt FreeBSD: http://www.secnetix.de/bsd
> Any opinions expressed in this message may be personal to the author
> and may not necessarily reflect the opinions of secnetix in any way.
> 
> "A language that doesn't have everything is actually easier
> to program in than some that do."
>         -- Dennis M. Ritchie
>