Milter-greylist

Hello,

Since it is quiet a big change, is it possible to consider changes in the logic what to store in the database?

More precisely: What about merging the pending and the autowhite lists into one data structure?

I will try to describe the way I imagine it.

In addition to the fields IP address, recipient and sender (this is the tupplet) there are two timestamp fields needed:
 firstseen: when this tupplet was encountered the first time.
 lastseen: when this tupplet was encountered the last time.

In the new list there is no implicit information stored any more about the configuration timeout (and other policy) parameters. The only information which is stored is referring only to external events.

For the MX sync no need any more for two commands, only one 'encounter'. Each peer is going to do the garbage collection individually applying its own configuration timeout parameters independently of the source of the tupplet: coming from an SMTP connection or from its MX peer.

In this pseudocode example the actual parameters connecting IP, sender, recipient are implicitly in the 'actualtupplet':


now = time();
foreach ( tupplet in database ) {
  if ( tupplet match actualtupplet) {
    found = 1;
    tupplet.lastseen = now;
    if (tupplet.lastseen - tupplet.firstseen < conf.delay)
      stillpending = 1;
    else
      stillpending = 0;
  }
  /* here is some maintenance */
  if (tupplet.lastseen - tupplet.firstseen < conf.delay) {
    /* pending */
    if (tupplet.lastseen + conf.timeout < now)
       removetupplet(tupplet)
  } else {
    /* autowhitelist */
    if (tupplet.lastseen + conf.autowhite_validity <= now)
       removetupplet(tupplet)
  }
}
if (found)
  return stillpending; 
addactualtupplettodatabase ( firstseen = lastseen = now );
return 0;

Please let me know what do you think about this idea.

Bests,
Attila

> -----Original Message-----
> From: milter-greylist@yahoogroups.com
> [mailto:milter-greylist@yahoogroups.com]On Behalf Of Emmanuel Dreyfus
> Sent: vendredi, 21. octobre 2005 14:40
> To: milter-greylist@yahoogroups.com
> Subject: Re: [milter-greylist] some strange behavior with mxsync
> 
> 
> On Fri, Oct 21, 2005 at 02:18:36PM +0300, Fredrik Nyberg DC wrote:
> > OK, I see your point. Of course, the problem could also be 
> solved using 
> > a persistent datastore. SQLLite has been mentioned, I also 
> saw it on the 
> > todo list on the milter-greylist homepage. Where do you 
> think efforts 
> > should be concentrated?
> 
> IMO the first step is to change autowhite.c and pending.c so that
> there is a clean separation between the storage backend and 
> milter-greylist.
> 
> The two files shouldbe merged into a unique file for the local storage
> backend. it could be named storage_local.c for instance.
> 
> Then we would define a structure containing function pointers for the
> various methods. storage_local.c would define an instance of 
> the structure 
> for the local storage backend, and we could add a file for a 
> SQLite backend.
> 
> An idea for the structure, after a quick look at the code
> struct storage {
> 	/* Initialization */
> 	void (*s_init)(void); 
> 
> 	/* Autowhitelist management */
> 	struct autowhite *(*s_awget)(struct sockaddr *, 
> 				     socklen_t, char *, char *, 
> time_t *);
> 	void (*s_awput)(struct autowhite *);
> 	void (*s_awadd)(struct sockaddr *, socklen_t, char *, 
> 			char *, time_t *, char *);
> 	int (*s_awcheck)(struct sockaddr *, socklen_t, char *, 
> char *, char *);
> 	int (*s_awtextdump)(FILE *);
> 
> 	/* greylist management */
> 	struct pending *(*s_pget)(struct sockaddr *, socklen_t, 
> 				  char *, char *, time_t);
> 	int (*s_pcheck)(struct sockaddr *, socklen_t, char *, 
> 		        char *, time_t *, time_t *, char *);
> 	void (*s_pdel)(struct sockaddr *, socklen_t, char *, 
> char *, time_t);  
> 	void (*s_pput)(struct pending *);
> 	int (*s_ptextdump)(FILE *);
> };
> 
> storage_local.c would define
> struct storage storage_local = {
> 	local_init, 	/* Would call former autowhite_init and 
> pending_init */
> 	local_awget,	/* former autowhite_get */
> 	local_awput,	/* former autowhite_put */
> 	...
> };
> 
> Next step is to have a field in struct conf for the storage:
> 	struct storage *c_storage;
> 
> And set defconf so that this field defaults to &storage_local.
> 
> Then you have to change any call to autowhite_* or pending_* in the
> code to a reference through the the conf.c_storage structure:
> 	pending = pending_get(sa, salen, from, rcpt, (time_t)0) 
> turns into
> 	pending = (*conf.c_storage->s_pget)(sa, salen, from, 
> rcpt, (time_t)0);
> 
> 
> Then you can start writing the SQLite backend, in 
> storage_sqlite.c. At some 
> point you'll have to change the config file parser to add a 
> storage keyword.
> 
> Of course you might discover that the API I just defined 
> isn't suited for 
> other use, or that it bounds to a lot of code duplication. We 
> might have 
> to refactor the code to avoid that.
> 
> Do you want to give it a try?
> 
> -- 
> Emmanuel Dreyfus
> manu@...
> 
> 
> ------------------------ Yahoo! Groups Sponsor 
> --------------------~--> 
> Fair play? Video games influencing politics. Click and talk back!
> http://us.click.yahoo.com/T8sf5C/tzNLAA/TtwFAA/W4wwlB/TM
> --------------------------------------------------------------
> ------~-> 
> 
>  
> Yahoo! Groups Links
> 
> 
> 
>  
> 
> 
>