Milter-greylist

Hi Eduardo,
>
> I agree that adding a simple embedded database backend such as SQLite
> does not add much to milter-greylist. The current code is more than
> adequate for a single server.
>
> The intention is to make milter-greylist more scalable. I am assuming
> that the 20 mail servers you are running do not serve the same domain;
> please confirm.

They are distributed in 5 countries and they process over 300 domains in clusters, not all servers process all domains.

>
> Once you get into the realm of 20 servers for a single domain, the
> amount of peer-to-peer traffic to keep the servers synchronized
> quickly grows out of control.
>
> With a backend that has the ability to connect to a scalable database
> architecture, the peer-to-peer communication issue disappears, and
> milter-greylist becomes truly scalable to any number of servers for a
> single domain.

But the peer2peer traffic evolutionates to DB traffic, ok having 20 servers updating generates a lot of traffic, but adding a db, also add troubles, you cant use Myisam db. You need something transactional (innodb) with row level block, imagine 20 servers writing/reading to the same table.

>
> I choose MySQL as an example, because it can run high-volume and
> high-availability clusters.

I would prefer to have a "dummy greylist daemon" with something like this http://www.ispras.ru/~knizhnik/fastdb/readme.htm as DB. All the 20 greylists have this dummy greylist configured as peer (only 1) and they send to this dummy peer all the transaccions. As the transactions gets into the dummy, it's replyes to the other 19 clients.

It will work as central node, behaving as a normal greylisting daemon (so no need to alter p2p code to much) but acting as redistribution daemon. I see as benefits, lightweight code, no need to involve a huge db daemon, it can be reused all code of p2p trafic.

Hope the idea is clear enough.

>
> Let me know if anyone feels this makes sense or not.
>
> --- In milter-greylist@yahoogroups.com, Eduardo Casarero
> wrote:
> >
> >
> > ----- "Sven Holz" wrote:
> > >
> > >
> > >
> > >
> >
> > Hi all
> > >
> > > > For scalability, the next step up for Milter-greylist will be to
> > > > implement a database backend. MySQL is a good candidate.
> > > >
> > > > In my discussions with Emmanuel, we have identified the following
> > > > steps to address this challenge, without breaking any current
> setups
> > > > in the mean time. The proposed steps are:
> > >
> > > if an db backend is included it would be nice to place config parts
> > > like whitelists, peers and so on also in the db...
> > >
> > > This would make it possible to code a central nice an sweet
> > > webfrontend for milter-greylist :-)
> > >
> > > Mit freundlichen Gruessen || with kind regards
> > >
> > > Sven Holz
> > >
> > >
> >
> >
> >
> >
> >
> > Is really so important to have a DB backend? i've more than 20
> servers running milter-greylist and they dump the tuples only once a day.
> >
> >
> >
> >
> >
> > How scalability would be improved? Accessing a mysqldb or any db
> stored on Hard Disk is slower than accesing memory directly (only
> mysql memory tables (HEAP) are fast enough.
> >
> >
> >
> >
> >
> > In the only case i think it might help is in configuration data, but
> the text file is not so complicated, and adding layers might add
> problems.
> >
> >
> >
> >
> > I think that milter-greylisting should focus in performance and the
> smtp transaction (not body neither headers). In my setup i only use it
> with spf and some acl to stop DDoS, harvesting, etc. Behind
> greylisting, i have mailscanner, spamassassin and everything.
> >
> >
> >
> >
> > Just my 2 cents
> >
> >
> >
> >
> > Eduardo.
> >
> >
> >
> >
> > --
> > -----
> > Eduardo Casarero
> > Informatica Avanzada SRL
> > Email: eduardo.casarero@...
> > Tel: (054) 011-52353939 int 115
> >
>
>