Emmanuel Dreyfus wrote:
> Fabien Tassin wrote:
> > Emmanuel Dreyfus wrote:
> > > I droped it because we had no garantee of not loosing data when the
> > > milter would be killed. How are you going to address that problem?
> >
> > What about SQLite ? I'd love to have that. No need to maintain a local DNSRBL
> > then. Any tool could update a table very easily from the outside.
>
> How does it resist to a kill -9?
You need to use a real transactional database (such as
PostgreSQL) in order to support that. If the transaction
is aborted for any reason (e.g. SIGKILL), it is guaranteed
that the database server rolls it back and the database is
always in a consistent state. No data is lost except (of
course) for the actual current transaction that has been
aborted.
Using the C client library of PostgreSQL isn't difficult.
There's a very detailed documentation for the current
version here:
http://www.postgresql.org/docs/8.2/static/libpq.html
In fact, only a small subset of the API is required for
the simple things that milter-greylist would need to do
(i.e. just "INSERT" and "SELECT" SQL statements). The
docs include several example programs on the last page.
Using PostgreSQL also has the advantage that it is very
resistant against crashes and other kinds of forced
shutdowns. In my daily job as a consultant I often have
to work with various databases, most of them are MySQL
and PostgreSQL. When a live MySQL database crashes,
it's quite likely that one or more tables are corrupted
so you have to repair them manually, sometimes they're
even damaged to the point that you have to drop them and
restart from the most recent backup. That never happened
with PostgreSQL, never ever. Even after a hard reboot
(e.g. after a power outage) the database engine detects
automatically which transactions had been committed to
disk completely and which had not. It uses techniques
similar to a journalled file system (called "WAL" =
write-ahead log) to keep track of its data.
That's one of the reasons (but not the only one) I always
suggest PostgreSQL as the number one open-source database.
If I had a bit of free time, I would have a look at adding
the support to milter-greylist myself. It really isn't a
big deal if you have an idea what SQL is. Unfortunately
my job and my real life don't let me have enough time for
it right now. :-(
By the way, using an SQL database would also provide more
solutions to the MX synchronization problem. For example,
instead of using milter-greylist's own MX sync feature,
you could simply let several instances of milter-greylist
(on different MX servers) access the same SQL database.
Or -- for improved redundancy -- let each MX server run its
own SQL server instance, and use SQL replication features
to distribute all changes ("INSERT" commands) to the other
SQL servers. There are probably even more possibilities,
but the ones mentioned are the most obvious ones.
It would also be a solution to the memory (RAM) problem,
I think. The milter-greylist processes wouldn't need to
hold any data anymore, so they require very little RAM.
An SQL database, on the other hand, is optimized for
managing huge amounts of data in an efficient way. That's
the main purpose of a database, after all.
Best regards
Oliver
--
Oliver Fromme, secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing
Dienstleistungen mit Schwerpunkt FreeBSD: http://www.secnetix.de/bsd
Any opinions expressed in this message may be personal to the author
and may not necessarily reflect the opinions of secnetix in any way.
"I started using PostgreSQL around a month ago, and the feeling is
similar to the switch from Linux to FreeBSD in '96 -- 'wow!'."
-- Oddbjorn Steffensen