Emmanuel Dreyfus wrote:
> Oliver Fromme wrote:
> > Shouldn't it be sufficient to open() the dump file right
> > from the start and never close() it? Only always write to
> > it, then fsync()+lseek(0). (And fflush()+rewind() if stdio
> > functions are used to write the file.)
>
> The idea is to preserve the file in the event of a crash during the
> dump. So the dump is currently done in a temporary file, which is
> moved to replace the original file once the dump completes.
That makes a lot of sense.
> rename(2) can buy us that. I'm not sure it's possible by using the
> same file.
Certainly. rename(2) just changes the name of a directory
entry, it does _not_ change the underlying inode: You can
see in "ls -li" that the inode stays the same when you mv(1)
or rename(2) a file. The same happens when you hardlink
a file, except that the source name is not removed.
Also, any open file descriptors associated with the inode
don't change (neither on rename(2) nor on link(2)), they
will just work with the new name. You can see that with
daemons like Apache: When you rotate the logfiles (e.g.
"mv access.log access.log.0") and forget to send the daemon
a SIGHUP, it will continue to write into the file under the
new name.
So, the solution for milter-greylist would be to open two
files at the start, and use them alternating to write the
dump file and then hardlink them to the final name. So,
in pseudo-code, it would look like this:
start:
name1 = "greylist.db.new1"
name2 = "greylist.db.new2"
fd1 = open(name1, O_CREAT | O_TRUNC)
fd2 = open(name2, O_CREAT | O_TRUNC)
...
dump:
write db to fd1
fflush(fd1) // only if stdio used
fsync(fd1)
rewind(fd1) // only if stdio used
lseek(fd1,0)
link("greylist.db", "greylist.db.old")
link(name1, "greylist.db")
swap fd1 and fd2
swap name1 and name2
That's from the top of my head and I haven't implemented
it, but it should work (I'm familiar with UNIX/POSIX file
semantics for many years). link() first removes the target
if it already exists, and it's an atomic operation, so it
ensures that always a good copy of greylist.sb exists.
The first dump is written to *.new1, and after the link(2)
functions, these files exist:
,--> greylist.db
`--> greylist.db.new1
greylist.db.old
where greylist.db and greylist.db.new1 are hardlinked.
The second dump is written to *.new2. After the linking,
The files look like this:
,--> greylist.db
| greylist.db.new1
`--> greylist.db.new2
greylist.db.old
where greylist.db and greylist.db.new2 are hardlinked.
The third dump is written to *.new1 again (which is not
hardlinked to greylist.db at this time). Afterwards:
,--> greylist.db
`--> greylist.db.new1
greylist.db.new2
greylist.db.old
again, greylist.db and greylist.db.new1 are hardlinked.
The alternation of fd1/fd2 and name1/name2 ensures that
the dump is always written to the file which is _not_
currently hardlinked with greylist.db, so it will be
preserved if the process crashes.
In my opinion, that would be a very clean solution.
Best regards
Oliver
PS: fflush() and rewind() are only required when stdio
functions (those that use a "FILE *") are used to write
to the file, e.g. fwrite(3), fputs(3), fprintf(3) etc.
But you will also need fsync() in this case! Note that
the fileno(3) function can be used to get the file
descriptor from a "FILE *", which is needed for fsync().
If only file descriptor functions are used (such as
write(2)), then fsync()+lseek() is sufficient.
Please don't be offended if you already knew all of that
(I'm sure you knew). I mention it just to be safe. :-)
--
Oliver Fromme, secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing
Dienstleistungen mit Schwerpunkt FreeBSD: http://www.secnetix.de/bsd
Any opinions expressed in this message may be personal to the author
and may not necessarily reflect the opinions of secnetix in any way.
Passwords are like underwear. You don't share them,
you don't hang them on your monitor or under your keyboard,
you don't email them, or put them on a web site,
and you must change them very often.