Milter-greylist

Hi!

About fclose/fsync behavior:

fclose() guaranees that internal stdio lib buffers are "flushed" to disk 
using write() and close() syscalls. Years ago that was a guarantee that 
data are in fact made it's way to persistent storage (in the early days 
of Unix read/write syscalls were translated to actual read/write 
operations of a storage device).

In our days, we have at least two more buffering layers between 
write() syscall and actual device:
- VFS/filesystem/io-scheduler kernel buffers
- Internal storage device buffers

So, fclose() only pushes down user-level buffers into kernel but does 
not guarantee that changes are really written to disk. Attempting to 
guarantee than makes necessary not only write any dirty kernel 
buffers but also to issue bus/drive-specific command to flush drvice 
internal buffer. And belive me, you don't want to do it on every file 
close(). Sove devices tend to execute flush command far longer then 
others, and some flush not only write-back but also read-ahead caches 
which impacts subsequent read() operations.

That is why we need explicit fsync()/fdatasync() to ensure actual disk 
writes and internal buffer flushes.

About fclose() macro:

I belive there is no need to fsync() data before each close(). Most of 
files we write are of no value to us after a system restart. The only 
file that matters - is a database dump. I think that the only one
fsync() we need is before closing a database dump file.

On Fri, Mar 20, 2009 at 07:21:28AM +0100, manu@... wrote:
> Hello
> 
> Following up this:
> http://www.h-online.com/open/Ext4-data-loss-explanations-and-workarounds
> --/news/112892
> 
> It seems ext4fs people do not assume that closing a file will make the
> data flushed to disk. It is claimed fsync(2) should be called first. 
> 
> The standards say "Any unwritten buffered data for the stream shall be
> written to the file" on a fclose(3) call:
> http://www.opengroup.org/onlinepubs/000095399/functions/fclose.html
> 
> I may be missing something, but I have the feeling this behavior is a
> standard violation. However, milter-greylist does not run on a standard,
> it runs on actual implementation, so we may need a fix for it. 
> 
> Request for comments: do we need to modify our Fclose() macro so that it
> calls fsync() before fclose()  on Linux? Is it useful? Can it harm?
> 
> -- 
> Emmanuel Dreyfus
> http://hcpnet.free.fr/pubz
> manu@...