ecryptfs team mailing list archive

[Chris Newman <chris@xxxxxxxxxxxxxxxxx>]

@Theodore,

As a scalable server developer with 25 years experience, I am fully
aware of the purpose of fsync, fdatasync and use them if and only if the
semantics I want are "really commit to disk right now".  To use them at
any other time would be an implementation error.

I further agree delayed allocation is a good thing and believe
application developers who use the first command sequence you describe
above get what they deserve and that is it a mistake for the filesystem
to perform an implicit sync in that case.

Where I strongly disagree with you is for the open-write-close-rename
call sequence (your second scenario).  It is very common for an
application to need "atomic replace, defer ok" semantics when updating a
file (more common, in fact, than cases where fsync is really needed).
The only way to express that semantic is open-write-close-rename, and
furthermore that semantic is the only useful interpretation of that call
sequence.  Adding an fsync expresses a different and less useful
semantic.  For example, when I do "atomic replace, defer ok" twice in a
flush interval I would expect an optimal filesystem to discard the
intermediate version without ever committing it to disk.  So I find the
workaround you've implemented undesirable as it results in non-optimal
and unnecessary disk commits.

Now your not-useful interpretation of open-write-close-rename is Posix
compliant under a narrow interpretation.  But I can interpret any
standard in a not-useful way.  An IMAP server that delivers all new mail
to a mailbox "NEWMAIL" and has no "INBOX" would be strictly compliant
with the spec and also not useful.  Any reasonable IMAP client vendor
will simply state they don't support that server.  And that's exactly
what will happen to EXT4, XFS and other filesystems that interpret the
open-write-close-rename call sequence in a not useful way.  You will
find applications declare your filesystem unsupported because you
interpret a useful call sequence in a not-useful fashion.

The right interpretation of open-write-close-rename is "atomic replace,
defer ok".  There is no reason to spin up the disk or fsync until the
next flush interval.  What's important is that the rename is not
committed until after the file data is committed.

If you disagree, I invite you to suggest how you would express "atomic
replace, defer ok" using Posix APIs when writing an application.

-- 
Ext4 data loss
https://bugs.launchpad.net/bugs/317781
You received this bug notification because you are a member of eCryptfs,
which is subscribed to ecryptfs-utils in ubuntu.

Status in “ecryptfs-utils” source package in Ubuntu: Invalid
Status in “linux” source package in Ubuntu: Confirmed
Status in ecryptfs-utils in Ubuntu Jaunty: Invalid
Status in linux in Ubuntu Jaunty: Confirmed

Bug description:
I recently installed Kubuntu Jaunty on a new drive, using Ext4 for all my data.

The first time i had this problem was a few days ago when after a power loss ktimetracker's config file was replaced by a 0 byte version . No idea if anything else was affected.. I just noticed ktimetracker right away.

Today, I was experimenting with some BIOS settings that made the system crash right after loading the desktop. After a clean reboot pretty much any file written to by any application (during the previous boot) was 0 bytes.
For example Plasma and some of the KDE core config files were reset. Also some of my MySQL databases were killed...

My EXT4 partitions all use the default settings with no performance tweaks. Barriers on, extents on, ordered data mode..

I used Ext3 for 2 years and I never had any problems after power losses or system crashes.

Jaunty has all the recent updates except for the kernel that i don't upgrade because of bug #315006

ProblemType: Bug
Architecture: amd64
DistroRelease: Ubuntu 9.04
NonfreeKernelModules: nvidia
Package: linux-image-2.6.28-4-generic 2.6.28-4.6
ProcCmdLine: root=UUID=81942248-db70-46ef-97df-836006aad399 ro rootfstype=ext4 vga=791 all_generic_ide elevator=anticipatory
ProcEnviron:
 LANGUAGE=
 LANG=en_US.UTF-8
 SHELL=/bin/bash
ProcVersionSignature: Ubuntu 2.6.28-4.6-generic
SourcePackage: linux