openstack team mailing list archive

[Michael Barton <mike-launchpad@xxxxxxxxxxxxxxxx>]

What I've been playing with is having a manifest that contains hashes
of (4mb) chunks for the volume's backups.  When a user initiates a new
backup, dm-snapshot does its thing and gives me a block device.  I
read and hash chunks from that block device and compare them to the
manifest, uploading any that differ to Swift, then update the manifest
with the new backup.

The restore uses fuse with some basic bitmap logic to lazy load chunks
from Swift on demand, plus a background thread that fills them in
autonomously.  I've been pretty happy with fuse's performance and
stability (python-fuse that is; fusepy is really slow).

The NBD solution isn't really any different logic-wise from the fuse
version, but requires a lot more wrangling of server and client
processes.  And actually we weren't too impressed with the performance
of a basic NBD server in some (non-scientific) tests.

All of this is sort of at the proof of concept stage at the moment.

-- Michael Barton