← Back to team overview

openstack team mailing list archive

Re: swift -- object layout on storage

 

It's pretty simple. Swift uses the underlying filesystem to store the data on disk, and so you can use normal FS tools to find and inspect your data.

For the object server, the "magic" happens here: https://github.com/openstack/swift/blob/master/swift/obj/server.py#L117

The end result is that the data is stored here:

/path/to/mount/points/device/objects/partition/hash_suffix/hash/

That directory is the object. Inside the directory, there is normally just one file (named <timestamp>.data). The object's data is stored in the file, and the object's metadata is stored in the xattrs of the file.

In some cases (mostly around failure handling), there may be more than one file in that directory, but for the general case, all the .data files are sorted (by filename) and the last is chosen (ie the most recent). As I said, there is normally just the one file in there.

If you delete the object, the .data file is deleted and a <timestamp>.ts ("ts" for "tombstone") file is created as a zero-byte file. This is a delete marker that will be eventually reaped, but it exists to ensure that the delete properly propagates to all replicas in the cluster.

--John

 

On Jan 4, 2013, at 10:14 AM, "Snider, Tim" <Tim.Snider@xxxxxxxxxx> wrote:

> I’d like to understand more on how Swift lays out objects on the underlaying storage. I can’t seem to find out  much about this in the openstack / swift documentation itself or in associated web searchs.
> Thanks for pointers / links.
> Tim
> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to     : openstack@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp

Attachment: smime.p7s
Description: S/MIME cryptographic signature


Follow ups

References