openstack team mailing list archive

[Caitlin Bestler <Caitlin.Bestler@xxxxxxxxxxx>]

Paulo,

I believe you'll find that we're thinking along the same lines. Please review my proposal at http://etherpad.openstack.org/P9MMYSWE6U

One quick observation is that SHA-1 is totally inadequate for fingerprinting objects in a public object store. An attacker could easily
predict the fingerprint of content likely to be posted, generate alternate content that had the same SHA-1 fingerprint and pre-empt
the signature. For example: an ISO of an open source OS distribution. If I get my false content with the same fingerprint into the
repository first then everyone who downloads that ISO will get my altered copy.

SHA-256 is really needed to make this type of attack infeasible.

I also think that distributed deduplication works very well with object versioning. Your comments on the proposal cited above
would be great to hear.

From: openstack-bounces+caitlin.bestler=nexenta.com@xxxxxxxxxxxxxxxxxxx [mailto:openstack-bounces+caitlin.bestler=nexenta.com@xxxxxxxxxxxxxxxxxxx] On Behalf Of Paulo Ricardo Motta Gomes
Sent: Thursday, March 08, 2012 1:19 PM
To: openstack@xxxxxxxxxxxxxxxxxxx
Subject: [Openstack] Enabling data deduplication on Swift

Hello everyone,

I'm a student of the European Master in Distributed Computing (EMDC) currently working on my master thesis on distributed content-addressable storage/deduplication.

I'm happy to announce I will be contributing the outcome of my thesis work to OpenStack by enabling both object-level and block-level deduplication functionality on Swift (https://answers.launchpad.net/swift/+question/156862).

I have written a detailed blog post where I describe the initial architecture of my solution: http://paulormg.com/2012/03/05/enabling-deduplication-in-a-distributed-object-storage/

Feedback from the OpenStack/Swift community would be very appreciated.

Cheers,

Paulo

--
European Master in Distributed Computing - www.kth.se/emdc<http://www.kth.se/emdc>
Royal Institute of Technology - KTH
Instituto Superior Técnico - IST
http://paulormg.com