← Back to team overview

openstack team mailing list archive

Re: suggestion for data location compliance in swift


I think, from an audit and security standpoint, whatever component is
responsible for the placement of data needs to be solely responsible, and
needs to fail in a clean and transactional way. If the ring is modified to
guarantee that the *first* copy of an object is written into a specific zone
(to support a hybrid local/remote cloud, for instance), then I could see
such an extension passing muster. However, in any given failure scenario, we
still need to ensure that the data is limited to the appropriate zones, and
I don't think that working around the existing ring architecture is going to
achieve that.

The goal of making the implementation pluggable was simply to take the
existing methods, codify them somewhat, and then support alternate
implementations of those methods. Not real-time pluggable or necessarily
even exposed as a separate web service.

I think, if you start looking at the intersection of location compliance, a
hybrid of local and remote zones (LAN/WAN), and multiple tiers of storage
hardware (SATA, SSD, etc) - patching rings and zones together feels a bit
limited. I'd like to support an arbitrarily intelligent/policy-driven

Of course, my use case is somewhat outside the "Service Provider
Appropriate" mission statement of the core OpenStack project, hence the
drive to support alternate ring components, rather than a proposal to make
the existing one arbitrarily complex. As you point out, it *works*.


On Tue, May 31, 2011 at 6:37 AM, Rostyslav Slipetskyy <rslipetskyy@xxxxxxxxx
> wrote:

> The idea to make ring implementation pluggable seems nice.
> At the same time I am thinking that many developers might not will feel
> comfortable with modifying existing ring structure, since it *works*
> :) Probably, the other viable option for allowing data location compliance
> is to implement it *outside* of ring file structure (but maybe *inside*the future Ring component/service).
> As an example I look at how replication works, "If a replicator detects
> that a remote drive is has failed, it will use the ring's “get_more_nodes”
> interface to choose an alternate node to synchronize with." It seems that
> "Ring#get_more_nodes" can be used in a similar manner to select
> alternative nodes in other zones for storing objects once some of the zones
> are banned for specific accounts.
> - Rostik
> ------------------------------
> *From:* Joshua McKenty <josh@xxxxxxxxx>
> *To:* Rostyslav Slipetskyy <rslipetskyy@xxxxxxxxx>
> *Cc:* OpenStack <openstack@xxxxxxxxxxxxxxxxxxx>
> *Sent:* Tue, May 31, 2011 1:45:01 AM
> *Subject:* Re: [Openstack] suggestion for data location compliance in
> swift
> This was one of the use cases that drove the design discussion on
> decoupling the swift ring implementation from the rest of swift (along with
> supporting multiple tiers of hardware). See
> https://blueprints.launchpad.net/swift/+spec/swift-pluggable-hashing-ring for
> the basic proposal, however, and you'll note that we could definitely use
> some additional developers :)
> Joshua
> Joshua McKenty
> Piston Cloud Computing, Inc.
> (650) 283-6846
> joshua@xxxxxxxxx
> On 2011-05-30, at 4:18 PM, Rostyslav Slipetskyy wrote:
> Some of the data stored in the cloud has legal requirements to be stored
> physically within certain geographical boundary (for example within a
> country).
> Currently OpenStack does not allow to impose restrictions on data location.
> It looks like zones can be very handy to achieve data location compliance
> (according to the docs they can be used to group devices based on physical
> location). For example, suppose that a provider has servers in USA (zones
> 1-5)
> and Canada (zones 6-10). Let's imagine that a customer has some legal
> requirements to store its data on the servers in the USA. What I imagine
> doing
> is to restrict data for customer accounts to zones 1-5.
> Most probably, ring modifications will be necessary in order to implement
> this.
> Best Regards,
> Rostik
> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to     : openstack@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp


Joshua McKenty

Piston Cloud Computing, Inc.

(650) 283-6846