← Back to team overview

openstack team mailing list archive

Re: A single cross-zone database?


Coming back from a long break and getting back up to speed.

I believe Sandy and I spoke about this at decent length before, the proposal
that I understood we both walked away happy with was this:

1. As a first step, implement it with a single db, because that is what we
already have and what is the easiest first step. Zones are an additional
column or join table and implemented as a filter on the query for instances
and the like. Zones in this case being used largely to mean "grouping of
servers with some common, possibly distinct, property." We only support a
single level of 'zones' for the moment, no arbitrary nesting.
2. See whether that works, it probably will for the current scaling targets
and use cases.
3. To implement bursting/hybrid stuff, we then work on a data abstraction
for aggregating the results of remote calls, understanding that we expect
remote calls to have lower performance.

The blueprint as it stands (http://wiki.openstack.org/MultiClusterZones) is
following a 'turtles all the way down' approach that probably isn't really
appropriate for real world usage. In reality there are things that are
global level, things that are region level and things that are cluster
 * A global service will be in charge of specific requests that need to be
global and have very specific and constrained scaling properties, perhaps
global dns management.
 * A region represents the largest unit of scaling, the maximum number of
nodes and services that can managed at time while meeting SLAs, including
the public endpoints, most things above region level need to be managed
client side: one does not expect to list all instances across all regions in
a single call, one expects to make three requests to the three regions one's
instances are in.
 * A cluster represents a grouping unit with common properties, either rack
commonality, capability commonality, or simply special case customers.
Subgroupings here may still be interesting or useful but are unlikely to be
necessary. Clusters are likely to share many things at the region level as
well as have individual cluster level concerns.

This design handles the needs of plenty of large organizations already and
is easy to move to piecemeal from our current codebase, the first step being
that zones are clusters and can share a regional registry (database) and
public endpoint while still having per-zone services (scheduler, et al).


On Wed, Mar 16, 2011 at 4:51 PM, Sandy Walsh <sandy.walsh@xxxxxxxxxxxxx>wrote:

>  Yup, that's a fair assessment.
>  That said, even without SDB and caching it's going to be tight for Cactus
> anyway. There are lots of little issues that are cropping up once I got down
> to 100'.
>  -S
>  ------------------------------
>  From: Justin Santa Barbara <justin@xxxxxxxxxxxx>
>  *
> *
> Sandy: Have I got the wrong end of the stick here?  Are these our choices?
>  Justin
>   Confidentiality Notice: This e-mail message (including any attached or
> embedded documents) is intended for the exclusive and confidential use of the
> individual or entity to which this message is addressed, and unless otherwise
> expressly indicated, is confidential and privileged information of Rackspace.
> Any dissemination, distribution or copying of the enclosed material is prohibited.
> If you receive this transmission in error, please notify us immediately by e-mail
> at abuse@xxxxxxxxxxxxx, and delete the original message.
> Your cooperation is appreciated.
> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to     : openstack@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp

Follow ups