nova team mailing list archive

[Jay Pipes <jaypipes@xxxxxxxxx>]

Yes, we are.  All discussion groups at the summit will have a wiki or
Etherpad page as well as one or more blueprints that will be started,
divided among assignees, and spec'd out at the summit.

-jay

On Tue, Oct 5, 2010 at 1:44 PM, Ewan Mellor <ewan.mellor@xxxxxxxxxxxxx> wrote:
> Are you both (and anyone else interested) going to work on a draft blueprint
> before the summit?  I'm keen to see discussion drafts in place before we
> all meet in person, otherwise we'll waste a lot of time just getting on the
> same page.
>
> Thanks,
>
> Ewan.
>
> On Mon, Oct 04, 2010 at 07:01:57PM +0100, Eric Day wrote:
>
>> ++ on all points. When it was decided to implement a Redis backend for
>> the DB API, I tried to explain my concerns about recreating SQLAlchemy
>> with a Redis back-end, and then some. We should decide what data
>> model we are using (relational, hierarchical, flat key-value, graph,
>> ...) and decide on a native API to use. If we wan to stay relational,
>> using SQLAlchemy natively and using it's pluggable backend system
>> may be sufficient.
>>
>> There is also the distributed storage and privacy concerns to discuss
>> as well at the summit. For example, each compute/network/volume worker
>> could keep it's own store and push any required information up to
>> the other layers (scheduler/API). This would ensure a compromised
>> worker machine could not touch other workers or their data stores,
>> and scaling a central data store is no longer an issue. With these
>> concerns in mind, this may also sway which data model and API we want
>> to expose to the applications.
>>
>> -Eric
>>
>> On Mon, Oct 04, 2010 at 01:34:27PM -0400, Jay Pipes wrote:
>> > I agree with Soren that big changes like this should occur after some
>> > discussion (a thought that I rose on IRC last week).
>> >
>> > I look forward to the discussion at the next summit about the ORM in
>> > general, and specifically, these concerns about it I have:
>> >
>> > 1) Inconsistent (and sometimes incorrect) use of session boundaries in
>> > the abstraction layer for sqlalchemy.  Peculiar bugs show up when
>> > sessions aren't handled (passed between functions) properly or when
>> > transactions are not reliable (NoSQL or when the RDBMS doesn't support
>> > transactional statements like SELECT ... FOR UPDATE.
>> > 2) Shared code between Glance and Nova (and possibly Swift, but it's
>> > less of an issue IMHO).  There is now a copy of the datastore adapter
>> > from Nova in Glance (this was done to save time).  We will want to
>> > pull this code out into openstack-common ASAP, as I really don't feel
>> > like maintaining two copies of the datastore adapter layer ;)
>> > 3) The Model class itself.  The biggest issue is that the model class
>> > does not live in the API layer; instead, it must be re-implemented for
>> > each adapter.  I experienced the joy of trying to re-create the
>> > existing SQLAlchemy-based NovaBase model class in the Redis adapter.
>> > This was less than enjoyable.  If we choose to use a base model class,
>> > it needs to address these fundamental issues *at a bare minimum*:
>> >   a) The definition of data fields (versus non-data fields, private
>> > fields, or calculated fields)
>> >   b) The casting of SQL-like data types (CHAR, INT, BLOB, DATETIME,
>> > etc) into data type that Python understands consistently.
>> >   c) A model's understanding (or lack of understanding) of its
>> > relationships with other types of models.
>> >   d) The aforementioned debate about what is appropriate as a primary
>> > key identifier for each model (UUID, URI, auto-inc, etc)
>> >
>> > -jay
>> >
>> > On Sun, Oct 3, 2010 at 5:10 PM, Devin Carlen <devin.carlen@xxxxxxxxx> wrote:
>> > > Soren,
>> > >
>> > > This comment was with regards to using declarative_base(), which was used to quickly implement the data layer and save some time.  The next iteration we've been discussing internally, is to drop the use of declarative_base and using sqlalchemy table definitions instead.
>> > >
>> > > declarative_base() provides value primarily when you are planning on actually using sqlalchemy models as the middle tier.  We only use them as a stepping stone in the data layer, and then treat everything as dicts currently.  (Whether we should be using dicts everywhere is part of a larger discussion I hope to bring up at the next design summit.)
>> > >
>> > > By using the table definitions directly and bypassing declarative_base(), we will be able to cut out two superfluous and unnecessary transformations before the data is passed back to the middle tier.
>> > >
>> > > There are no immediate plans to do this, but you are right - it is worthy of further discussion.
>> > >
>> > >
>> > > Devin
>> > >
>> > >
>> > > This only was meant to indicate that we would be changing from u
>> > >
>> > > On Oct 2, 2010, at 2:52 AM, Soren Hansen wrote:
>> > >
>> > >> Looking at the commit message for r316 on trunk, I see this (emphasis mine):
>> > >>
>> > >>    Also refactored NovaBase and removed several methods since they
>> > >>    would have to be changed *when we move away from sqlalchemy models*
>> > >>    and begin using sqlalchemy table definitions.
>> > >>
>> > >> I find it worrisome that such things are stated as though they are
>> > >> naturally given or generally agreed upon. I have no recollection of a
>> > >> discussion to that effect, let alone a decision.
>> > >>
>> > >> I certainly hope a big patch to do this doesn't suddenly turn up without
>> > >> prior discussion and a related blueprint..
>> > >>
>> > >> --
>> > >> Soren Hansen
>> > >> Ubuntu Developer    http://www.ubuntu.com/
>> > >> OpenStack Developer http://www.openstack.org/
>> > >>
>> > >> _______________________________________________
>> > >> Mailing list: https://launchpad.net/~nova
>> > >> Post to     : nova@xxxxxxxxxxxxxxxxxxx
>> > >> Unsubscribe : https://launchpad.net/~nova
>> > >> More help   : https://help.launchpad.net/ListHelp
>> > >
>> > >
>> > > _______________________________________________
>> > > Mailing list: https://launchpad.net/~nova
>> > > Post to     : nova@xxxxxxxxxxxxxxxxxxx
>> > > Unsubscribe : https://launchpad.net/~nova
>> > > More help   : https://help.launchpad.net/ListHelp
>> > >
>> >
>> > _______________________________________________
>> > Mailing list: https://launchpad.net/~nova
>> > Post to     : nova@xxxxxxxxxxxxxxxxxxx
>> > Unsubscribe : https://launchpad.net/~nova
>> > More help   : https://help.launchpad.net/ListHelp
>>
>> _______________________________________________
>> Mailing list: https://launchpad.net/~nova
>> Post to     : nova@xxxxxxxxxxxxxxxxxxx
>> Unsubscribe : https://launchpad.net/~nova
>> More help   : https://help.launchpad.net/ListHelp
>