openstack team mailing list archive

[Eric Day <eday@xxxxxxxxxxxx>]

I agree. I propose we always keep lp:nova (or lp:<project>) stable,
and instead create "trunks" like lp:nova/testing that all the
test/regression systems can be run against. This is pretty similar to
how we did things with Drizzle, a commit would bounce down the line,
finlly landing in lp:<project> when it was verified.

-Eric

On Thu, Feb 24, 2011 at 10:41:02AM -0800, Justin Santa Barbara wrote:
>    Sounds like we're actually in agreement, just disagreeing about the
>    topology of the branches!
>    My preferred topology is actually a series of increasingly stable trunks,
>    each of which has passed each level of the smoke tests (one platform,
>    multi platform, torture tests). *The nice thing about this is that then a
>    release can simply be seen as the next level up in the stability
>    hierarchy, with a manual pull. *I think Google Chrome has a particularly
>    sophisticated process which we could learn from:
>    http://techcrunch.com/2011/01/11/google-chrome-release-cycle-slideshow/
>    I still think we should start simple with two trunks - stable and
>    unstable. *But I'll be happy if with per-branch testing also - that just
>    seems a lot harder to me!
>    Justin
> 
>    On Thu, Feb 24, 2011 at 10:24 AM, Trey Morris <trey.morris@xxxxxxxxxxxxx>
>    wrote:
> 
>      Instead of an unstable trunk, i think code should just be better vetted
>      before it lands in the trunk. If the difference between trunk and your
>      proposed unstable trunk is a set of automated tests, then those tests
>      can just as easily be run on a LP branch before it gets into current
>      trunk. We just need the tests. No matter how many stability levels of
>      trunk we have, without better tests we can't guarantee stability.
> 
>      On Thu, Feb 24, 2011 at 11:15 AM, Justin Santa Barbara
>      <justin@xxxxxxxxxxxx> wrote:
> 
>        Hi Jay,
>        I couldn't agree more. *I had another bug come up yesterday on another
>        of my patches (I know - not a good day for me!) where I again broke
>        the OpenStack API by requiring the metadata attribute.
>        In this case, it was missed by the unit tests. *I believe I was always
>        passing metadata, so simply missed the real world case. *Here's the
>        bug report:
>        https://bugs.launchpad.net/nova/+bug/724143
>        This brings up a number of points though:
>        On testing
>         1. The bug reporter apparently knows how to program, but instead of
>            us getting a test case which we could immediately use, we got a
>            Ruby test case. *I think we should do whatever we can so that
>            people that are moderately comfortable with code also feel
>            comfortable submitting a failing test case that we can use. *I
>            think this means having some version of an API client, or even a
>            Reference-Implementation API client, in the source tree, and using
>            it.
>         2. It's not clear to me how we deal with unit test cases that are
>            failing - cases that represent a found bug that is not yet fixed.
>            *Maybe it should be submitted on a bugNNN branch, with failing
>            tests, and then whoever works on the bug can branch from it? *(Of
>            course, we'd be lucky if all bugs went like this, but it does
>            happen. *I often find myself fixing tangential issues, and I'd
>            like to know the 'right thing to do' is)
>         3. On this particular metadata bug, it was my fault. *I therefore
>            submitted a very rapid hotfix which just fixes the issue. *I then
>            coded up unit tests that use the OpenStack API, and did in fact
>            hit the issue naturally, and then included my fix which resolved
>            it. *I submitted the 'full patch' as a separate branch.
>         4. This 'full patch' branch includes unit tests that bring up the
>            OpenStack API and various services in-process, and runs tests just
>            like a user. *It would have caught the metadata issue.*
>         5. The hope is that we can reuse the same tests as smoke tests, by
>            simply tweaking them to work against real services instead of
>            bringing up in-process stub-services. *These could be (some of)
>            the smoke tests in Hudson.
>         6. I'd hope that we could have two levels of these smoke tests, one
>            that runs on a single configuration (e.g. KVM, OpenISCSI, Glance);
>            another that runs a matrix of configurations (and might take an
>            hour or more to run)
>         7. Ideally we'd have a torture test that would run overnight and be
>            randomized and try to find obscure bugs, even if issues found are
>            not necessarily repeatable in the way that non-randomized tests
>            are.
>        We need an unstable trunk:
>         1. In general, it seems that our end-users are using trunk for
>            unreleased functionality and treating it as if it were released.
>            *I don't think we should be encouraging that, because I know I'll
>            make more mistakes in future and some of them will make it past
>            the reviewers' defensive line into trunk; it's also simply not
>            realistic to require reviewers to review every combination - e.g.
>            how can a reviewer really review my HP SAN patch without an HP
>            SAN? *There will be issues in trunk, and if we have to revert them
>            rather than just fixing them it will slow us down. *The current
>            situation is bad for our users and bad for developers.*
>         2. One way we could keep everyone happy is by using our test suite to
>            auto-merge from an 'unstable trunk' into 'stable trunk', only once
>            code passes tests. *Commits would initially merge into 'unstable
>            trunk', and we would try to keep that branch moving forwards
>            rather than reverting things that go wrong. *Of course,
>            maintaining a good 'stable trunk' relies on having good tests, but
>            I think we're getting there. *It's also great incentive to write
>            good smoke tests.
>         3. Jay: I believe you've done this to great success on the Drizzle
>            project?
>        Justin
> 
>        On Thu, Feb 24, 2011 at 6:13 AM, Jay Pipes <jaypipes@xxxxxxxxx> wrote:
> 
>          Hi all,
> 
>          I'd like to bring up an alternate reason why it was approved and
>          subsequently reverted.
> 
>          The test cases for the OpenStack API (and much of the EC2 API)
>          assume
>          way too many things and mock out too many things. In addition, since
>          there are zero smoketests for the OpenStack API, there were no
>          functional tests that would have *immediately* highlighted this
>          problem (and many other recent EC2 vs OS API problems).
> 
>          In other words, sure, we should revert the patch to "fix things",
>          however the priority should *not* be on refactoring the auth API or
>          the way the auth layer in Nova is handled. The priority should be on
>          writing a smoketest for the OpenStack API so that we can link it
>          into
>          Hudson and these types of issues can be automatically caught.
>          -jay
>          On Wed, Feb 23, 2011 at 10:03 PM, Paul Voccio
>          <paul.voccio@xxxxxxxxxxxxx> wrote:
>          > Justin,
>          > I think you hit upon the reason of why I think it was approved and
>          reverted.
>          > Because it hadn't been talked about in a blueprint or a mail sent
>          to the
>          > list (I think I'm up to date on the threads) and a patch landed
>          means other
>          > alternatives weren't considered before pushing it through to begin
>          with. I
>          > think we're all open to talking about how to better the auth
>          system and make
>          > improvements. Dragon has already discussed some alternatives and
>          suggestions
>          > on the BP page below. I think this is the right way to continue
>          the dialog
>          > and we all can agree on a good way forward.
>          > I'm confident we can figure it out.
>          > If I missed a conversation, my apologies.
>          > pvo
>          > From: Vishvananda Ishaya <vishvananda@xxxxxxxxx>
>          > Date: Wed, 23 Feb 2011 18:19:41 -0800
>          > To: Justin Santa Barbara <justin@xxxxxxxxxxxx>
>          > Cc: <openstack@xxxxxxxxxxxxxxxxxxx>
>          > Subject: Re: [Openstack] Should the OpenStack API re-use the EC2
>          > credentials?
>          >
>          > Hey Justin,
>          > Does it make any difference that the way the auth is
>          (theoretically)
>          > supposed to work with the os api is that the user gets an auth
>          token from an
>          > external auth server and then uses username / authtoken to
>          actually contact
>          > the api? *I think it is just faked out right now to use the
>          access_key
>          > instead of doing external auth, but I think the reason it works
>          like it does
>          > is because the plan was to switch to external auth eventually.
>          > Vish
>          > On Feb 23, 2011, at 5:56 PM, Justin Santa Barbara wrote:
>          >
>          > I previously fixed OpenStack authentication so it would use the
>          same
>          > credentials as EC2. *This bugfix was just reverted, because it
>          caused
>          > OpenStack API users to have to enter in different credentials
>          (sorry!), but
>          > primarily because it hadn't been*discussed*on the mailing list.
>          *So here
>          > goes!
>          > Here's a
>          >
>          blueprint:*https://blueprints.launchpad.net/nova/+spec/authentication-consistency
>          > Here's an overview of the problem:
>          > EC2 uses an (api_key, api_secret) pair. *Post-revert, OpenStack
>          uses the
>          > api_key(!) as the password, but a different value entirely as the
>          username:
>          > (username, api_key). *The bugfix made it so that both APIs used
>          the EC2
>          > credentials (api_key, api_secret) . *This did mean that anyone
>          that had
>          > saved the 'bad' OpenStack credentials was unable to continue to
>          use those
>          > credentials. *I also overlooked exporting the updated credentials
>          in novarc
>          > (though a merge request was pending).
>          > I actually thought originally that this was a straight-up bug,
>          rather than a
>          > design 'decision', so I should definitely have flagged it better.
>          *Again,
>          > sorry to those I impacted.
>          > As things stand now, post-revert, this is probably a security
>          flaw, because
>          > the EC2 API does not treat the api_key as a secret. *The EC2 API
>          can
>          > (relatively) safely be run over non-SSL, because it uses
>          signatures instead
>          > of passing the shared secret directly.
>          > This is also not very user-friendly. *Post-revert, an end-user
>          must know
>          > whether any particular cloud tool uses the EC2 API or the
>          OpenStack API, so
>          > that they can enter in the correct pair of credentials. *That
>          doesn't seem
>          > like a good idea; I think there should be one set of credentials.
>          >
>          > There is some discussion about the idea of having the api_key be
>          > user-friendly. *I don't think it buys us anything, because the
>          api_secret is
>          > still going to be un-friendly, but I have no objection as long as
>          it is does
>          > in a way that does not break existing users of the EC2 API.
>          > I propose that:
>          > *(1) the OpenStack API and EC2 credentials should be the same as
>          each other
>          > (whatever they are) for the sake of our collective sanity and
>          > *(2) we have to change the current configuration anyway for
>          security
>          > reasons.
>          > *(3) We should not change the EC2 credentials, because we've
>          shipped the EC2
>          > API and our users have an expectation that we won't break them
>          without good
>          > reason, so
>          > *(4) we must change the credentials for users of the (non-shipped)
>          OpenStack
>          > API.
>          > Estimated user impact: I believe there are two people that will be
>          affected,
>          > and it will take them ~1 minute each, so total impact ~2 minutes.
>          > The longer we delay fixing this, the more people we break and the
>          bigger the
>          > impact. *It seems that we have no choice but to do a
>          > non-backwards-compatible authentication change, but I believe this
>          is OK at
>          > the moment because the OpenStack API is not yet stable/released -
>          i.e. we
>          > can still make fixes without worrying about backwards
>          compatibility shims.
>          > We're not in "The Old New Thing" land yet :-)
>          >
>          >
>          > As an aside, I am very unhappy about the way this revert was
>          pushed through
>          > by Rackspace team-members, seemingly without much consideration of
>          > alternatives. *Perhaps we should consider changing from needing
>          two
>          > core-approves, to needing one Rackspace core-approve and one
>          non-Rackspace
>          > core-approve.
>          >
>          > Justin
>          >
>          >
>          > _______________________________________________
>          > Mailing list: https://launchpad.net/~openstack
>          > Post to ****: openstack@xxxxxxxxxxxxxxxxxxx
>          > Unsubscribe : https://launchpad.net/~openstack
>          > More help **: https://help.launchpad.net/ListHelp
>          >
>          > _______________________________________________ Mailing list:
>          > https://launchpad.net/~openstack Post to :
>          openstack@xxxxxxxxxxxxxxxxxxx
>          > Unsubscribe : https://launchpad.net/~openstack More help :
>          > https://help.launchpad.net/ListHelp
>          >
>          > Confidentiality Notice: This e-mail message (including any
>          attached or
>          > embedded documents) is intended for the exclusive and confidential
>          use of
>          > the
>          > individual or entity to which this message is addressed, and
>          unless
>          > otherwise
>          > expressly indicated, is confidential and privileged information of
>          > Rackspace.
>          > Any dissemination, distribution or copying of the enclosed
>          material is
>          > prohibited.
>          > If you receive this transmission in error, please notify us
>          immediately by
>          > e-mail
>          > at abuse@xxxxxxxxxxxxx, and delete the original message.
>          > Your cooperation is appreciated.
>          >
>          > _______________________________________________
>          > Mailing list: https://launchpad.net/~openstack
>          > Post to * * : openstack@xxxxxxxxxxxxxxxxxxx
>          > Unsubscribe : https://launchpad.net/~openstack
>          > More help * : https://help.launchpad.net/ListHelp
>          >
>          >
> 
>        _______________________________________________
>        Mailing list: https://launchpad.net/~openstack
>        Post to * * : openstack@xxxxxxxxxxxxxxxxxxx
>        Unsubscribe : https://launchpad.net/~openstack
>        More help * : https://help.launchpad.net/ListHelp
> 
>  Confidentiality Notice: This e-mail message (including any attached or
>  embedded documents) is intended for the exclusive and confidential use of the
>  individual or entity to which this message is addressed, and unless otherwise
>  expressly indicated, is confidential and privileged information of Rackspace.
>  Any dissemination, distribution or copying of the enclosed material is prohibited.
>  If you receive this transmission in error, please notify us immediately by e-mail
>  at abuse@xxxxxxxxxxxxx, and delete the original message.
>  Your cooperation is appreciated.

> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to     : openstack@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp