openstack team mailing list archive

[Thierry Carrez <thierry@xxxxxxxxxxxxx>]

Mark McLoughlin wrote:
> We discussed this during the "baking area for features" design summit
> session. I found that discussion fairly frustrating because there were
> so many of us involved and we all were either wanting to discuss
> slightly different things or had a slightly different understanding of
> what we were discussing. So, here's my attempt to put some more
> structure on the discussion.

Thanks for that ! I'd not yet found the time to restart that discussion.
The session was a bit frustrating indeed: I've been pushing those ideas
around the end of the Essex cycle, but with release work we couldn't
come to a ready-to-implement solution that we could have switched to for
Folsom. The session was not completely useless though: there was general
consensus that this was the way to go, but that we had to carefully
experiment with this in the current cycle in order to be potentially
ready to propose a significant model evolution for the next cycle.

> Firstly, problem definition:
> 
>   - Nova is big, complex and has a fairly massive rate of churn. While 
>     the nova-core team is big, there isn't enough careful review going 
>     on by experts in particular areas and there's a consistently large
>     backlog of reviews.

I think smaller teams of dedicated area-oriented reviewers will indeed
go a long way to solve the issues of speed and quality of review.

>   - Developers working on features are very keen to have their work 
>     land somewhere and this leads to half-finished features being 
>     merged onto master rather than developers collaborating to get a 
>     feature to a level of completeness and polish before merging into 
>     master.

Yes, the absence of state between "in master" and "nowhere" led to both
half-implemented (incomplete) and half-baked (not release-ready yet)
features landing in master, milestones and potentially the release.

> Some assumptions about the solution:
> [...]

Agree on all points.

> Ok, what are "subsystem branches" and how would they work?
> [...]
>   - It would be up to the project dictators to help drive patches 
>     through the right subsystem branches - e.g. they might object if 
>     one subsystem maintainer merged a patch that inappropriately cut 
>     into another subsystem or they might refuse to merge a given patch
>     into the main branch unless it went through the appropriate 
>     subsystem branch.
> 
>     (In gerrit, this would mean a small team or single person who can 
>     +1 approve merge proposals on master. They would -1 proposals
>     submitted against master which should have been submitted against a 
>     subsystem branch.)

IIUC under this model we would still accept changes to be directly
proposed to master (think bug fixes, or features that don't belong into
a specific subsystem) ? For those we would still need a review team
(separate from the "project dictators" who do the second-stage review)
with wide expertise on all things Nova.

> [...]
>   - Plausible subsystem branches are e.g.:
> 
>       - OpenStack APIs
>       - EC2 API
>       - virt
>          - libvirt driver
>          - xenapi driver
>          - vmware driver
>       - networking
>       - volumes
>       - scheduler
>       - RPC
> 
>     Deciding which areas make sense as a subsystem branch is 
>     non-trivial.
> 
>     Should there be a "DB" subsystem? Probably not, because that would 
>     mean every new feature needs to come through this branch or, 
>     alternatively, the DB maintainer would need to accept DB schema 
>     additions without the context of how it's being used higher up the 
>     stack.
> 
>     Ok, so why does it make sense to have an "OpenStack APIs" 
>     subsystem? Don't all features affect that branch too? Well, maybe, 
>     but the APIs really do need strong oversight. Perhaps we can be 
>     confident that we can add e.g. a new scheduler feature through the
>     scheduler branch and then later merge any API additions through the 
>     APIs branch.

I think there are two types of subsystems making sense: those areas of
code that are already separated into a self-contained module, and those
areas of the code where editorial control / extra expertise is needed.

As an example, sometimes I wish that all proposed run-as-root commands
would be through a security review before being accepted in master
(because sometimes they are overkill and needlessly increase the
run-as-root surface, and some other times they annihilate our efforts at
filtering nova->root escalation). This is difficult to do with a common
team of 25+ reviewers.

> And how about feature branches?
> 
>   - Feature branches are relatively short-lived (i.e. weeks or months
>     rather than years) branches for a specific feature. They are a
>     mechanism for developers to work on a patch series in the open until
>     the feature is complete enough to be merged into a subsystem branch
>     or master.
> 
>     (I'm not sure gerrit is right for this. Why not just do it in 
>     folk's github forks? I think all people are looking for is for 
>     people to be more aware of feature branches. How about if you put 
>     details of your feature branch in the blueprint for the feature?)
> 
>     (If not using gerrit, can developers configure Jenkins to CI their 
>     branch? Or is Smokestack the right tool?)

I think preserving the ability to run your branch through integration
testing is a necessary prerequisite of the new model.

Thanks again for starting the discussion on this. I'll meet with Jim
Blair (and hopefully Monty) next week to brainstorm solutions, so
discussing the needed properties of the system is a very nice step forward.

Regards,

-- 
Thierry Carrez (ttx)
Release Manager, OpenStack