openstack team mailing list archive

[Vishvananda Ishaya <vishvananda@xxxxxxxxx>]

Hello everyone,

Sorry I didn't respond to this earlier.  I have been extremely busy trying to tie things up for e-4 so that we can start producing and testing release candidates for essex. Soren, you make some excellent points that we should address regardless of the results of the election.  Most of them I have been thinking a lot about myself.  I've put some of my main thoughts on my election wiki page here: http://wiki.openstack.org/Governance/ElectionsSpring2012/Vishvananda_Ishaya

I am really proud of the progress we have made during Essex towards greater stability. Some of the high points:
 * We switched to a completely new code review and merge system
 * We added over 1000 unit tests
 * We got novaclient into the review system instead of managing it separately
 * We are gating every merge running the whole system with keystone glance and the clients
 * Disk Configuration Parity (this was a big one)

There is always progress to be made, but while we are toiling away working on testing, technical debt, and code smell, we have to keep our users in mind.

I also have some specific comments inline.

On Feb 23, 2012, at 6:17 AM, Soren Hansen wrote:

> I've put my name on the ballot for Nova PTL, and I'd like to explain
> what I expect to do (my platform, if you will).
> 
> Nova is facing many separate, but related problems.
> 
> * Nova is too big.
>   Very few (if any) core developers are comfortable reviewing every
>   part of the code base.  In itself, this isn't necessarily a problem,
>   but I think it would be valuable to try to somehow acknowledge that
>   the average focus is much narrower than "all of nova".

I strongly agree. I have been trying to break out volume and networking for some time now.  The subteam plan was my approach to combat this during essex, and it has worked to some degree but we need a new approach here.

> * Lots of things in Nova that should be orthogonal are not.
>   This problem is especially prevalent in the virtualisation layer. The
>   layout and number of disks you get attached to instances shouldn't
>   depend on the hypervisor you've chosen, but it does. There is lots
>   and lots and lots of logic embedded in both the libvirt and XenServer
>   drivers that isn't related to the hypervisor, but is a result of the
>   origin of these drivers.

I agree with this as well. 

> * The overall quality is decreasing
>   There's an almost unilateral focus on features across the board. The
>   topic of almost every session at the summit is some new feature.
>   There is very little focus on stability, predictability and
>   operation. Personally, I think that shows very clearly in the final
>   product.

I disagree with this point. There was a huge amount of effort from a lot of parties on getting things cleaned up during essex. Most of the "features" that were worked on were getting things aligned and consistent behind our api. We have to make sure that Nova is usable as well as stable. It is tempting to think of things from the developer/code perspective, but we have to put on our user/operator hats as well.

That said, we can always improve in this area.

> 
> I'd like to try to shift our focus and turn the proverbial ship around.
> 
> I'd like to remove any incentive to rush things into Nova trunk.
> 
> 1. A much shorter release cycle (as Thierry also suggests[1]) would be
> very beneficial. Noone wants to have to wait an extra 6 months getting
> some new feature in just because it missed the feature freeze.  However,
> just a single month of delay... That should be manageable in most cases.

I like this idea, although logistically it seems difficult. We should discuss this at the summit.

> 
> 2. I'd like to make it more straight forward to have things mature
> somewhere separete from Nova trunk, but still make it easy to
> collaborate on them or get people to test them.

I'm all for this, assuming we can work out the strategy for shared code, feature branches, and merging.

> 
> 3. I'd like to encourage a stronger focus on QA and testing.
> Specifically, I'd love to have more people focused on making it easier
> to test things in Nova. Tempest is a great effort, but the unit test
> suite is our first line of defence. It should be fast and comprehensive.
> Right now, it's neither.

We added a huge amounts of unit tests during the last cycle.  I would love to see more cleanup and consistency across tests. I was hoping the testing subteam would help this, but we didn't make as much progress as I hoped.  It would be great to improve this.

> 
> 4. I'd like a stronger focus on extensibility and plugability.

+1 to this one. I was hoping to organize a summit discussion around how we can have a sane plugin model that allows us to have plugins managed separately from core, but allows us to integration test them so we don't break, and makes it easy for packagers to ship plugins easily.

> 
> 5. I'd like us to rethink our configuration management strategy. So far,
> we've punted on it and deferred to deployers to choose between Puppet,
> Chef or whatever else to handle this. However, many things will crash
> and burn if the configuration of various components is out of sync with
> each other or with the database. This is particularly clear in the
> networking area.

Design summit discussion?  This is important but it seems like something that a small group of people could focus on.


Overall these are great points, and we should discuss them further at the summit. Nova has grown from a team of 6 developers to over 100 in less than two years. There are growing pains as we figure out how to work together, but as a whole we are doing very well. Some changes are definitely required, but drastic changes in direction will only alienate parts of the wonderful community that we have built.

Vish