openstack team mailing list archive

[Monty Taylor <mordred@xxxxxxxxxxxx>]

On 08/25/2011 12:52 AM, Thierry Carrez wrote:
> What a good way to start a long milestone release day.

Tell me about it. Thank you, however, for your well written and thought
out response.

> Disclaimer: I used to work for Canonical as the Technical Lead for
> Ubuntu Server -- and I'm now OpenStack Release Manager.
> 
> Like Monty said, this is not the first time the "integrated
> distribution" model conflicts with deployers needs. This has been
> discussed quite a few times already, and I happen to know the subject
> quite well.
> 
> Integrated distributions (think Debian or Ubuntu) want you to run the
> pristine packages in their official repositories. They don't want you to
> run your own (and submit bugs when it fails), they don't want you to run
> a custom backport of a required new library. They want you to stay with
> a given version on a given OS, and to upgrade the whole package from
> time to time (Note: This does not affect that much RPM-based
> distributions, where taking a random RPM from a vendor site is
> apparently more accepted, due to most packages missing from the distro
> anyway).

(I think it may have been a while since you worked with or on the
RPM-based distros. They are actually quite complete and much more
debian-like in the way you are describing above these days. The
inclusion/use of yum in fedora and centos truly changed many things)

> Deployers want to be able to run the exact version they want, on the
> exact OS they want. Use the base packages in the OS from a given
> reference point (generally Debian stable or Ubuntu LTS), and then deploy
> key packages at a chosen version, along with any dependency.
> 
> Those goals are not compatible.
> 
> So far we tried to reach both targets. We tightly integrated with the
> Ubuntu release under development. The absence of territorial
> maintainership in Ubuntu allowed us to efficiently push patches or bump
> versions on dependencies so that the future Ubuntu release would have
> all we need, and OpenStack packages could run directly on the pristine
> OS. We aligned the release schedules so that Ubuntu could actually ship
> the latest version of OpenStack packages in their official repositories.
> For all the other use cases, we would provide PPAs (last release, last
> milestone, trunk) x (lucid, maverick, natty, oneiric).
> 
> This is very elegant, especially for ex-Ubuntu developers now working on
> upstream, like Soren or me. But I agree it has a few drawbacks, due to
> the fact it's pretty centered around Ubuntu: it makes the other distros
> second-class citizens, and it makes the "deployer" use case above a
> second-class citizen as well.
> 
> The end result being that some critical deployers ended up not using our
> packages. We can bitch forever how wrong that is, and what they should
> do instead -- I don't think we'll change them. Monty's proposal is
> trying to address that.

Yes. I agree with all of the above. (with the exception of the tiny bit
of fud that snuck in)

> I think his proposal makes sense, but I also see how much we'll lose if
> we follow it. Reading between the lines, Monty's proposal is about
> losing the Ubuntu-integrated development, and providing our own packages
> (+ any needed dep) in our own repositories for a set of supported
> platforms. Then distributions are free to try to take those packages and
> ship them in their own repositories, or not.

This is where I either disagree with you, or where I may not have spoken
clearly enough.

I do NOT think we need to lose the Ubuntu-integrated development. And I
do not think that we should just willy-nilly go in to a free for all. I
believe that, as you say below, there are excellent things to be gained
from it. One of the reasons that I've forced all of the jenkins and
tarmac builders to use packages rather than virtualenv from day one is
so that we know that it's POSSIBLE to get packaged for upcoming ubuntu.
I believe we should still maintain that constraint... I just don't think
that it's my job to actually make that happen.

> The first and most immediate drawback in my opinion is that we lose the
> very experienced Debian packaging team that we had set up. By closely
> working with Ubuntu upstream, we could leverage their manpower and
> experience to benefit our packaging. Bugs reported against Ubuntu
> packaging were in fact reported against our common work, and we got
> their bugfixes for free. If we move forward with this plan, the Ubuntu
> devs will retreat into being pure consumers (like with every other
> upstream), adding patches for their needs in their own version of the
> packaging. It's a loss that shouldn't be underestimated, especially by
> people that so far contributed nothing to the existing packaging.

I do not believe that we need to lose the very experienced Debian
packaging team. I highly value them and their work, as much as I value
the work of pvo and gholt and the other deployers. In a properly set up
system, it should be the most attractive thing to submit packaging
patches to the core, and only to maintain differences when it's just not
possible for the things to go in to core. I believe with all of the
tools at our disposal, this is totally workable. (even if we move
packaging to git-buildpackage base repos - launchpad can still track
upstream git repos,

> The second drawback is that without integration, the incentive to make
> the environments converge is limited. So far when we added a dependency
> there were discussions with Ubuntu as to what the common version should
> be. Now OpenStack will just pick what it prefers and shove it into its
> repos. Using packages from official repositories is more stable: their
> combination with the OS is more tested, and we benefit from OS bugfixes.
> So I expect the end result will be a less-integrated and more buggy
> environment.

I strongly strongly strongly disagree with you here. I believe that the
incentive is quite high - because the more we can keep the environments
convergent, the more we can have different people working on the core
packaging.

Remember that, although I believe we will be FORCED to maintain
backports of packages, it's still work to do so. Therefor, just randomly
grabbing a bazillion new things from PyPI isn't going to stop being
annoying.

> The last (and not least) drawback is that by following this plan, we
> become distributors. That means we have to care about support
> timeframes, point releases and security updates. We can no longer
> pretend that the downstream distributions will do it for us. This is a
> huge undertaking... who exactly is going to do that maintenance ? Is it
> really the best way to spend our time ?

We are already distributors, and we already have to care about support
timeframes, point releases and security updates. That's the reason that
we have things set up in release series. If we were not distributors, we
could just have a linus-style rolling trunk and say screw it. We have
not done that.

> Last remark, to my fellow Frenchman Thomas, who sounds very supportive
> of Monty's proposal: I'm far from certain the end result would simplify
> your life. You'll still have to chase all dependency maintainers so that
> they align their version and patches with the ones OpenStack will
> require. Debian's use case is the same as Ubuntu's above -- you want
> people to use the main repository. Having an upstream ship its own
> overlapping debs should scare you. Unless your personal need, as an
> OpenStack user, is more on the deployer use case ?
> 

I do not think anyone should be scared of anything. Certainly no one
should fear upstreams shipping debs.

However, I'm asserting that, using your excellent terminology, most of
the folks deploying openstack are going to fall in to the deployer use
case. We aren't exactly talking about a web browser here. :)

Monty

PS. Even web browsers are skipping the integrated distro model and
moving to a app-centric deployer model. What version of Chromium does
Ubuntnu ship? I don't care. The real question is what is the latest
version of Chromium from Google.