openstack team mailing list archive

[Thomas Goirand <thomas@xxxxxxxxxx>]

Hi,

Thanks Ola, for this work.

I am Cc-ing the launchpad list of Openstack, to put the Ubuntu devs in
the loop. For those who don't know about Debian Openstack packaging
team, here's the team:
- Thomas Goirand (zigo): myself
- Loic Dachary <loic@xxxxxxxxxxxx>
- Ghe Rivero <ghe@xxxxxxxxxx>
- Julien Danjou <acid@xxxxxxxxxx>
- Ola Lundqvist <opal@xxxxxxxxxx>

There may have been some other contributors (especially Debconf
translations), but that's the main team. I'm not sure if Julien still
wants to get involved in the Debian packaging of Openstack, since he is
busy on the Ceilometer project.

What happened is that Debian and Ubuntu packages have drifted in
different direction, since Julien did his work about a year ago. This
isn't necessary a bad thing, we can both learn from this, and enhance
both packages. But at some point, this has to stop: we are duplicating
our work, and that's not a good thing.

One of the problem we have in Debian, is that we have a strong allergy
to bzr. We use Git, and we are happy to do so, and I don't believe that
any of us wishes to go back to using bzr, for the same reasons Openstack
itself moved to Github. Also, since we use Git, it's easy for us to
import "upstream" code, which is a way better.

To the Ubuntu packaging team: would you consider working with Git for
the Openstack packages?

This way, we would have a chance to merge our work. If this doesn't
happen, I don't think any of the Debian Developers working on Openstack
would like to use bzr, which would make collaboration quite difficult.

If Ubuntu people move to Git, then at least *I* would be interested in
working together with Ubuntu folks to re-merge the packaging work. This
would be very beneficial to both Ubuntu and Debian, and what's below
proves it. At least, Ubuntu would benefit from our Debconf work AND all
the l10n translations, making it a lot more easy to install Openstack.

Let me give my comments what I see/know in the differences commented by
Ola below.

On 09/05/2012 03:42 AM, Ola Lundqvist wrote:
> High level summary:
> ===================
>  - The diff is very large. You can get a feeling about the difference by just
>    reading debian/control below.
>  - Debian dependency for nova-compute-xen, nova-api-* should probably be adjusted.

nova-compute-xen are ok, it's the Ubuntu side which is wrongly using
libvirt. I'm not sure for nova-api-*, can anyone comment?

>  - Adjustment to the rest should be considered.

It's a little bit too late in the release cycle, I'm fearing the release
team reaction here.

>  - Debian has usr/bin/nova-consoleauth part of nova-console package while ubuntu
>    has it in the nova-consoleauth package. Debian still provide the nova-consoleauth
>    package but it seems to be quite empty.
>  - Quite different .init files.

Why did you even bother commenting the .init / .upstart files? It's
clear enough: Ubuntu uses upstart, and we use sysv-rc. If Ubuntu still
has some .init files, it's because I did the effort, about a year ago,
to have my work merged there. But since, it hasn't been merged, so what
you see in Ubuntu is a *very* old version of what I did.

>  - Different patch set.

That's is a lot more worrisome.

> Detailed summary below:
> =======================
> 
> debian/control
> --------------
>  - build dependency differences. The difference is large, so it is
>    better to look into the detailed diff file instead. In general
>    ubuntu has a large list of Build-Depends-Indep and just two
>    small ones for Build-Depends. Debian has the opposite.

I don't think that's a huge problem. Maybe we should fix that in
Wheezy+1. Also, it's to be noted that there's so many build dependency,
only to satisfy the unit tests.

>  general:
>   - Different descriptions for the binary packages
>   - When ubuntu use ${ostack-lsb-base}

That also comes from a year and a half ago, when I tried to merge stuff.
Then Julien came along, and deleted all my Ubuntu compat work.

> debian use lsb-base (>= 3.0.6) or lsb-base or
>     ${ostack-lsb-base}

There shouldn't be a version depend on lsb-base. Any lsb-base is fine, IMO.

>  python-nova:
>  - Dependency difference for python-nova. Ubuntu has the following depds
>     that debian is lacking:
>       openssh-client, python-netaddr, python-feedparser,
>       python-xattr, python-daemon, python-suds, python-iso8601

Is this with the package in Precise? If so, then we *must* fix this.

>  - ubuntu provide XB-Python-Version: ${python:Versions} for python-nova.
>  nova-common:

We don't care about XB-Python-Version, IMO.

>  - Dependency difference for nova-common where debian has the following
>     that ubuntu is lacking.
>       python-iso8601, dbconfig-common, sqlite3
>    Debian suggest: python-pysqlite2 | python-mysqldb | python-pygresql while
>     ubuntu suggests: python-keystone

We shouldn't really care about suggests...

>  - Ubuntu provides Provides: ${python:Provides} for nova-common, debian
>    does not do that

IMO, not a big deal, even though this should be fixed, so that
dh_python2 can do its work.

>  nova-compute:
>  - Debian depend on adduser, ubuntu does not.

Sure, since we use adduser, we *must* depend on it.

>  - Debian has a replaces and breaks that ubuntu is lacking. The breaks is
>    not really necessary, the replaces should be sufficient.

Agreed.

>  nova-compute-lxc:
>  - Dependency difference:
>    ubuntu: nova-compute (= ${binary:Version})
>    debian: adduser, nova-common

Has to be fixed (eg: we should add nova-compute as dependency).

>  - Conflict differences:
>    ubuntu: nova-compute-hypervisor
>    debian: nova-compute-kvm, nova-compute-qemu, nova-compute-uml, nova-compute-xen
>  nova-compute-uml:
>  - Same diff as for nova-compute-lxc except the conflicts where you replace uml with lxc.
>  nova-compute-qemu:
>  - Same diff as for nova-compute-lxc except the conflicts where you replace qemu with lxc.
>  nova-compute-kvm:
>  - Same diff as for nova-compute-lxc except the conflicts where you replace kvm with lxc.

These are IMO ok.

>  nova-compute-xen:
>  - Dependency difference:
>    ubuntu: nova-compute (= ${binary:Version}), python-libvirt, libvirt-bin, xen-hypervisor-4.1-amd64 | xen-hypervisor-4.1-i386
>    debian: python-xenapi, adduser, nova-common

We got it right, and Ubuntu got it wrong. Openstack is completely
untested with libvirt + Xen. It's supposed to work with XCP, and even
though, I now believe that it wouldn't work well, and that XenServer
would be a better choice than XCP in Debian.

>  nova-compute-xcp:
>  - Debian does not provide this package at all. Ubuntu does.

It's completely stupid to have nova-compute-xcp and nova-compute-xen.
Only nova-compute-xcp should exist. Probably, we should have named our
package this way instead.

Maybe this has changed and I'm all wrong, and we do have libvirt support
for Xen. I'd be happy to have returns from Ubuntu people on this.

>  nova-ajax-console-proxy:
>  - Debian does not provide this package at all. Ubuntu does.
>  nova-network:
>  - Debian has a replaces & breaks against python-nova (<< 2012.1~rc1-1)
>  nova-volume:
>  - Debian has a replaces & breaks against python-nova (<< 2012.1~rc1-1)
>  nova-vncproxy & nova-xvpvncproxy:
>  - Ubuntu has nova-vncproxy while debian has nova-xvpvncproxy (that provides nova-vncproxy)
>  - Ubuntu has a conflict against novnc, while debian does not.
>  nova-consoleauth:
>  - Ubuntu provides this pacakge. Debian does not, instead nova-console provide
>    nova-consoleauth in debian.
>  nova-console:
>  - Ubuntu merely recommend nova-consoleauth, where debian include it.
>  - ubuntu has a breaks replaces on nova-console (<< 2012.1~rc1-0ubuntu2)

I have no experience with the above. Can anyone comment?

>  nova-doc:
>  - Debian adds libjs-jquery, libjs-underscore to the dependencies.

If Ubuntu doesn't have it, it means the doc package is using the
embedded jquery of openstack, which is *wrong*.

>  nova-api-metadata:
>  - Debian adds reaplces and breaks.
>  nova-api-os-compute, nova-api-os-volume, nova-api-ec2:
>  - Ubuntu has an unversioned breaks to nova-api which looks odd. It should be versioned
>    as in the Debian package variant.
>  - The debian replaces, should be versioned. This is a bug.
>  nova-api-metadata & nova-api-os-compute & nova-api-os-volume & nova-api-ec2:
>  - ubuntu has a Breaks: nova-api. It should be versioned.
>  - debian has a versioned breaks but has a replaces which is unversioned. It should
>    be versioned.

Can you fix the above in our Git?

Also, in your general description, you missed the fact that I did the
work of packaging nova-xcp-network and nova-xcp-plugins which are
missing from Ubuntu.

> debian/rules
> ------------
>  - different clean rules
>  - different dh command
>     ubuntu: dh $@ ${WITH_PYTHON2}
>     debian: dh $@ --with python2

This is the first time I see ${WITH_PYTHON2}. Does it do the same as
--with python2?

>  - ubuntu package generates nova-compute.postinst from nova-compute.postinst.in

That's also a remaining of my work a year and a half ago. I wish this
stayed in our Debian packages, but it didn't.

>  - debian has special handling for nova-compute-xen.conf
>  - different auto-test command
>  - differences in what .upstart and .init files that are installed.

Of course. Before, it use to have dpkg-vendor stuff for that, so we
could use the same packages.

>  - debian has special handling for nova-consoleauth init file, see also below.
>  - debian has rules for po handling, ubuntu does not.
>  - ubuntu has rules to override dh_installlogrotate, debian does not.
> 
> debian/copyright:
> -----------------
>  - Some differences. Debian has added a specific for bin/nova-manage.
> 
> debian/source_nova.py
> ---------------------
>  - ubuntu list nova-cert in "attach_related_packages", ubuntu does not.
> 
> debian/mans/nova-network.8
> --------------------------
>  - spelling correction in debian
> 
> debian/*.upstart*
> -----------------
>  - ubuntu provides these, debian does not

Of course... (see above)

> debian/nova-api.init
> debian/nova-console.init
> debian/nova-network.init
> debian/nova-objectstore.init
> debian/nova-scheduler.init
> debian/nova-xcp-network.init
> debian/nova-xvpvncproxy.init
> ------------------------
>  - Quite large difference but is possible to summarize.
>  - Debian adds lsb init functions.
>  - Debian loads defaults file, and has adjustments for the $NOVA_ENABLE flag.

Ubuntu doesn't care about these, the old versions in Ubuntu are from me.

> debian/nova-api-metadata.ini
> debian/nova-api-os-compute.init
> debian/nova-api-ec2.init
> debian/nova-api-os-volume.init
> debian/nova-cert.init
> debian/nova-compute.init
> debian/nova-volume.init
> ----------------------------
>  - large differences, see the detailed diff

Same remarks.

> debian/nova-api.postrm
> debian/nova-cert.postrm
> ----------------------
>  - Debian provides this file. It removes log files on purge.

It's an Ubuntu bug (eg: Debian policy violation, owned files after purge).

> debian/nova-common.install
> --------------------------
>  - ubuntu install debian/nova.conf to etc/nova, debian does not.
>    Instead this is handled in .postinst file, see below.

Probably, we are missing the etc/nova/policy.json file. What is this one
for?

> debian/nova-common.nova-manage.logrotate
> ----------------------------------------
>  - ubuntu provides this, debian does not.

This is a bug in our package that should be fixed.

> debian/nova-common.postinst
> ---------------------------
>  - Ubuntu checks for the existance of an user/group before creating.
>    Debian should do the same.

You didn't read the script correctly then. The debian package uses
getent, which is the correct way (tm) to do things. It's the Ubuntu
package that is wrong here, IMO.

Also, the Ubuntu postinst feels very "hackish". So many dpkg
--compare-version calls...

>  - Handling for nova.conf file instead of marking it as a conffile.
>  - ubuntu alter group permissions and file permissions.

Well, we do as well, but I believe that both are wrong here, as we are
here in a race condition attack scenario. Also, the Debian package is
missing the chmod for nova-compute.conf, and this could potentially
includes a password for XenAPI, so that's *bad*.

>  - debian setup database and change configuration accordinly.
> 
> debian/nova-common.postrm
> -------------------------
>  - Debian provides this to handle purge, including purge from db.
> 
> debian/nova-common.prerm
> ------------------------
>  - Debian provides this for db handling.
> 
> debian/nova-common.templates
> ----------------------------
>  - Debian provides this file.
> 
> debian/nova-compute.install
> debian/nova-network.install
> debian/nova-volume.install
> ---------------------------
>  - Debian provides the compute.py files in the package. Ubuntu has them elsewhere.
> 
> debian/nova-compute.pyinstall
> debian/nova-network.pyinstall
> debian/nova-volume.pyinstall
> -----------------------------
>  - Ubuntu has this file. Is this a way to do the same as in the .install file above?
> 
> debian/nova-compute-*.conf
> --------------------------
>  - Debian has [DEFAULT] set with connection type and libvirt_type.

That's Ghe who did it, I still don't know what is the purpose of this.

> nova-compute-lxc.postinst
> -------------------------
>  - Ubuntu set permissions to nova user and group for nova-compute.conf.
>  - Debian adds a libvirt user, but does not set any permissions for it.
>    Also debian does not check for the existance of that user before
>    creating it.

Probably, we are wrong in Debian, since nobody tested with LXC.

> nova/debian/nova-compute.postinst.in
> ------------------------------------
>  - Ubuntu creates a nova user. The odd thing is that it checks for
>    the nova group and if it does not exist, it creates the nova user...
> debian/nova-compute-qemu.postinst
> ---------------------------------
>  - ubuntu change the the permission for /etc/nova/nova-compute.conf
>    to nova user and change its permission to user readable.
> 
> debian/nova-compute-uml.postinst
> --------------------------------
>  - ubuntu change the the permission for /etc/nova/nova-compute.conf
>    to nova user and change its permission to user readable.
>  - Debian add nova user to libvirt group
> 
> debian/nova-compute-xen.config
> ------------------------------
>  - Debian provides this file, ubuntu does not.
> 
> debian/nova-compute-xen.postinst
> --------------------------------
>  - ubuntu change the the permission for /etc/nova/nova-compute.conf
>    to nova user and change its permission to user readable.
>  - debian add nova user to nova group
>  - debian has other handling of config file
>  - debian has support for configuration questions using debconf
> 
> debian/nova-compute-xen.templates
> ---------------------------------
>  - debian has support for configuration questions using debconf
> 
> debian/nova-compute-xen.postrm
> ------------------------------
>  - debian remove /etc/nova/nova-compute.conf at purge

Since it's a conffile in Ubuntu, that's fine.

> debian/nova-compute-xen.conf
> debian/nova.conf
> ----------------------------
>  - ubuntu provides these files, debian does not

Handled in our postinst. That's normal.

> debian/nova.conf.dist
> ---------------------
>  - debian provide a few more default options
> 
> debian/nova-consoleauth.*
> debian/nova-console.nova-consoleauth.init
> -----------------------------------------
>  - ubuntu provide these files, debian does not
>    The exception is that debian has a nova-consoleauth.init file
>    but on the nova-console package instead,
>    nova/debian/nova-console.nova-consoleauth.init. This seems to be
>    handled in debian/rules file in an uncommon way.
> 
> debian/nova-console.install
> ---------------------------
>  - debian has usr/bin/nova-consoleauth as part of this package.
> 
> debian/nova-console.postrm
> debian/nova-network.postrm
> debian/nova-objectstore.postrm
> debian/nova-scheduler.postrm
> debian/nova-compute.postrm
> debian/nova-volume.postrm
> debian/nova-xvpvncproxy.postrm
> --------------------------
>  - debian remove log file at purge, ubuntu dues not

Then Ubuntu does a Debian policy violation. Probably, we added these
after the tests from Lucas Nussbaum.

> debian/nova-network.logrotate
> -----------------------------
>  - debian has the /var/log/nova/nova-dhcpbridge.log in nova-network.logrotate
>    while ubuntu has it in its own configuration file
>    nova-network.nova-dhcpbridge.logrotate.

Both approaches are fine, IMO.

> debian/nova-objectstore.logrotate
> ---------------------------------
>  - different way to restart nova-objectstore

Because Ubuntu uses upstart...

> debian/nova_sudoers
> -------------------
>  - ubuntu has the line
>    Defaults:nova !requiretty
> debian/nova*vncproxy*
> ---------------------
>  - ubuntu has the package name nova-vncproxy while
>    debian name is nova-xvpvncproxy.
>  - The detailed diff per file is not provided.
> 
> debian/nova-xcp-network.postinst
> debian/nova-xcp-plugins.postinst
> --------------------------------
>  - debian package makes a chmod on a number of plugin files to
>    make sure they are executeable. This is a bit of an unusual
>    practice to do this way.
>    The plugins to maek executable differ between packages.
> 
> debian/python-nova.postinst
> ---------------------------
>  - debian has "update-python-modules --post-install"
> 
> debian/nova-xvpvncproxy.manpages
> debian/mans/nova-vncproxy.8
> ---------------------------------
>  - debian provides this, ubuntu does not
> 
> debian/patches/*
> ----------------
>  ubuntu:
>    kombu_tests_timeout.patch
>    fix-ubuntu-tests.patch
>    fix-docs-build-without-network.patch
>    0001-fix-useexisting-deprecation-warnings.patch
>    fix-pep8-errors.patch
>  debian:
>    stable_essex_20120710.patch
>    iscsiadm_path.patch

We really need to address these, especially before Wheezy is released.
Does anyone know which one should urgently be added in Debian?

> debian/po/*
> ------------
>  - debian provides this, ubuntu does not
> 
> You have made it! You read it though. :-) The even more detailed
> diff is attached. Please note that some differences are just
> differences because files are re-ordered or renamed.
> 
> Best regards,
> 
> // Ola
> 
> PS. I'm subscribed to the list, but I do not read it every day
> so please Cc me if you remember. :-)
> DS.
> 
>