openstack team mailing list archive

[Thomas Goirand <thomas@xxxxxxxxxx>]

On 12/04/2012 09:06 PM, Thierry Carrez wrote:
> A bit of history here:
> 
> We used to rely on MANIFEST.in to list all files, but people routinely
> forgot to add new files to it. Apparently "every Python developer"
> doesn't know (or care) about this. The end result was that we
> discovered very late (sometimes after the release itself) that we
> built incomplete tarballs. As a quick search[1] shows, I have
> personally filed 27 bugs so far on the subject, so it's not a corner case.
> 
> [1] http://bit.ly/TDim7U
> 
> Relying on setuptools_git instead allows to avoid that issue
> altogether. The projects that adopted it became a non-issue. The
> projects that didn't adopt it yet are still a problem. I was about to
> push setuptools_git support to projects that don't use it yet.
> 
> In summary, I would hate it if we went back to the previous situation.
> I'm not personally attached to setuptools_git, but any proposed
> replacement solution should keep its simplicity.

With my Debian Developer in charge of Openstack, I have to tell that
this was a real problem for us, in Debian, as well.

On Debian, we don't use released tarballs like they do on Ubuntu. We use
git-buildpackage, and do our packaging exclusively with Git (no bzr at
all here...). All of our packaging work is stored on alioth.debian.org,
on the pkg-openstack git repository. We don't even store the master
branch, it is picked-up doing:

./debian/rules get-vcs-source

which on top of that, creates a .orig.tar.xz out of the upstream source
code. (yes! Because xz is so much nicer than gzip...)

The problem isn't to use setuptools_git for building, you're fine to do
that, the problem is having the Git repositories not holding the correct
MANIFEST.in file, and fooling the people doing packaging using Git. This
means that absolutely all of our packages have to embed a patch in
debian/patches to "fix" the "wrong" MANIFEST.in.

We've spent quite some time on that. Or rather, should I say: it's a
real time waster.

While I do agree that the MANIFEST.in should be generated automatically,
I don't think it should be stored in a "wrong" way on github.

The magic solution could be to have some automatic tools to take care of
it. Like, for example, jenkins, or maybe a git hook? I'm not sure, how,
but I'm quite convince it should be workable.

Also, we've been having troubles with clean targets accessing the
network at build time, because they see a ".git" folder. That's wrong
too, when I do "git-buildpackage", I do have a .git folder, though I
don't want to have any network access, even less a "git fetch origin" or
something like this. This can be mitigated by doing:

echo "cleaner = true" >>debian/gbp.conf

I've done that, but that's quite annoying to have to disable the clean
before the build (note: it's only not cleaning my git repo, but it's
really calling clean later on at build time in ../build-area).

[No pun intended, but it'd be nice if stackers had a bit more
consideration for our work in Debian, and stop thinking only with
Ubuntu, Ubuntu, Ubuntu, Ubuntu, Ubuntu, Ubuntu, ... in mind.]

Cheers,

Thomas Goirand (zigo)

P.S: I'm currently uploading Folsom to debian/experimental, after 2
months of hard work, adding debconf and dbconfig-common support. Half
already uploaded, I expect the 2nd half to be done this week, pending of
course, approval from ftp-masters. Please have a look at our howto:
https://wiki.debian.org/OpenStackHowto/Folsom