bzr-packagers team mailing list archive

[Sidnei da Silva <sidnei.da.silva@xxxxxxxxxxxxx>]

On Tue, Mar 30, 2010 at 10:06 PM, Ian Clatworthy
<ian.clatworthy@xxxxxxxxxxxxx> wrote:
> Hi Sidnei and other buildout gurus,
>
> I want to love buildout - I really do - but I'm struggling to find much
> affection for it right now. :-( My primary issue is that I don't fully
> understand how it works and, despite reasonably good documentation, I'm not
> sure where to look.

Ok. I will try to give the 10.000 feet level overview.

Basically buildout itself doesn't do much, other than providing some
infrastructure around easy_install, for downloading eggs, and doing
stuff with recipes. All the useful stuff is done by recipes. A recipe
exposes entry points (please refer to setuptools if you don't know
what this is) to buildout, namely 'install' and 'uninstall', which
point to callables.

As you might have noticed, there are many 'parts' inside a
buildout.cfg. Generally you list all the parts in order under the
'parts' key on the [buildout] section. When you call 'bin/buildout',
buildout itself will look for .installed.cfg to see if it was called
before and if so, what might have been installed. It compares the
current configuration (as specified in buildout.cfg) with whatever is
in .installed.cfg to detect if some configuration changed. If so, it
will call the 'uninstall' entry point to remove what might have been
installed by the previous invocation, so that it can then call
'install' with a (supposedly) clean environment.

This is where things start to get tricky. As you might have guessed,
your experience will only be as good with buildout as the recipes you
are using. Recipes are free to have a noop 'uninstall' for example,
which many do. This can be good or bad, depending on what you want to
do. The specific bzr recipe we are using tries to play it safe,
refusing to 'uninstall' stuff if there are changes in the checkout it
performed when installing, to avoid losing data. As I said before,
there might be an option provided by the recipe itself to disable this
behavior (or we could add one, I have commit rights).

> Here's the problem. buildout generates a build directory with the various
> parts in it based on what's in a config file, e.g.:
>
> build-win32/
>  bzr/
>    bzr-2.2/
>  bzrtools/
>    bzr-2.2/
>  bzr-foo/
>    bzr-2.2/
>
> So far, so good. If I change the config file though so that bzr-foo is no
> longer included in a particular build, then buildout insists on trying to
> uninstall it. This nearly always fails, breaking the installer build
> altogether. Likewise, if I have bzr-2.1 directories created under each part
> (from a 2.1 installer build run), then it wants to uninstall those
> directories as well when I build 2.2. Again, the installer build falls over
> on something that has nothing to do with building a 2.2 installer. :-(
>
> Why does it do this? Why can't it just leave stuff it doesn't need for a
> particular build there? There's a magic .installed.cfg file inside the
> build-win32 directory. Is it safe to delete that so that buildout loses its
> knowledge of what's already there?

It sounds like you do not want the recipe to uninstall anything, ever,
instead relying on manually deleting directories that might be on the
way. If that's what you want it to do, we can make it do it.

> I've attached the buildout cfg files I'm using if they are any help. Is the
> problem in how I'm using buildout or in how I'm using the bzr recipe? In our
> case, we're using buildout as a tool for collecting pieces rather than as a
> tool for reproducing a disk image. Maybe it's just not designed for that?

Per my description above, I wouldn't say that it was not designed for
that. It might be somewhat too generic with a little bit of naiveness
on top though.

As for the configuration, IIRC, we added the ':strict' postfix so that
it would use a variation of the recipe that would *not* delete
branches containing uncomitted changes.

Although, from your description of the problem, it seems like your
problem is actually that you have 2 different buildout.cfg files and
you are running buildout on the same directory, each time with a
different configuration file. So what happens is that it looks at
.installed.cfg and always considers that the configuration has
changed.

It sounds to me like it might be just problem of unintended usage, as
opposed to a configuration problem or a recipe problem.

I'm happy to answer more questions. I'm writing mostly from memory
here so one or other fact might not be completely accurate.

-- Sidnei