launchpad-dev team mailing list archive

[Julian Edwards <julian.edwards@xxxxxxxxxxxxx>]

On Wednesday 16 September 2009 13:31:56 William Grant wrote:
> Hi all,

Hi William,

> I'm currently attempting to finish bugs #285205 (Soyuz needs to be able
> to process and publish ddebs) and #156575 (PPA builds do not create
> -dbgsym packages). There is a large and fairly complicated issue with
> the current implementation: the ddebs will live a life quite separate
> from their debs (and even their sources!), in an archive untouchable by
> any current tools, which breaks lots of assumptions.

Thank you for working on this.  It is indeed a tough problem and your further 
analysis has exposed a lot of areas we hadn't originally thought about.

[snip]

> (1) can probably be fixed by assuming that every ddeb has a
> corresponding deb with the same name, but with '-dbgsym' stripped off
> the end. I'm not sure how that would go if any of the debs themselves
> were NEW, however.

This is probably a good assumption, it will make things easier, although I 
worry that it is easily broken.

I think that ddebs should not skip NEW and should do whatever the 
corresponding deb does, including overrides.

> To fix (2) the scripts themselves will have to be changed in fairly
> obvious ways (as long as we can make the naming assumption).

Yes.  Any binary manipulation must also deal with ddebs in step.  We need to 
review the code base and try and encapsulate stuff better, probably in the 
publishing classes, so that future code automatically benefits and does the 
right thing.

> To fix (3), at least SPPH.getBuiltBinaries,
> PS._getSourceBinaryJoinForSources, PS.getFilesForSources,
> PS.getBinaryPublicationsForSources will need to include the
> corresponding debug archive in their search if looking in a primary
> archive. I haven't had a thorough look for any other relevant methods.

We can do something similar to what we had to do for the partner archive here, 
and that's extend the list of MAIN_ARCHIVE_PURPOSES.  This will fix some 
stuff, and break other stuff.  Some of those methods above assume a single 
archive; they will need fixing.

Again, if we can encapsulate this behaviour inside the publishing classes I'll 
be really happy.

> For (4), I think delayed copies might already be OK. They grab entire
> builds and run them through the queue again, so things should work fine.

For copies from PPAs it should be ok.  For copies within the primary archive 
it will get a little hairier because you hit unwanted upload checks for 
duplicate files.  (I'm currently adding a feature to do this, mainly for 
copying custom files around, so I'll bear it in mind.)

> Direct copies are another matter: they currently look just at binary
> publications in the source, and create new publications in the target
> archive. If the target is a primary archive, ddebs will need to be
> forced into the debug archive.

Yes, the copying should be encapsulated in the publishing classes' copyTo() so 
call sites don't need to think about it.  We'd need to fix the CopyChecker 
class as well (which will also benefit delayed copies).

> While direct copies *into* a primary archive are probably not supported,

Actually this is possible.

> I believe that the issues raised in (4) apply equally to
> inter-pocket/series copies within a primary archive.

Yes - see above :)

> There are probably other things that I'll find to be broken. Any
> thoughts about any of this, Soyuz guys?

Almost certainly.  It's a hard problem and we should take every opportunity to 
fix our code so that we don't have separate pieces of code all making the same 
assumptions, and abstract this behaviour into a single place.

I'll try and help you a bit more with this once 3.0 is out the door.

Cheers
J