launchpad-dev team mailing list archive

[Julian Edwards <julian.edwards@xxxxxxxxxxxxx>]

Finally, I get a chance to reply to this.  Sorry for the delay.  This is also 
a long reply :)

On Thursday 26 November 2009 02:02:36 Michael Hudson wrote:
> Michael Hudson wrote:
> > The branch at ~mwhudson/launchpad/recipe-db-schema adds a patch to add
> > SourcePackageRecipe, SourcePackageRecipeBranch and
> > BuildSourcePackageFromRecipeJob tables.  You can also see it at
> > http://pastebin.ubuntu.com/327343/.
> 
> I talked to Jono and Julian a bit, and have made some changes.  New
> patch at http://pastebin.ubuntu.com/328140/ .  It's certainly a fairly
> large patch and I'd feel a little uncomfortable about landing it without
> some model code (at least) to back it up.

Some comments:

 * Should SourcePackageRecipeData have an owner or created_by?
 * Why is SourcePackageRecipe a separate table to SourcePackageRecipeData? 
Since one references the other, the duplicated columns are not needed are 
they?  In fact the only additional column is the recipe text. Maybe I don't 
understand what you're trying to do with this?
 * SourcePackageRecipeDataBranch isn't really needed yet, we're not creating 
branches for daily builds.  It's not a problem adding it though I guess.
 * in SourcePackageBuild:
  * you have a comment that you dropped "processor" but it's still there :)
  * I don't think you need "archive", this is not pertinent until the package 
is actually uploaded.  At least, I think we can share these across archives.  
Can anyone think of a reason why we would not do that?
  * you don't need "dependencies"
  * you probably still want upload_log as on Build - this records the log of 
when the package is uploaded back into Soyuz.  This stage can go wrong if the 
resulting package is no good and the log says why.
  * I see you have a column called "manifest" which references 
SourcePackageRecipeData.  Now I'm really confused about what that is!  I 
thought recipes and manifests were separate things?  If it needs to be 
separate why is it not SourcePackageRecipeManifest?
 * BuildSourcePackageFromRecipeJob - this might as well contain everything in 
SourcePackageBuild unless you have a good reason for separating them?

> 
> > I think it's mostly fairly uncontroversial.  Some remaining points:
> >
> >  - My unique index assumes that the url will be
> > https://launchpad.net/ubuntu/sourcepackage/+recipe/name or very similar.
> >  Arguments welcome about whether the owner should be in there :-)
> 
> I think we should keep the name in the url.

Yes, definitely.

The owner is another thing though :)  I'm not sure, and replied to the other 
emails about this separately.

> >  - The big missing part is where to store the log file and manifest that
> > are produced by the job's execution.  We could be more like the existing
> > Soyuz code and have a separate "Build" table that points at these
> > things, or they could live on BuildSourcePackageFromRecipeJob
> 
> I think a separate SourcePackageRecipeBuild table mimicing Build (hey,
> anyone want to rename that BinaryPackageBuild or something? ;-) makes
> most sense, so I've added that.
> 
> >  - This doesn't get into the mess of SourcePackageRecipe subscriptions.
> >
> >  - My patch assumes that a built sourcepackage will be uploaded uploaded
> > to exactly one archive.  Julian expressed some concerns about wanting to
> >  be able to upload to more than one, but I think we can probably survive
> > without this for now.
> 
> This is still true and I'd still like someone to shout if they disagree :)

Actually, having "archive" on the build record is fine, as builds are always 
targeted to a particular archive.  I was more worried about putting it on a 
recipe or a sourcepackage.

Having said that, I would still vastly prefer a separate table linking 
SourcePackageBuild and Archives.  This allows the possibility of re-uploading 
the same source recipe build if we want.  I think that will be important for 2 
reasons:
 1. re-creating upload issues
 2. re-creating genuine package bugs in a different PPA environment

Talking of which, I noticed that this patch doesn't have anything for 
scheduling recipe builds.  Are you doing that separately?

> >  - There was talk earlier about needing to record who requested a source
> > package build; it turns out that Job already has a requester field, so
> > we don't need one on BuildSourcePackageFromRecipeJob.
> 
> However, if the Job row goes away before the SourcePackageRecipeBuild
> row, then we'd lose that.

I would expect Job rows to live forever*.  At least, we've refactored the 
existing buildqueue stuff with that expectation.  Why would they be removed?

(* where forever == until we remove the whole set of related rows for whatever 
reason)

> I still don't know where to store it.  In the
> existing model I think this sort of thing is tracked in the
> SourcePackageRelease table (correct?)

Sort of.  We record the source creator on the SourcePackageRelease and the 
source uploader on the PackageUpload.

You can see why I keep banging on about why creation and uploading are 
orthogonal :)

> which we don't have an analogue
> for yet in the build from recipe world -- I don't know if we need one
> though as SourcePackageRelease <-> Build is 1-to-many in a way that
> doesn't apply to recipes.

Also, if they're automated daily builds, who is the requester?

> What I think I'm trying to say is that who requested the recipe be built
> into a source package can be stored in the SourcePackageRecipeBuild table.

That sounds perfect to me.


> > Comments welcome -- it would be nice to land this this cycle so we can
> > start working on things for realz next cycle.
> 
> Screw this part: we can just land it on db-devel.  Rushing is bad

Agreed.

Cheers
J