launchpad-dev team mailing list archive

[Muharem Hrnjadovic <muharem@xxxxxxxxxx>]

Michael Hudson wrote:
> This got a bit long (parts of it should probably go into a spec).  I
> hope it's still interesting.

Hello Michael,

thanks very much for writing this up. I have a few quick comments,
please see below.

> 
> Julian Edwards wrote:
>> Howdy folks
>>
>> Next week is UDS, which I am attending, so I won't be spending much
>> time on the generic build farm jobs work.
>>
>> Muharem has been working on a branch to convert the schema (which jml
>> approved today) and to change our code that uses it to work with the
>> new schema.  It is largely done, and we hope to land it today.
>>
>> https://code.edge.launchpad.net/~al-maisan/launchpad/bqim-479079
>>
>> Once it has landed, this opens the door for some Translations and
>> Code work.  
>> Namely:
>>
>>  * New tables, BuildPackageBranchJob, BuildPackageRecipeJob and 
>> BuildTranslationsJob (or similar names as you prefer)
> 
> So, we finally get to think concretely about some things :-)
> 
> Let's start with a recipe.  In the abstract, a recipe is roughly a thing
> that specifies how to combine branches into something that debuild could
> turn into a source package -- a "debianized source tree".  Recipes today
> are specified as text files that the 'bzr-builder' plugin parses and
> acts on.  An obvious open question in choosing how to represent these is
> either to store them in a parsed form, with all the structure of a
> recipe in the schema or unparsed, as lumps of text.
> 
> Or, and there are actually reasons for doing this, we could do something
> in between: store it mostly as text but replace the references to
> branches in the text with references to database objects (probably the
> id of entries in some RecipeBranch linking table).  This would let us
> (a) check that the branches exist at parsing time (b) keep the
> references up to date if the branch is moved or renamed (c) prevent
> branches that are referenced in Recipes from being deleted.
I like this. How would we reference the linking table rows in the recipe
text?

> Floating around in a very abstract way is the idea of recipe
> inheritance.  I don't know how this works.
> 
> Separately, we need to decide where a recipe lives.  The current
> thinking is
> "https://launchpad.net/ubuntu/karmic/+source/some-package/+recipe/recipe-name";
> which seems OK to me (we'd have to trust a bit that this recipe would
> build a recipe for some-package in karmic, but that doesn't seem any
> different to say branches today).
> 
> Finally, we could stick an archive on the recipe, but maybe we don't
> want to.  I'll talk about this a bit more later in the mail.
> 
> This leads to a schema a bit like:
> 
> Recipe:
>  - id, registrant, date_created, owner, date_last_modified
>    - all standard launchpad fields.  the owner would be able to edit
>      the recipe.
>  - name
>    - the last bit of the url
>  - distroseries, sourcepackagename
>    - provides the rest of the url
>  - recipe
>    - a text field containing the text of the recipe (probably with
> mangled branch references so lp:foo would be replaced with lp:21435)
> 
> RecipeBranch:
>  - id, recipe, branch
>    - all obvious i hope
> 
> What follows hopefully doesn't depend too much on how the above gets
> decided in the end.
> 
> For the job of building a recipe into a source package we'll have a
> BuildSourcePackageFromRecipeJob table.  I foresee this table looking like:
> 
> BuildSourcePackageFromRecipeJob
>  - job
>  - recipe
>  - archive?
Storing the archive here is a good idea IMHO. The question in my mind
is: should this really be "BuildPackageFromRecipeJob" i.e. should it span
both stages, the generation of the souce package and the subsequent
"upload" and build from source package to a binary (.deb file)?

In that case we'd probably want a `Build` foreign key as well. It
would also give us a more holistic view of the job. Whether that's
desirable or cumbersome remains to be seen :)

[..]

>> The model code should implement the interface ISoyuzJob (although this is a 
>> terrible name, it will be changed) which is declared in 
>> lib/lp/soyuz/interfaces/soyuzjob.py.
> 
> This file doesn't seem to exist?

We renamed the interface and moved the files around, please see:

lib/lp/buildmaster/model/buildfarmjob.py
lib/lp/buildmaster/interfaces/buildfarmjob.py

>> See lib/lp/soyuz/model/buildpackagejob.py for an example of something that 
>> implements it.

[..]

> [1] I guess the fact that these packages aren't signed will bite us in
>    the ass somehow or other at some point, but I don't know how much it
>    affects how this bit would work.  We don't *have* to get the source
>    package files into Soyuz via the Librarian I guess.

I have a similar gut feeling :) The "upload" of a source package
generated from a recipe occurs withing the confines of the data center
but signing it with a "build farm" key would still be a good idea IMHO.

Best regards

-- 
Muharem Hrnjadovic <muharem@xxxxxxxxxx>
Public key id   : B2BBFCFC
Key fingerprint : A5A3 CC67 2B87 D641 103F  5602 219F 6B60 B2BB FCFC

Attachment: signature.asc
Description: OpenPGP digital signature