launchpad-dev team mailing list archive

[Michael Hudson <michael.hudson@xxxxxxxxxxxxx>]

Michael Hudson wrote:
> Julian Edwards wrote:
>> On Wednesday 02 December 2009 03:50:50 Michael Hudson wrote:
>>>>  * Should SourcePackageRecipeData have an owner or created_by?
>>> I don't think so.  SourcePackageRecipeData is really just a shared
>>> implementation detail of SourcePackageBuild and SourcePackageRecipe, and
>>> both of those should record all the interesting stuff.
>> Ok, so the same recipe data can be re-used in multiple recipes.  Makes sense.
> 
> No... I see that I've not explained how I envision this bit working very
> well at all.
> 
> Quoting from my mail of a few weeks ago in the "Immediate plan for Build
> Farm generic jobs" thread
> (http://www.mail-archive.com/launchpad-dev@xxxxxxxxxxxxxxxxxxx/msg01594.html):
> 
>> Let's start with a recipe.  In the abstract, a recipe is roughly a thing
>> that specifies how to combine branches into something that debuild could
>> turn into a source package -- a "debianized source tree".  Recipes today
>> are specified as text files that the 'bzr-builder' plugin parses and
>> acts on.  An obvious open question in choosing how to represent these is
>> either to store them in a parsed form, with all the structure of a
>> recipe in the schema or unparsed, as lumps of text.
>>
>> Or, and there are actually reasons for doing this, we could do something
>> in between: store it mostly as text but replace the references to
>> branches in the text with references to database objects (probably the
>> id of entries in some RecipeBranch linking table).  This would let us
>> (a) check that the branches exist at parsing time (b) keep the
>> references up to date if the branch is moved or renamed (c) prevent
>> branches that are referenced in Recipes from being deleted.
> 
> The only thing that has really changed about this is that I realized a
> manifest needs to be stored in essentially the same way as the recipe
> text.  So I should explain properly what a manifest is!
> 
> A "manifest" is part of the output of building a recipe into a
> debianized source tree.  Syntactically speaking, it's a recipe, but it's
> a "frozen" recipe where all the references it includes specify precisely
> which revision of which branch was used in the build, so if you build
> from that recipe again you will[1] get the same debianized source tree
> and thus same source package again.
> 
> The name and idea of a "manifest" are both not my fault :-)
> 
> [1] Modulo the "run" command being used to do evil things, I guess.
> 
> So instead of the pseudo-schema in the mail linked above, where the
> SourcePackageRecipe table had a text column and there was a table for
> linking this text with the branches it references, I have
> SourcePackageRecipeData and SourcePackageRecipeDataBranch tables which
> represent the text of a recipe, and link to these from the
> SourcePackageRecipe and SourcePackageBuild branches.
> 
> As I start to write the model code, I'm thinking that all this will be
> hidden from the Python level -- ISourcePackageRecipe will probably have
> a 'recipe' field that gives you the recipe as the user will see it.
> 
> I hope this makes sense now, and I apologize for not being clear on this
> before.
> 
>> So will some code attempt to factor the recipe data as much as possible into 
>> as few rows as possible when the recipes are created?
> 
> No..., see above.
> 
>>>>  * Why is SourcePackageRecipe a separate table to
>>>> SourcePackageRecipeData? Since one references the other, the duplicated
>>>> columns are not needed are they?  In fact the only additional column is
>>>> the recipe text. Maybe I don't understand what you're trying to do with
>>>> this?
>>> Ah right, I meant to remove those duplicate columns from SPRecipeData.
>>> Essentially the reason they're separate is because SPRecipes have access
>>> control and a location in the UI, but SPRecipeDatas don't as many of
>>> them will be read-only versions referenced from SPBuild rows.
>> OK.
>>
>>>>  * SourcePackageRecipeDataBranch isn't really needed yet, we're not
>>>> creating branches for daily builds.  It's not a problem adding it though
>>>> I guess.
>>> It's meant to be there to record the links between the recipe and the
>>> branches it references.  I guess I don't understand what you mean.
>> My understanding was that we're not going to be creating branches at all, for 
>> daily builds.  The package file tree will be built from the recipe and then 
>> uploaded to Soyuz.
>>
>> This table will be needed when we start building branches from recipes though!
> 
> I hope this part makes sense by now.
> 
>>>>   * I don't think you need "archive", this is not pertinent until the
>>>> package is actually uploaded.  At least, I think we can share these
>>>> across archives. Can anyone think of a reason why we would not do that?
>>> Well, somehow we need to record the archive(s?) to which the resulting
>>> source pacakge will be built.  Maybe not in this table though, indeed.
>>> More on this below.
>> OK
>>
>>>>   * I see you have a column called "manifest" which references
>>>> SourcePackageRecipeData.  Now I'm really confused about what that is!  I
>>>> thought recipes and manifests were separate things?  If it needs to be
>>>> separate why is it not SourcePackageRecipeManifest?
>>> I hope this makes sense by now?  If we just stored recipe/manifests as
>>> text, both SPRecipe and SPBuild would just have a text column, but we're
>>> planning on being a bit more sophisticated than that.
>> It sorta does, but I don't get the name "manifest".
> 
> See above.
> 
>> Also, why is it
>> referencing there as opposed to SourcePackageRecipe?
> 
> I hope this is clear now -- it's an output of the build, like the build log.
> 
>> Sorry if I am being thick!
> 
> The name is very far from obviously indicating what it is, so no, I
> don't think so...
> 
>>>>  * BuildSourcePackageFromRecipeJob - this might as well contain
>>>> everything in SourcePackageBuild unless you have a good reason for
>>>> separating them?
>>> Why would it make sense to duplicate the columns across the tables?
>> I meant that we should move them, not duplicate them.
>>
>>> I thought we went over why BuildSourcePackageFromRecipeJob and SPBuild
>>> are separate last week.
>> Maybe.
> 
> XXX

Oops, I didn't write this section before hitting send.  I meant to say
something like this:

I thought this was what we were talking about in the mails about this
point of this thread:
http://www.mail-archive.com/launchpad-dev@xxxxxxxxxxxxxxxxxxx/msg01662.html

Cheers,
mwh