launchpad-dev team mailing list archive

[Julian Edwards <julian.edwards@xxxxxxxxxxxxx>]

I had a chat with many, many people yeterday and the upshot is that we'd like 
to make the API into the Soyuz build system look like IJob, which is what the 
Code folks use.

Aaron was kind enough to spend some time explaining IJob things to me 
yesterday so I have a basic understanding of that now.

I will start by explaining how the current build farm schema works, which has 
a large bearing on this new schema.

== Existing schema  ==

|| Build || <build meta data> ... 
|| BuildQueue || Build (FK) | lastscore (int) | builder (FK) | logtail | ... 

The basic premise is that BuildQueue rows only exist as long as the job is 
waiting to be run, or running.  As I've said previously, I want to retain this 
behaviour so that the changes to Soyuz code are not as invasive.

Running jobs have the `builder` FK populated.

BuildQueue is sorted by "lastscore" and then ID.  This is important as build 
jobs can be rescored and we rely on a pretty hairy query to fetch stuff from 
the queue in the right order (there are other constraints, especially where 
private builds are concerned).

== Proposed changes ==

The first stage is to retain the existing functionality for Soyuz package 
builds, but allow other types of jobs to run later.

`Job` rows will be created in tandem with the build jobs, however log files 
will continue to be stored in the librarian (they get big).

|| BuildPackageJob || job (FK) | build (FK) | ... 
|| BuildQueue || job (FK) | job_type (enum) | lastscore (int) | ... 

BuildPackageJob will link the build record to the Job record.  BuildQueue 
loses its build FK and now links to the build via the BuildPackageJob record 
with the same job ID.

For other jobs we can add some more tables:

|| BuildPackageBranchJob || job (FK) | sourcepackagebranch (FK) | ...
|| BuildPackageRecipeJob || job (FK) | ...
|| BuildTranslationsJob || job (FK) | ...	

There will be no doubt more clean-ups we can do in the Soyuz data/schema, but 
that can wait for later.  For now, I just want to unblock everyone else.

I welcome all comments on this schema, I want to start implementing it next 
Monday.

Thanks
J