launchpad-dev team mailing list archive

[Michael Nelson <michael.nelson@xxxxxxxxxxxxx>]

On Thu, May 6, 2010 at 11:36 PM, William Grant <wgrant@xxxxxxxxxx> wrote:
> On Thu, 2010-05-06 at 18:08 +0200, Michael Nelson wrote:
>> On Thu, May 6, 2010 at 5:34 PM, Aaron Bentley <aaron@xxxxxxxxxxxxx> wrote:
>> > -----BEGIN PGP SIGNED MESSAGE-----
>> > Hash: SHA1
>> >
>> > Hi all,
>> ...
>> > I bring this up because Soyuz is currently in the middle of redesigning
>> > the build system, and this is therefore a good opportunity to bring more
>> > similarity to our systems.  The Code Import system is also a candidate
>> > for convergence, but the Code team isn't currently redesigning it.
>> >
>> > AIUI, part of the redesign of the build system involves creating a
>> > Buildjob table with many fields that are similar to the Job table.  I
>> > propose that the buildjob table should hold only fields that are not
>> > generally relevant to jobs, plus a reference to the Job table.  This may
>> > entail adjusting the Job table to better fit the build system's needs.
>>
>> Hi Aaron,
>>
>> I created the following a few weeks ago for that exact reason:
>> https://bugs.launchpad.net/bugs/570939
>> "Replace BPJ and SPRBJ with IBuildFarmJob.job"
>>
>> The new IBuildFarmJob model will certainly have many fields similar to
>> the Job table, and as we spoke on the phone the other day, I hope we
>> can replace those fields simply with a reference to the job (as in the
>> above bug).
>>
>> But it isn't as straight-forward yet as simply switching the fields as
>> the other temporary tables mentioned in the above bug reference the
>> job already and are deleted (along with the job) after the queued
>> build job is processed. The current refactoring is (IMO), a big step
>> towards being able to do what you've suggested, but it's aim is to
>> generalise the builds (which will always sit on top of services.Jobs),
>> without touching the current queuing infrastructure which links to the
>> job in strange ways (and deletes them afterwards).
>>
>> That said, another intermediate option that might allow us to simply
>> use something like IBuildFarmJob.services_job (and remove the relevant
>> fields from IBuildFarmJob) without affecting the current queuing
>> infrastructure might be to simply to ensure that when the current
>> queue records are deleted after the build finishes that we just don't
>> delete the job. Julian? William?
>
> Why do we want an intermediate option? I don't imagine the refactoring
> of the queue infrastructure is too far off, and we should be able to
> losslessly migrate permanent data into a services Job later. I think
> that introducing a separate permanent Job in between is just going to
> unnecessarily complicate things.

Yeah, so the idea was that we *don't* create a separate permanent job,
but just modify the current queue infrastructure code so that rather
than deleting the BuildQueue, BPJ and Job, it just deletes the first
two and leaves the job in tact (and we reference it from
IBuildFarmJob).

We can then later refactor the queue infrastructure without
complicating things? What do you think?

>
> Apart from that, I am all for unifying the two systems. I agree that it
> would be much nicer if builds were really just a special class of jobs.
>
>