launchpad-dev team mailing list archive

["Francis J. Lacoste" <francis.lacoste@xxxxxxxxxx>]

Hi Aaron,

On 12-09-21 02:38 PM, Aaron Bentley wrote:
> 
> That duration is similar to bzr's test suite.  With bzr, we've always
> been able to run in the traditional PQM mode where PQM runs the tests,
> and if they fail, the code doesn't land.
> 
> I bet that would work with Launchpad too, which would allow us to
> simplify a lot of our infrastructure-- launchpad/devel and
> launchpad/stable become a single branch, we don't need buildbot
> anymore, and we don't need to use "ec2 land".  We can just lp-land
> directly.
> 
> When test runs were longer, this mode caused unfortunate queueing, but
> with 35-minute test runs, we can do 41 test runs a day, and the most
> landings I see this month is 11 (on Sept 12).  Most days are more like 6.
> 

I agree that going back to pre-commit merge is one thing we could try.
There is a caveat in your data though. You are counting the number of
test runs processed by buildbot in a day. The number that is truly
important is the number of ec2test submission daily. Because only once a
successful ec2 test run has happened does it gets send to buildbot.

What you are proposing (and what was happening before we switched to
buildbot) is that developers simply use the landing architecture as a
convenient test runner. One problem in the old days was that a lot of
queued landings would fail simply because the tests hadn't been run, not
because of failing tests because of a integration error or intermittent
failure. (Although we had also some of those.)

I don't know what's the effort required to set a tarmac instance that
can run parallelized tests. (Unfortunately, it probably requires scarce
webops resource also). But I'd be willing to try an experiment around
that if it's cheap.

To achieve the similar flow you want, we can also make ec2test run tests
in parallel in EC2. On the big instances with 32 cores, Yellow was
seeing a ~50 mins test run in EC2. That would put us well into range of
writing and deploying code in the same day.  (And deploying this
requires 0 webops involvement).

(And one doesn't exclude the other.)




-- 
Francis J. Lacoste
francis.lacoste@xxxxxxxxxx

Attachment: signature.asc
Description: OpenPGP digital signature