← Back to team overview

ubuntu-phone team mailing list archive

IMPORTANT ANNOUNCEMENT: CI Train changes

 

Hello!

So today has been an exciting and terrifying day for the train, as I've
attempted to fix some issues discovered during my vacation, and created
more problems, and then worked to fix those, too.

First, the bad news: I've made a horrible, horrible mistake that resulted
in at least 10 silos being deleted. I've done my best to restore & rebuild
them but as usual I'm only human. Everybody should check over their silos,
and if possible do a quick smoketest to make sure nothing has exploded.
Most of the silos I rebuilt succeeded but in particular there are these 4
failures:

timo-jyrinki wellark charles dobey, ubuntu/landing-026: Build failed:
pay-service failed to build:
https://requests.ci-train.ubuntu.com/#/ticket/284

oSoMoN, ubuntu/landing-035: Build failed: webbrowser-app failed to build:
https://requests.ci-train.ubuntu.com/#/ticket/386

mandel, ubuntu/landing-030: Build failed: ubuntu-download-manager failed to
build. unity-scope-click failed to build:
https://requests.ci-train.ubuntu.com/#/ticket/394

bzoltan, ubuntu/landing-038: Build failed: ubuntu-ui-toolkit failed to
build: https://requests.ci-train.ubuntu.com/#/ticket/369

Since all of the failures are PPA build failures, this means most likely
they were broken before I accidentally deleted them and you're already
aware of these failures and are already working on them anyway, but still
best to double-check that I didn't make things worse.

There is also a possibility that I've resurrected extra silos by mistake,
so everybody should check their requests to make sure that things you
thought were landed/abandoned aren't back from the dead.

Here is the full list of rebuilds I performed:
http://paste.ubuntu.com/12518903/

Sorry for the inconvenience! Now with that out of the way, there is some
good news:

* The prepare-silo job now can no longer fail/explode when somebody else's
request is malformed, so silo assignments should be bulletproof at this
point. The only failure mode should be if we're out of silos, if you see
anything else please inform me. Actually, inform me if we're out of silos
as well and I'll try to free some up ;-)

* When assigning silos, instead of always getting the lowest-numbered
available one, you will get a random one. This isn't perfect, but can be
helpful in the situation where you've uploaded an incorrectly high version
number to a PPA and want to downgrade your version number in a rebuild.
PPAs don't allow this, so in order to do this you need to free your silo
then reassign. Previously since assignment always gave you the
lexically-first silo, you'd find yourself freeing & reassigning and ending
up back in the same silo. Now there's at least a chance you'll get a
different silo.

* There is a new 'abandon' job that is more along the lines of what people
have been expecting. If you want to free a silo, you no longer run the
"Merge & Clean" job with ONLY_FREE_SILO, there's now a separate "Abandon"
link in Bileto. And this works whether or not the request has a silo
assigned or not. So if you're done with a request, just click Abandon then
Build on the jenkins form, if you had a silo it will be released, and then
the status will unconditionally be set to 'Abandoned' which hides the
request.


Any questions or concerns please let me know, that's all for now! The
iterations continue!!

Follow ups