launchpad-dev team mailing list archive

[Michael Nelson <michael.nelson@xxxxxxxxxxxxx>]

Michael Nelson wrote:
> And remember, flacoste (and BjornT in his timezone) are also available
> for reviewing any r-c merge proposals.

Looks like my original email didn't make it...

Hi Launchpad developers!

Just an update regarding the current 3.1.10 release - it's been a pretty
wild day with a number of rollout-process-related issues (see below)
which are they're all being worked on.

Michael Nelson wrote:
> > Friday 22:00 UTC - PQM will close.  All subsequent submissions are
> >    release-critical only. Note: this Friday 22:00 closure of PQM makes
> >    it super-important to get your QA done now :)

Initially this looked doubtful - as we had a cherry-pick blocking pqm on
Friday, but kiko got it booted and thanks to the losas we were able to
close PQM later on Friday night after the queue had emptied. Yay.

So far, QA has been going well with all teams having < 8 items left to
process (although I'm not sure how to check translations progress as
they're using a different system... danilos? How are you guys going?).

Let's get these all down to zero as soon as possible. We have some
issues with staging not being updated automatically, which is hurting
some teams trying to QA on db-stable.


Buildbot badness
================

Our main issue is that build_bot is not in a healthy state, our db_lp
buildbot has died, and hasn't yet been resurrected. BjornT, allenap and
gary have been working all day with the losas to find out why, but so
far it's not resolved.

This had left us in testfix mode, so as a work-around (suggested by
BjornT), I've tried submitting an empty testfix to db-devel just so pqm
will move out of testfix and we can continuing landing on devel, but it
turns out we can't do that either - due to an email server issue. So
currently submitting to PQM doesn't hit the queue and no email response
is returned (both deryck and I have been trying). Francis will update
the status of this as soon as he knows more.


Staging updates
===============

Staging is currently on db-stable 8620 (after a manual update on
Friday). Although we can't do the normal daily updates with new data, we
should be able to have automatic updates of code/schema changes, but
these are not happening either (at least, they're not listed on
https://staging.launchpad.net/successful-updates.txt)

Francis will try to ensure that staging gets updated asap so people,
particularly the translations guys, can continue QA.


BTW: There was mention of using a new process for the release
management, but I wasn't sure of the details, so I'm still using:

https://dev.launchpad.net/CurrentRolloutBlockers

and am also using this page to document the above current process issues
too.

/me -> EODs

Thanks!

-- 
Michael