ubuntu-phone team mailing list archive

[Didier Roche <didrocks@xxxxxxxxxx>]

Hey,

Since Monday, we are fighting to get an image promoted. We had to block 
any Touch landings because of the multiple regressions spotted and only 
focus on the fixes. We still let desktop-specific fixes carefully tested 
on Touch landing to mitigate the situation.

Most of the fixes are now in (last ones are expected for tomorrow 
morning, EU time), but we have a Qt crash in a lot of places in the 
automation. With Qt5.2 landing coming soon Ubuntu Engineering made the 
decision to not invest our engineering time in trying to fix this issue, 
but instead rather focus on getting the 5.2 landing in shape. This was a 
difficult decision (landing one big change on top of another one) as the 
Qt 5.2 work is not completely ready yet and we'll need to get all our 
energy on it to land it.

Meanwhile, we don't want to block landings any longer. Even if the 
results are puzzling on the dashboard. We consequently decided to have 
this special deokgraded (it's like Raid 1 with only one disk :p) landing 
process. What does this mean?

1. NOW - Friday: We are getting latest regression fixes in + the trusty 
beta unfreeze block hit.
-> We hope that remaining issues will then be known one (Qt crashes, 
screen doesn't always unblock and need to rerun the tests).
We expect to reach that state by tomorrow mid-day.

2. Friday - Qt 5.2 landing (earliest Tuesday) We resume landings with 
extra care:
-> We'll try to land components in small chunks, kicking an image and 
getting test results in between (producing 3 images a day)
-> Every landing will be double checked by the landing team, rerunning 
the tests on their side to try to get as much insurance that we don't 
introduce extra issues to the image.
-> We ask every landers to be even more careful in their testing than 
usual and don't dismiss any failing test. If you have your application 
crashing, please:
     * ensure that in your core .crash file, that the stacktrace 
contains in frame 0 *****d8f8 in the address (this is the signature of 
the known V8 crash)
     * rerun your test until you don't have that crash on the tests 
that failed to ensure that this wasn't hiding a regression

3. Qt 5.2 landing:
Once we start doing final landing steps on Qt 5.2, we will stop other 
landings again to ensure we can focus on getting this in (some of your 
components will be included in that transition as well). Any help will 
be welcome to smooth that transition.
Tuesday is a target for now, we'll have more infos as we get progress. 
The Qt 5.2 started to build this morning on a silo to prepare that.

Note that we have been considering hard whether we can do promotions 
once we only have the Qt V8 crasher on the image, but since we don't 
want to have any user regressions, we will need more time for thorough 
investigation on the exact impact. This investigation is conducted 
jointly with the QA team to give us a clear GO/NOGO.

Hope that this special landing mode will help to not decrease our 
velocity, mitigate risk and gain so that we don't pile up too many 
changes and get the next image broken for sure.
Cheers,
Didier