ubuntu-phone team mailing list archive
-
ubuntu-phone team
-
Mailing list archive
-
Message #05517
Re: Landing team 10.12.13
On Wed, Dec 11, 2013 at 12:47 PM, Didier Roche <didrocks@xxxxxxxxxx> wrote:
> Le 11/12/2013 12:38, Olivier Tilloy a écrit :
>
> * notes-app (Bill)
>> One test failing on both mako and maguro:
>>
>> http://ci.ubuntu.com/smokeng/trusty/touch/maguro/55:20131209.1:20131203/5355/notes-app-autopilot/558827/
>>
>> http://ci.ubuntu.com/smokeng/trusty/touch/mako/55:20131209.1:20131203/5356/notes-app-autopilot/558857/
>> 3 flaky tests on mako:
>>
>> http://ci.ubuntu.com/smokeng/trusty/touch/mako/56:20131210:20131203/5360/notes-app-autopilot/
>>
>
> I am able to reproduce the failure on maguro, although not reliably.
> It’s being looked at.
> I don’t have a nexus 4 handy to look into the failures on mako,
> unfortunately.
>
>
> Does anyone on your team have a nexus 4? As it seems to be flacky and not
> being reliably reproducible, looking at the code can help already to see if
> a race happened.
>
Bill has a nexus 4, he’ll be running the tests on his device.
We agreed on a plan with Ugo to tackle those flaky tests as best as we can,
starting with the ones with can reliably reproduce on maguro.
> * webbrowser-app (Bill)
>> 2 flaky tests on maguro only:
>>
>> http://ci.ubuntu.com/smokeng/trusty/touch/maguro/56:20131210:20131203/5363/webbrowser-app-autopilot/
>>
>
> Looks like this one has been re-run, it’s all green now. The
> webbrowser-app tests have been green for weeks (if not months) in a row. Do
> we have access to the logs of the previous run (with the two failures
> you’re mentioning)?
>
>
> Argh, I did ask to not rerun the test to the CI team. I'll reash it.
> Webbrowser-app is flaky for quite some time, but the CI team generally
> rerun the tests, that's why you did see it green for weeks/months in a row.
>
Is it common for the webbrowser-app tests to fail, be re-run, and then
pass? Does that happen weekly, daily, more often? Has it always been the
case, or did it start happening recently?
When it does happen, does the second run always pass, or how many re-runs
are necessary to reach the green state?
> For instance, here is on image 57 webbrowser with a lot of failing tests:
> http://ci.ubuntu.com/smokeng/trusty/touch/mako/57:20131210.1:20131203/5375/webbrowser-app-autopilot/
> .
> Would be interesting to know the root case, it seems something is screwed
> at some point, and then all tests are failing, can you have a look at that
> case and try to be more robust? (or pass to the CI team if it's an infra
> issue).
>
This specific run looks like an infrastructure issue to me, after a given
test all the subsequent tests fail when starting the app (it might be that
the app actually starts, but autopilot fails to find it). There are no
crash files in the artifacts, so it doesn’t look like an issue in the app
itself.
Follow ups
References