ubuntu-phone team mailing list archive

[Sergio Schvezov <sergio.schvezov@xxxxxxxxxxxxx>]

On 20/12/13 04:49, Olivier Tilloy wrote:
> On Fri, Dec 20, 2013 at 5:17 AM, Paul Larson 
> <paul.larson@xxxxxxxxxxxxx <mailto:paul.larson@xxxxxxxxxxxxx>> wrote:
>
>     On Thu, Dec 19, 2013 at 3:30 PM, Olivier Tilloy
>     <olivier.tilloy@xxxxxxxxxxxxx
>     <mailto:olivier.tilloy@xxxxxxxxxxxxx>> wrote:
>     > On Thu, Dec 19, 2013 at 8:07 PM, Paul Larson
>     <paul.larson@xxxxxxxxxxxxx <mailto:paul.larson@xxxxxxxxxxxxx>>
>     > wrote:
>     >>
>     >> image 77 initial results were:
>     >> Mako 99.0%
>     >> Maguro 98.7%
>     >>
>     >> Here were the things I reran:
>     >
>     >
>     > I thought we were not re-running anything, to focus on fixing
>     flaky tests?
>     This was modified again recently.  If we see something that's known to
>     be flaky, we'll typically just leave it as-is.  For new failures
>     especially though, we need to rerun to see if it's consistently
>     failing or if it will sometimes pass.
>
>     >
>     >> Mako:
>     >> gallery-app (passed on rerun)
>     >>  *
>     >>
>     gallery_app.tests.test_photo_viewer.TestPhotoViewer.test_photo_delete_works
>     >> webbrowser (got worse on try 2, passed on try 3
>     >>  *
>     >>
>     webbrowser_app.tests.test_addressbar_states.TestAddressBarStates.test_cancel_state_loading
>     >>  *
>     >>
>     webbrowser_app.tests.test_addressbar_states.TestAddressBarStates.test_state_editing
>     >>  *
>     >>
>     webbrowser_app.tests.test_addressbar_states.TestAddressBarStates.test_state_loading_then_idle
>     >
>     >
>     > Can I access the artifacts of those failed tests?
>     Yes, you'll need to go to the jenkins page to do that though. You can
>     click on the job from the dashboard page, or find them on
>     jenkins.qa.ubuntu.com <http://jenkins.qa.ubuntu.com>.  If you have
>     trouble locating them, just let me
>     know and I'll be happy to help.
>
>
> Right, I know how to access the artifacts of a failed run, but are 
> they still available even if the job has been re-run and passed?

If no one deletes the jenkins job run itself, for the case of webbrowser 
app, all runs for mako are here:
https://jenkins.qa.ubuntu.com/job/trusty-touch-mako-smoke-webbrowser-app-autopilot/

> For example, for webbrowser-app on image 77 on mako, the dashboard [1] 
> is all green, and so is the jenkins view [2].
> Have the artifacts of the first two tries been kept somewhere?

I'm not sure how this is configured, the policy on when to drop 
artifacts themselves in jenkins, but build 87 has failed runs
https://jenkins.qa.ubuntu.com/job/trusty-touch-mako-smoke-webbrowser-app-autopilot/87/
and it's artifacts, under 'Build Artifacts' are still there.

The only thing that doesn't look that easy to grab is match the jenkins 
job run with a specific build attempt from the ci side, unless it's 
already in one of the collected logs and I just can't find it.

Cheers