← Back to team overview

launchpad-dev team mailing list archive

Re: qa-needstesting and qa-bad status

 

On Tue, Oct 12, 2010 at 11:30 PM, Edwin Grubbs
<edwin.grubbs@xxxxxxxxxxxxx>wrote:

> On Tue, Oct 12, 2010 at 3:46 PM, Edwin Grubbs
> <edwin.grubbs@xxxxxxxxxxxxx> wrote:
> > On Mon, Oct 11, 2010 at 2:29 PM, Robert Collins
> > <robert.collins@xxxxxxxxxxxxx> wrote:
> >> On Tue, Oct 12, 2010 at 7:54 AM, Edwin Grubbs
> >> <edwin.grubbs@xxxxxxxxxxxxx> wrote:
> >>> Bug 427263 (Impossible to edit bug descriptions with Konqueror) is
> >>> currently tagged qa-bad. Does this need to be added to
> >>> https://dev.launchpad.net/CurrentRolloutBlockers ?
> >>
> >>
> https://devpad.canonical.com/~lpqateam/qa_reports/deployment-stable.html<https://devpad.canonical.com/%7Elpqateam/qa_reports/deployment-stable.html>
> >> and
> >>
> https://devpad.canonical.com/~lpqateam/qa_reports/deployment-db-stable.html<https://devpad.canonical.com/%7Elpqateam/qa_reports/deployment-db-stable.html>
> >>
> >> are both authoritative for deployments these days; we can't deploy
> >> anything at the moment - we're stalled on QA.
> >
> >
> > I think these two reports may have exposed a serious problem with
> > handling QA with the qa-needstesting tag. When we had a bunch of bugs
> > with qa-needstesting, it didn't surprise me that the above reports
> > showed that deployment is blocked on untested bugs. However, we now
> > only have one bug with qa-needstesting, and the reports for both
> > stable and db-stable show revisions that have no qa tags at all. Since
> > the reports only show the oldest revision that hasn't been QA'ed, I
> > have no idea if this is just a couple of unusual bugs, or if there are
> > a bunch that we have missed.
> >
> > -Edwin
>
>
> Now that I have looked at it more closely. It looks like the qa-tagger
> report is pulling some bogus data out of the revisions. For example,
> revision 11684 of devel says that it fixes bug 1544, but it is
> obviously not related to that bug. Since launchpad doesn't even a show
> a link between that revision and bug, I assume that qa-tagger is
> getting the fixes list from the revision metadata, which is added with
> "bzr commit --fixes".


qa-tagger gets bugs' info from the commit message, from the bugs linked to
branches linked to the mp, and from the merged branch nick.



> There are also problems when qatagger checks a
> bogus bug number listed in the pqm message since there is no way to
> unlink that. lifeless and I have opened bugs 659618 and 659629 to
> track the problems with qatagger.
>
> The good news is that we have been able to get the reports to display
> the correct status by removing incorrect bug links and by adding
> qa-untestable to the bug listed in the pqm commit message. I'll remove
> that tag after the release.
>
> -Edwin
>
>
>
> >>> We still have several bugs that need to be QA'ed. Since staging is
> >>> currently unavailable due to problems updating the db (see Tom
> >>> Haddon's email), I have split up the bugs that can be QA'ed now on
> >>> EDGE versus those that will have to wait until staging is working.
> >>>
> >>> Bugs that can be QA'ed on EDGE
> >>>
> ------------------------------------------------------------------------
> >>>    645403 TooNewRecipeFormat raised editing a recipe
> >>>        ON EDGE ~rockstar/launchpad/recipe-too-new
> >>>             merged into ~launchpad-pqm/launchpad/devel at rev 11612
> >>
> >> This is one of them :)
> >>
> >>>    645702 oops in holdMessage storing large message
> >>>        ON EDGE ~sinzui/launchpad/hold-message
> >>>             merged into ~launchpad-pqm/launchpad/devel at rev 11685
> >>>
> >>>    652626 Windmill egg needs to be upgraded to fix test breakages
> >>>        ON EDGE ~wallyworld/launchpad/windmill-1.3r1544
> >>>             merged into ~launchpad-pqm/launchpad/devel at rev 11684
> >>
> >> This doesn't have any bearing on deployment, I'll mark it ok (because
> >> the test suite which it affects is working).
> >>
> >>>    627701 Make it possible to use feature flags to override the
> >>> global timeout for specific pages
> >>>        ON EDGE
> ~matsubara/launchpad/fix-bug-627701-override-global-timeout
> >>>
> >>>        ON EDGE ~lifeless/launchpad/bug-627701
> >>>             merged into ~launchpad-pqm/launchpad/devel at rev 11668
> >>
> >> This cannot be QA'ed on edge. (The way to QA it is to drop the timeout
> >> way low, or way high on a page that times out) - both will have
> >> negative impact on our users).
> >>
> >> -Rob
> >>
> >
>
> _______________________________________________
> Mailing list: https://launchpad.net/~launchpad-dev<https://launchpad.net/%7Elaunchpad-dev>
> Post to     : launchpad-dev@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~launchpad-dev<https://launchpad.net/%7Elaunchpad-dev>
> More help   : https://help.launchpad.net/ListHelp
>

References