launchpad-dev team mailing list archive

[Ian Booth <ian.m.booth@xxxxxxxxx>]

Hi Aaron

> 
>> The bug's tags (as used in the current
>> discussion to reflect QA status of the associated revision) and status
>> attributes (as used for bug lifecycle) should be orthogonal I think.
> 
> I agree, and I've written up a LEP, which jml has approved, for
> deployability-oriented QA that tracks the status of revisions, not bugs:
> 
> https://dev.launchpad.net/LEP/DeploymentQA
>

Ah thanks. I missed that one. I saw the partial done
TrackingQualityAssurance LEP and didn't realise there was another.

>> To
>> me, it's not acceptable in some cases to merely have to look at the QA
>> tags to get the relevant information (eg tag="qa-bad" means what it
>> means without having to look elsewhere), whereas in other cases, you
>> need to also look at the value of what should be an unrelated bug
>> attribute to understand what's happening
> 
> I think the deployment report could be a good place to get all this
> information, but it needs improvement.
>

+1 on both counts.


>> (so currently tag="qa-ok" has
>> different meanings depending on the value of the bug's status
>> attribute).
> 
> Not in the release manager's perspective, IMHO.  As RM, I didn't really
> care about which bugs were fixed, I cared about which revisions were
> deployable.  As long as QA-ok means "deployable", I'm happy.  If
> renaming QA-ok to QA-deployable or changing the definition gets us
> there, I'm happy.
>

One issue for me is that the
https://dev.launchpad.net/QAProcessContinuousRollouts page lists qa-ok
as meaning:

qa-ok: The branch is verified and implements successfully what it proposes.

which is very different to "this branch is deployable from a RM
perspective but may not actually fix what it says it does". So there's
really two different concerns here - the QA perspective and the RM
perspective - and the one tag (qa-ok) is perhaps being used for both. We
should decide what we want and if it's to indicate deployability then I
think the tag should be renamed (to avoid misinterpretation based on the
"obvious" meaning of qa-ok) and the QAProcessContinuousRollouts page
updated to describe the true semantic meaning of the tag.

> 
> To me, it looks like database denormalization.  It means we have to
> track the "fixed" status of the bug in two places, the bug "status"
> attribute and the QA tags.  I would much rather completely remove the
> "fixes the bug" aspect of QA tags entirely, since we already have a
> place for that information that is standardised in Launchpad.
>

That could work, but for me a rename and definition change is required.
Both tag names "qa-ok" and "qa-bad" imply, and are currently defined to
be as per QAProcessContinuousRollouts, associated more with the QA
verification process and bug resolution. So if we want them to be used
to indicate "safe to deploy", then their names should be different and
that wiki page updated.


>> And
>> it's then perhaps easier to automate bug handling etc For example,
>> perhaps the qa tagger could automatically set the status of a bug back
>> to Confirmed
> 
> We don't use Confirmed.  Did you mean "Triaged", or are you suggesting
> we start using it?
>

Ah, I misunderstood our bug handling guidelines. To me, Triaged and
Confirmed are different (according to the dictionary definition and how
I've worked previously). What I was trying to indicate above was an
automated mechanism to put the bug into a state where it would show up
on someone's todo list to get fixed if it failed QA, assuming tagging a
bug as "qa-deployable" implied some sort of QA failure as per the
original discussion.


>> (from In Progess) if the qa-deployable tag is set, since
>> the implication is that there's still and issue to fix, else it would
>> have been tagged qa-ok.
> 
> I agree that the bug status and QA tags should be orthagonal, and
> therefore I disagree with this.  I say QA tags should only indicate
> deployability.  We can *rename* qa-ok to qa-deployable if that helps.
> 

And I think I agree with this last bit. My summary for this part of the
dicussion would be:
- QA tags should only indicate deployability
- their names should better reflect their purpose; I would go for:
  * qa-deployable
  * qa-undeployable
  * qa-needstesting
  * qa-untestable
- their definition on QAProcessContinuousRollouts should be corrected to
match their meaning and intended use
- there should be process automation (say using qa tagger) to update a
bug's status (if required) in response to a qa tag being applied