ubuntu-appstore-developers team mailing list archive

[Ricardo Kirkner <ricardo.kirkner@xxxxxxxxxxxxx>]

On Wed, Sep 17, 2014 at 11:48 AM, Jamie Strandboge <jamie@xxxxxxxxxxxxx>
wrote:

> On 09/17/2014 09:31 AM, Jamie Strandboge wrote:
> ...
>
> >> All of this sounds pretty reasonable, however I'm now somewhat
> confused. I think
> >> we're talking about different things here. So to disambiguate,
> >>
> >> 1. the process is to override errors so to not automatically reject,
> but force a
> >> manual review, and *not* to override errors to allow an automatic
> approval, right?
> >
> > What you have implemented thus far is perfect. What Daniel is wanting to
> do is
> > further reduce the number of manual reviews for apps where the app was
> already
> > manually reviewed, so it doesn't have to be next time. What I have
> discussed
> > allows for the overrides to allow warnings and errors that would
> normally prompt
> > a manual review to be ignored. I also discussed that we could have logic
> around
> > this so that sometimes errors could get a pass, but other times not and
> if not,
> > there is extra information that is available for display to the reviewer.
> >
> I reread your previous email and see where the confusion may lie. The click
> reviewers team is used to "manually reviewing" any apps that have warnings
> and/or errors. This "manual review" does typically include an in depth
> review.
> Meanwhile, the click-reviewers-tools outputs errors and warnings and a
> subset of
> those errors have the 'MANUAL REVIEW' string (and now the '"manual_review":
> True' json) which indicates that the reviewer should perform an in depth
> review
> of the app. As such, the term "manual review" is a bit overloaded in this
> conversation.
>
> Daniel is hoping to have an override mechanism for any errors and warnings
> so
> that your server logic becomes:
>
> 1. If there are non-overridden errors in the automated review results and
> none
>    of them are marked as requiring manual review, the app is automatically
>    rejected
> 2. If there are non-overridden errors in the automated review results and
> some
>    of them are marked as requiring manual review, the app is left in the
> review
>    queue waiting for a reviewer to pick it up and start a review (just
> like it
>    is now)
> 3. If there are errors and warnings in the automated review results and
> all of
>    them are overridden, the app is automatically approved
> 4. If there are no errors and no warnings in the automated review results,
> the
>    app is automatically approved.
>
> Note-- you said only 'errors' in portions of your email (which is captured
> above) which got me thinking: what does the server do when an app has no
> errors
> but there are warnings?
>

This sounds good.  I presume the click-reviewers-tools will be marking
checks with overrides somehow in the check results json so that the server
can figure out what to do.

Regarding warnings, if there are warnings but no errors, the app is left in
the review queue for a reviewer to pick it up (ie, manual review is implied)


>
>
> --
> Jamie Strandboge                 http://www.ubuntu.com/
>
>
> --
> Mailing list: https://launchpad.net/~ubuntu-appstore-developers
> Post to     : ubuntu-appstore-developers@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~ubuntu-appstore-developers
> More help   : https://help.launchpad.net/ListHelp
>
>