ubuntu-phone team mailing list archive

[Alan Pope <alan.pope@xxxxxxxxxxxxx>]

On Fri, Mar 21, 2014 at 5:57 AM, Alex Chiang <achiang@xxxxxxxxxxxxx> wrote:
> Is this resource bottleneck (debug, fix, test) going to affect 3rd
> party apps too, or is it just the case that the core apps get special
> attention/gating because they are core apps?
>

It's specific to core apps. 3rd party apps in the store are uploaded
by the developers themselves and only get the (soon to be automated)
click reviewers tools run against them before approval/rejection.

We (Canonical) upload the core apps to the store. In addition we have
an extra testing step before approving into the store which 3rd party
apps don't get. We do that because the core apps are pre-installed.

In general Sergio or Nick will upload to the store and then ping me. I
will get the click package and run the (currently manual but soon to
be automated) click reviewers tools against the click. If it fails I
bounce it back. This can happen for various reasons, most often pilot
error (typos). If it passes (or is resubmitted and then passes) then I
will locally shove the click package onto a device and run all the
autopilot tests for that application before either approving into the
store or bouncing it back for fixing and resubmission.

Running the AP tests can be pretty time consuming, especially if my
devices have been used earlier in the day to do some other bisecting
or debugging. We also sometimes have flaky tests or broken images
which means I have to run the tests repeatedly to get a 100% pass. I
only approve into the store if I'm confident I can get 100% pass, even
if the app works perfectly, but fails the tests - as happened this
week with Sudoku.

One way to speed that process up would be for the submitter to run the
click reviewers tools before uploading or pinging me, as that would
eliminate some wait & turnaround time.

Another would be to have multiple people able to approve into the
store - we already have this, but not everyone in every timezone has
time and a spare device kicking around which they can run the AP tests
on.

Further we could automate the autopilot testing for core click apps
submitted to the store.

I am keen on automating myself (and others) out of this process.

Cheers,
-- 
Alan Pope
Engineering Manager

Canonical - Product Strategy
+44 (0) 7973 620 164
alan.pope@xxxxxxxxxxxxx
http://ubuntu.com/