ubuntu-phone team mailing list archive

[Dimitri John Ledkov <xnox@xxxxxxxxxx>]

On 7 January 2014 15:58, Ted Gould <ted@xxxxxxxxxx> wrote:
> On Tue, 2014-01-07 at 14:50 +0000, Dimitri John Ledkov wrote:
>
> Whilst the emulator improvement work is on-going, I've spend some time
> getting autopilot tests execution using the emulator running.
>
>
> Awesome.  I knew something cool was brewing from your questions in IRC ;-)
>
>
> I've also wrote a juju charm which does provisioning of the emulator
> on a cloud instance, after which one can ssh in and use run-tests.sh /
> run-test.sh script straight away. 2 CPU cores & 2GB ram (min) is
> recommended. See jemjem charm in the charm store.
>
>
> I find this a bit odd, it seems that I want to SSH in less and more have the
> charm execute the tests with a given new package and return the results.  Is
> that something you're considering for the future or are you just trying to
> automate the setup?  What I'd love is to be able to do something like
> (simplified):
>
> $ juju deploy jemjem
> $ juju set jemjem test_branches=lp:~ted/foo/bar,lp:~ted/bar/foo
>
>
> And then check in later for results with those packages.
>

Yeap, at the moment jemjem charm is very basic. Essentially it just
gives me N amount of units, and then I just scrape the IP addresses
and use GNU Parallel to do remote batch execution of all test-suites
in parallel. Since executing all tests sequentially takes a long time
on a single instance.

Something like "TESTSUITES=calculator,calendar,unity8" would be nice.
Or have like "PPA=ppa:ted/featurePPA TESTSUITES=all" charm settings
would be nice. Do note that emulator is armhf, so to e.g. to support
branch, build, and test workflow it should either be crosscompilable
or packages generated / available of somewhere else (e.g. from the
merge proposal generated jenkins job URL ;-) ) The test runner already
supports installing custom / additional debs, so this just needs
hooking up in the charm.

Patches welcome, this was just a rough start, driven to the point of
generating useful results with a significantly high pass rate.

-- 
Regards,

Dimitri.