← Back to team overview

unity-dev team mailing list archive

Re: [ANNOUNCE] unity-autopilot-wrapper - Run autopilot tests headless or in xephyr without messing up your session

 

Hey Thomi,

In my experience, I've found the thing to cause the most problems so
far has been when the mouse fails to move somewhere on the launcher.
I'm not sure why that happens yet, but I think it might be more of a
problem with the resolution xephyr is running at more than anything
else. I've not really seen anything llvmpipe related - unity works
just fine with it.

As for running xephyr using actual graphics drivers - I've seen
something that came out of the meego camp a while back here[1] but
those changes aren't yet upstream.

Right now, I don't see too much of a usecase for running the entire
testsuite of 861~ tests inside of xephyr - given that this takes a
long time anyway, the best place for that is probably in continuous
integration. The main goal of unity-autopilot-wrapper is to provide a
way for developers to quickly get feedback on whether or not changes
break or fix particular autopilot tests without the fuss of having to
set up a session just for autopilot.

At the moment, that's a little more painful than it should be with the
sample binary, we can't have dynamically generated test names in
Google Test, so you can only really filter by test category (eg,
LauncherKeyNav). However, the helper function which calls out to
autopilot for the tests does generate the actual tests themselves at
runtime, so it should be fairly easy to create a main () where you can
pass in a filter via an environment variable. For example,
TEST_FILTER=test_switcher_rises_current_window etc.

[1] http://gitorious.org/xserver-with-gl-accelerated-xephyr

On Tue, Jun 4, 2013 at 6:45 AM, Thomi Richards
<thomi.richards@xxxxxxxxxxxxx> wrote:
> Hi Sam,
>
> That's seriously cool - I looked in to doing something very similar myself,
> but ended up not having the time for it.
>
> My experience is that unity3D does not work well on llvmpipe, to the point
> where the number of autopilot failures is very high. I wonder if there's
> some way around this?
>
> Cheers,
>
>
> On Mon, Jun 3, 2013 at 2:26 AM, Sam Spilsbury <smspillaz@xxxxxxxxx> wrote:
>>
>> Hey unity (7.0) hackers,
>>
>> I put together something this weekend that I thought people might find
>> useful if they're running autopilot tests.
>>
>> unity-autopilot-wrapper is effectively a Google Test
>> ::testing::Environment (with a few other related tools) which will
>> allow you to spawn a self-contained instance of Unity in a headless X
>> Server or Xephyr using the ubuntu default settings. This should make
>> it a lot easier to run those tests should you need to do so without
>> having to worry about changing all the settings back to their defaults
>> and closing open windows etc.
>>
>> Currently, its features are:
>>  * - Spawns a new dbus-session bus and exports its address as the one
>> to use for the test
>>  * - Spawns a headless X Server or Xephyr
>>  * - Spawns compiz inside of that, connected to the new dbus session
>> (this in turn causes fresh instances of bamfdaemon,
>> unity-panel-service, hud-service, lenses, etc to spawn just for that
>> dbus session).
>>  * - Enumerates through the available autopilot tests and runs each
>> autopilot test as a separate test in the Google Test binary (so you
>> can track the progress of the tests)
>>  * - Allows filtering through those tests on startup (so you can just
>> run one autopilot test in a self-contained session to check it
>> quickly)
>>  * - Allows you to specify a custom compizconfig profile to load,
>> compiz binary to launch and even where compiz plugins should be loaded
>> from, so you don't have to use the installed compiz and unity to run
>> the tests.
>>  * - A sample of the environment can be installed and run from
>> anywhere, which runs all the autopilot tests installed inside of a
>> Xephyr window. Perhaps ideal for autopkgtests.
>>
>> The source code can be found at my GitHub repository:
>>  * - git://github.com/smspillaz/unity-autopilot-wrapper
>>
>> At the moment, there isn't an ubuntu package. I'd like to do a daily
>> build, but launchpad imports don't work with git repositories that
>> have submodules and this one does. Perhaps I'll get a PPA up and
>> running soon enough.
>>
>> Caveats:
>>  * - It renders using llvmpipe which is pretty CPU heavy. I've not
>> been able to complete a whole run of the tests without heat problems
>> on my machine. YMMV.
>>  * - There may be *some* false negatives. I've seen a few failures but
>> so far I've been able to reproduce them in a real session too.
>>  * - Xephyr seems to create two windows at the moment. Not sure why,
>> just minimize the other one.
>>
>> Screenshot, in case anybody is interested:
>> http://www.ucc.asn.au/~smspillaz/autopilot-xephyr.png
>>
>> Best,
>>
>> Sam
>>
>> --
>> Sam Spilsbury
>>
>> --
>> Mailing list: https://launchpad.net/~unity-dev
>> Post to     : unity-dev@xxxxxxxxxxxxxxxxxxx
>> Unsubscribe : https://launchpad.net/~unity-dev
>> More help   : https://help.launchpad.net/ListHelp
>
>
>
>
> --
> Thomi Richards
> thomi.richards@xxxxxxxxxxxxx



-- 
Sam Spilsbury


Follow ups

References