ubuntu-appstore-developers team mailing list archive

[James Tait <james.tait@xxxxxxxxxxxxx>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 25/07/13 09:37, Daniel Holbach wrote:
> Hello everybody,
> 
> I think it'd be good to talk a bit about continuous integration for
> a bit and see what kind of testing we already have in place, both
> on the server-side, on the client side, which example data we use,
> if we can run tests on actual phone hardware easily, etc.
> 
> For your projects, what do you have in place already? What is
> planned next? What's possible?

For click-package-index, right now we have Jenkins automatically
running our unit tests before merging branches to trunk.  When we want
to create a new test case we write:

 - A JSON file representing the data SCA will POST to us
 - A JSON file representing the data we will POST to Solr (via sunburnt)
 - A Python dictionary representing what Solr will return to us (ditto)
 - A JSON file representing what we will return to the client

This allows us to test the transformations at each stage in isolation,
(i.e. "this POST from SCA should result in this call to sunburnt" -
mocking out the Solr interface) while simultaneously ensuring that any
schema changes don't completely break our ability to index data - we
*will* need to re-index when the schema changes, but we have a
load-sample-data Makefile target that will POST our sample files into
Solr and complain if it fails, so we know we need to adjust our
transformations.  Our small test suite currently runs in under 0.1
seconds.

Łukasz is currently working on acceptance tests that will use real
components - i.e. POSTing real data to the service, indexing it in a
real Solr instance, then performing real queries against the service
and verifying the results.  We still need to discuss it some more, but
the intention is that we have a representative sample of the kinds of
queries the Dash will end up sending us and that the tests ensure the
responses are usable as designed.  Since these will be expensive to
set up and run, I expect they'll only be run nightly.

HTH,

JT
- -- 
James Tait, BSc. | https://launchpad.net/~jamestait/
Software Engineer, Canonical Online Services, Web and Ops Team
Ubuntu - Linux for human beings | www.ubuntu.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)
Comment: Using GnuPG with undefined - http://www.enigmail.net/

iEYEARECAAYFAlHxP+EACgkQyDo4xMNTLiY6sQCgkrirKiONQfF612/N2CgmcRrW
YA4An1+dqAFCps/kk2YtYeS3pYVjh5lV
=b1AR
-----END PGP SIGNATURE-----