linaro-release team mailing list archive

[James Tunnicliffe <launchpad@xxxxxxxxxxxxx>]

Is LAVA the right tool for this? Clearly we need something to run tests,
but we can use other tools if they are a better fit.

I would rather this was an automated tool that didn't run on developer
machines. We should expect most bugs to be visible on developer machines
and use an automated tool (on check-in / nightly) to run extra
validation.

I guess the next question is about hosting of VMs. If we have a physical
box, it would make sense to have it in the validation lab. We could rent
time on EC2 / Rackspace Cloud / some other cloud compute platform. EC2
is quite good for validation since, as I understand it, you get a fresh
instance every time and have to install any extra software you need.
There are also Amazon Machine Images going back to the last LTS
available for us to use (see
https://help.ubuntu.com/community/EC2StartersGuide). That said, creating
a few VMs on a box that we own shouldn't be a big deal.

I don't know how LAVA will cope with using EC2 machines as the clients.
If it wants a permanent connection, that won't work well (since we will
want to drop the instance after the end of each run to reset it and save
money).

-- 
You received this bug notification because you are a member of linaro-
infrastructure-drivers, which is the registrant for Linaro Image Tools.
https://bugs.launchpad.net/bugs/888555

Title:
  Set up l-i-t testing on multiple archs and releases

Status in Linaro Image Tools:
  New

Bug description:
  We need to test l-i-t on all supported releases and architectures. We
  have seen a number of cases where new features are later found to not
  work on old releases or older versions of Python. We just had a
  problem with linaro-hwpack create not working on Oneiric 64 bit, but
  happily work on 32 bit.

  For testing new code and especially for release testing we would need
  to set something up to enable this since a single developer won't have
  access to all variants of installations.

  We could perhaps automate something to set up virtual machines on
  local dev PC:s with possible impact on TDD since tests would take many
  times longer to run. Perhaps we can use LAVA?

  For full coverage we'd need to test on all possible combinations of
  the below components (please add to them as you see fit), but that is
  probably overkill.

  Python:
      2.6
      2.7

  Ubuntu:
      Precise
      Oneiric
      Natty
      Maverick
      (how far back do we want to ensure support?)

  Archs:
      i386
      x86_64

To manage notifications about this bug go to:
https://bugs.launchpad.net/linaro-image-tools/+bug/888555/+subscriptions