kubuntu-council team mailing list archive

[Dave Jones <mp+408427@xxxxxxxxxxxxxxxxxx>]

Certainly! At the moment, the Pi test cases (and from a quick browse a few other test cases) are *mostly* but not *entirely* duplicates of each other. For example, the ethernet tests are mostly universal, but don't apply to the 3A+ (which has no ethernet port). Likewise, the wifi tests are mostly universal, but don't apply to the 2B (which has no wifi). Similar situations for the aux audio port (not on the compute modules), or USB ports (no USB3 prior to the 4B series).

As a result, consider a couple of scenarios:

* I want to update the wifi test case to clarify the instructions. They're not defined in a single place, but nor are they universal across all the Pi test files so I can't perform (say) a blanket sed operation to update all the necessary files without first digging through them to find exactly which would be affected. However, with the "test_case_gen" script the wifi tests *are* defined in a single place and only included in the relevant Pi test files, so I'd update "pi_image_cases.xml", re-run "test_case_gen" and commit the results

* I want to add tests for a new model of Pi which we need to support. Again, it's not necessarily as simple as copying an existing test file, unless I'm absolutely certain it has exactly the same hardware as an existing model. With the "pi_image_cases.xml" configuration however, I'd copy the closest "ut:case" element, edit it as necessary (include / exclude tests as necessary), re-run "test_case_gen" and commit the results (which should simply be the addition of a single file)

In short, it's intended to be run whenever we either need to define a new set of tests similar (but not necessarily exactly the same) as an existing set of tests (the "new Pi model scenario"), or when we want to update tests which are duplicated over a wide swathe of models (the "clarify existing instructions scenario").

However, there's nothing specific to Pi models about the "test_case_gen" script; it could be used (with a different XML input file) to simplify generation (and therefore maintenance and consistency) of the existing test cases which contain large areas of duplication. 1311 (non-english installation full network support) and 1312 (non-english installation no network), for example.

I'm happy to extend the MP with another XML configuration covering some such cases if that would make things clearer?
-- 
https://code.launchpad.net/~waveform/ubuntu-manual-tests/+git/ubuntu-manual-tests/+merge/408427
Your team Ubuntu Testcase Admins is subscribed to branch ubuntu-manual-tests:master.