checkbox-dev team mailing list archive

[Zygmunt Krynicki <zygmunt.krynicki@xxxxxxxxxxxxx>]

Hi.

This is a quick summary of the discussion about system states that we
just had in a hangout. The core idea behind system states is to
simplify job maintenance by modelling "states" and state transitions
that affect the DUT.

The classic example of this problem is the duplication (or
quadruplication if you include the manual and automatic variant) of
the before/after anything jobs. Those jobs express the requirement to
test a particular component both before and after suspend that, as you
know, often breaks something that works well on cold boot.

The idea that Yung came up with works is composed of the following new
bits of information.

- A set of states, so perhaps (my interpretation) a new state unit
- A new annotation on certain jobs that they cause a state transition
(e.g. that the suspend job causes the system to go from
not_seen_suspend to did_see_suspend state, hypothetically). Similarly
the reboot job can cause the did_see_suspend state to reset to its
nominal value, to express what is really happening with the DUT
- A new annotation, either at a test plan level or at a job definition
level, saying that we wish a job to run for all possible combinations
of given states. In practice this can be a test for GPU that runs
before and after suspend and before and after GPU switch.
- A tier annotation on states that assists plainbox in ordering
transitions. For example, we can say that we prefer to reboot first
then suspend or vice versa. It doesn't change the semantics of
anything but it does change the effective ordering of the sequence of
state transition events.

Having this information plainbox can synthesize a run list that
contains additional jobs, derived from the base definition, and can
ensure that all the required transitions occur and that all the tests
are in the right spot in the list. Currently this is a very time
consuming and fragile, manual process. This also means that we can
drastically reduce the number of existing jobs by combining many of
the before/after definitions into one definition.

Lastly, we can take advantage of the existing part of the unit
template system. All units can be parametric in plainbox. A template
is simply a parametric unit with non-zero list of parameters. We can
use that system to define non-template parametric units that have a
parameter for each modelled system state.

I hope this captures our discussions accurately. If not please post
corrections for everyone to see. Let's keep discussing this on the
mailing list.

Thanks
ZK