← Back to team overview

checkbox-dev team mailing list archive

New Checkbox Release (Week 37)

 

Hello,

A new release of the Checkbox packages has been uploaded to the Hardware
Certification Public PPA
(https://launchpad.net/~hardware-certification/+archive/public).

This release is based on the following bzr tags of lp:checkbox:

plainbox-v0.12
checkbox-support-v0.9
checkbox-ng-v0.10
checkbox-gui-v0.24
plainbox-provider-resource-generic-v0.8
plainbox-provider-checkbox-v0.11
plainbox-provider-canonical-certification-v0.7

Bugs fixed:
http://goo.gl/2tfmtF

The roadmap for the next 2 releases is as follows:

Checkbox release (2014-09-26):
http://goo.gl/pfNANf

Checkbox release (2014-10-10):
http://goo.gl/mI0oCw

And finally, at the end of this e-mail, find  a detailed log of all the changes
in this stable release (the changelog versus the last stable checkbox release
from 2014-09-12). Note that the change log is still experimental.


Thanks,

Sylvain Pineau

-------------------------------------------------------
No Debian packaging changes for checkbox-gui
-------------------------------------------------------
No Debian packaging changes for checkbox-ng
-------------------------------------------------------
No Debian packaging changes for checkbox-support
-------------------------------------------------------
No Debian packaging changes for plainbox
-------------------------------------------------------
Debian packaging changes for plainbox-provider-canonical-certification
  [ Jeff Lane ]
  * removed packaging for ppcs-soc
  * add canonical-certification-submit as recommends for p-p-c-server
    so that it is always installed to SUTs and available to testers
    without additionall work

-------------------------------------------------------
No Debian packaging changes for plainbox-provider-checkbox
-------------------------------------------------------
No Debian packaging changes for plainbox-provider-resource-generic
-------------------------------------------------------
Features and bugfixes in checkbox trunk code
  [ Jeff Lane ]
  * Failure Ouput should be a little more informative now. Also added
    iperf test example to help intro message
  * semantics
  * Changed test in miscellanea/get-maas-version to -s instead of -e
    to catch cases where the file is created, but left empty for some
    reason. Also improved the output on failure cases
  * Removed soc provider stuff, moved a single whitelist to p-p-c-s
    which will now be the home for the single soc whitelist

  [ Zygmunt Krynicki ]
  * plainbox:provider_manager: add an option to use the new validation
    stack
  * plainbox:providers: add support for keyword arguments on
    InsecureProvider1PlugInCollection This patch makes
    InsecureProvider1PlugInCollection accept arbitrary keyword
    arguments. Those arguments are simply forwarded to the base
    FsPlugInCollection class. The intent is to allow anyone to
    construct a plugin collection that will not, for example, validate
    or check the loaded providers.
  * plainbox:secure:providers: make Provider1PlugIn() honor new-style
    validation arguments This patch adds support for the new-style
    validation arguments to Provider1Plugin(). The new arguments,
    check and context, are now being forwarded to the
    Provider1.from_definition() class method.
  * plainbox:secure:providers: make Provider1.from_definition() honor
    new-style validation arguments
  * plainbox:secure:providers: add check= and context= to Provider1()
    This patch adds two new optional keyword-only arguments to the
    Provider1 class. They describe the new-style validation
    configuration for this provider's units (this complements existing
    old-style validation arguments) The new arguments are passed to
    Provider1._load_units() where they are finally passed to
    UnitPlugIn()
  * plainbox:secure:providers: add support for keyword arguments on
    SecureProvider1PlugInCollection This patch makes
    SsecureProvider1PlugInCollection accept arbitrary keyword
    arguments. Those arguments are simply forwarded to the base
    FsPlugInCollection class. The intent is to allow anyone to
    construct a plugin collection that will not, for example, validate
    or check the loaded providers.
  * plainbox:secure:providers: add a way to use new validation stack
    via UnitPlugIn
  * plainbox:unit:category: add __all__
  * plainbox:unit:category: replace UnitCategory.fields with
    .Meta.fields
  * plainbox:unit:category: add new validation meta-data
  * plainbox:unit:category: fix PEP-8 issue
  * plainbox:unit:category: remove CategoryUnit.get_unit_type() This
    method is now implemented by the base UnitLegacyAPI, based on the
    Unit.tr_unit() method that CategoryUnit now correctly overrides
  * plainbox:unit:category: fix/add several docstrings
  * plainbox:unit:category: silence validate() deprecation warnings in
    tests
  * plainbox:unit:template: simplify
    TemplateUnit.get_target_unit_cls()
  * plainbox:unit:template: document
    TemplateUnit.get_target_unit_cls() exceptions
  * plainbox:unit:template: tweak TemplateUnit.template_unit
    documentation bork bork bork!
  * plainbox:unit:template: add TemplateUnit.instantiate_template()
  * plainbox:unit:template: add TemplateUnit(parameters=) argument
    This patch restores the intially-gone parameters= argument to the
    TemplateUnit initializer. This argument was gone because it was
    not thought useful to have templates that generate templates. For
    unit tests, it is useful to have a consistent interface to all
    units hence the parameters= field should be supported, even we
    don't plan to use it
  * plainbox:unit:template: alias .Meta.fields as .fields
  * plainbox:unit:template: add custom TemplateUnitValidator
  * plainbox:unit:template: add new-style validation meta-data
  * plainbox:unit:template: base TemplateUnit.fields on Unit.fields
    This patch corrects an earlier mistake in how TemplateUnit.fields
    were defined. In the olden days, the UnitTemplate used to be
    called JobTemplate (that's before we got units as a concept) and
    thus the only fields that the template could have were the fields
    of the job definition. Since now a template can define units of
    any kind inheritance makes no sense.
  * plainbox:unit:template: fix PEP-8 issue
  * plainbox:unit:template: fix PEP-8 issue
  * plainbox:unit:template: remove TemplateUnit.get_unit_type() This
    method is now implemented by the base UnitLegacyAPI, based on the
    Unit.tr_unit() method that TemplateUnit now correctly overrides
  * plainbox:unit:template: add TemplateUnit.tr_unit()
  * plainbox:unit:template: don't refer to private attributes in id()
  * plainbox:unit:template: refer to JobDefinition.fields This patch
    corrects an earlier mistake where TemplateUnit.fields would
    inherit from JobDefinition.fields. Now that templates can
    instantiate all kinds of units (except themselves) this makes no
    sense and should be avoided. This will be a requirement later on
    once fields become automatically handled by UnitType, the Unit
    metaclass.
  * plainbox:unit:job: add __all__
  * plainbox:unit:job: replace JobDefinition.fields with a reference
  * plainbox:unit:job: silence warnings from
    JobDefinition.estimated_duration This patch silences the annoying
    warning that was printed if the 'estimated_duration' field was
    defined but incorrect. That case is now checked by the new job
    validator. The error case is still "valid" as in, no exception is
    being raised as not everything is currently using the new
    validator.
  * plainbox:unit:job: add new validation meta-data This patch adds a
    number of validators that thoroughly check job definition unit for
    correctness. A lot new checks are being performed (for things that
    were not looked at before in the old stack) so new errors and
    warnings may be reported (for valid issues). In addition the new
    validators will report a large number of advice issues (aka,
    hints).
  * plainbox:unit:job: move _PluginValues to module level This patch
    moves the private _PluginValues symbol definition class to top of
    the module. This makes it possible to access from within the
    definition of the JobDefinition class.
  * plainbox:unit.job: add JobDefinition.get_category_id()
  * plainbox:unit:job: use Unit.qualify_id() instead of a custom copy
  * plainbox:unit:job: remove JobDefinition.id This property is
    implemented by the base UnitWithId class so we don't need an
    identical copy here.
  * plainbox:unit:job: remove JobDefinition.get_unit_type() This
    method is now implemented by the base UnitLegacyAPI, based on the
    Unit.tr_unit() method that JobDefinition now correctly overrides
  * plainbox:unit:job: fix PEP-8 issue
  * plainbox:unit:job: fix typo
  * plainbox:unit:job: fix JobDefinition.Meta inheritance This patch
    is briefly required before all of the Meta + LegacyAPI inheritance
    mess becomes handled by the UnitType metaclass Here the goal is to
    simply see the JobDefinitionLegacyAPI.Meta's version of
    template_constraints (otherwise they are shadowed by
    UnitWithId.Meta's copy that only has checks for 'id' and 'unit'.
  * plainbox:unit:job: remove copyright typo
  * plainbox:unit:unit_with_id: add new-style validation metadata This
    patch adds new-style validation metadata to the UnitWithId class.
    The meta-data consists of a custom validator class
    (UnitWithIdValidator) and a validator for the 'id' field. The set
    of checks performed on the id field includes: - making sure nobody
    wants to translate 'id' - making sure that 'id' is present -
    making sure that 'id' is a variant in template context - making
    sure that 'id' is unique across all units - making sure that 'id'
    is just a partial_id without the namespace
  * plainbox:unit:unit_with_id: add UnitWithId.tr_unit() This patch
    adds UnitWithId.tr_unit(). This method should be implemented by
    all Unit subclasses so let's just do it even though we don't
    expect to see any unit-with-id in the wild.
  * plainbox:unit:unit_with_id: add UnitWithIdValidator This patch
    adds a custom validator subclass for UnitWithId. It offers a
    better explain() method that can use the partial_id property to
    help users understand to which unit each problem is related to
  * plainbox:unit:unit_with_id: fix logger definition
  * plainbox:unit:unit_with_id: fix UnitWithId.id
  * plainbox:unit:unit_with_id: add docstring for the id property
  * plainbox:unit:unit_with_id: fix partial_id docstring
  * plainbox:unit:unit_with_id: fix typo
  * plainbox:unit:unit: drop default value from Unit.unit This patch
    removes the default value of the Unit.unit property. This allows
    us to validate the actual value of the unit field.
  * plainbox:unit:unit: add Unit.Meta meta-data for the new validation
    code This patch adds validation meta-data to the base Unit class.
    The meta-data consists of three attributes: fields: this is a
    symbol definition class that has a symbol for each field that may
    be present in the unit. It is used by the validation stack and by
    the field_validators attribute. validator_cls: this attribute
    refers to the validator class to use for all new-style checks.
    Here it refers to the UnitValdidator base class. field_validators:
    this attribute contains a dictionary mapping from each field (from
    fields symbol def) to a list of validator classes that are used to
    check that particular field.
  * plainbox:unit:unit: add IssueMixIn
  * plainbox:unit:unit: add Unit.check() This patch adds a high-level
    API for checking units for correctness. It entirely replaces the
    old validate() method and offers more features with a simpler
    interface. The main difference is that *all* issues related to a
    unit are reported. The checker code no longer stops at the first
    encountered problem. The method offers two keyword-only arguments:
    the context argument allows to perform context-aware checks
    (global validation) and the live argument allows to return issues
    as soon as they are found instead of buffering them to an
    intermediate list.
  * plainbox:unit:unit: add Unit.qualify_id() This patch adds a method
    that can resolve relative unit identifiers based on the namespace
    of the resolving unit's provider reference.
  * plainbox:unit:unit: remove Unit.get_unit_type() This method is now
    also avialable via the UnitLegacyAPI so we should not need a
    second copy here.
  * plainbox:unit:unit: add Unit.is_translatable_field() This patch
    adds a method that can be used to check if a given field in the
    unit data was marked as translatable or not.
  * plainbox:unit:unit: fix PEP-8 issue
  * plainbox:unit:unit: make UnitValidator.make_field_validators()
    deterministic
  * plainbox:unit:unit: add new UnitValidator class This patch adds
    the new base unit validator class. Unlike in the past, the new
    class should almost never have to be subclassed and extended as it
    relies on field validators to do most of the heavy lifting. The
    core APIs are the two new methods UnitValidator.check(unit) and
    UnitValidator.check_in_context(unit, context) which look at a
    particular unit and generate subsequent Issue instances that
    describe all of the detected issues. The check_in_context() method
    gets an additional UnitValidationContext instance that allows to
    perform global analysis. This allows to verify correctness of
    units in context of other units (checking that references are
    correct, that identifiers are unique, etc). In addition to the
    check() methods there are report_issue() family of methods
    (including advice(), warning() and error() short-cuts) that
    collaborate with the particular field validators to simplify the
    process of reporting issues. All of the issues detected by a
    particular validator are now collected and exposed as the new
    `all_issues` attribute. Lastly there is the new explain() method
    that collaborates with report_issue() to provide human-readable
    strings for various Problem constants. It may be overridden in
    other classes to provide more accurate error messages
  * plainbox:unit:unit: correct module docstring
  * plainbox:unit:unit: silence validate() deprecation warnings in
    tests
  * plainbox:unit:validators: don't use collections.abc This patch
    removes the need for collections.abc that doesn't exist on
    python3.2 that we still support.
  * plainbox:unit:validators: make UnitValidationContext.provider_list
    a property
  * plainbox:unit:validators: tweak UnitReferenceValidator This patch
    adds a new intermediate class, ReferenceConstraint, that couples a
    constraint_fn (function), message (str) and onlyif (function)
  * plainbox:unit:validators: tweak UnitReferenceValidator error
    message
  * plainbox:unit:validators: fix ShellProgramValidator not to crash
    in rare cases This patch fixes a NameError that would be raised if
    token was never set, which can happen if the syntax error happens
    on the first token.
  * plainbox:unit:legacy: super call validate() everywhere
  * plainbox:unit: remove useless logging
  * plainbox:symbol: make Symbol comparable via
    functools.total_ordering
  * plainbox:abc: add IProvider1.get_units() This patch exposes the
    Provider1.get_units() method via the IProvider1 interface. This is
    required to mock some providers with mocks that use
    spec_set=IProvider1
  * plainbox:unit: split Unit and UnitWithId to separate modules This
    (large) patch simply splits the Unit and UnitWithId to separate
    modules. The set of affected modules is pretty large but most
    changes are trivial.
  * plainbox:unit: add unit field validators This patch adds a new
    module full of small field validators. It is relatively untested
    (no unit tests) but was tested during development and turned out
    to work OK. It's still a poor excuse for not having tests. My bad
  * plainbox:unit: add get_accessed_parameters() (function) This patch
    moves the core logic of Unit.get_accessed_parameters() out to be a
    standalone function.
  * plainbox:unit: copy all the "legacy" code to new _legacy module
    This patch takes legacy validation code paths from *each* existing
    unit and copies it to the new _legacy module. The module contains
    a collection of classes ${name}ValidatorLegacyAPI and
    ${name}LegacyAPI that contain legacy code for the validator and
    unit classes respectively. All of the validators were unified to
    be normal classes with normal methods and not a random collection
    of classes with normal and static methods. Some test code was
    adjusted not to talk to the validator directly and instead call
    unit.validate()
  * plainbox:provider_manager: PEP-8 fixes
  * plainbox:secure:providers: remove unused import
  * plainbox:secure:config: PEP-8 fix
  * plainbox:validation: add new Problem types This patch adds a few
    new possible problem types. They are currently unused but will
    soon show up in the new validation core branch.
  * plainbox:validation: add Severity This patch adds a simple symbol
    definition with a few symbols that describe the severity of an
    issue. Available choices are 'advice', 'warning' and 'error'.
  * plainbox:validation: add Issue This patch adds a new base class
    for all "issues" with a generic, high-level interface that can
    specify the description of the problem (message), its severity
    (severity), location (origin) and the kind of the problem (kind).
    It is meant to be used as a base class for concrete problem
    objects.
  * plainbox:symbol: fix typo
  * plainbox:symbol: add a allow_outer feature This patch adds a queer
    feature to the SymbolDef class which allows a symbol definition to
    "see" variables defined in the outer scope. It is especially
    useful for using N_(...) to mark each symbol value up for
    translation without having it translated.
  * providers:resource: don't validate deprecated features
  * providers:checkbox: fix trailing space
  * providers:checkbox: don't validate deprecated features
  * providers:checkbox: mark description as translatable
  * providers:categories: move category provider to new namespace This
    patch moves the "category" provider into the plainbox namespace.
    This has to be done because it otherwise clashes with some of the
    existing checkbox job definitions that have equally generic names.
  * providers:templates-demo: update Polish translations
  * providers:templates-demo: update POT file
  * providers:templates-demo: add description to all jobs This was
    caught by the new validation core ;-)

  [ Maciej Kisielewski ]
  * checkbox-touch: fix screens not properly stacking Ensure we're not
    not registering to a signal multiple times. Unwinding is done only
    on verification screen and when test is skipped.
  * checkbox-touch: add user-interact page
  * checkbox-touch: margin fix and code cleanup
  * checkbox-touch: make intro-page reusable to other types of tests
  * checkbox-touch: add manual test page
  * checkbox-touch: make verification-page reusable to other types of
    tests
  * checkbox-touch: make get-libs reuse same cache for set of packages
    get-libs now gets url of all packages from given source. This
    speeds up the process significantly.
  * checkbox-touch: make get-libs find correct uris for packages get-
    libs now creates environment for apt to work in. It configures
    target architecture, updates apt-cache and gets URI of the package
    to download. It uses ports.ubuntu.com for armhf and
    archive.ubuntu.com for i386 and amd64.
  * checkbox-touch: move python libraries to ./lib/py Store python
    libraries in ./lib/py which is git-ignored
  * checkbox-touch: [trivial] update get-libs - new libpython URI
  * checkbox-touch: add user-interact-verify pages This patch adds two
    components to checkbox-touch. Both are pages necessary to run
    user-interact-verify types of tests. 7b279ca checkbox-touch: add
    user-interact-verify pages f80867f checkbox-touch: add user-
    interact-verify pages 7b52689 checkbox-touch: change signalling in
    UIV pages 04ac55b checkbox-touch: UIV pages layout and structure
    improvements

  [ Sylvain Pineau ]
  * checkbox-ng: increment version to 0.10
  * plainbox: increment version to 0.12
  * checkbox-support: increment version to 0.9
  * plainbox: increment version to 0.12c2
  * checkbox-ng: increment version to 0.10c1
  * plainbox: increment version to 0.12c1
  * checkbox-support: increment version to 0.9c1

  [ Daniel Manrique ]
  * " plainbox:exporter:xml: Sort job results before converting to
    XML. As long as other consumers of our XML reports don't want to
    reorder questions, this guarantees test results will be output in
    alphabetical order Previously we'd show test results as they came
    straight from the result_map. This makes for hard-to-read HTML
    reports (and any report that derives from the XML one).
    [r=zkrynicki][bug=1260271][author=roadmr]"
  * plainbox:exporter:xml: Sort job results before converting to XML.
    As long as other consumers of our XML reports don't want to
    reorder questions, this guarantees test results will be output in
    alphabetical order.
  * plainbox:exporter:xml Test for alphabetical ordering of test
    results. Previously we'd show test results as they came straight
    from the result_map. This makes for hard-to-read HTML reports (and
    any report that derives from the XML one). This verifies that
    results (called 'questions' in our XML schema) are ordered
    alphabetically in the XML result file, regardless of their order
    in the result_map. The sample .json data dump, used to produce the
    mock result_map, was conveniently alphabetically ordered, so I
    flipped it to have this test fail. Interestingly it also made a
    few other perfect-xml-match tests fail, since they also relied on
    sample .json and .xml files having the same ordering.
  * "providers:checkbox: Add buildstamp attachment job
    [r=zkrynicki][bug=1362602][author=roadmr]"
  * providers:checkbox: Add buildstamp attachment job
  * "plainbox: Document deprecation of user-verify plugin
    [r=zkrynicki][bug=][author=roadmr]"
  * plainbox: Document deprecation of user-verify plugin
  * "plainbox, checkbox-ng: fixed grammar in --not-interactive
    [r=zkrynicki][bug=1349996][author=roadmr]"
  * plainbox, checkbox-ng: fixed grammar in --not-interactive
  * " checkbox-support:parsers:udevadm: Add heuristics, data files and
    tests for QEMU/KVM virtual machines, with an emphasis on virtio
    disks. These heuristics properly categorize these devices
    (category DISK). By their nature they're very bare and have few
    identifying features, but the driver and bus are quite
    distinctive, and the rest of the attributes used should also be
    present and reliable. The rest of the tests in the test suite were
    not modified, to ensure the new heuristics don't miscategorize any
    existing or previously-seen systems and/or devices.
    [r=zkrynicki][bug=1355282][author=roadmr]"
  * checkbox-support:parsers:udevadm: Add heuristics for QEMU/KVM
    virtio disks. These heuristics properly categorize these devices
    (category DISK). By their nature they're very bare and have few
    identifying features, but the driver and bus are quite
    distinctive, and the rest of the attributes used should also be
    present and reliable. The rest of the tests in the test suite were
    not modified, to ensure the new heuristics don't miscategorize any
    existing or previously-seen systems and/or devices.
  * checkbox-support:parsers:udev: add expectations for qemu-kvm
    system
  * checkbox-support:parsers:udev: Added test file for QEMU_KVM
    virtual machine
  * " checkbox-ng: Added reference to launcher/profile tutorial to doc
    index. checkbox-ng: Updated the profiles document. checkbox-ng:
    Added launcher tutorial page. Added information on launchers for
    CLI (using checkbox-launcher). Added some undocumented settings
    and converted setting descriptions to RST definition lists. The
    tutorial page describes old checkbox configuration (perhaps in too
    much detail) and explains the concept and rationale for launchers,
    providing and analyzing a few examples of launcher. [r=sylvain-
    pineau][bug=][author=roadmr]"
  * checkbox-ng: Added reference to launcher/profile tutorial to doc
    index.
  * checkbox-ng: Updated the profiles document. Added information on
    launchers for CLI (using checkbox-launcher). Added some
    undocumented settings and converted setting descriptions to RST
    definition lists.
  * checkbox-ng: Added launcher tutorial page. This page describes old
    checkbox configuration (perhaps in too much detail) and explains
    the concept and rationale for launchers, providing and analyzing a
    few examples of launcher.
  * "Release_2014_Week35 [r=roadmr][bug=1331302,1341769][author
    =checkbox-dev]"
  * Fix conflict with trunk
  * checkbox-ng: increment version to 0.10.dev
  * providers/plainbox-provider-resource-generic: increment version to
    0.9.dev
  * providers/plainbox-provider-checkbox: increment version to
    0.11.dev
  * plainbox: increment version to 0.12.dev
  * checkbox-support: increment version to 0.9.dev
  * checkbox-ng: increment version to 0.9

  [ Launchpad Translations on behalf of checkbox-dev ]
  * Launchpad automatic translations update.