firmware-testing-team team mailing list archive

[Colin Ian King <colin.king@xxxxxxxxxxxxx>]

Hi there Firmware Testing Team!

This week brings a couple of additions to the firmware test suite in the
form of "Interactive Tests".  Up until now, the test suite has been
running in batch mode, that is, you start the tests and leave fwts to
run until completion.  Interactive tests require some form of user
intervention, for example, pressing hot keys, opening/closing a laptop
lid or observing state change such as LCD brightness changes.

By default, fwts runs just the batch mode (non-interactive) tests - this
is equivalent to running fwts with the --batch option.  However, one can
turn on just the interactive tests by using the --interactive option.
If you want both these tests, use both --batch and --interactive option
flags.  Specifying an interactive test by name will also run the test
too. Examples:

Run all interactive tests:

$ sudo fwts --interactive

Run all batch mode tests (default)

$ sudo fwts
or
$ sudo fwts --batch

Run all tests:

$ sudo fwts --batch --interactive

Run just the brightness interactive test:

$ sudo fwts brightness

This week I've added a chunk of functionality to do acpid event
monitoring behind the scenes and added some more interactive keyboard
handling library routines.

=== This weeks new interactive tests ===

1. LID mode sensing

For this test, one needs to disable the gnome power management settings
that suspend or hibernate a laptop if you close the laptop lid before
running this test.

$ sudo fwts lid

The test will emit some instructions that need following to run the test
to completion. It will time out after 20 seconds if inactive.

2. Laptop Screen Brightness and Brightness hotkey tests

The "brightness" test exercises several layers of functionality:

a) kernel /sys/devices/virtual/backlight interface
b) Min/Max brightness levels
c) All brightness levels
d) ACPI hotkeys that increase/decrease brightness levels

=== Other changes to fwts this week ===

1.  Regex scanning of kernel message log.

The original kernel message log was fast, but clumsy. I've now
re-written the scanner to use regular expressions. This leads to more
flexible pattern matching, however it does run slower. However, for
most kernel logs, this is not noticeable. It only becomes an issue when
scanning thousands of kernel logs.  I believe the added flexibility of
regex pattern matching outweighs the reduced speed.

2. More kernel message log pattern matching.

To compliment the regex feature, I've also added in many more kernel log
error and warning messages to diagnose. I ran through ~2000 kernel
message logs from Launch Pad and identified several hundred common
messages and plugged the patterns into the kernel log scanning.

3. In addition to finer message checking, I've also added more advice
feedback when a kernel error or warning is found. Some of the advice re-
states the error message in a more understandable form.  Some of the
advice feedback suggests workarounds or BIOS features than need fixing.
I hope to expand on this once I've dug into the kernel and figured out
what all the kernel messages exactly mean and sorted out possible
workarounds/fixes.

As ever, this tool is only good if it's used and tested. So please keep
testing and please feed back any bugs you find or improvements you want
in this tool.

Thanks!

Colin