checkbox-dev team mailing list archive

[Daniel Manrique <daniel.manrique@xxxxxxxxxxxxx>]

On Fri, Apr 24, 2015 at 7:16 AM, Zygmunt Krynicki
<zygmunt.krynicki@xxxxxxxxxxxxx> wrote:
> Hey Pierre.
>
> I've just released a new version (0.2) that contains a command to
> lookup sub-devices. Have a look at it (and at the corresponding,
> trivial-to-use API). The release changelog can be found on [1]. The
> package can be installed with "pip3 install --user libpci". Enjoy!
> Please tell me if you'd like to use Ubuntu packages instead.
>
> For a quick example on how it can be used:
>
> zyga@g580:~/libpci$ ./pci-lookup subsystem-device 14e4 4727 14e4 0587
> vendor-id:   0x14e4
> device-id:   0x4727
> subvendor-id:0x14e4
> subdevice-id:0x0587
> subsystem-device-name: BCM4313 802.11bgn Wireless Network Adapter
>
> [1] http://libpci.readthedocs.org/en/latest/history.html#id1

This library is awesome! Just a bit of implementation data on how
checkbox does this. Device names come from the udev_resource script,
which in turn simply parses udevadm info --export-db. SO if a name is
missing there, it will also be missing in the information checkbox
sees.

Maybe we can integrate Zygmunt's library in the udev_resource script
so it can look up and augment devices for which it has no name
information. If we do this, it'd be ideal to do it in the udev parser
so that the server side components can also benefit from the
improvement. Though I wonder if we could also look at why udevadm is
not properly showing this information. I see udev maintains its own
hardware database, but I'm not sure if this is the reason for what you
saw.


>
> Best regards
> ZK
>
> On Fri, Apr 24, 2015 at 9:31 AM, Zygmunt Krynicki
> <zygmunt.krynicki@xxxxxxxxxxxxx> wrote:
>> Hey, just a quick update - the four argument version is not
>> implemented yet. I just didn't finish it yet :-) It's very easy though
>> (just call lookup4 with four arguments) if you want to try.
>>
>> I'll post an update soon.
>>
>> On Fri, Apr 24, 2015 at 5:01 AM, Pierre Equoy
>> <pierre.equoy@xxxxxxxxxxxxx> wrote:
>>>
>>>
>>> On Fri, Apr 24, 2015 at 3:37 AM, Zygmunt Krynicki
>>> <zygmunt.krynicki@xxxxxxxxxxxxx> wrote:
>>>>
>>>>
>>>>
>>>> W dniu 23.04.2015 o 21:25, Daniel Manrique pisze:
>>>>>
>>>>> On Thu, Apr 23, 2015 at 3:17 PM, Zygmunt Krynicki
>>>>> <zygmunt.krynicki@xxxxxxxxxxxxx> wrote:
>>>>>>
>>>>>> Hey
>>>>>>
>>>>>> A while ago I talked to Pierre about a bug where we would not see
>>>>>> product name for a certain PCI graphics card. We looked, read, and
>>>>>> searched and realized how the PCI database works. We thought about
>>>>>> adding a local database or about doing something better than just not
>>>>>> working at all.
>>>>>
>>>>>
>>>>> Just to confirm, the recent hack so that if there's no product name we
>>>>> show the PCI ID, was that helpful at all?
>>>>
>>>>
>>>> There is an entry for the vendor:device but there's no entry for
>>>> vendor:device subsystem:subvendor. There was no entry in the upstream
>>>> repository.
>>>
>>>
>>> OK, a bit of story here:
>>>
>>> We often test relatively new hardware (graphic cards), so they often don't
>>> appear in the pci.ids database. [1] For instance, ChickletW 12 [2] has a
>>> dual AMD+Intel GPU. If I type 'lspci -k', here is what I get for the graphic
>>> cards:
>>>
>>> 00:02.0 VGA compatible controller: Intel Corporation Broadwell-U Integrated
>>> Graphics (rev 09)
>>>     Subsystem: Hewlett-Packard Company Device 2216
>>>     Kernel driver in use: i915
>>> 04:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI]
>>> Opal XT [Radeon R7 M265]
>>>     Subsystem: Hewlett-Packard Company Device 8006
>>>     Kernel driver in use: fglrx_pci
>>>
>>> Running the associated plainbox job leaves me with a very vague result:
>>>
>>> $ plainbox run -i "2013.com.canonical.certification::graphics_card_resource"
>>> --dont-suppress-output
>>> (...)
>>>
>>> bus: pci
>>> category: VIDEO
>>> driver: i915
>>> index: 1
>>> path: /devices/pci0000:00/0000:00:02.0
>>> product: PCI ID 0x1616
>>> product_id: 5654
>>> subproduct_id: 8726
>>> subvendor_id: 4156
>>> vendor: Intel Corporation
>>> vendor_id: 32902
>>>
>>> bus: pci
>>> category: VIDEO
>>> driver: fglrx_pci
>>> index: 2
>>> path: /devices/pci0000:00/0000:00:1c.4/0000:04:00.0
>>> product: PCI ID 0x6604
>>> product_id: 26116
>>> subproduct_id: 32774
>>> subvendor_id: 4156
>>> vendor: Advanced Micro Devices, Inc. [AMD/ATI]
>>> vendor_id: 4098
>>>
>>>
>>> The names that are going to be used through the whole testing process are
>>> "PCI ID 0x1616" and "PCI ID 0x6604". It's better than nothing, but it's
>>> still error-prone. (I know because it happened to me: I was not careful and
>>> in every test it says something like "switch to this card if it's not the
>>> one currently in use", so I mistakenly switched once or twice, it messed up
>>> the test results and I had to restart the tests).
>>>
>>> Now, if I edit the subsystems in the pci.ids files and re-run 'lspci -k', I
>>> get this, which is slightly better:
>>>
>>> 00:02.0 VGA compatible controller: Intel Corporation Broadwell-U Integrated
>>> Graphics (rev 09)
>>>     Subsystem: Hewlett-Packard Company Broadwell-U Integrated Graphics
>>>     Kernel driver in use: i915
>>> 04:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI]
>>> Opal XT [Radeon R7 M265]
>>>     Subsystem: Hewlett-Packard Company FirePro M4170
>>>     Kernel driver in use: fglrx_pci
>>>
>>> However, the plainbox job doesn't see this and still provides the same
>>> output:
>>>
>>> bus: pci
>>> category: VIDEO
>>> driver: i915
>>> index: 1
>>> path: /devices/pci0000:00/0000:00:02.0
>>> product: PCI ID 0x1616
>>> product_id: 5654
>>> subproduct_id: 8726
>>> subvendor_id: 4156
>>> vendor: Intel Corporation
>>> vendor_id: 32902
>>>
>>> bus: pci
>>> category: VIDEO
>>> driver: fglrx_pci
>>> index: 2
>>> path: /devices/pci0000:00/0000:00:1c.4/0000:04:00.0
>>> product: PCI ID 0x6604
>>> product_id: 26116
>>> subproduct_id: 32774
>>> subvendor_id: 4156
>>> vendor: Advanced Micro Devices, Inc. [AMD/ATI]
>>> vendor_id: 4098
>>>
>>> That said, I hope Zygmunt's lib can be helpful to access the subsystem's
>>> information. I quickly tried libpci using the 'pci-lookup' script but I
>>> couldn't get it to work with a "vendor:product subvendor:subproduct"
>>> configuration.
>>>
>>> We should push vendors to update this database but if they don't, we can
>>> also update it ourselves. I will discuss about this with the rest of the
>>> Stella QA team on Monday.
>>>
>>>
>>> Cheers!
>>>
>>>
>>>>
>>>>> The bug number would be useful to see more detail about this :)
>>>>
>>>>
>>>> Sorry, I was too lazy to look it up. It is
>>>> https://bugs.launchpad.net/plainbox-provider-checkbox/+bug/1343089
>>>>
>>>>>> The canonical way to look up those names is through libpci. Since
>>>>>> libpci is written in C and has a simple API I thought about making
>>>>>> some bindings so that we can use that in checkbox.
>>>>>>
>>>>>> And so I did, this is an example program that does vendor or device
>>>>>> name lookup [1, 2, 3]. Note that currently it only works on Python 3.4
>>>>>> as I used the enum module (it doesn't work on Ubuntu 12.04).
>>>>>>
>>>>>> One interesting observation is the ability to query the central server
>>>>>> when a given identifier cannot be found in a local database. You can
>>>>>> use that by setting pci.flag_network = True (or call the pci-lookup
>>>>>> script with --network argument at the end).
>>>>>
>>>>>
>>>>> Without more data on the bug I can only speculate, but if this
>>>>> happened on a SUT, the solution that looks up stuff on the central
>>>>> server may not work (tm) if the network is down for some reason. In
>>>>> this case you'll still get no product name, which can be an issue if
>>>>> you're running tests locally. If this is about something else, then
>>>>> perhaps I'm wrong :)
>>>>
>>>>
>>>> There are many possibilities. Libpci can generate phony device names,
>>>> mixed names, can use a local cache, can lookup data over network, etc. It's
>>>> quite flexible.
>>>>
>>>> Personally, I think we should just use the network lookup feature in HEXR.
>>>> This would remove any reason to update PCI database periodically as it would
>>>> always be up-to-date.
>>>>
>>>>>> Please tell me what you think.
>>>>>
>>>>>
>>>>> I think this is awesome! and should be quite useful.
>>>>
>>>>
>>>> I looked at the API and we can use it to enumerate devices. We can use
>>>> various constants to filter devices (perhaps better than checkbox-support
>>>> currently does). Have a look at this list
>>>>
>>>> http://libpci.readthedocs.org/en/latest/usage.html#module-libpci._macros
>>>>
>>>> Thanks
>>>> ZK
>>>
>>>
>>>
>>> [1]: https://pci-ids.ucw.cz/
>>> [2]: https://certification.canonical.com/hardware/201411-16142/
>>>
>>> --
>>> Pierre Equoy
>>> QA Engineer | Canonical
>>> www.canonical.com | www.ubuntu.com