yahoo-eng-team team mailing list archive

[Lee Yarwood <lyarwood@xxxxxxxxxx>]

** No longer affects: nova

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1831866

Title:
  tempest.api.compute.servers.test_servers.ServerShowV263Test only
  passing when image has already been cached

Status in tempest:
  In Progress

Bug description:
  tempest.api.compute.servers.test_servers.ServerShowV263Test aims to
  test the optional trusted_image_certificates parameter introduced into
  n-api in microversion 2.63:

  Add trusted_image_certificates to REST API
  https://review.opendev.org/#/c/486204/

  Add new schema for Nova microversion 2.63
  https://review.opendev.org/#/c/526485/

  However this test currently relies on other tests having cached the
  image used to create the test instance before it is run. Otherwise the
  underlying logic within Nova [1] to validate the image using the
  provided certificates will be executed and fail as the image is
  missing image properties such as img_signature_hash_method etc [2].

  [1] https://github.com/openstack/nova/blob/6009cdaa47a10f6d06a31b68fdc6a263bced13ad/nova/image/glance.py#L402-L460
  [2] https://logs.rdoproject.org/openstack-periodic-24hr/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-7-ovb-1ctlr_2comp-featureset020-rocky/207e8c1/logs/overcloud-novacompute-0/var/log/containers/nova/nova-compute.log.txt.gz?level=ERROR#_2019-05-31_07_06_11_779

  This behaviour can be easily reproduced in devstack by cleaning out
  the local image cache and running
  tempest.api.compute.servers.test_servers.ServerShowV263Test in
  isolation:

  $ rm -rf ~/data/nova/instances/_base/*
  $ tempest run --regex tempest.api.compute.servers.test_servers.ServerShowV263Test
  [..]
  ==============================
  Failed 1 tests - output below:
  ==============================

  tempest.api.compute.servers.test_servers.ServerShowV263Test.test_show_update_rebuild_list_server[id-71b8e3d5-11d2-494f-b917-b094a4afed3c]
  -------------------------------------------------------------------------------------------------

  Captured traceback:
  ~~~~~~~~~~~~~~~~~~~
      Traceback (most recent call last):
        File "tempest/api/compute/servers/test_servers.py", line 194, in test_show_update_rebuild_list_server
          wait_until='ACTIVE')
        File "tempest/api/compute/base.py", line 246, in create_test_server
          **kwargs)
        File "tempest/common/compute.py", line 265, in create_test_server
          server['id'])
        File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
          self.force_reraise()
        File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
          six.reraise(self.type_, self.value, self.tb)
        File "tempest/common/compute.py", line 236, in create_test_server
          clients.servers_client, server['id'], wait_until)
        File "tempest/common/waiters.py", line 76, in wait_for_server_status
          server_id=server_id)
      tempest.exceptions.BuildErrorException: Server 254fb4a1-74a8-4681-ad67-a287bb3cde66 failed to build and is in ERROR status
      Details: {u'message': u'Build of instance 254fb4a1-74a8-4681-ad67-a287bb3cde66 aborted: Signature verification for the image failed: Required image properties for signature verification do not exist. Cannot verify signature. Missing property: img_signature_hash_method.', u'code': 500, u'created': u'2019-06-06T08:38:11Z'}

  Running all of the test_servers tests together however typically
  allows this to pass:

  $ rm -rf ~/data/nova/instances/_base/*
  $ tempest run --regex tempest.api.compute.servers.test_servers
  [..]
  ======
  Totals
  ======
  Ran: 59 tests in 150.0000 sec.
   - Passed: 58
   - Skipped: 1
   - Expected Fail: 0
   - Unexpected Success: 0
   - Failed: 0
  Sum of execute time for each test: 522.7800 sec.

  I'm going to tag nova into this bug report as well as I'm not sure
  that the current behaviour of n-api is correct. At present we attempt
  to verify the image when the optional trusted_image_certificates
  parameter is provided, regardless of
  CONF.glance.verify_glance_signatures being enabled within nova.conf:

  https://github.com/openstack/nova/blob/6009cdaa47a10f6d06a31b68fdc6a263bced13ad/nova/image/glance.py#L414

  While documented that behaviour doesn't seem right and I believe led
  to some incorrect assumptions being made by the initial writer of this
  test.

  https://docs.openstack.org/nova/latest/user/certificate-
  validation.html

  "Certificate validation will only be performed if image signature
  validation is enabled. However, the presence of trusted certificate
  IDs overrides the enable_certificate_validation and
  verify_glance_signatures settings. In other words, if a list of
  trusted certificate IDs is provided to the instance create or rebuild
  commands, signature verification and certificate validation will be
  performed, regardless of their settings in the Nova configurations.
  See Using Signature Verification for details."

To manage notifications about this bug go to:
https://bugs.launchpad.net/tempest/+bug/1831866/+subscriptions