yahoo-eng-team team mailing list archive

[OpenStack Infra <1592963@xxxxxxxxxxxxxxxxxx>]

Reviewed:  https://review.openstack.org/330229
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=ffe2487cf60e91659b6cb8a09b10de5455d9a3ed
Submitter: Jenkins
Branch:    master

commit ffe2487cf60e91659b6cb8a09b10de5455d9a3ed
Author: Matt Riedemann <mriedem@xxxxxxxxxx>
Date:   Wed Jun 15 17:21:23 2016 -0400

    Pre-load tags when showing server details
    
    When microversion>=2.26 is requested to show server details
    we should pre-load 'tags' when getting the instance from the
    database. Listing server details already does this, but show
    didn't.
    
    Failing to do this can also result in a 500 response from the
    server. The scenario is you delete a server and then poll for
    it to be gone using 'nova show'. In between the time that the
    server is retrieved from the database and the view builder
    creates the response, the instance is deleted and since
    instance.tags is not set it will attempt to lazy-load the tags
    which will fail with an InstanceNotFound.
    
    We need to also fix the lazy-load issue in the instance object
    but that will come in a follow-on change.
    
    Change-Id: Iae6551028179e31699c06d06284ca4c1660be240
    Closes-Bug: #1592963


** Changed in: nova
       Status: In Progress => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1592963

Title:
  Showing server details on a deleted instance fails for >=v2.26

Status in OpenStack Compute (nova):
  Fix Released

Bug description:
  This failed in a python-novaclient functional CI run for newton
  (current master):

  http://logs.openstack.org/11/328211/5/gate/gate-novaclient-dsvm-
  functional/cd6a3d2/logs/screen-n-api.txt.gz?level=TRACE

  2016-06-15 19:42:01.313 12360 ERROR nova.api.openstack.extensions [req-7b05c7c9-153d-4571-9d56-5a421159af1a admin admin] Unexpected exception in API method
  2016-06-15 19:42:01.313 12360 ERROR nova.api.openstack.extensions Traceback (most recent call last):
  2016-06-15 19:42:01.313 12360 ERROR nova.api.openstack.extensions   File "/opt/stack/new/nova/nova/api/openstack/extensions.py", line 453, in wrapped
  2016-06-15 19:42:01.313 12360 ERROR nova.api.openstack.extensions     return f(*args, **kwargs)
  2016-06-15 19:42:01.313 12360 ERROR nova.api.openstack.extensions   File "/opt/stack/new/nova/nova/api/openstack/compute/servers.py", line 543, in show
  2016-06-15 19:42:01.313 12360 ERROR nova.api.openstack.extensions     return self._view_builder.show(req, instance)
  2016-06-15 19:42:01.313 12360 ERROR nova.api.openstack.extensions   File "/opt/stack/new/nova/nova/api/openstack/compute/views/servers.py", line 317, in show
  2016-06-15 19:42:01.313 12360 ERROR nova.api.openstack.extensions     server["server"]["tags"] = [t.tag for t in instance.tags]
  2016-06-15 19:42:01.313 12360 ERROR nova.api.openstack.extensions   File "/usr/local/lib/python2.7/dist-packages/oslo_versionedobjects/base.py", line 67, in getter
  2016-06-15 19:42:01.313 12360 ERROR nova.api.openstack.extensions     self.obj_load_attr(name)
  2016-06-15 19:42:01.313 12360 ERROR nova.api.openstack.extensions   File "/opt/stack/new/nova/nova/objects/instance.py", line 985, in obj_load_attr
  2016-06-15 19:42:01.313 12360 ERROR nova.api.openstack.extensions     self._load_generic(attrname)
  2016-06-15 19:42:01.313 12360 ERROR nova.api.openstack.extensions   File "/opt/stack/new/nova/nova/objects/instance.py", line 765, in _load_generic
  2016-06-15 19:42:01.313 12360 ERROR nova.api.openstack.extensions     expected_attrs=[attrname])
  2016-06-15 19:42:01.313 12360 ERROR nova.api.openstack.extensions   File "/usr/local/lib/python2.7/dist-packages/oslo_versionedobjects/base.py", line 181, in wrapper
  2016-06-15 19:42:01.313 12360 ERROR nova.api.openstack.extensions     result = fn(cls, context, *args, **kwargs)
  2016-06-15 19:42:01.313 12360 ERROR nova.api.openstack.extensions   File "/opt/stack/new/nova/nova/objects/instance.py", line 416, in get_by_uuid
  2016-06-15 19:42:01.313 12360 ERROR nova.api.openstack.extensions     use_slave=use_slave)
  2016-06-15 19:42:01.313 12360 ERROR nova.api.openstack.extensions   File "/opt/stack/new/nova/nova/db/sqlalchemy/api.py", line 225, in wrapper
  2016-06-15 19:42:01.313 12360 ERROR nova.api.openstack.extensions     return f(*args, **kwargs)
  2016-06-15 19:42:01.313 12360 ERROR nova.api.openstack.extensions   File "/opt/stack/new/nova/nova/objects/instance.py", line 408, in _db_instance_get_by_uuid
  2016-06-15 19:42:01.313 12360 ERROR nova.api.openstack.extensions     columns_to_join=columns_to_join)
  2016-06-15 19:42:01.313 12360 ERROR nova.api.openstack.extensions   File "/opt/stack/new/nova/nova/db/api.py", line 692, in instance_get_by_uuid
  2016-06-15 19:42:01.313 12360 ERROR nova.api.openstack.extensions     return IMPL.instance_get_by_uuid(context, uuid, columns_to_join)
  2016-06-15 19:42:01.313 12360 ERROR nova.api.openstack.extensions   File "/opt/stack/new/nova/nova/db/sqlalchemy/api.py", line 169, in wrapper
  2016-06-15 19:42:01.313 12360 ERROR nova.api.openstack.extensions     return f(*args, **kwargs)
  2016-06-15 19:42:01.313 12360 ERROR nova.api.openstack.extensions   File "/opt/stack/new/nova/nova/db/sqlalchemy/api.py", line 270, in wrapped
  2016-06-15 19:42:01.313 12360 ERROR nova.api.openstack.extensions     return f(context, *args, **kwargs)
  2016-06-15 19:42:01.313 12360 ERROR nova.api.openstack.extensions   File "/opt/stack/new/nova/nova/db/sqlalchemy/api.py", line 1814, in instance_get_by_uuid
  2016-06-15 19:42:01.313 12360 ERROR nova.api.openstack.extensions     columns_to_join=columns_to_join)
  2016-06-15 19:42:01.313 12360 ERROR nova.api.openstack.extensions   File "/opt/stack/new/nova/nova/db/sqlalchemy/api.py", line 1823, in _instance_get_by_uuid
  2016-06-15 19:42:01.313 12360 ERROR nova.api.openstack.extensions     raise exception.InstanceNotFound(instance_id=uuid)
  2016-06-15 19:42:01.313 12360 ERROR nova.api.openstack.extensions InstanceNotFound: Instance a7c3ed25-c0dd-47a9-8eb5-b268dd0c3ad4 could not be found.
  2016-06-15 19:42:01.313 12360 ERROR nova.api.openstack.extensions 

  The test is waiting for the server to be deleted. It blows up here
  because we didn't lazy-load the instance tags:

  https://github.com/openstack/nova/blob/537df23d85e0f7c461643efe6b6501d267ae99d0/nova/api/openstack/compute/views/servers.py#L316

  And because we don't handle lazy-loading 'tags' specifically it does
  the load-generic with a join on the tags table, but the instance is
  deleted so it blows up with InstanceNotFound:

  https://github.com/openstack/nova/blob/537df23d85e0f7c461643efe6b6501d267ae99d0/nova/objects/instance.py#L753

  We should just treat it like we do for instance.services:

  https://github.com/openstack/nova/blob/537df23d85e0f7c461643efe6b6501d267ae99d0/nova/objects/instance.py#L921

  Especially because instance tags are deleted when the instance is
  deleted:

  https://github.com/openstack/nova/blob/537df23d85e0f7c461643efe6b6501d267ae99d0/nova/db/sqlalchemy/api.py#L2037

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1592963/+subscriptions