← Back to team overview

yahoo-eng-team team mailing list archive

  1. yahoo-eng-team team
  2. Mailing list archive
  3. Message #78051

[Bug 1825034] [NEW] listing deleted servers from the API fails after running fill_virtual_interface_list online data migration

  Thread Previous • Date Previous • • Thread Next 
Public bug reported:

I found this bug while trying to recreate bug 1825018 with a functional
test.

The fill_virtual_interface_list online data migration creates a fake
mostly empty instance record to satisfy a foreign key constraint in the
virtual_interfaces table which is used as a marker when paging across
cells to fulfill the migration. The problem is if you list deleted
servers (as admin) with the all_tenants=1 and deleted=1 filters, the API
will fail with a 500 error trying to load the instance.flavor field:

    b'2019-04-16 15:08:53,720 ERROR [nova.api.openstack.wsgi] Unexpected exception in API method'
    b'Traceback (most recent call last):'
    b'  File "/home/osboxes/git/nova/.tox/functional-py36/lib/python3.6/site-packages/urllib3/connectionpool.py", line 377, in _make_request'
    b'    httplib_response = conn.getresponse(buffering=True)'
    b"TypeError: getresponse() got an unexpected keyword argument 'buffering'"
    b''
    b'During handling of the above exception, another exception occurred:'
    b''
    b'Traceback (most recent call last):'
    b'  File "/home/osboxes/git/nova/nova/api/openstack/wsgi.py", line 671, in wrapped'
    b'    return f(*args, **kwargs)'
    b'  File "/home/osboxes/git/nova/nova/api/validation/__init__.py", line 192, in wrapper'
    b'    return func(*args, **kwargs)'
    b'  File "/home/osboxes/git/nova/nova/api/validation/__init__.py", line 192, in wrapper'
    b'    return func(*args, **kwargs)'
    b'  File "/home/osboxes/git/nova/nova/api/validation/__init__.py", line 192, in wrapper'
    b'    return func(*args, **kwargs)'
    b'  File "/home/osboxes/git/nova/nova/api/openstack/compute/servers.py", line 136, in detail'
    b'    servers = self._get_servers(req, is_detail=True)'
    b'  File "/home/osboxes/git/nova/nova/api/openstack/compute/servers.py", line 330, in _get_servers'
    b'    req, instance_list, cell_down_support=cell_down_support)'
    b'  File "/home/osboxes/git/nova/nova/api/openstack/compute/views/servers.py", line 390, in detail'
    b'    cell_down_support=cell_down_support)'
    b'  File "/home/osboxes/git/nova/nova/api/openstack/compute/views/servers.py", line 425, in _list_view'
    b'    for server in servers]'
    b'  File "/home/osboxes/git/nova/nova/api/openstack/compute/views/servers.py", line 425, in <listcomp>'
    b'    for server in servers]'
    b'  File "/home/osboxes/git/nova/nova/api/openstack/compute/views/servers.py", line 222, in show'
    b'    show_extra_specs),'
    b'  File "/home/osboxes/git/nova/nova/api/openstack/compute/views/servers.py", line 494, in _get_flavor'
    b'    instance_type = instance.get_flavor()'
    b'  File "/home/osboxes/git/nova/nova/objects/instance.py", line 1191, in get_flavor'
    b'    return getattr(self, attr)'
    b'  File "/home/osboxes/git/nova/.tox/functional-py36/lib/python3.6/site-packages/oslo_versionedobjects/base.py", line 67, in getter'
    b'    self.obj_load_attr(name)'
    b'  File "/home/osboxes/git/nova/nova/objects/instance.py", line 1114, in obj_load_attr'
    b'    self._obj_load_attr(attrname)'
    b'  File "/home/osboxes/git/nova/nova/objects/instance.py", line 1158, in _obj_load_attr'
    b'    self._load_flavor()'
    b'  File "/home/osboxes/git/nova/nova/objects/instance.py", line 967, in _load_flavor'
    b'    self.flavor = instance.flavor'
    b'  File "/home/osboxes/git/nova/.tox/functional-py36/lib/python3.6/site-packages/oslo_versionedobjects/base.py", line 67, in getter'
    b'    self.obj_load_attr(name)'
    b'  File "/home/osboxes/git/nova/nova/objects/instance.py", line 1101, in obj_load_attr'
    b'    objtype=self.obj_name())'
    b'nova.exception.OrphanedObjectError: Cannot call obj_load_attr on orphaned Instance object'
    b'2019-04-16 15:08:53,722 INFO [nova.api.openstack.wsgi] HTTP exception thrown: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.'
    b"<class 'nova.exception.OrphanedObjectError'>"
    b'2019-04-16 15:08:53,723 INFO [nova.api.openstack.requestlog] 127.0.0.1 "GET /v2.1/6f70656e737461636b20342065766572/servers/detail?all_tenants=1&deleted=1" status: 500 len: 208 microversion: 2.1 time: 0.138964'

** Affects: nova
     Importance: High
         Status: Confirmed


** Tags: upgrade

** Changed in: nova
       Status: New => Confirmed

** Changed in: nova
   Importance: Undecided => High

** Tags added: upgrade

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1825034

Title:
  listing deleted servers from the API fails after running
  fill_virtual_interface_list online data migration

Status in OpenStack Compute (nova):
  Confirmed

Bug description:
  I found this bug while trying to recreate bug 1825018 with a
  functional test.

  The fill_virtual_interface_list online data migration creates a fake
  mostly empty instance record to satisfy a foreign key constraint in
  the virtual_interfaces table which is used as a marker when paging
  across cells to fulfill the migration. The problem is if you list
  deleted servers (as admin) with the all_tenants=1 and deleted=1
  filters, the API will fail with a 500 error trying to load the
  instance.flavor field:

      b'2019-04-16 15:08:53,720 ERROR [nova.api.openstack.wsgi] Unexpected exception in API method'
      b'Traceback (most recent call last):'
      b'  File "/home/osboxes/git/nova/.tox/functional-py36/lib/python3.6/site-packages/urllib3/connectionpool.py", line 377, in _make_request'
      b'    httplib_response = conn.getresponse(buffering=True)'
      b"TypeError: getresponse() got an unexpected keyword argument 'buffering'"
      b''
      b'During handling of the above exception, another exception occurred:'
      b''
      b'Traceback (most recent call last):'
      b'  File "/home/osboxes/git/nova/nova/api/openstack/wsgi.py", line 671, in wrapped'
      b'    return f(*args, **kwargs)'
      b'  File "/home/osboxes/git/nova/nova/api/validation/__init__.py", line 192, in wrapper'
      b'    return func(*args, **kwargs)'
      b'  File "/home/osboxes/git/nova/nova/api/validation/__init__.py", line 192, in wrapper'
      b'    return func(*args, **kwargs)'
      b'  File "/home/osboxes/git/nova/nova/api/validation/__init__.py", line 192, in wrapper'
      b'    return func(*args, **kwargs)'
      b'  File "/home/osboxes/git/nova/nova/api/openstack/compute/servers.py", line 136, in detail'
      b'    servers = self._get_servers(req, is_detail=True)'
      b'  File "/home/osboxes/git/nova/nova/api/openstack/compute/servers.py", line 330, in _get_servers'
      b'    req, instance_list, cell_down_support=cell_down_support)'
      b'  File "/home/osboxes/git/nova/nova/api/openstack/compute/views/servers.py", line 390, in detail'
      b'    cell_down_support=cell_down_support)'
      b'  File "/home/osboxes/git/nova/nova/api/openstack/compute/views/servers.py", line 425, in _list_view'
      b'    for server in servers]'
      b'  File "/home/osboxes/git/nova/nova/api/openstack/compute/views/servers.py", line 425, in <listcomp>'
      b'    for server in servers]'
      b'  File "/home/osboxes/git/nova/nova/api/openstack/compute/views/servers.py", line 222, in show'
      b'    show_extra_specs),'
      b'  File "/home/osboxes/git/nova/nova/api/openstack/compute/views/servers.py", line 494, in _get_flavor'
      b'    instance_type = instance.get_flavor()'
      b'  File "/home/osboxes/git/nova/nova/objects/instance.py", line 1191, in get_flavor'
      b'    return getattr(self, attr)'
      b'  File "/home/osboxes/git/nova/.tox/functional-py36/lib/python3.6/site-packages/oslo_versionedobjects/base.py", line 67, in getter'
      b'    self.obj_load_attr(name)'
      b'  File "/home/osboxes/git/nova/nova/objects/instance.py", line 1114, in obj_load_attr'
      b'    self._obj_load_attr(attrname)'
      b'  File "/home/osboxes/git/nova/nova/objects/instance.py", line 1158, in _obj_load_attr'
      b'    self._load_flavor()'
      b'  File "/home/osboxes/git/nova/nova/objects/instance.py", line 967, in _load_flavor'
      b'    self.flavor = instance.flavor'
      b'  File "/home/osboxes/git/nova/.tox/functional-py36/lib/python3.6/site-packages/oslo_versionedobjects/base.py", line 67, in getter'
      b'    self.obj_load_attr(name)'
      b'  File "/home/osboxes/git/nova/nova/objects/instance.py", line 1101, in obj_load_attr'
      b'    objtype=self.obj_name())'
      b'nova.exception.OrphanedObjectError: Cannot call obj_load_attr on orphaned Instance object'
      b'2019-04-16 15:08:53,722 INFO [nova.api.openstack.wsgi] HTTP exception thrown: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.'
      b"<class 'nova.exception.OrphanedObjectError'>"
      b'2019-04-16 15:08:53,723 INFO [nova.api.openstack.requestlog] 127.0.0.1 "GET /v2.1/6f70656e737461636b20342065766572/servers/detail?all_tenants=1&deleted=1" status: 500 len: 208 microversion: 2.1 time: 0.138964'

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1825034/+subscriptions

Follow ups

  Thread Previous • Date Previous • • Thread Next