yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1828862] [NEW] Listing servers with the "--all-tenants" and "--deleted" flag fails due to a bad marker

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Surya Seetharaman <1828862@xxxxxxxxxxxxxxxxxx>
Date: Mon, 13 May 2019 16:48:52 -0000
Reply-to: Bug 1828862 <1828862@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
Public bug reported:

If "nova list --all-tenants --deleted" is run after the
"virtual_interface_obj.fill_virtual_interface_list" migration that was
added in Stein it will fail with a "<class
'nova.exception.OrphanedObjectError'> (HTTP 500)" error because of
encountering a bad marker. This will be a problem until the archival is
run I guess. Anyhow while admin listing this marker should not even show
up under the list of deleted servers. I guess this should be filtered
out in some way.

I am also not sure if the operator is supposed to just archive the
nuisance marker because it defeats the purpose of the persistent marker.

========Traceback================
May 13 16:41:51 surya001 devstack@n-api.service[19544]: DEBUG nova.objects.instance [None req-df297b54-82b1-43ea-83c1-fac0f27705a9 admin admin] Lazy-loading 'flavor' on Instance uuid 00000000-0000-0000-0000-000000000000 {{(pid=19555) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1110}}
May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi [None req-df297b54-82b1-43ea-83c1-fac0f27705a9 admin admin] Unexpected exception in API method: OrphanedObjectError: Cannot call obj_load_attr on orphaned Instance object
May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi Traceback (most recent call last):
May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi   File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 671, in wrapped
May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi     return f(*args, **kwargs)
May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi   File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper
May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi     return func(*args, **kwargs)
May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi   File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper
May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi     return func(*args, **kwargs)
May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi   File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper
May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi     return func(*args, **kwargs)
May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi   File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 136, in detail
May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi     servers = self._get_servers(req, is_detail=True)
May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi   File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 330, in _get_servers
May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi     req, instance_list, cell_down_support=cell_down_support)
May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi   File "/opt/stack/nova/nova/api/openstack/compute/views/servers.py", line 390, in detail
May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi     cell_down_support=cell_down_support)
May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi   File "/opt/stack/nova/nova/api/openstack/compute/views/servers.py", line 425, in _list_view
May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi     for server in servers]
May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi   File "/opt/stack/nova/nova/api/openstack/compute/views/servers.py", line 222, in show
May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi     show_extra_specs),
May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi   File "/opt/stack/nova/nova/api/openstack/compute/views/servers.py", line 494, in _get_flavor
May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi     instance_type = instance.get_flavor()
May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi   File "/opt/stack/nova/nova/objects/instance.py", line 1191, in get_flavor
May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi     return getattr(self, attr)
May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi   File "/usr/local/lib/python2.7/dist-packages/oslo_versionedobjects/base.py", line 67, in getter
May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi     self.obj_load_attr(name)
May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi   File "/opt/stack/nova/nova/objects/instance.py", line 1114, in obj_load_attr
May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi     self._obj_load_attr(attrname)
May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi   File "/opt/stack/nova/nova/objects/instance.py", line 1158, in _obj_load_attr
May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi     self._load_flavor()
May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi   File "/opt/stack/nova/nova/objects/instance.py", line 967, in _load_flavor
May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi     self.flavor = instance.flavor
May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi   File "/usr/local/lib/python2.7/dist-packages/oslo_versionedobjects/base.py", line 67, in getter
May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi     self.obj_load_attr(name)
May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi   File "/opt/stack/nova/nova/objects/instance.py", line 1101, in obj_load_attr
May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi     objtype=self.obj_name())
May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi OrphanedObjectError: Cannot call obj_load_attr on orphaned Instance object
May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi
May 13 16:41:51 surya001 devstack@n-api.service[19544]: INFO nova.api.openstack.wsgi [None req-df297b54-82b1-43ea-83c1-fac0f27705a9 admin admin] HTTP exception thrown: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
May 13 16:41:51 surya001 devstack@n-api.service[19544]: <class 'nova.exception.OrphanedObjectError'>
May 13 16:41:51 surya001 devstack@n-api.service[19544]: DEBUG nova.api.openstack.wsgi [None req-df297b54-82b1-43ea-83c1-fac0f27705a9 admin admin] Returning 500 to user: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
May 13 16:41:51 surya001 devstack@n-api.service[19544]: <class 'nova.exception.OrphanedObjectError'> {{(pid=19555) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:943}}

** Affects: nova
     Importance: Undecided
         Status: New

** Affects: nova/stein
     Importance: Undecided
         Status: New


** Tags: api

** Also affects: nova/stein
   Importance: Undecided
       Status: New

** Description changed:

  If "nova list --all-tenants --deleted" is run after the
  "virtual_interface_obj.fill_virtual_interface_list" migration that was
  added in Stein it will fail with a "<class
  'nova.exception.OrphanedObjectError'> (HTTP 500)" error because of
- encountering a bad marker. I am not sure if the operator is supposed to
- just delete the nuisance marker because it defeats the purpose of the
- persistent marker.
+ encountering a bad marker. This will be a problem until the archival is
+ run I guess. Anyhow while admin listing this marker should not even show
+ up under the list of deleted servers. I guess this should be filtered
+ out in some way.
+ 
+ 
+ I am not sure if the operator is supposed to just delete the nuisance marker because it defeats the purpose of the persistent marker.
  
  ========Traceback================
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: DEBUG nova.objects.instance [None req-df297b54-82b1-43ea-83c1-fac0f27705a9 admin admin] Lazy-loading 'flavor' on Instance uuid 00000000-0000-0000-0000-000000000000 {{(pid=19555) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1110}}
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi [None req-df297b54-82b1-43ea-83c1-fac0f27705a9 admin admin] Unexpected exception in API method: OrphanedObjectError: Cannot call obj_load_attr on orphaned Instance object
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi Traceback (most recent call last):
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi   File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 671, in wrapped
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi     return f(*args, **kwargs)
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi   File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi     return func(*args, **kwargs)
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi   File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi     return func(*args, **kwargs)
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi   File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi     return func(*args, **kwargs)
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi   File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 136, in detail
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi     servers = self._get_servers(req, is_detail=True)
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi   File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 330, in _get_servers
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi     req, instance_list, cell_down_support=cell_down_support)
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi   File "/opt/stack/nova/nova/api/openstack/compute/views/servers.py", line 390, in detail
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi     cell_down_support=cell_down_support)
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi   File "/opt/stack/nova/nova/api/openstack/compute/views/servers.py", line 425, in _list_view
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi     for server in servers]
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi   File "/opt/stack/nova/nova/api/openstack/compute/views/servers.py", line 222, in show
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi     show_extra_specs),
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi   File "/opt/stack/nova/nova/api/openstack/compute/views/servers.py", line 494, in _get_flavor
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi     instance_type = instance.get_flavor()
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi   File "/opt/stack/nova/nova/objects/instance.py", line 1191, in get_flavor
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi     return getattr(self, attr)
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi   File "/usr/local/lib/python2.7/dist-packages/oslo_versionedobjects/base.py", line 67, in getter
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi     self.obj_load_attr(name)
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi   File "/opt/stack/nova/nova/objects/instance.py", line 1114, in obj_load_attr
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi     self._obj_load_attr(attrname)
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi   File "/opt/stack/nova/nova/objects/instance.py", line 1158, in _obj_load_attr
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi     self._load_flavor()
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi   File "/opt/stack/nova/nova/objects/instance.py", line 967, in _load_flavor
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi     self.flavor = instance.flavor
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi   File "/usr/local/lib/python2.7/dist-packages/oslo_versionedobjects/base.py", line 67, in getter
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi     self.obj_load_attr(name)
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi   File "/opt/stack/nova/nova/objects/instance.py", line 1101, in obj_load_attr
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi     objtype=self.obj_name())
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi OrphanedObjectError: Cannot call obj_load_attr on orphaned Instance object
- May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi 
+ May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: INFO nova.api.openstack.wsgi [None req-df297b54-82b1-43ea-83c1-fac0f27705a9 admin admin] HTTP exception thrown: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: <class 'nova.exception.OrphanedObjectError'>
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: DEBUG nova.api.openstack.wsgi [None req-df297b54-82b1-43ea-83c1-fac0f27705a9 admin admin] Returning 500 to user: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: <class 'nova.exception.OrphanedObjectError'> {{(pid=19555) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:943}}

** Description changed:

  If "nova list --all-tenants --deleted" is run after the
  "virtual_interface_obj.fill_virtual_interface_list" migration that was
  added in Stein it will fail with a "<class
  'nova.exception.OrphanedObjectError'> (HTTP 500)" error because of
  encountering a bad marker. This will be a problem until the archival is
  run I guess. Anyhow while admin listing this marker should not even show
  up under the list of deleted servers. I guess this should be filtered
  out in some way.
  
- 
- I am not sure if the operator is supposed to just delete the nuisance marker because it defeats the purpose of the persistent marker.
+ I am also not sure if the operator is supposed to just archive the
+ nuisance marker because it defeats the purpose of the persistent marker.
  
  ========Traceback================
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: DEBUG nova.objects.instance [None req-df297b54-82b1-43ea-83c1-fac0f27705a9 admin admin] Lazy-loading 'flavor' on Instance uuid 00000000-0000-0000-0000-000000000000 {{(pid=19555) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1110}}
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi [None req-df297b54-82b1-43ea-83c1-fac0f27705a9 admin admin] Unexpected exception in API method: OrphanedObjectError: Cannot call obj_load_attr on orphaned Instance object
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi Traceback (most recent call last):
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi   File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 671, in wrapped
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi     return f(*args, **kwargs)
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi   File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi     return func(*args, **kwargs)
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi   File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi     return func(*args, **kwargs)
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi   File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi     return func(*args, **kwargs)
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi   File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 136, in detail
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi     servers = self._get_servers(req, is_detail=True)
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi   File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 330, in _get_servers
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi     req, instance_list, cell_down_support=cell_down_support)
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi   File "/opt/stack/nova/nova/api/openstack/compute/views/servers.py", line 390, in detail
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi     cell_down_support=cell_down_support)
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi   File "/opt/stack/nova/nova/api/openstack/compute/views/servers.py", line 425, in _list_view
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi     for server in servers]
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi   File "/opt/stack/nova/nova/api/openstack/compute/views/servers.py", line 222, in show
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi     show_extra_specs),
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi   File "/opt/stack/nova/nova/api/openstack/compute/views/servers.py", line 494, in _get_flavor
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi     instance_type = instance.get_flavor()
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi   File "/opt/stack/nova/nova/objects/instance.py", line 1191, in get_flavor
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi     return getattr(self, attr)
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi   File "/usr/local/lib/python2.7/dist-packages/oslo_versionedobjects/base.py", line 67, in getter
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi     self.obj_load_attr(name)
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi   File "/opt/stack/nova/nova/objects/instance.py", line 1114, in obj_load_attr
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi     self._obj_load_attr(attrname)
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi   File "/opt/stack/nova/nova/objects/instance.py", line 1158, in _obj_load_attr
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi     self._load_flavor()
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi   File "/opt/stack/nova/nova/objects/instance.py", line 967, in _load_flavor
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi     self.flavor = instance.flavor
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi   File "/usr/local/lib/python2.7/dist-packages/oslo_versionedobjects/base.py", line 67, in getter
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi     self.obj_load_attr(name)
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi   File "/opt/stack/nova/nova/objects/instance.py", line 1101, in obj_load_attr
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi     objtype=self.obj_name())
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi OrphanedObjectError: Cannot call obj_load_attr on orphaned Instance object
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: INFO nova.api.openstack.wsgi [None req-df297b54-82b1-43ea-83c1-fac0f27705a9 admin admin] HTTP exception thrown: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: <class 'nova.exception.OrphanedObjectError'>
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: DEBUG nova.api.openstack.wsgi [None req-df297b54-82b1-43ea-83c1-fac0f27705a9 admin admin] Returning 500 to user: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: <class 'nova.exception.OrphanedObjectError'> {{(pid=19555) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:943}}

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1828862

Title:
  Listing servers with the "--all-tenants" and "--deleted" flag fails
  due to a bad marker

Status in OpenStack Compute (nova):
  New
Status in OpenStack Compute (nova) stein series:
  New

Bug description:
  If "nova list --all-tenants --deleted" is run after the
  "virtual_interface_obj.fill_virtual_interface_list" migration that was
  added in Stein it will fail with a "<class
  'nova.exception.OrphanedObjectError'> (HTTP 500)" error because of
  encountering a bad marker. This will be a problem until the archival
  is run I guess. Anyhow while admin listing this marker should not even
  show up under the list of deleted servers. I guess this should be
  filtered out in some way.

  I am also not sure if the operator is supposed to just archive the
  nuisance marker because it defeats the purpose of the persistent
  marker.

  ========Traceback================
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: DEBUG nova.objects.instance [None req-df297b54-82b1-43ea-83c1-fac0f27705a9 admin admin] Lazy-loading 'flavor' on Instance uuid 00000000-0000-0000-0000-000000000000 {{(pid=19555) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1110}}
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi [None req-df297b54-82b1-43ea-83c1-fac0f27705a9 admin admin] Unexpected exception in API method: OrphanedObjectError: Cannot call obj_load_attr on orphaned Instance object
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi Traceback (most recent call last):
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi   File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 671, in wrapped
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi     return f(*args, **kwargs)
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi   File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi     return func(*args, **kwargs)
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi   File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi     return func(*args, **kwargs)
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi   File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi     return func(*args, **kwargs)
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi   File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 136, in detail
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi     servers = self._get_servers(req, is_detail=True)
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi   File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 330, in _get_servers
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi     req, instance_list, cell_down_support=cell_down_support)
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi   File "/opt/stack/nova/nova/api/openstack/compute/views/servers.py", line 390, in detail
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi     cell_down_support=cell_down_support)
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi   File "/opt/stack/nova/nova/api/openstack/compute/views/servers.py", line 425, in _list_view
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi     for server in servers]
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi   File "/opt/stack/nova/nova/api/openstack/compute/views/servers.py", line 222, in show
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi     show_extra_specs),
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi   File "/opt/stack/nova/nova/api/openstack/compute/views/servers.py", line 494, in _get_flavor
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi     instance_type = instance.get_flavor()
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi   File "/opt/stack/nova/nova/objects/instance.py", line 1191, in get_flavor
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi     return getattr(self, attr)
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi   File "/usr/local/lib/python2.7/dist-packages/oslo_versionedobjects/base.py", line 67, in getter
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi     self.obj_load_attr(name)
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi   File "/opt/stack/nova/nova/objects/instance.py", line 1114, in obj_load_attr
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi     self._obj_load_attr(attrname)
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi   File "/opt/stack/nova/nova/objects/instance.py", line 1158, in _obj_load_attr
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi     self._load_flavor()
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi   File "/opt/stack/nova/nova/objects/instance.py", line 967, in _load_flavor
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi     self.flavor = instance.flavor
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi   File "/usr/local/lib/python2.7/dist-packages/oslo_versionedobjects/base.py", line 67, in getter
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi     self.obj_load_attr(name)
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi   File "/opt/stack/nova/nova/objects/instance.py", line 1101, in obj_load_attr
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi     objtype=self.obj_name())
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi OrphanedObjectError: Cannot call obj_load_attr on orphaned Instance object
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: ERROR nova.api.openstack.wsgi
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: INFO nova.api.openstack.wsgi [None req-df297b54-82b1-43ea-83c1-fac0f27705a9 admin admin] HTTP exception thrown: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: <class 'nova.exception.OrphanedObjectError'>
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: DEBUG nova.api.openstack.wsgi [None req-df297b54-82b1-43ea-83c1-fac0f27705a9 admin admin] Returning 500 to user: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
  May 13 16:41:51 surya001 devstack@n-api.service[19544]: <class 'nova.exception.OrphanedObjectError'> {{(pid=19555) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:943}}

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1828862/+subscriptions
Follow ups

[Bug 1828862] Re: Listing servers with the "--all-tenants" and "--deleted" flag fails due to a bad marker
From: Surya Seetharaman, 2019-05-14