yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1860312] Re: compute service failed to delete

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Launchpad Bug Tracker <1860312@xxxxxxxxxxxxxxxxxx>
Date: Sun, 21 Jun 2020 04:17:29 -0000
Reply-to: Bug 1860312 <1860312@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
[Expired for OpenStack Compute (nova) because there has been no activity
for 60 days.]

** Changed in: nova
       Status: Incomplete => Expired

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1860312

Title:
  compute service failed to delete

Status in OpenStack Compute (nova):
  Expired

Bug description:
  Description
  ===========
  I deployed openstack with openstack-helm on kubernetes.When one of the nova-compute service(driver=ironic replica of the deployment is 1) breakdown.It may be scheduled to another node by kubernetes.When I try to delete the old compute service(status down), it failed.

  Steps to reproduce
  ==================
  Firstly, openstack was deployed in kubernetes cluster, and the replica of the nova-compute-ironic is 1.
  * I deleted the pod nova-compute-ironic-xxxxx
  * then wait for the new pod to start
  * then exec openstack compute service list, there will be two compute service for ironic, the status of the old one would be down.
  * then I try to delete the old compute service

  Expected result
  ===============
  the old compute service could be deleted successfully

  Actual result
  =============
  failed to delete, and returned an http 500

  Environment
  ===========
  1. Exact version of OpenStack you are running. See the following
     18.2.2, rocky

  2. Which hypervisor did you use?
     Libvirt + KVM

  2. Which storage type did you use?
     ceph

  3. Which networking type did you use?
     Neutron with OpenVSwitch

  Logs & Configs
  ==============
  2020-01-20 06:44:53.480 1 ERROR nova.api.openstack.wsgi [req-922cc601-9aa1-4c3d-ad9c-71f73a341c28 40e7b8c3d59943e08a52acd24fe30652 d13f1690c08d41ac854d720ea510a710 - default default] Unexpected exception in API method: ComputeHostNotFound: Compute host mgt-slave03 could not be found.
  2020-01-20 06:44:53.480 1 ERROR nova.api.openstack.wsgi Traceback (most recent call last):
  2020-01-20 06:44:53.480 1 ERROR nova.api.openstack.wsgi   File "/var/lib/openstack/local/lib/python2.7/site-packages/nova/api/openstack/wsgi.py", line 801, in wrapped
  2020-01-20 06:44:53.480 1 ERROR nova.api.openstack.wsgi     return f(*args, **kwargs)
  2020-01-20 06:44:53.480 1 ERROR nova.api.openstack.wsgi   File "/var/lib/openstack/local/lib/python2.7/site-packages/nova/api/openstack/compute/services.py", line 252, in delete
  2020-01-20 06:44:53.480 1 ERROR nova.api.openstack.wsgi     context, service.host)
  2020-01-20 06:44:53.480 1 ERROR nova.api.openstack.wsgi   File "/var/lib/openstack/local/lib/python2.7/site-packages/oslo_versionedobjects/base.py", line 184, in wrapper
  2020-01-20 06:44:53.480 1 ERROR nova.api.openstack.wsgi     result = fn(cls, context, *args, **kwargs)
  2020-01-20 06:44:53.480 1 ERROR nova.api.openstack.wsgi   File "/var/lib/openstack/local/lib/python2.7/site-packages/nova/objects/compute_node.py", line 443, in get_all_by_host
  2020-01-20 06:44:53.480 1 ERROR nova.api.openstack.wsgi     use_slave=use_slave)
  2020-01-20 06:44:53.480 1 ERROR nova.api.openstack.wsgi   File "/var/lib/openstack/local/lib/python2.7/site-packages/nova/db/sqlalchemy/api.py", line 213, in wrapper
  2020-01-20 06:44:53.480 1 ERROR nova.api.openstack.wsgi     return f(*args, **kwargs)
  2020-01-20 06:44:53.480 1 ERROR nova.api.openstack.wsgi   File "/var/lib/openstack/local/lib/python2.7/site-packages/nova/objects/compute_node.py", line 438, in _db_compute_node_get_all_by_host
  2020-01-20 06:44:53.480 1 ERROR nova.api.openstack.wsgi     return db.compute_node_get_all_by_host(context, host)
  2020-01-20 06:44:53.480 1 ERROR nova.api.openstack.wsgi   File "/var/lib/openstack/local/lib/python2.7/site-packages/nova/db/api.py", line 291, in compute_node_get_all_by_host
  2020-01-20 06:44:53.480 1 ERROR nova.api.openstack.wsgi     return IMPL.compute_node_get_all_by_host(context, host)
  2020-01-20 06:44:53.480 1 ERROR nova.api.openstack.wsgi   File "/var/lib/openstack/local/lib/python2.7/site-packages/nova/db/sqlalchemy/api.py", line 258, in wrapped
  2020-01-20 06:44:53.480 1 ERROR nova.api.openstack.wsgi     return f(context, *args, **kwargs)
  2020-01-20 06:44:53.480 1 ERROR nova.api.openstack.wsgi   File "/var/lib/openstack/local/lib/python2.7/site-packages/nova/db/sqlalchemy/api.py", line 659, in compute_node_get_all_by_host
  2020-01-20 06:44:53.480 1 ERROR nova.api.openstack.wsgi     raise exception.ComputeHostNotFound(host=host)
  2020-01-20 06:44:53.480 1 ERROR nova.api.openstack.wsgi ComputeHostNotFound: Compute host mgt-slave03 could not be found.
  2020-01-20 06:44:53.480 1 ERROR nova.api.openstack.wsgi 
  2020-01-20 06:44:53.480 1 ERROR nova.api.openstack.wsgi [req-922cc601-9aa1-4c3d-ad9c-71f73a341c28 40e7b8c3d59943e08a52acd24fe30652 d13f1690c08d41ac854d720ea510a710 - default default] Unexpected exception in API method: ComputeHostNotFound: Compute host mgt-slave03 could not be found.
  2020-01-20 06:44:53.480 1 ERROR nova.api.openstack.wsgi Traceback (most recent call last):
  2020-01-20 06:44:53.480 1 ERROR nova.api.openstack.wsgi   File "/var/lib/openstack/local/lib/python2.7/site-packages/nova/api/openstack/wsgi.py", line 801, in wrapped
  2020-01-20 06:44:53.480 1 ERROR nova.api.openstack.wsgi     return f(*args, **kwargs)
  2020-01-20 06:44:53.480 1 ERROR nova.api.openstack.wsgi   File "/var/lib/openstack/local/lib/python2.7/site-packages/nova/api/openstack/compute/services.py", line 252, in delete
  2020-01-20 06:44:53.480 1 ERROR nova.api.openstack.wsgi     context, service.host)
  2020-01-20 06:44:53.480 1 ERROR nova.api.openstack.wsgi   File "/var/lib/openstack/local/lib/python2.7/site-packages/oslo_versionedobjects/base.py", line 184, in wrapper
  2020-01-20 06:44:53.480 1 ERROR nova.api.openstack.wsgi     result = fn(cls, context, *args, **kwargs)
  2020-01-20 06:44:53.480 1 ERROR nova.api.openstack.wsgi   File "/var/lib/openstack/local/lib/python2.7/site-packages/nova/objects/compute_node.py", line 443, in get_all_by_host
  2020-01-20 06:44:53.480 1 ERROR nova.api.openstack.wsgi     use_slave=use_slave)
  2020-01-20 06:44:53.480 1 ERROR nova.api.openstack.wsgi   File "/var/lib/openstack/local/lib/python2.7/site-packages/nova/db/sqlalchemy/api.py", line 213, in wrapper
  2020-01-20 06:44:53.480 1 ERROR nova.api.openstack.wsgi     return f(*args, **kwargs)
  2020-01-20 06:44:53.480 1 ERROR nova.api.openstack.wsgi   File "/var/lib/openstack/local/lib/python2.7/site-packages/nova/objects/compute_node.py", line 438, in _db_compute_node_get_all_by_host
  2020-01-20 06:44:53.480 1 ERROR nova.api.openstack.wsgi     return db.compute_node_get_all_by_host(context, host)
  2020-01-20 06:44:53.480 1 ERROR nova.api.openstack.wsgi   File "/var/lib/openstack/local/lib/python2.7/site-packages/nova/db/api.py", line 291, in compute_node_get_all_by_host
  2020-01-20 06:44:53.480 1 ERROR nova.api.openstack.wsgi     return IMPL.compute_node_get_all_by_host(context, host)
  2020-01-20 06:44:53.480 1 ERROR nova.api.openstack.wsgi   File "/var/lib/openstack/local/lib/python2.7/site-packages/nova/db/sqlalchemy/api.py", line 258, in wrapped
  2020-01-20 06:44:53.480 1 ERROR nova.api.openstack.wsgi     return f(context, *args, **kwargs)
  2020-01-20 06:44:53.480 1 ERROR nova.api.openstack.wsgi   File "/var/lib/openstack/local/lib/python2.7/site-packages/nova/db/sqlalchemy/api.py", line 659, in compute_node_get_all_by_host
  2020-01-20 06:44:53.480 1 ERROR nova.api.openstack.wsgi     raise exception.ComputeHostNotFound(host=host)
  2020-01-20 06:44:53.480 1 ERROR nova.api.openstack.wsgi ComputeHostNotFound: Compute host mgt-slave03 could not be found

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1860312/+subscriptions
References

[Bug 1860312] [NEW] compute service failed to delete
From: tanghang, 2020-01-20