← Back to team overview

yahoo-eng-team team mailing list archive

  1. yahoo-eng-team team
  2. Mailing list archive
  3. Message #88436

[Bug 1964137] [NEW] Failed to delete compute service

  Thread Previous • Date Previous • • Thread Next 
Public bug reported:


I have temporally added a new compute host to my openstack wallaby cloud.  It was not the first time I have done this, but it is the first time it fails. 

To add the compute host I installed nova-compute and neutron-linuxbridge
services in it, and then updated the cloud using the command "nova-
manage cell_v2 discover_hosts" in my controller.

It worked well, but then I decided to remove the host, so I get the id
of the host to remove with the command

openstack compute service list

Executed this command to delete the host from the cloud:

openstack compute service delete 43


and got this error:

Failed to delete compute service with ID '43': Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
<class 'ValueError'> (HTTP 500) (Request-ID: req-82511bfa-660d-41de-ab8a-c1b2dc78d623)
1 of 1 compute services failed to delete.

What can I do?


The last lines of my /var/log/nova/nova-api.log file are:

2022-03-08 15:23:31.136 2569818 INFO nova.osapi_compute.wsgi.server [req-b0f02718-bbbc-4cd5-a936-312aa47cca97 d2fba82966a44a82b2624f75f65f9ad0 3de2426941b943fd95633b0dbaedfd3f - default default] 10.100.133.11 "GET /v2.1/3de2426941b943fd95633b0dbaedfd3f/os-services HTTP/1.1" status: 200 len: 3113 time: 0.1815200
2022-03-08 15:23:40.545 2569811 ERROR nova.api.openstack.wsgi [req-82511bfa-660d-41de-ab8a-c1b2dc78d623 d2fba82966a44a82b2624f75f65f9ad0 3de2426941b943fd95633b0dbaedfd3f - default default] Unexpected exception in API method: ValueError: No such provider e224cb92-c659-4fed-81e9-5ce7ee2832fd
2022-03-08 15:23:40.545 2569811 ERROR nova.api.openstack.wsgi Traceback (most recent call last):
2022-03-08 15:23:40.545 2569811 ERROR nova.api.openstack.wsgi   File "/usr/lib/python3/dist-packages/nova/api/openstack/wsgi.py", line 658, in wrapped
2022-03-08 15:23:40.545 2569811 ERROR nova.api.openstack.wsgi     return f(*args, **kwargs)
2022-03-08 15:23:40.545 2569811 ERROR nova.api.openstack.wsgi   File "/usr/lib/python3/dist-packages/nova/api/openstack/compute/services.py", line 286, in delete
2022-03-08 15:23:40.545 2569811 ERROR nova.api.openstack.wsgi     self.placementclient.delete_resource_provider(
2022-03-08 15:23:40.545 2569811 ERROR nova.api.openstack.wsgi   File "/usr/lib/python3/dist-packages/nova/scheduler/client/report.py", line 2257, in delete_resource_provider
2022-03-08 15:23:40.545 2569811 ERROR nova.api.openstack.wsgi     provider_uuids = self._provider_tree.get_provider_uuids_in_tree(
2022-03-08 15:23:40.545 2569811 ERROR nova.api.openstack.wsgi   File "/usr/lib/python3/dist-packages/nova/compute/provider_tree.py", line 288, in get_provider_uuids_in_tree
2022-03-08 15:23:40.545 2569811 ERROR nova.api.openstack.wsgi     return self._find_with_lock(
2022-03-08 15:23:40.545 2569811 ERROR nova.api.openstack.wsgi   File "/usr/lib/python3/dist-packages/nova/compute/provider_tree.py", line 439, in _find_with_lock
2022-03-08 15:23:40.545 2569811 ERROR nova.api.openstack.wsgi     raise ValueError(_("No such provider %s") % name_or_uuid)
2022-03-08 15:23:40.545 2569811 ERROR nova.api.openstack.wsgi ValueError: No such provider e224cb92-c659-4fed-81e9-5ce7ee2832fd
2022-03-08 15:23:40.545 2569811 ERROR nova.api.openstack.wsgi 
2022-03-08 15:23:40.546 2569811 INFO nova.api.openstack.wsgi [req-82511bfa-660d-41de-ab8a-c1b2dc78d623 d2fba82966a44a82b2624f75f65f9ad0 3de2426941b943fd95633b0dbaedfd3f - default default] HTTP exception thrown: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
<class 'ValueError'>
2022-03-08 15:23:40.548 2569811 INFO nova.osapi_compute.wsgi.server [req-82511bfa-660d-41de-ab8a-c1b2dc78d623 d2fba82966a44a82b2624f75f65f9ad0 3de2426941b943fd95633b0dbaedfd3f - default default] 10.100.133.11 "DELETE /v2.1/3de2426941b943fd95633b0dbaedfd3f/os-services/43 HTTP/1.1" status: 500 len: 617 time: 1.2662396


The output of the command openstack --debug compute service delete 43 is
(I have deleted first lines, which seems ok):

Instantiating compute client for API Version Major: 2, Minor: 1
Instantiating compute api: <class 'openstackclient.api.compute_v2.APIv2'>
REQ: curl -g -i -X DELETE http://10.100.133.11:8774/v2.1/3de2426941b943fd95633b0dbaedfd3f/os-services/43 -H "Accept: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}8487771e5f65fa77e5306e561573afd56b1810bea648198c2f05e70e6bae3a9c" -H "X-OpenStack-Nova-API-Version: 2.1"
Starting new HTTP connection (1): 10.100.133.11:8774
http://10.100.133.11:8774 "DELETE /v2.1/3de2426941b943fd95633b0dbaedfd3f/os-services/43 HTTP/1.1" 500 184
RESP: [500] Connection: keep-alive Content-Length: 184 Content-Type: application/json; charset=UTF-8 Date: Tue, 08 Mar 2022 14:42:22 GMT Openstack-Api-Version: compute 2.1 Vary: OpenStack-API-Version, X-OpenStack-Nova-API-Version X-Compute-Request-Id: req-01d6510b-7299-4713-92cc-8ec0315fc4fb X-Openstack-Nova-Api-Version: 2.1 X-Openstack-Request-Id: req-01d6510b-7299-4713-92cc-8ec0315fc4fb
RESP BODY: {"computeFault": {"code": 500, "message": "Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.\n<class 'ValueError'>"}}
DELETE call to compute for http://10.100.133.11:8774/v2.1/3de2426941b943fd95633b0dbaedfd3f/os-services/43 used request id req-01d6510b-7299-4713-92cc-8ec0315fc4fb
Failed to delete compute service with ID '43': Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
<class 'ValueError'> (HTTP 500) (Request-ID: req-01d6510b-7299-4713-92cc-8ec0315fc4fb)
1 of 1 compute services failed to delete.
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/cliff/app.py", line 401, in run_subcommand
    result = cmd.run(parsed_args)
  File "/usr/lib/python3/dist-packages/osc_lib/command/command.py", line 39, in run
    return super(Command, self).run(parsed_args)
  File "/usr/lib/python3/dist-packages/cliff/command.py", line 185, in run
    return_code = self.take_action(parsed_args) or 0
  File "/usr/lib/python3/dist-packages/openstackclient/compute/v2/service.py", line 62, in take_action
    raise exceptions.CommandError(msg)
osc_lib.exceptions.CommandError: 1 of 1 compute services failed to delete.
clean_up DeleteService: 1 of 1 compute services failed to delete.
END return value: 1

the output of the command "dpkg -l | grep nova" is:

ii  nova-api             3:23.0.0-0ubuntu1~cloud0  OpenStack Compute - API frontend
ii  nova-common          3:23.0.0-0ubuntu1~cloud0  OpenStack Compute - common files
ii  nova-compute         3:23.0.0-0ubuntu1~cloud0  OpenStack Compute - compute node base
ii  nova-compute-kvm     3:23.0.0-0ubuntu1~cloud0  OpenStack Compute - compute node (KVM)
ii  nova-compute-libvirt 3:23.0.0-0ubuntu1~cloud0  OpenStack Compute - compute node libvirt support
ii  nova-conductor       3:23.0.0-0ubuntu1~cloud0  OpenStack Compute - conductor service
ii  nova-novncproxy      3:23.0.0-0ubuntu1~cloud0  OpenStack Compute - NoVNC proxy
ii  nova-scheduler       3:23.0.0-0ubuntu1~cloud0  OpenStack Compute - virtual machine scheduler
ii  python3-nova         3:23.0.0-0ubuntu1~cloud0  OpenStack Compute Python 3 libraries
ii  python3-novaclient   2:17.4.0-0ubuntu1~cloud0  client library for OpenStack Compute API - 3.x

** Affects: nova
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1964137

Title:
  Failed to delete compute service

Status in OpenStack Compute (nova):
  New

Bug description:
  
  I have temporally added a new compute host to my openstack wallaby cloud.  It was not the first time I have done this, but it is the first time it fails. 

  To add the compute host I installed nova-compute and neutron-
  linuxbridge services in it, and then updated the cloud using the
  command "nova-manage cell_v2 discover_hosts" in my controller.

  It worked well, but then I decided to remove the host, so I get the id
  of the host to remove with the command

  openstack compute service list

  Executed this command to delete the host from the cloud:

  openstack compute service delete 43

  
  and got this error:

  Failed to delete compute service with ID '43': Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
  <class 'ValueError'> (HTTP 500) (Request-ID: req-82511bfa-660d-41de-ab8a-c1b2dc78d623)
  1 of 1 compute services failed to delete.

  What can I do?

  
  The last lines of my /var/log/nova/nova-api.log file are:

  2022-03-08 15:23:31.136 2569818 INFO nova.osapi_compute.wsgi.server [req-b0f02718-bbbc-4cd5-a936-312aa47cca97 d2fba82966a44a82b2624f75f65f9ad0 3de2426941b943fd95633b0dbaedfd3f - default default] 10.100.133.11 "GET /v2.1/3de2426941b943fd95633b0dbaedfd3f/os-services HTTP/1.1" status: 200 len: 3113 time: 0.1815200
  2022-03-08 15:23:40.545 2569811 ERROR nova.api.openstack.wsgi [req-82511bfa-660d-41de-ab8a-c1b2dc78d623 d2fba82966a44a82b2624f75f65f9ad0 3de2426941b943fd95633b0dbaedfd3f - default default] Unexpected exception in API method: ValueError: No such provider e224cb92-c659-4fed-81e9-5ce7ee2832fd
  2022-03-08 15:23:40.545 2569811 ERROR nova.api.openstack.wsgi Traceback (most recent call last):
  2022-03-08 15:23:40.545 2569811 ERROR nova.api.openstack.wsgi   File "/usr/lib/python3/dist-packages/nova/api/openstack/wsgi.py", line 658, in wrapped
  2022-03-08 15:23:40.545 2569811 ERROR nova.api.openstack.wsgi     return f(*args, **kwargs)
  2022-03-08 15:23:40.545 2569811 ERROR nova.api.openstack.wsgi   File "/usr/lib/python3/dist-packages/nova/api/openstack/compute/services.py", line 286, in delete
  2022-03-08 15:23:40.545 2569811 ERROR nova.api.openstack.wsgi     self.placementclient.delete_resource_provider(
  2022-03-08 15:23:40.545 2569811 ERROR nova.api.openstack.wsgi   File "/usr/lib/python3/dist-packages/nova/scheduler/client/report.py", line 2257, in delete_resource_provider
  2022-03-08 15:23:40.545 2569811 ERROR nova.api.openstack.wsgi     provider_uuids = self._provider_tree.get_provider_uuids_in_tree(
  2022-03-08 15:23:40.545 2569811 ERROR nova.api.openstack.wsgi   File "/usr/lib/python3/dist-packages/nova/compute/provider_tree.py", line 288, in get_provider_uuids_in_tree
  2022-03-08 15:23:40.545 2569811 ERROR nova.api.openstack.wsgi     return self._find_with_lock(
  2022-03-08 15:23:40.545 2569811 ERROR nova.api.openstack.wsgi   File "/usr/lib/python3/dist-packages/nova/compute/provider_tree.py", line 439, in _find_with_lock
  2022-03-08 15:23:40.545 2569811 ERROR nova.api.openstack.wsgi     raise ValueError(_("No such provider %s") % name_or_uuid)
  2022-03-08 15:23:40.545 2569811 ERROR nova.api.openstack.wsgi ValueError: No such provider e224cb92-c659-4fed-81e9-5ce7ee2832fd
  2022-03-08 15:23:40.545 2569811 ERROR nova.api.openstack.wsgi 
  2022-03-08 15:23:40.546 2569811 INFO nova.api.openstack.wsgi [req-82511bfa-660d-41de-ab8a-c1b2dc78d623 d2fba82966a44a82b2624f75f65f9ad0 3de2426941b943fd95633b0dbaedfd3f - default default] HTTP exception thrown: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
  <class 'ValueError'>
  2022-03-08 15:23:40.548 2569811 INFO nova.osapi_compute.wsgi.server [req-82511bfa-660d-41de-ab8a-c1b2dc78d623 d2fba82966a44a82b2624f75f65f9ad0 3de2426941b943fd95633b0dbaedfd3f - default default] 10.100.133.11 "DELETE /v2.1/3de2426941b943fd95633b0dbaedfd3f/os-services/43 HTTP/1.1" status: 500 len: 617 time: 1.2662396


  The output of the command openstack --debug compute service delete 43
  is (I have deleted first lines, which seems ok):

  Instantiating compute client for API Version Major: 2, Minor: 1
  Instantiating compute api: <class 'openstackclient.api.compute_v2.APIv2'>
  REQ: curl -g -i -X DELETE http://10.100.133.11:8774/v2.1/3de2426941b943fd95633b0dbaedfd3f/os-services/43 -H "Accept: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}8487771e5f65fa77e5306e561573afd56b1810bea648198c2f05e70e6bae3a9c" -H "X-OpenStack-Nova-API-Version: 2.1"
  Starting new HTTP connection (1): 10.100.133.11:8774
  http://10.100.133.11:8774 "DELETE /v2.1/3de2426941b943fd95633b0dbaedfd3f/os-services/43 HTTP/1.1" 500 184
  RESP: [500] Connection: keep-alive Content-Length: 184 Content-Type: application/json; charset=UTF-8 Date: Tue, 08 Mar 2022 14:42:22 GMT Openstack-Api-Version: compute 2.1 Vary: OpenStack-API-Version, X-OpenStack-Nova-API-Version X-Compute-Request-Id: req-01d6510b-7299-4713-92cc-8ec0315fc4fb X-Openstack-Nova-Api-Version: 2.1 X-Openstack-Request-Id: req-01d6510b-7299-4713-92cc-8ec0315fc4fb
  RESP BODY: {"computeFault": {"code": 500, "message": "Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.\n<class 'ValueError'>"}}
  DELETE call to compute for http://10.100.133.11:8774/v2.1/3de2426941b943fd95633b0dbaedfd3f/os-services/43 used request id req-01d6510b-7299-4713-92cc-8ec0315fc4fb
  Failed to delete compute service with ID '43': Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
  <class 'ValueError'> (HTTP 500) (Request-ID: req-01d6510b-7299-4713-92cc-8ec0315fc4fb)
  1 of 1 compute services failed to delete.
  Traceback (most recent call last):
    File "/usr/lib/python3/dist-packages/cliff/app.py", line 401, in run_subcommand
      result = cmd.run(parsed_args)
    File "/usr/lib/python3/dist-packages/osc_lib/command/command.py", line 39, in run
      return super(Command, self).run(parsed_args)
    File "/usr/lib/python3/dist-packages/cliff/command.py", line 185, in run
      return_code = self.take_action(parsed_args) or 0
    File "/usr/lib/python3/dist-packages/openstackclient/compute/v2/service.py", line 62, in take_action
      raise exceptions.CommandError(msg)
  osc_lib.exceptions.CommandError: 1 of 1 compute services failed to delete.
  clean_up DeleteService: 1 of 1 compute services failed to delete.
  END return value: 1

  the output of the command "dpkg -l | grep nova" is:

  ii  nova-api             3:23.0.0-0ubuntu1~cloud0  OpenStack Compute - API frontend
  ii  nova-common          3:23.0.0-0ubuntu1~cloud0  OpenStack Compute - common files
  ii  nova-compute         3:23.0.0-0ubuntu1~cloud0  OpenStack Compute - compute node base
  ii  nova-compute-kvm     3:23.0.0-0ubuntu1~cloud0  OpenStack Compute - compute node (KVM)
  ii  nova-compute-libvirt 3:23.0.0-0ubuntu1~cloud0  OpenStack Compute - compute node libvirt support
  ii  nova-conductor       3:23.0.0-0ubuntu1~cloud0  OpenStack Compute - conductor service
  ii  nova-novncproxy      3:23.0.0-0ubuntu1~cloud0  OpenStack Compute - NoVNC proxy
  ii  nova-scheduler       3:23.0.0-0ubuntu1~cloud0  OpenStack Compute - virtual machine scheduler
  ii  python3-nova         3:23.0.0-0ubuntu1~cloud0  OpenStack Compute Python 3 libraries
  ii  python3-novaclient   2:17.4.0-0ubuntu1~cloud0  client library for OpenStack Compute API - 3.x

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1964137/+subscriptions

Follow ups

  Thread Previous • Date Previous • • Thread Next