yahoo-eng-team team mailing list archive

[Thierry Carrez <thierry.carrez+lp@xxxxxxxxx>]

** Changed in: nova
       Status: Fix Committed => Fix Released

** Changed in: nova
    Milestone: None => kilo-2

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1361186

Title:
  nova service-delete fails for services on non-child (top) cell

Status in OpenStack Compute (Nova):
  Fix Released

Bug description:
  Nova service-delete fails for services on non-child (top) cell.

  How to reproduce:

  $ nova --os-username admin service-list

  +----------------+------------------+---------------------+----------+---------+-------+----------------------------+-----------------+
  | Id             | Binary           | Host                | Zone     | Status  | State | Updated_at                 | Disabled Reason |
  +----------------+------------------+---------------------+----------+---------+-------+----------------------------+-----------------+
  | region!child@1 | nova-conductor   | region!child@ubuntu | internal | enabled | up    | 2014-08-18T06:06:56.000000 | -               |
  | region!child@2 | nova-compute     | region!child@ubuntu | nova     | enabled | up    | 2014-08-18T06:06:55.000000 | -               |
  | region!child@3 | nova-cells       | region!child@ubuntu | internal | enabled | up    | 2014-08-18T06:06:59.000000 | -               |
  | region!child@4 | nova-scheduler   | region!child@ubuntu | internal | enabled | up    | 2014-08-18T06:06:50.000000 | -               |
  | region@1       | nova-cells       | region@ubuntu       | internal | enabled | up    | 2014-08-18T06:06:59.000000 | -               |
  | region@2       | nova-cert        | region@ubuntu       | internal | enabled | up    | 2014-08-18T06:06:58.000000 | -               |
  | region@3       | nova-consoleauth | region@ubuntu       | internal | enabled | up    | 2014-08-18T06:06:57.000000 | -               |
  +----------------+------------------+---------------------+----------+---------+-------+----------------------------+-----------------+

  Stop one of the services on top cell (e.g. nova-cert).

  $ nova --os-username admin service-list

  +----------------+------------------+---------------------+----------+---------+-------+----------------------------+-----------------+
  | Id             | Binary           | Host                | Zone     | Status  | State | Updated_at                 | Disabled Reason |
  +----------------+------------------+---------------------+----------+---------+-------+----------------------------+-----------------+
  | region!child@1 | nova-conductor   | region!child@ubuntu | internal | enabled | up    | 2014-08-18T06:09:26.000000 | -               |
  | region!child@2 | nova-compute     | region!child@ubuntu | nova     | enabled | up    | 2014-08-18T06:09:25.000000 | -               |
  | region!child@3 | nova-cells       | region!child@ubuntu | internal | enabled | up    | 2014-08-18T06:09:19.000000 | -               |
  | region!child@4 | nova-scheduler   | region!child@ubuntu | internal | enabled | up    | 2014-08-18T06:09:20.000000 | -               |
  | region@1       | nova-cells       | region@ubuntu       | internal | enabled | up    | 2014-08-18T06:09:19.000000 | -               |
  | region@2       | nova-cert        | region@ubuntu       | internal | enabled | down  | 2014-08-18T06:08:28.000000 | -               |
  | region@3       | nova-consoleauth | region@ubuntu       | internal | enabled | up    | 2014-08-18T06:09:27.000000 | -               |
  +----------------+------------------+---------------------+----------+---------+-------+----------------------------+-----------------+

  Nova service-delete:
  $ nova --os-username admin service-delete 'region@2'

  Check the request id from nova-api.log:

  2014-08-18 15:10:23.491 INFO nova.osapi_compute.wsgi.server [req-
  e134d915-ad66-41ba-a6f8-33ec51b7daee admin demo] 192.168.101.31
  "DELETE /v2/d66804d2e78549cd8f5efcedd0abecb2/os-services/region@2
  HTTP/1.1" status: 204 len: 179 time: 0.1334069

  Error log in n-cell-region service:

  2014-08-18 15:10:23.464 ERROR nova.cells.messaging [req-e134d915-ad66-41ba-a6f8-33ec51b7daee admin demo] Error locating next hop for message: 'NoneType' object has no attribute 'count'
  2014-08-18 15:10:23.464 TRACE nova.cells.messaging Traceback (most recent call last):
  2014-08-18 15:10:23.464 TRACE nova.cells.messaging   File "/opt/stack/nova/nova/cells/messaging.py", line 406, in process
  2014-08-18 15:10:23.464 TRACE nova.cells.messaging     next_hop = self._get_next_hop()
  2014-08-18 15:10:23.464 TRACE nova.cells.messaging   File "/opt/stack/nova/nova/cells/messaging.py", line 361, in _get_next_hop
  2014-08-18 15:10:23.464 TRACE nova.cells.messaging     dest_hops = target_cell.count(_PATH_CELL_SEP)
  2014-08-18 15:10:23.464 TRACE nova.cells.messaging AttributeError: 'NoneType' object has no attribute 'count'

  
  Appendix:
  In case of services on child cell, no issues.

  $ nova --os-username admin service-list

  +----------------+------------------+---------------------+----------+---------+-------+----------------------------+-----------------+
  | Id             | Binary           | Host                | Zone     | Status  | State | Updated_at                 | Disabled Reason |
  +----------------+------------------+---------------------+----------+---------+-------+----------------------------+-----------------+
  | region!child@1 | nova-conductor   | region!child@ubuntu | internal | enabled | up    | 2014-08-18T06:14:46.000000 | -               |
  | region!child@2 | nova-compute     | region!child@ubuntu | nova     | enabled | down  | 2014-08-18T06:13:15.000000 | -               |
  | region!child@3 | nova-cells       | region!child@ubuntu | internal | enabled | up    | 2014-08-18T06:14:49.000000 | -               |
  | region!child@4 | nova-scheduler   | region!child@ubuntu | internal | enabled | up    | 2014-08-18T06:14:50.000000 | -               |
  | region@1       | nova-cells       | region@ubuntu       | internal | enabled | up    | 2014-08-18T06:14:49.000000 | -               |
  | region@2       | nova-cert        | region@ubuntu       | internal | enabled | down  | 2014-08-18T06:08:28.000000 | -               |
  | region@3       | nova-consoleauth | region@ubuntu       | internal | enabled | up    | 2014-08-18T06:14:47.000000 | -               |
  +----------------+------------------+---------------------+----------+---------+-------+----------------------------+-----------------+

  Delete child cell service:
  $ nova --os-username admin service-delete 'region!child@2'

  $ nova --os-username admin service-list

  +----------------+------------------+---------------------+----------+---------+-------+----------------------------+-----------------+
  | Id             | Binary           | Host                | Zone     | Status  | State | Updated_at                 | Disabled Reason |
  +----------------+------------------+---------------------+----------+---------+-------+----------------------------+-----------------+
  | region!child@1 | nova-conductor   | region!child@ubuntu | internal | enabled | up    | 2014-08-18T06:15:46.000000 | -               |
  | region!child@3 | nova-cells       | region!child@ubuntu | internal | enabled | up    | 2014-08-18T06:15:39.000000 | -               |
  | region!child@4 | nova-scheduler   | region!child@ubuntu | internal | enabled | up    | 2014-08-18T06:15:40.000000 | -               |
  | region@1       | nova-cells       | region@ubuntu       | internal | enabled | up    | 2014-08-18T06:15:39.000000 | -               |
  | region@2       | nova-cert        | region@ubuntu       | internal | enabled | down  | 2014-08-18T06:08:28.000000 | -               |
  | region@3       | nova-consoleauth | region@ubuntu       | internal | enabled | up    | 2014-08-18T06:15:47.000000 | -               |
  +----------------+------------------+---------------------+----------+---------+-------+----------------------------+-----------------+

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1361186/+subscriptions