yahoo-eng-team team mailing list archive

[Jorge Niedbalski <1768094@xxxxxxxxxxxxxxxxxx>]

** Also affects: nova
   Importance: Undecided
       Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1768094

Title:
  nova_api nova-manage cell_v2 list_hosts not displaying all hypervisors

Status in kolla-ansible:
  In Progress
Status in OpenStack Compute (nova):
  New

Bug description:
  [Environment]

  Kernel 4.14
  Queens release

  [Description]

  While doing a multinode deployment [0] ,not all the hypervisors are
  registered on the default nova cell right after the deployment has
  completed.

  The machines that fails to be enlisted are coincidentally part of
  different services (controllers/compute) groups, however, the machines
  that are exclusive part of the compute group, are enlisted correctly.

  /var/log/kolla/nova/nova-scheduler.log:2018-04-30 14:45:28.516 7 INFO nova.scheduler.host_manager [req-c719e3c6-ceb0-4a39-9997-59799f785a72 - - - - -] Host mapping not found for host xxx-09. Not tracking instance info for this host.
  /var/log/kolla/nova/nova-scheduler.log:2018-04-30 14:45:51.388 7 INFO nova.scheduler.host_manager [req-5ba217bb-f449-4a65-a875-8251191df434 - - - - -] Host mapping not found for host xxx-04. Not tracking instance info for this host.
  /var/log/kolla/nova/nova-scheduler.log:2018-04-30 14:45:56.353 7 INFO nova.scheduler.host_manager [req-6e44a9e0-9b90-41aa-826d-65ead111a1ed - - - - -] Host mapping not found for host xxx-10. Not tracking instance info for this host.
  /var/log/kolla/nova/nova-scheduler.log:2018-04-30 14:46:16.451 7 INFO nova.scheduler.host_manager [req-b9755302-3eca-47a8-90c3-7e2b72537956 - - - - -] Host mapping not found for host xxx-02. Not tracking instance info for this host.
  /var/log/kolla/nova/nova-scheduler.log:2018-04-30 14:46:37.436 7 INFO nova.scheduler.host_manager [req-86348de2-ee5c-4a38-9851-caecdfd5374e - - - - -] Host mapping not found for host xxx-08. Not tracking instance info for this host.
  /var/log/kolla/nova/nova-scheduler.log:2018-04-30 14:46:37.779 7 INFO nova.scheduler.host_manager [req-93183d20-8ac7-4c02-9d45-288939679a09 - - - - -] Host mapping not found for host xxx-03. Not tracking instance info for this host.
  /var/log/kolla/nova/nova-scheduler.log:2018-04-30 14:47:10.784 7 INFO nova.scheduler.host_manager [req-0c008505-c0b5-4f44-8a04-569d2013d63b - - - - -] Host mapping not found for host xxx-01. Not tracking instance info for this host.
  /var/log/kolla/nova/nova-scheduler.log:2018-04-30 14:47:18.598 7 INFO nova.scheduler.host_manager [req-7afb8034-470a-408e-b9c8-b1611df1b5de - - - - -] Host mapping not found for host xxx-07. Not tracking instance info for this host.
  /var/log/kolla/nova/nova-scheduler.log:2018-04-30 14:47:19.196 7 INFO nova.scheduler.host_manager [req-7dfb3e0c-a734-49b3-865e-1856a7939e55 - - - - -] Host mapping not found for host xxx-05. Not tracking instance info for this host.
  /var/log/kolla/nova/nova-scheduler.log:2018-04-30 14:47:21.354 7 INFO nova.scheduler.host_manager [req-6fce72dc-8716-4ac4-b2bc-cd13af812dd8 - - - - -] Host mapping not found for host xxx-06. Not tracking instance info for this host.
  /var/log/kolla/nova/nova-scheduler.log.1:2018-04-28 04:22:17.212 7 INFO nova.scheduler.host_manager [req-9795b7c4-9ce2-4299-b198-5697eef8b3bf - - - - -] Host mapping not found for host xxx-05. Not tracking instance info for this host.
  /var/log/kolla/nova/nova-scheduler.log.1:2018-04-28 04:22:17.356 7 INFO nova.scheduler.host_manager [req-6f01945d-9405-4364-9597-d7444a58bd91 - - - - -] Host mapping not found for host xxx-06. Not tracking instance info for this host.
  /var/log/kolla/nova/nova-scheduler.log.1:2018-04-28 04:22:17.385 7 INFO nova.scheduler.host_manager [req-f5de3e7c-5034-4980-9897-59b56a9257ee - - - - -] Host mapping not found for host xxx-04. Not tracking instance info for this host.
  /var/log/kolla/nova/nova-scheduler.log.1:2018-04-28 04:22:17.600 7 INFO nova.scheduler.host_manager [req-c506d111-d309-422a-b4fb-afe6620dd838 - - - - -] Host mapping not found for host xxx-07. Not tracking instance info for this host.
  /var/log/kolla/nova/nova-scheduler.log.1:2018-04-28 04:22:17.744 7 INFO nova.scheduler.host_manager [req-7d136730-e3f4-49db-bd3d-260516eae96a - - - - -] Host mapping not found for host xxx-03. Not tracking instance info for this host.
  /var/log/kolla/nova/nova-scheduler.log.1:2018-04-28 04:22:32.841 7 INFO nova.scheduler.host_manager [req-d227c40d-9a4b-4fbc-84ed-87e00cb01f4e - - - - -] Host mapping not found for host uk-dc-moonshot-cartridge-02. Not tracking instance info for this host.
  /var/log/kolla/nova/nova-scheduler.log.1:2018-04-28 04:22:37.443 7 INFO nova.scheduler.host_manager [req-11730392-be04-4d32-8aef-9384da570cdf - - - - -] Host mapping not found for host xxx-08. Not tracking instance info for this host.
  /var/log/kolla/nova/nova-scheduler.log.1:2018-04-28 04:22:37.462 7 INFO nova.scheduler.host_manager [req-11730392-be04-4d32-8aef-9384da570cdf - - - - -] Host mapping not found for host xxx-02. Not tracking instance info for this host.
  /var/log/kolla/nova/nova-scheduler.log.1:2018-04-28 04:22:37.517 7 INFO nova.scheduler.host_manager [req-318845ef-bc1f-4479-b43e-dd7eb6068c19 - - - - -] Host mapping not found for host xxx-09. Not tracking instance info for this host.
  /var/log/kolla/nova/nova-scheduler.log.1:2018-04-28 04:22:37.787 7 INFO nova.scheduler.host_manager [req-494c4cf8-4769-49a9-9e89-37c4ee33da93 - - - - -] Host mapping not found for host xxx-01. Not tracking instance info for this host.
  /var/log/kolla/nova/nova-scheduler.log.1:2018-04-28 04:22:52.770 7 INFO nova.scheduler.host_manager [req-90c650a5-85d8-44a1-868c-12e6980504d4 - - - - -] Host mapping not found for host xxx-01. Not tracking instance info for this host.
  /var/log/kolla/nova/nova-scheduler.log.1:2018-04-28 04:22:52.791 7 INFO nova.scheduler.host_manager [req-90c650a5-85d8-44a1-868c-12e6980504d4 - - - - -] Host mapping not found for host xxx-04. Not tracking instance info for this host.

  For the not enlisted machines, these are the entries at the nova-api
  logs, with a reference to
  http://git.openstack.org/cgit/openstack/nova/tree/nova/scheduler/host_manager.py#n745

  "It's possible to hit this when the compute service first starts
   up and casts to update_instance_info with an empty list but
   before the host is mapped in the API database."

  The problem is that even considering that theoretically this should self-heal once new instances
  arrive into this compute host, the cells doesn't return the correct information after deployment, so any automation validation/check  will fail.

  As a side point, the machines are listed when nova service-list is invoked, and
  the nova-compute service is indeed up.

  [Possible solution]

  Implement the by-service parameter
  https://github.com/openstack/nova/blob/37b4f77b5ed2c3606ebeae42ae2b1aedb0eda775/nova/objects/host_mapping.py

To manage notifications about this bug go to:
https://bugs.launchpad.net/kolla-ansible/+bug/1768094/+subscriptions