yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1782349] [NEW] Failed to schedule instances: NoValidHost_Remote: No valid host was found.

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: men <1782349@xxxxxxxxxxxxxxxxxx>
Date: Wed, 18 Jul 2018 12:37:57 -0000
Reply-to: Bug 1782349 <1782349@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
Public bug reported:

openstack Q：

(openstack) compute service list
+----+------------------+------------+----------+---------+-------+----------------------------+
| ID | Binary           | Host       | Zone     | Status  | State | Updated At                 |
+----+------------------+------------+----------+---------+-------+----------------------------+
|  1 | nova-consoleauth | controller | internal | enabled | up    | 2018-07-18T12:23:51.000000 |
|  2 | nova-conductor   | controller | internal | enabled | up    | 2018-07-18T12:23:47.000000 |
|  3 | nova-scheduler   | controller | internal | enabled | up    | 2018-07-18T12:23:49.000000 |
|  6 | nova-compute     | controller | nova     | enabled | up    | 2018-07-18T12:23:53.000000 |
|  7 | nova-compute     | compute01  | nova     | enabled | up    | 2018-07-18T12:23:47.000000 |
+----+------------------+------------+----------+---------+-------+----------------------------+


(openstack) flavor list
+--------------------------------------+------+-----+------+-----------+-------+-----------+
| ID                                   | Name | RAM | Disk | Ephemeral | VCPUs | Is Public |
+--------------------------------------+------+-----+------+-----------+-------+-----------+
| 2340e418-8214-440f-a5b6-39a1e3648cf1 | f1   | 512 |   10 |         0 |     1 | True      |
| f1c886da-b5ab-4519-b0ca-b0703e18d629 | f0   | 512 |    1 |         0 |     1 | True      |
+--------------------------------------+------+-----+------+-----------+-------+-----------+


(openstack) hypervisor stats show
+----------------------+-------+
| Field                | Value |
+----------------------+-------+
| count                | 2     |
| current_workload     | 0     |
| disk_available_least | 81    |
| free_disk_gb         | 87    |
| free_ram_mb          | 30718 |
| local_gb             | 98    |
| local_gb_used        | 11    |
| memory_mb            | 32766 |
| memory_mb_used       | 2048  |
| running_vms          | 2     |
| vcpus                | 20    |
| vcpus_used           | 2     |
+----------------------+-------+


[root@controller ~]# tail -f /var/log/nova/nova-conductor.log
2018-07-18 20:21:22.904 13466 ERROR nova.conductor.manager [req-f360123a-42b8-4941-8fd6-496baa0742bb 93978126a56848db8a0cb177311c3076 11bd52001a0a45fd87f1d6e44949abaa - default default] Failed to schedule instances: NoValidHost_Remote: No valid host was found. 
Traceback (most recent call last):

  File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 226, in inner
    return func(*args, **kwargs)

  File "/usr/lib/python2.7/site-packages/nova/scheduler/manager.py", line 139, in select_destinations
    raise exception.NoValidHost(reason="")

NoValidHost: No valid host was found.
2018-07-18 20:21:22.904 13466 ERROR nova.conductor.manager Traceback (most recent call last):
2018-07-18 20:21:22.904 13466 ERROR nova.conductor.manager   File "/usr/lib/python2.7/site-packages/nova/conductor/manager.py", line 1116, in schedule_and_build_instances
2018-07-18 20:21:22.904 13466 ERROR nova.conductor.manager     instance_uuids, return_alternates=True)
2018-07-18 20:21:22.904 13466 ERROR nova.conductor.manager   File "/usr/lib/python2.7/site-packages/nova/conductor/manager.py", line 716, in _schedule_instances
2018-07-18 20:21:22.904 13466 ERROR nova.conductor.manager     return_alternates=return_alternates)
2018-07-18 20:21:22.904 13466 ERROR nova.conductor.manager   File "/usr/lib/python2.7/site-packages/nova/scheduler/utils.py", line 726, in wrapped
2018-07-18 20:21:22.904 13466 ERROR nova.conductor.manager     return func(*args, **kwargs)
2018-07-18 20:21:22.904 13466 ERROR nova.conductor.manager   File "/usr/lib/python2.7/site-packages/nova/scheduler/client/__init__.py", line 53, in select_destinations
2018-07-18 20:21:22.904 13466 ERROR nova.conductor.manager     instance_uuids, return_objects, return_alternates)
2018-07-18 20:21:22.904 13466 ERROR nova.conductor.manager   File "/usr/lib/python2.7/site-packages/nova/scheduler/client/__init__.py", line 37, in __run_method
2018-07-18 20:21:22.904 13466 ERROR nova.conductor.manager     return getattr(self.instance, __name)(*args, **kwargs)
2018-07-18 20:21:22.904 13466 ERROR nova.conductor.manager   File "/usr/lib/python2.7/site-packages/nova/scheduler/client/query.py", line 42, in select_destinations
2018-07-18 20:21:22.904 13466 ERROR nova.conductor.manager     instance_uuids, return_objects, return_alternates)
2018-07-18 20:21:22.904 13466 ERROR nova.conductor.manager   File "/usr/lib/python2.7/site-packages/nova/scheduler/rpcapi.py", line 158, in select_destinations
2018-07-18 20:21:22.904 13466 ERROR nova.conductor.manager     return cctxt.call(ctxt, 'select_destinations', **msg_args)
2018-07-18 20:21:22.904 13466 ERROR nova.conductor.manager   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/client.py", line 174, in call
2018-07-18 20:21:22.904 13466 ERROR nova.conductor.manager     retry=self.retry)
2018-07-18 20:21:22.904 13466 ERROR nova.conductor.manager   File "/usr/lib/python2.7/site-packages/oslo_messaging/transport.py", line 131, in _send
2018-07-18 20:21:22.904 13466 ERROR nova.conductor.manager     timeout=timeout, retry=retry)
2018-07-18 20:21:22.904 13466 ERROR nova.conductor.manager   File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 559, in send
2018-07-18 20:21:22.904 13466 ERROR nova.conductor.manager     retry=retry)
2018-07-18 20:21:22.904 13466 ERROR nova.conductor.manager   File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 550, in _send
2018-07-18 20:21:22.904 13466 ERROR nova.conductor.manager     raise result
2018-07-18 20:21:22.904 13466 ERROR nova.conductor.manager NoValidHost_Remote: No valid host was found. 
2018-07-18 20:21:22.904 13466 ERROR nova.conductor.manager Traceback (most recent call last):
2018-07-18 20:21:22.904 13466 ERROR nova.conductor.manager 
2018-07-18 20:21:22.904 13466 ERROR nova.conductor.manager   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 226, in inner
2018-07-18 20:21:22.904 13466 ERROR nova.conductor.manager     return func(*args, **kwargs)
2018-07-18 20:21:22.904 13466 ERROR nova.conductor.manager 
2018-07-18 20:21:22.904 13466 ERROR nova.conductor.manager   File "/usr/lib/python2.7/site-packages/nova/scheduler/manager.py", line 139, in select_destinations
2018-07-18 20:21:22.904 13466 ERROR nova.conductor.manager     raise exception.NoValidHost(reason="")
2018-07-18 20:21:22.904 13466 ERROR nova.conductor.manager 
2018-07-18 20:21:22.904 13466 ERROR nova.conductor.manager NoValidHost: No valid host was found. 
2018-07-18 20:21:22.904 13466 ERROR nova.conductor.manager 
2018-07-18 20:21:22.904 13466 ERROR nova.conductor.manager 
2018-07-18 20:21:23.009 13466 WARNING nova.scheduler.utils [req-f360123a-42b8-4941-8fd6-496baa0742bb 93978126a56848db8a0cb177311c3076 11bd52001a0a45fd87f1d6e44949abaa - default default] Failed to compute_task_build_instances: No valid host was found. 
Traceback (most recent call last):

  File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 226, in inner
    return func(*args, **kwargs)

  File "/usr/lib/python2.7/site-packages/nova/scheduler/manager.py", line 139, in select_destinations
    raise exception.NoValidHost(reason="")

NoValidHost: No valid host was found. 
: NoValidHost_Remote: No valid host was found. 
2018-07-18 20:21:23.013 13466 WARNING nova.scheduler.utils [req-f360123a-42b8-4941-8fd6-496baa0742bb 93978126a56848db8a0cb177311c3076 11bd52001a0a45fd87f1d6e44949abaa - default default] [instance: 860337fd-7571-4f04-a7f2-a672be57c57e] Setting instance to ERROR state.: NoValidHost_Remote: No valid host was found. 


Using f1 flavor  can't create the above error, the resource is very
good, why not let it be created?

** Affects: nova
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1782349

Title:
   Failed to schedule instances: NoValidHost_Remote: No valid host was
  found.

Status in OpenStack Compute (nova):
  New

Bug description:
  openstack Q：

  (openstack) compute service list
  +----+------------------+------------+----------+---------+-------+----------------------------+
  | ID | Binary           | Host       | Zone     | Status  | State | Updated At                 |
  +----+------------------+------------+----------+---------+-------+----------------------------+
  |  1 | nova-consoleauth | controller | internal | enabled | up    | 2018-07-18T12:23:51.000000 |
  |  2 | nova-conductor   | controller | internal | enabled | up    | 2018-07-18T12:23:47.000000 |
  |  3 | nova-scheduler   | controller | internal | enabled | up    | 2018-07-18T12:23:49.000000 |
  |  6 | nova-compute     | controller | nova     | enabled | up    | 2018-07-18T12:23:53.000000 |
  |  7 | nova-compute     | compute01  | nova     | enabled | up    | 2018-07-18T12:23:47.000000 |
  +----+------------------+------------+----------+---------+-------+----------------------------+


  (openstack) flavor list
  +--------------------------------------+------+-----+------+-----------+-------+-----------+
  | ID                                   | Name | RAM | Disk | Ephemeral | VCPUs | Is Public |
  +--------------------------------------+------+-----+------+-----------+-------+-----------+
  | 2340e418-8214-440f-a5b6-39a1e3648cf1 | f1   | 512 |   10 |         0 |     1 | True      |
  | f1c886da-b5ab-4519-b0ca-b0703e18d629 | f0   | 512 |    1 |         0 |     1 | True      |
  +--------------------------------------+------+-----+------+-----------+-------+-----------+

  
  (openstack) hypervisor stats show
  +----------------------+-------+
  | Field                | Value |
  +----------------------+-------+
  | count                | 2     |
  | current_workload     | 0     |
  | disk_available_least | 81    |
  | free_disk_gb         | 87    |
  | free_ram_mb          | 30718 |
  | local_gb             | 98    |
  | local_gb_used        | 11    |
  | memory_mb            | 32766 |
  | memory_mb_used       | 2048  |
  | running_vms          | 2     |
  | vcpus                | 20    |
  | vcpus_used           | 2     |
  +----------------------+-------+

  
  [root@controller ~]# tail -f /var/log/nova/nova-conductor.log
  2018-07-18 20:21:22.904 13466 ERROR nova.conductor.manager [req-f360123a-42b8-4941-8fd6-496baa0742bb 93978126a56848db8a0cb177311c3076 11bd52001a0a45fd87f1d6e44949abaa - default default] Failed to schedule instances: NoValidHost_Remote: No valid host was found. 
  Traceback (most recent call last):

    File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 226, in inner
      return func(*args, **kwargs)

    File "/usr/lib/python2.7/site-packages/nova/scheduler/manager.py", line 139, in select_destinations
      raise exception.NoValidHost(reason="")

  NoValidHost: No valid host was found.
  2018-07-18 20:21:22.904 13466 ERROR nova.conductor.manager Traceback (most recent call last):
  2018-07-18 20:21:22.904 13466 ERROR nova.conductor.manager   File "/usr/lib/python2.7/site-packages/nova/conductor/manager.py", line 1116, in schedule_and_build_instances
  2018-07-18 20:21:22.904 13466 ERROR nova.conductor.manager     instance_uuids, return_alternates=True)
  2018-07-18 20:21:22.904 13466 ERROR nova.conductor.manager   File "/usr/lib/python2.7/site-packages/nova/conductor/manager.py", line 716, in _schedule_instances
  2018-07-18 20:21:22.904 13466 ERROR nova.conductor.manager     return_alternates=return_alternates)
  2018-07-18 20:21:22.904 13466 ERROR nova.conductor.manager   File "/usr/lib/python2.7/site-packages/nova/scheduler/utils.py", line 726, in wrapped
  2018-07-18 20:21:22.904 13466 ERROR nova.conductor.manager     return func(*args, **kwargs)
  2018-07-18 20:21:22.904 13466 ERROR nova.conductor.manager   File "/usr/lib/python2.7/site-packages/nova/scheduler/client/__init__.py", line 53, in select_destinations
  2018-07-18 20:21:22.904 13466 ERROR nova.conductor.manager     instance_uuids, return_objects, return_alternates)
  2018-07-18 20:21:22.904 13466 ERROR nova.conductor.manager   File "/usr/lib/python2.7/site-packages/nova/scheduler/client/__init__.py", line 37, in __run_method
  2018-07-18 20:21:22.904 13466 ERROR nova.conductor.manager     return getattr(self.instance, __name)(*args, **kwargs)
  2018-07-18 20:21:22.904 13466 ERROR nova.conductor.manager   File "/usr/lib/python2.7/site-packages/nova/scheduler/client/query.py", line 42, in select_destinations
  2018-07-18 20:21:22.904 13466 ERROR nova.conductor.manager     instance_uuids, return_objects, return_alternates)
  2018-07-18 20:21:22.904 13466 ERROR nova.conductor.manager   File "/usr/lib/python2.7/site-packages/nova/scheduler/rpcapi.py", line 158, in select_destinations
  2018-07-18 20:21:22.904 13466 ERROR nova.conductor.manager     return cctxt.call(ctxt, 'select_destinations', **msg_args)
  2018-07-18 20:21:22.904 13466 ERROR nova.conductor.manager   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/client.py", line 174, in call
  2018-07-18 20:21:22.904 13466 ERROR nova.conductor.manager     retry=self.retry)
  2018-07-18 20:21:22.904 13466 ERROR nova.conductor.manager   File "/usr/lib/python2.7/site-packages/oslo_messaging/transport.py", line 131, in _send
  2018-07-18 20:21:22.904 13466 ERROR nova.conductor.manager     timeout=timeout, retry=retry)
  2018-07-18 20:21:22.904 13466 ERROR nova.conductor.manager   File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 559, in send
  2018-07-18 20:21:22.904 13466 ERROR nova.conductor.manager     retry=retry)
  2018-07-18 20:21:22.904 13466 ERROR nova.conductor.manager   File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 550, in _send
  2018-07-18 20:21:22.904 13466 ERROR nova.conductor.manager     raise result
  2018-07-18 20:21:22.904 13466 ERROR nova.conductor.manager NoValidHost_Remote: No valid host was found. 
  2018-07-18 20:21:22.904 13466 ERROR nova.conductor.manager Traceback (most recent call last):
  2018-07-18 20:21:22.904 13466 ERROR nova.conductor.manager 
  2018-07-18 20:21:22.904 13466 ERROR nova.conductor.manager   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 226, in inner
  2018-07-18 20:21:22.904 13466 ERROR nova.conductor.manager     return func(*args, **kwargs)
  2018-07-18 20:21:22.904 13466 ERROR nova.conductor.manager 
  2018-07-18 20:21:22.904 13466 ERROR nova.conductor.manager   File "/usr/lib/python2.7/site-packages/nova/scheduler/manager.py", line 139, in select_destinations
  2018-07-18 20:21:22.904 13466 ERROR nova.conductor.manager     raise exception.NoValidHost(reason="")
  2018-07-18 20:21:22.904 13466 ERROR nova.conductor.manager 
  2018-07-18 20:21:22.904 13466 ERROR nova.conductor.manager NoValidHost: No valid host was found. 
  2018-07-18 20:21:22.904 13466 ERROR nova.conductor.manager 
  2018-07-18 20:21:22.904 13466 ERROR nova.conductor.manager 
  2018-07-18 20:21:23.009 13466 WARNING nova.scheduler.utils [req-f360123a-42b8-4941-8fd6-496baa0742bb 93978126a56848db8a0cb177311c3076 11bd52001a0a45fd87f1d6e44949abaa - default default] Failed to compute_task_build_instances: No valid host was found. 
  Traceback (most recent call last):

    File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 226, in inner
      return func(*args, **kwargs)

    File "/usr/lib/python2.7/site-packages/nova/scheduler/manager.py", line 139, in select_destinations
      raise exception.NoValidHost(reason="")

  NoValidHost: No valid host was found. 
  : NoValidHost_Remote: No valid host was found. 
  2018-07-18 20:21:23.013 13466 WARNING nova.scheduler.utils [req-f360123a-42b8-4941-8fd6-496baa0742bb 93978126a56848db8a0cb177311c3076 11bd52001a0a45fd87f1d6e44949abaa - default default] [instance: 860337fd-7571-4f04-a7f2-a672be57c57e] Setting instance to ERROR state.: NoValidHost_Remote: No valid host was found. 


  Using f1 flavor  can't create the above error, the resource is very
  good, why not let it be created?

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1782349/+subscriptions
Follow ups

[Bug 1782349] Re: Failed to schedule instances: NoValidHost_Remote: No valid host was found.
From: Launchpad Bug Tracker, 2018-11-03