yahoo-eng-team team mailing list archive
-
yahoo-eng-team team
-
Mailing list archive
-
Message #64288
[Bug 1693481] Re: VM launch is failing with message "No valid host was found. There are not enough hosts available.", "code": 500
I used the instance uuid to find the request id in the nova-conductor
logs and then looked for that request id in the nova-scheduler logs and
found this:
2017-05-25 21:12:36.102 2562 WARNING nova.scheduler.client.report [req-8359c011-587a-4c60-95a1-ded9e50b26af 15d62a6cd8db41f48d9a85d9aaff958f f85c91e401304cb8a0b78654352d13f6 - - -] Placement service credentials do not work. Placement is optional in Newton, but required in Ocata. Please enable the placement service before upgrading.
2017-05-25 21:12:36.104 2562 INFO nova.scheduler.host_manager [req-8359c011-587a-4c60-95a1-ded9e50b26af 15d62a6cd8db41f48d9a85d9aaff958f f85c91e401304cb8a0b78654352d13f6 - - -] No hosts matched due to not matching 'force_hosts' value of 'controlNode'
2017-05-25 21:12:36.104 2562 DEBUG nova.scheduler.filter_scheduler [req-8359c011-587a-4c60-95a1-ded9e50b26af 15d62a6cd8db41f48d9a85d9aaff958f f85c91e401304cb8a0b78654352d13f6 - - -] There are 0 hosts available but 1 instances requested to build. select_destinations /usr/lib/python2.7/dist-packages/nova/scheduler/filter_scheduler.py:75
You don't have placement running or configured correctly for nova-scheduler to ask placement for hosts. So this is an invalid bug as you have an incomplete setup.
See:
https://docs.openstack.org/ocata/install-guide-ubuntu/nova-controller-
install.html
https://docs.openstack.org/developer/nova/placement.html
** Changed in: nova
Status: New => Invalid
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1693481
Title:
VM launch is failing with message "No valid host was found. There are
not enough hosts available.", "code": 500
Status in OpenStack Compute (nova):
Invalid
Bug description:
Description
===========
I have installed multinode node setup using (sudo add-apt-repository ppa:openstack-ubuntu-testing/pike) where controller & compute on one node and neutron on other node
When i tried to launch VM using cirros image with flavour m1.tiny
gettig
message": "No valid host was found. There are not enough hosts available.", "code": 500, "details": " File \"/usr/lib/python2.7/dist-packages/nova/conductor/manager.py\", line 915, in schedule_and_build_instances |
| | request_specs[0].to_legacy_filter_properties_dict())
Steps to reproduce
==================
1. Install the multinode setup on virutalbox using the procedure mentioned in
https://docs.openstack.org/newton/install-guide-ubuntu/keystone-
verify.html
VM1: ControlNode & Computenode [ RAM- 8GB, Hardisck - 100GB]
VM2: NeutronS1 - neutron server [RAM - 8GB, Hardisck - 10GB]
Note: Used below repository
sudo add-apt-repository ppa:openstack-ubuntu-testing/pike
2.create image cirros and flavor m1.tiny
#openstack image create "cirros" \
--file cirros-0.3.4-x86_64-disk.img \
--disk-format qcow2 --container-format bare \
--public
# openstack flavor create --public m1.tiny --id auto --ram 512 --disk
1 --vcpus 1 --rxtx-factor 1
3.Once all the nodes are up, create network, subnet and launch VM
openstack network create net3
openstack subnet create subnet3 --network net3 --subnet-range
30.30.30.0/24
nova boot --nic net-id=5d51d10b-9776-4617-8963-018639ed78aa --image
cirros --flavor m1.tiny VM5 --availability-zone nova:controlNode
Expected result
===============
Creation of VMs should be successful and able to communicate between two VMs
Actual result
=============
VM launch is getting failed always with message "No valid host was found. There are not enough hosts available."
root@controlNode:~# nova show 8ff61689-11cd-46d6-a88a-fb26a15d7b83
/usr/lib/python2.7/dist-packages/novaclient/client.py:278: UserWarning: The 'tenant_id' argument is deprecated in Ocata and its use may result in errors in future releases. As 'project_id' is provided, the 'tenant_id' argument will be ignored.
warnings.warn(msg)
+--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Property | Value |
+--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| OS-DCF:diskConfig | MANUAL |
| OS-EXT-AZ:availability_zone | nova |
| OS-EXT-SRV-ATTR:host | - |
| OS-EXT-SRV-ATTR:hostname | vm5 |
| OS-EXT-SRV-ATTR:hypervisor_hostname | - |
| OS-EXT-SRV-ATTR:instance_name | instance-0000000c |
| OS-EXT-SRV-ATTR:kernel_id | |
| OS-EXT-SRV-ATTR:launch_index | 0 |
| OS-EXT-SRV-ATTR:ramdisk_id | |
| OS-EXT-SRV-ATTR:reservation_id | r-iijln3t0 |
| OS-EXT-SRV-ATTR:root_device_name | - |
| OS-EXT-SRV-ATTR:user_data | - |
| OS-EXT-STS:power_state | 0 |
| OS-EXT-STS:task_state | - |
| OS-EXT-STS:vm_state | error |
| OS-SRV-USG:launched_at | - |
| OS-SRV-USG:terminated_at | - |
| accessIPv4 | |
| accessIPv6 | |
| config_drive | |
| created | 2017-05-25T15:42:35Z |
| description | - |
| fault | {"message": "No valid host was found. There are not enough hosts available.", "code": 500, "details": " File \"/usr/lib/python2.7/dist-packages/nova/conductor/manager.py\", line 915, in schedule_and_build_instances |
| | request_specs[0].to_legacy_filter_properties_dict()) |
| | File \"/usr/lib/python2.7/dist-packages/nova/conductor/manager.py\", line 637, in _schedule_instances |
| | hosts = self.scheduler_client.select_destinations(context, spec_obj) |
| | File \"/usr/lib/python2.7/dist-packages/nova/scheduler/utils.py\", line 371, in wrapped |
| | return func(*args, **kwargs) |
| | File \"/usr/lib/python2.7/dist-packages/nova/scheduler/client/__init__.py\", line 51, in select_destinations |
| | return self.queryclient.select_destinations(context, spec_obj) |
| | File \"/usr/lib/python2.7/dist-packages/nova/scheduler/client/__init__.py\", line 37, in __run_method |
| | return getattr(self.instance, __name)(*args, **kwargs) |
| | File \"/usr/lib/python2.7/dist-packages/nova/scheduler/client/query.py\", line 32, in select_destinations |
| | return self.scheduler_rpcapi.select_destinations(context, spec_obj) |
| | File \"/usr/lib/python2.7/dist-packages/nova/scheduler/rpcapi.py\", line 130, in select_destinations |
| | return cctxt.call(ctxt, 'select_destinations', **msg_args) |
| | File \"/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/client.py\", line 169, in call |
| | retry=self.retry) |
| | File \"/usr/lib/python2.7/dist-packages/oslo_messaging/transport.py\", line 98, in _send |
| | timeout=timeout, retry=retry) |
| | File \"/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py\", line 505, in send |
| | retry=retry) |
| | File \"/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py\", line 496, in _send |
| | raise result |
| | ", "created": "2017-05-25T15:42:36Z"} |
| flavor | m1.tiny (e5bf6e30-0d88-49d0-be26-2fc0418bb7c9) |
| hostId | |
| host_status | |
| id | 8ff61689-11cd-46d6-a88a-fb26a15d7b83 |
| image | cirros (302e6c08-e0b0-43df-a6e8-4abad517d01f) |
| key_name | - |
| locked | False |
| metadata | {} |
| name | VM5 |
| os-extended-volumes:volumes_attached | [] |
| status | ERROR |
| tags | [] |
| tenant_id | f85c91e401304cb8a0b78654352d13f6 |
| updated | 2017-05-25T15:42:36Z |
| user_id | 15d62a6cd8db41f48d9a85d9aaff958f |
+--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
Environment
===========
1. Exact version of OpenStack you are running. See the following
list for all releases: http://docs.openstack.org/releases/
root@controlNode:~# dpkg -l |grep nova
ii nova-api 2:16.0.0.0a1.201704201428.xenial-0ubuntu1 all OpenStack Compute - API frontend
ii nova-common 2:16.0.0.0a1.201704201428.xenial-0ubuntu1 all OpenStack Compute - common files
ii nova-compute 2:16.0.0.0a1.201704201428.xenial-0ubuntu1 all OpenStack Compute - compute node base
ii nova-compute-kvm 2:16.0.0.0a1.201704201428.xenial-0ubuntu1 all OpenStack Compute - compute node (KVM)
ii nova-compute-libvirt 2:16.0.0.0a1.201704201428.xenial-0ubuntu1 all OpenStack Compute - compute node libvirt support
ii nova-conductor 2:16.0.0.0a1.201704201428.xenial-0ubuntu1 all OpenStack Compute - conductor service
ii nova-consoleauth 2:16.0.0.0a1.201704201428.xenial-0ubuntu1 all OpenStack Compute - Console Authenticator
ii nova-novncproxy 2:16.0.0.0a1.201704201428.xenial-0ubuntu1 all OpenStack Compute - NoVNC proxy
ii nova-placement-api 2:16.0.0.0a1.201704201428.xenial-0ubuntu1 all OpenStack Compute - placement API frontend
ii nova-scheduler 2:16.0.0.0a1.201704201428.xenial-0ubuntu1 all OpenStack Compute - virtual machine scheduler
ii python-nova 2:16.0.0.0a1.201704201428.xenial-0ubuntu1 all OpenStack Compute Python libraries
ii python-novaclient 2:7.1.0-0ubuntu1~cloud0 all client library for OpenStack Compute API - Python 2.7
2. Which hypervisor did you use?
root@controlNode:~# dpkg -l |grep libvirt
ii libvirt-bin 2.5.0-3ubuntu7~cloud0 amd64 programs for the libvirt library
ii libvirt-clients 2.5.0-3ubuntu7~cloud0 amd64 Programs for the libvirt library
ii libvirt-daemon 2.5.0-3ubuntu7~cloud0 amd64 Virtualization daemon
ii libvirt-daemon-system 2.5.0-3ubuntu7~cloud0 amd64 Libvirt daemon configuration files
ii libvirt0:amd64 2.5.0-3ubuntu7~cloud0 amd64 library for interfacing with different virtualization systems
ii nova-compute-libvirt 2:16.0.0.0a1.201704201428.xenial-0ubuntu1 all OpenStack Compute - compute node libvirt support
ii python-libvirt 3.0.0-2build1~cloud0 amd64 libvirt Python bindings
root@controlNode:~#
root@controlNode:~#
root@controlNode:~# dpkg -l |grep qemu
ii ipxe-qemu 1.0.0+git-20150424.a25a16d-1ubuntu1 all PXE boot firmware - ROM images for qemu
ii qemu-block-extra:amd64 1:2.8+dfsg-3ubuntu2.1~cloud0 amd64 extra block backend modules for qemu-system and qemu-utils
ii qemu-kvm 1:2.8+dfsg-3ubuntu2.1~cloud0 amd64 QEMU Full virtualization
ii qemu-system-common 1:2.8+dfsg-3ubuntu2.1~cloud0 amd64 QEMU full system emulation binaries (common files)
ii qemu-system-x86 1:2.8+dfsg-3ubuntu2.1~cloud0 amd64 QEMU full system emulation binaries (x86)
ii qemu-utils 1:2.8+dfsg-3ubuntu2.1~cloud0 amd64 QEMU utilities
root@controlNode:~#
root@controlNode:~#
root@controlNode:~# dpkg -l |grep kvm
ii nova-compute-kvm 2:16.0.0.0a1.201704201428.xenial-0ubuntu1 all OpenStack Compute - compute node (KVM)
ii qemu-kvm 1:2.8+dfsg-3ubuntu2.1~cloud0 amd64 QEMU Full virtualization
2. Which storage type did you use?
LVM
root@controlNode:~# dpkg -l |grep lvm
ii liblvm2app2.2:amd64 2.02.133-1ubuntu10 amd64 LVM2 application library
ii liblvm2cmd2.02:amd64 2.02.133-1ubuntu10 amd64 LVM2 command library
ii lvm2 2.02.133-1ubuntu10 amd64 Linux Logical Volume Manager
3. Which networking type did you use?
Neutron with openvswitch
root@controlNode:~# sudo ovs-vsctl show
8afa9151-19dd-4a76-b77c-3b2e87fc3c3d
Manager "tcp:192.168.56.102:6640"
is_connected: true
Bridge br-int
Controller "tcp:192.168.56.102:6653"
is_connected: true
fail_mode: secure
Port br-int
Interface br-int
type: internal
ovs_version: "2.7.0"
Logs & Configs
==============
Executed following command on compute node (enabled DEBUG mode) to collect the reports and same is attached
$ sudo sosreport -o openstack_nova --batch
File name: sosreport-controlNode-20170526000241.tar.xz
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1693481/+subscriptions
References