yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1446449] [NEW] ironic hypervisor resource should be released for booting failed case

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: "Haomeng,Wang" <whaom@xxxxxxxxxx>
Date: Tue, 21 Apr 2015 03:42:48 -0000
Reply-to: Bug 1446449 <1446449@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
Public bug reported:


nova boot failed in spawn step, the ironic hypervisor show the
mem/cpu/disk resource still be occpied by the nova instance which is in
error status, I understand for such boot failed case, the ironic
hypervisor resource should be released once the boot is completed with
error.

[root@hbcontrol ~]# nova hypervisor-stats
+----------------------+-------+
| Property             | Value |
+----------------------+-------+
| count                | 1     |
| current_workload     | 0     |
| disk_available_least | 30    |
| free_disk_gb         | 0     |
| free_ram_mb          | 0     |
| local_gb             | 30    |
| local_gb_used        | 30    |
| memory_mb            | 2048  |
| memory_mb_used       | 2048  |
| running_vms          | 1     |
| vcpus                | 2     |
| vcpus_used           | 2     |
+----------------------+-------+
[root@hbcontrol ~]#

[root@hbcontrol ~]# ironic node-list
+--------------------------------------+------+---------------+-------------+--------------------+-------------+
| UUID                                 | Name | Instance UUID | Power State | Provisioning State | Maintenance |
+--------------------------------------+------+---------------+-------------+--------------------+-------------+
| ccdce9d8-2f6a-4d7f-8c53-f89f289fd0a1 | None | None          | power on    | available          | False       |
+--------------------------------------+------+---------------+-------------+--------------------+-------------+
[root@hbcontrol ~]#


nova compute log
====================
2015-04-21 07:31:51.979 1337 INFO nova.compute.manager [req-9abbf97b-8bf0-495a-850e-7874dbb87a22 2b0fbc7394cf4459867f2957e268e2d2 db9f9ab6aef84239a0206c2bb810b55a - - -] [instance: 5597c25c-287e-420f-89d3-4f5a211471b8] Starting instance...
2015-04-21 07:31:52.923 1337 INFO nova.compute.claims [-] [instance: 5597c25c-287e-420f-89d3-4f5a211471b8] Attempting claim: memory 2048 MB, disk 30 GB
2015-04-21 07:31:52.928 1337 INFO nova.compute.claims [-] [instance: 5597c25c-287e-420f-89d3-4f5a211471b8] Total memory: 2048 MB, used: 0.00 MB
2015-04-21 07:31:52.928 1337 INFO nova.compute.claims [-] [instance: 5597c25c-287e-420f-89d3-4f5a211471b8] memory limit: 2048.00 MB, free: 2048.00 MB
2015-04-21 07:31:52.928 1337 INFO nova.compute.claims [-] [instance: 5597c25c-287e-420f-89d3-4f5a211471b8] Total disk: 30 GB, used: 0.00 GB
2015-04-21 07:31:52.928 1337 INFO nova.compute.claims [-] [instance: 5597c25c-287e-420f-89d3-4f5a211471b8] disk limit not specified, defaulting to unlimited
2015-04-21 07:31:53.002 1337 INFO nova.compute.claims [-] [instance: 5597c25c-287e-420f-89d3-4f5a211471b8] Claim successful

2015-04-21 07:32:06.331 1337 ERROR nova.servicegroup.drivers.db [req-a53a836a-18b5-4da4-9e1a-3ad4a95de788 - - - - -] model server went away
2015-04-21 07:32:06.331 1337 TRACE nova.servicegroup.drivers.db Traceback (most recent call last):
2015-04-21 07:32:06.331 1337 TRACE nova.servicegroup.drivers.db   File "/usr/lib/python2.7/site-packages/nova/servicegroup/drivers/db.py", line 112, in _report_state
2015-04-21 07:32:06.331 1337 TRACE nova.servicegroup.drivers.db     service.service_ref, state_catalog)
2015-04-21 07:32:06.331 1337 TRACE nova.servicegroup.drivers.db   File "/usr/lib/python2.7/site-packages/nova/conductor/api.py", line 164, in service_update
2015-04-21 07:32:06.331 1337 TRACE nova.servicegroup.drivers.db     return self._manager.service_update(context, service, values)
2015-04-21 07:32:06.331 1337 TRACE nova.servicegroup.drivers.db   File "/usr/lib/python2.7/site-packages/nova/conductor/rpcapi.py", line 284, in service_update
2015-04-21 07:32:06.331 1337 TRACE nova.servicegroup.drivers.db     service=service_p, values=values)
2015-04-21 07:32:06.331 1337 TRACE nova.servicegroup.drivers.db   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/client.py", line 156, in call
2015-04-21 07:32:06.331 1337 TRACE nova.servicegroup.drivers.db     retry=self.retry)
2015-04-21 07:32:06.331 1337 TRACE nova.servicegroup.drivers.db   File "/usr/lib/python2.7/site-packages/oslo_messaging/transport.py", line 90, in _send
2015-04-21 07:32:06.331 1337 TRACE nova.servicegroup.drivers.db     timeout=timeout, retry=retry)
2015-04-21 07:32:06.331 1337 TRACE nova.servicegroup.drivers.db   File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 349, in send
2015-04-21 07:32:06.331 1337 TRACE nova.servicegroup.drivers.db     retry=retry)
2015-04-21 07:32:06.331 1337 TRACE nova.servicegroup.drivers.db   File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 338, in _send
2015-04-21 07:32:06.331 1337 TRACE nova.servicegroup.drivers.db     result = self._waiter.wait(msg_id, timeout)
2015-04-21 07:32:06.331 1337 TRACE nova.servicegroup.drivers.db   File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 242, in wait
2015-04-21 07:32:06.331 1337 TRACE nova.servicegroup.drivers.db     message = self.waiters.get(msg_id, timeout=timeout)
2015-04-21 07:32:06.331 1337 TRACE nova.servicegroup.drivers.db   File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 148, in get
2015-04-21 07:32:06.331 1337 TRACE nova.servicegroup.drivers.db     'to message ID %s' % msg_id)
2015-04-21 07:32:06.331 1337 TRACE nova.servicegroup.drivers.db MessagingTimeout: Timed out waiting for a reply to message ID 2bda87fd61fd445ca3f39081977efda9
2015-04-21 07:32:06.331 1337 TRACE nova.servicegroup.drivers.db
2015-04-21 07:32:07.450 1337 INFO oslo_messaging._drivers.amqpdriver [-] No calling threads waiting for msg_id : 2bda87fd61fd445ca3f39081977efda9
2015-04-21 07:32:07.451 1337 INFO oslo_messaging._drivers.amqpdriver [-] No calling threads waiting for msg_id : 2bda87fd61fd445ca3f39081977efda9
2015-04-21 07:32:07.453 1337 ERROR nova.servicegroup.drivers.db [req-a53a836a-18b5-4da4-9e1a-3ad4a95de788 - - - - -] Recovered model server connection!


2015-04-21 07:32:10.240 1337 INFO nova.scheduler.client.report [-] Compute_service record updated for ('hbcontrol', u'ccdce9d8-2f6a-4d7f-8c53-f89f289fd0a1')
2015-04-21 07:32:13.702 1337 INFO nova.scheduler.client.report [-] Compute_service record updated for ('hbcontrol', u'ccdce9d8-2f6a-4d7f-8c53-f89f289fd0a1')
2015-04-21 07:32:15.364 1337 ERROR nova.compute.manager [-] Instance failed network setup after 1 attempt(s)
2015-04-21 07:32:15.364 1337 TRACE nova.compute.manager Traceback (most recent call last):
2015-04-21 07:32:15.364 1337 TRACE nova.compute.manager   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1736, in _allocate_network_async
2015-04-21 07:32:15.364 1337 TRACE nova.compute.manager     dhcp_options=dhcp_options)
2015-04-21 07:32:15.364 1337 TRACE nova.compute.manager   File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 576, in allocate_for_instance
2015-04-21 07:32:15.364 1337 TRACE nova.compute.manager     self._delete_ports(neutron, instance, created_port_ids)
2015-04-21 07:32:15.364 1337 TRACE nova.compute.manager   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 85, in __exit__
2015-04-21 07:32:15.364 1337 TRACE nova.compute.manager     six.reraise(self.type_, self.value, self.tb)
2015-04-21 07:32:15.364 1337 TRACE nova.compute.manager   File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 568, in allocate_for_instance
2015-04-21 07:32:15.364 1337 TRACE nova.compute.manager     security_group_ids, available_macs, dhcp_opts)
2015-04-21 07:32:15.364 1337 TRACE nova.compute.manager   File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 298, in _create_port
2015-04-21 07:32:15.364 1337 TRACE nova.compute.manager     instance=instance.uuid)
2015-04-21 07:32:15.364 1337 TRACE nova.compute.manager PortNotFree: No free port available for instance 5597c25c-287e-420f-89d3-4f5a211471b8.
2015-04-21 07:32:15.364 1337 TRACE nova.compute.manager
2015-04-21 07:32:17.390 1337 ERROR nova.virt.ironic.driver [-] Error preparing deploy for instance 5597c25c-287e-420f-89d3-4f5a211471b8 on baremetal node ccdce9d8-2f6a-4d7f-8c53-f89f289fd0a1.
2015-04-21 07:32:17.573 1337 ERROR nova.compute.manager [-] [instance: 5597c25c-287e-420f-89d3-4f5a211471b8] Instance failed to spawn
2015-04-21 07:32:17.573 1337 TRACE nova.compute.manager [instance: 5597c25c-287e-420f-89d3-4f5a211471b8] Traceback (most recent call last):
2015-04-21 07:32:17.573 1337 TRACE nova.compute.manager [instance: 5597c25c-287e-420f-89d3-4f5a211471b8]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2458, in _build_resources
2015-04-21 07:32:17.573 1337 TRACE nova.compute.manager [instance: 5597c25c-287e-420f-89d3-4f5a211471b8]     yield resources
2015-04-21 07:32:17.573 1337 TRACE nova.compute.manager [instance: 5597c25c-287e-420f-89d3-4f5a211471b8]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2322, in _build_and_run_instance
2015-04-21 07:32:17.573 1337 TRACE nova.compute.manager [instance: 5597c25c-287e-420f-89d3-4f5a211471b8]     block_device_info=block_device_info)
2015-04-21 07:32:17.573 1337 TRACE nova.compute.manager [instance: 5597c25c-287e-420f-89d3-4f5a211471b8]   File "/usr/lib/python2.7/site-packages/nova/virt/ironic/driver.py", line 695, in spawn
2015-04-21 07:32:17.573 1337 TRACE nova.compute.manager [instance: 5597c25c-287e-420f-89d3-4f5a211471b8]     flavor=flavor)
2015-04-21 07:32:17.573 1337 TRACE nova.compute.manager [instance: 5597c25c-287e-420f-89d3-4f5a211471b8]   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 85, in __exit__
2015-04-21 07:32:17.573 1337 TRACE nova.compute.manager [instance: 5597c25c-287e-420f-89d3-4f5a211471b8]     six.reraise(self.type_, self.value, self.tb)
2015-04-21 07:32:17.573 1337 TRACE nova.compute.manager [instance: 5597c25c-287e-420f-89d3-4f5a211471b8]   File "/usr/lib/python2.7/site-packages/nova/virt/ironic/driver.py", line 686, in spawn
2015-04-21 07:32:17.573 1337 TRACE nova.compute.manager [instance: 5597c25c-287e-420f-89d3-4f5a211471b8]     self._plug_vifs(node, instance, network_info)
2015-04-21 07:32:17.573 1337 TRACE nova.compute.manager [instance: 5597c25c-287e-420f-89d3-4f5a211471b8]   File "/usr/lib/python2.7/site-packages/nova/virt/ironic/driver.py", line 954, in _plug_vifs
2015-04-21 07:32:17.573 1337 TRACE nova.compute.manager [instance: 5597c25c-287e-420f-89d3-4f5a211471b8]     network_info_str = str(network_info)
2015-04-21 07:32:17.573 1337 TRACE nova.compute.manager [instance: 5597c25c-287e-420f-89d3-4f5a211471b8]   File "/usr/lib/python2.7/site-packages/nova/network/model.py", line 480, in __str__
2015-04-21 07:32:17.573 1337 TRACE nova.compute.manager [instance: 5597c25c-287e-420f-89d3-4f5a211471b8]     return self._sync_wrapper(fn, *args, **kwargs)
2015-04-21 07:32:17.573 1337 TRACE nova.compute.manager [instance: 5597c25c-287e-420f-89d3-4f5a211471b8]   File "/usr/lib/python2.7/site-packages/nova/network/model.py", line 463, in _sync_wrapper
2015-04-21 07:32:17.573 1337 TRACE nova.compute.manager [instance: 5597c25c-287e-420f-89d3-4f5a211471b8]     self.wait()
2015-04-21 07:32:17.573 1337 TRACE nova.compute.manager [instance: 5597c25c-287e-420f-89d3-4f5a211471b8]   File "/usr/lib/python2.7/site-packages/nova/network/model.py", line 495, in wait
2015-04-21 07:32:17.573 1337 TRACE nova.compute.manager [instance: 5597c25c-287e-420f-89d3-4f5a211471b8]     self[:] = self._gt.wait()
2015-04-21 07:32:17.573 1337 TRACE nova.compute.manager [instance: 5597c25c-287e-420f-89d3-4f5a211471b8]   File "/usr/lib/python2.7/site-packages/eventlet/greenthread.py", line 175, in wait
2015-04-21 07:32:17.573 1337 TRACE nova.compute.manager [instance: 5597c25c-287e-420f-89d3-4f5a211471b8]     return self._exit_event.wait()
2015-04-21 07:32:17.573 1337 TRACE nova.compute.manager [instance: 5597c25c-287e-420f-89d3-4f5a211471b8]   File "/usr/lib/python2.7/site-packages/eventlet/event.py", line 125, in wait
2015-04-21 07:32:17.573 1337 TRACE nova.compute.manager [instance: 5597c25c-287e-420f-89d3-4f5a211471b8]     current.throw(*self._exc)
2015-04-21 07:32:17.573 1337 TRACE nova.compute.manager [instance: 5597c25c-287e-420f-89d3-4f5a211471b8]   File "/usr/lib/python2.7/site-packages/eventlet/greenthread.py", line 214, in main
2015-04-21 07:32:17.573 1337 TRACE nova.compute.manager [instance: 5597c25c-287e-420f-89d3-4f5a211471b8]     result = function(*args, **kwargs)
2015-04-21 07:32:17.573 1337 TRACE nova.compute.manager [instance: 5597c25c-287e-420f-89d3-4f5a211471b8]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1736, in _allocate_network_async
2015-04-21 07:32:17.573 1337 TRACE nova.compute.manager [instance: 5597c25c-287e-420f-89d3-4f5a211471b8]     dhcp_options=dhcp_options)
2015-04-21 07:32:17.573 1337 TRACE nova.compute.manager [instance: 5597c25c-287e-420f-89d3-4f5a211471b8]   File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 576, in allocate_for_instance
2015-04-21 07:32:17.573 1337 TRACE nova.compute.manager [instance: 5597c25c-287e-420f-89d3-4f5a211471b8]     self._delete_ports(neutron, instance, created_port_ids)
2015-04-21 07:32:17.573 1337 TRACE nova.compute.manager [instance: 5597c25c-287e-420f-89d3-4f5a211471b8]   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 85, in __exit__
2015-04-21 07:32:17.573 1337 TRACE nova.compute.manager [instance: 5597c25c-287e-420f-89d3-4f5a211471b8]     six.reraise(self.type_, self.value, self.tb)
2015-04-21 07:32:17.573 1337 TRACE nova.compute.manager [instance: 5597c25c-287e-420f-89d3-4f5a211471b8]   File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 568, in allocate_for_instance
2015-04-21 07:32:17.573 1337 TRACE nova.compute.manager [instance: 5597c25c-287e-420f-89d3-4f5a211471b8]     security_group_ids, available_macs, dhcp_opts)
2015-04-21 07:32:17.573 1337 TRACE nova.compute.manager [instance: 5597c25c-287e-420f-89d3-4f5a211471b8]   File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 298, in _create_port
2015-04-21 07:32:17.573 1337 TRACE nova.compute.manager [instance: 5597c25c-287e-420f-89d3-4f5a211471b8]     instance=instance.uuid)
2015-04-21 07:32:17.573 1337 TRACE nova.compute.manager [instance: 5597c25c-287e-420f-89d3-4f5a211471b8] PortNotFree: No free port available for instance 5597c25c-287e-420f-89d3-4f5a211471b8.
2015-04-21 07:32:17.573 1337 TRACE nova.compute.manager [instance: 5597c25c-287e-420f-89d3-4f5a211471b8]
2015-04-21 07:32:17.594 1337 INFO nova.compute.manager [req-9abbf97b-8bf0-495a-850e-7874dbb87a22 2b0fbc7394cf4459867f2957e268e2d2 db9f9ab6aef84239a0206c2bb810b55a - - -] [instance: 5597c25c-287e-420f-89d3-4f5a211471b8] Terminating instance
2015-04-21 07:32:17.659 1337 WARNING nova.virt.ironic.driver [-] Destroy called on non-existing instance 5597c25c-287e-420f-89d3-4f5a211471b8.
2015-04-21 07:32:18.311 1337 ERROR nova.network.neutronv2.api [-] Unable to clear device ID for port 'None'
2015-04-21 07:32:18.311 1337 TRACE nova.network.neutronv2.api Traceback (most recent call last):
2015-04-21 07:32:18.311 1337 TRACE nova.network.neutronv2.api   File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 365, in _unbind_ports
2015-04-21 07:32:18.311 1337 TRACE nova.network.neutronv2.api     port_client.update_port(port_id, port_req_body)
2015-04-21 07:32:18.311 1337 TRACE nova.network.neutronv2.api   File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 99, in with_params
2015-04-21 07:32:18.311 1337 TRACE nova.network.neutronv2.api     ret = self.function(instance, *args, **kwargs)
2015-04-21 07:32:18.311 1337 TRACE nova.network.neutronv2.api   File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 512, in update_port
2015-04-21 07:32:18.311 1337 TRACE nova.network.neutronv2.api     return self.put(self.port_path % (port), body=body)
2015-04-21 07:32:18.311 1337 TRACE nova.network.neutronv2.api   File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 299, in put
2015-04-21 07:32:18.311 1337 TRACE nova.network.neutronv2.api     headers=headers, params=params)
2015-04-21 07:32:18.311 1337 TRACE nova.network.neutronv2.api   File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 267, in retry_request
2015-04-21 07:32:18.311 1337 TRACE nova.network.neutronv2.api     headers=headers, params=params)
2015-04-21 07:32:18.311 1337 TRACE nova.network.neutronv2.api   File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 208, in do_request
2015-04-21 07:32:18.311 1337 TRACE nova.network.neutronv2.api     self._handle_fault_response(status_code, replybody)
2015-04-21 07:32:18.311 1337 TRACE nova.network.neutronv2.api   File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 182, in _handle_fault_response
2015-04-21 07:32:18.311 1337 TRACE nova.network.neutronv2.api     exception_handler_v20(status_code, des_error_body)
2015-04-21 07:32:18.311 1337 TRACE nova.network.neutronv2.api   File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 80, in exception_handler_v20
2015-04-21 07:32:18.311 1337 TRACE nova.network.neutronv2.api     message=message)
2015-04-21 07:32:18.311 1337 TRACE nova.network.neutronv2.api NeutronClientException: 404 Not Found
2015-04-21 07:32:18.311 1337 TRACE nova.network.neutronv2.api
2015-04-21 07:32:18.311 1337 TRACE nova.network.neutronv2.api The resource could not be found.
2015-04-21 07:32:18.311 1337 TRACE nova.network.neutronv2.api
2015-04-21 07:32:18.311 1337 TRACE nova.network.neutronv2.api
2015-04-21 07:32:18.311 1337 TRACE nova.network.neutronv2.api
2015-04-21 07:32:18.485 1337 INFO nova.scheduler.client.report [-] Compute_service record updated for ('hbcontrol', u'ccdce9d8-2f6a-4d7f-8c53-f89f289fd0a1')
2015-04-21 07:32:18.580 1337 ERROR nova.network.neutronv2.api [-] Unable to clear device ID for port 'None'
2015-04-21 07:32:18.580 1337 TRACE nova.network.neutronv2.api Traceback (most recent call last):
2015-04-21 07:32:18.580 1337 TRACE nova.network.neutronv2.api   File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 365, in _unbind_ports
2015-04-21 07:32:18.580 1337 TRACE nova.network.neutronv2.api     port_client.update_port(port_id, port_req_body)
2015-04-21 07:32:18.580 1337 TRACE nova.network.neutronv2.api   File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 99, in with_params
2015-04-21 07:32:18.580 1337 TRACE nova.network.neutronv2.api     ret = self.function(instance, *args, **kwargs)
2015-04-21 07:32:18.580 1337 TRACE nova.network.neutronv2.api   File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 512, in update_port
2015-04-21 07:32:18.580 1337 TRACE nova.network.neutronv2.api     return self.put(self.port_path % (port), body=body)
2015-04-21 07:32:18.580 1337 TRACE nova.network.neutronv2.api   File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 299, in put
2015-04-21 07:32:18.580 1337 TRACE nova.network.neutronv2.api     headers=headers, params=params)
2015-04-21 07:32:18.580 1337 TRACE nova.network.neutronv2.api   File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 267, in retry_request
2015-04-21 07:32:18.580 1337 TRACE nova.network.neutronv2.api     headers=headers, params=params)
2015-04-21 07:32:18.580 1337 TRACE nova.network.neutronv2.api   File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 208, in do_request
2015-04-21 07:32:18.580 1337 TRACE nova.network.neutronv2.api     self._handle_fault_response(status_code, replybody)
2015-04-21 07:32:18.580 1337 TRACE nova.network.neutronv2.api   File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 182, in _handle_fault_response
2015-04-21 07:32:18.580 1337 TRACE nova.network.neutronv2.api     exception_handler_v20(status_code, des_error_body)
2015-04-21 07:32:18.580 1337 TRACE nova.network.neutronv2.api   File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 80, in exception_handler_v20
2015-04-21 07:32:18.580 1337 TRACE nova.network.neutronv2.api     message=message)
2015-04-21 07:32:18.580 1337 TRACE nova.network.neutronv2.api NeutronClientException: 404 Not Found
2015-04-21 07:32:18.580 1337 TRACE nova.network.neutronv2.api
2015-04-21 07:32:18.580 1337 TRACE nova.network.neutronv2.api The resource could not be found.
2015-04-21 07:32:18.580 1337 TRACE nova.network.neutronv2.api
2015-04-21 07:32:18.580 1337 TRACE nova.network.neutronv2.api
2015-04-21 07:32:18.580 1337 TRACE nova.network.neutronv2.api
^C
[root@hbcontrol ~]#

** Affects: ironic
     Importance: Undecided
         Status: New

** Affects: nova
     Importance: Undecided
         Status: New

** Also affects: nova
   Importance: Undecided
       Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1446449

Title:
  ironic hypervisor resource should be released for booting failed case

Status in OpenStack Bare Metal Provisioning Service (Ironic):
  New
Status in OpenStack Compute (Nova):
  New

Bug description:

  nova boot failed in spawn step, the ironic hypervisor show the
  mem/cpu/disk resource still be occpied by the nova instance which is
  in error status, I understand for such boot failed case, the ironic
  hypervisor resource should be released once the boot is completed with
  error.

  [root@hbcontrol ~]# nova hypervisor-stats
  +----------------------+-------+
  | Property             | Value |
  +----------------------+-------+
  | count                | 1     |
  | current_workload     | 0     |
  | disk_available_least | 30    |
  | free_disk_gb         | 0     |
  | free_ram_mb          | 0     |
  | local_gb             | 30    |
  | local_gb_used        | 30    |
  | memory_mb            | 2048  |
  | memory_mb_used       | 2048  |
  | running_vms          | 1     |
  | vcpus                | 2     |
  | vcpus_used           | 2     |
  +----------------------+-------+
  [root@hbcontrol ~]#

  [root@hbcontrol ~]# ironic node-list
  +--------------------------------------+------+---------------+-------------+--------------------+-------------+
  | UUID                                 | Name | Instance UUID | Power State | Provisioning State | Maintenance |
  +--------------------------------------+------+---------------+-------------+--------------------+-------------+
  | ccdce9d8-2f6a-4d7f-8c53-f89f289fd0a1 | None | None          | power on    | available          | False       |
  +--------------------------------------+------+---------------+-------------+--------------------+-------------+
  [root@hbcontrol ~]#

  
  nova compute log
  ====================
  2015-04-21 07:31:51.979 1337 INFO nova.compute.manager [req-9abbf97b-8bf0-495a-850e-7874dbb87a22 2b0fbc7394cf4459867f2957e268e2d2 db9f9ab6aef84239a0206c2bb810b55a - - -] [instance: 5597c25c-287e-420f-89d3-4f5a211471b8] Starting instance...
  2015-04-21 07:31:52.923 1337 INFO nova.compute.claims [-] [instance: 5597c25c-287e-420f-89d3-4f5a211471b8] Attempting claim: memory 2048 MB, disk 30 GB
  2015-04-21 07:31:52.928 1337 INFO nova.compute.claims [-] [instance: 5597c25c-287e-420f-89d3-4f5a211471b8] Total memory: 2048 MB, used: 0.00 MB
  2015-04-21 07:31:52.928 1337 INFO nova.compute.claims [-] [instance: 5597c25c-287e-420f-89d3-4f5a211471b8] memory limit: 2048.00 MB, free: 2048.00 MB
  2015-04-21 07:31:52.928 1337 INFO nova.compute.claims [-] [instance: 5597c25c-287e-420f-89d3-4f5a211471b8] Total disk: 30 GB, used: 0.00 GB
  2015-04-21 07:31:52.928 1337 INFO nova.compute.claims [-] [instance: 5597c25c-287e-420f-89d3-4f5a211471b8] disk limit not specified, defaulting to unlimited
  2015-04-21 07:31:53.002 1337 INFO nova.compute.claims [-] [instance: 5597c25c-287e-420f-89d3-4f5a211471b8] Claim successful

  2015-04-21 07:32:06.331 1337 ERROR nova.servicegroup.drivers.db [req-a53a836a-18b5-4da4-9e1a-3ad4a95de788 - - - - -] model server went away
  2015-04-21 07:32:06.331 1337 TRACE nova.servicegroup.drivers.db Traceback (most recent call last):
  2015-04-21 07:32:06.331 1337 TRACE nova.servicegroup.drivers.db   File "/usr/lib/python2.7/site-packages/nova/servicegroup/drivers/db.py", line 112, in _report_state
  2015-04-21 07:32:06.331 1337 TRACE nova.servicegroup.drivers.db     service.service_ref, state_catalog)
  2015-04-21 07:32:06.331 1337 TRACE nova.servicegroup.drivers.db   File "/usr/lib/python2.7/site-packages/nova/conductor/api.py", line 164, in service_update
  2015-04-21 07:32:06.331 1337 TRACE nova.servicegroup.drivers.db     return self._manager.service_update(context, service, values)
  2015-04-21 07:32:06.331 1337 TRACE nova.servicegroup.drivers.db   File "/usr/lib/python2.7/site-packages/nova/conductor/rpcapi.py", line 284, in service_update
  2015-04-21 07:32:06.331 1337 TRACE nova.servicegroup.drivers.db     service=service_p, values=values)
  2015-04-21 07:32:06.331 1337 TRACE nova.servicegroup.drivers.db   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/client.py", line 156, in call
  2015-04-21 07:32:06.331 1337 TRACE nova.servicegroup.drivers.db     retry=self.retry)
  2015-04-21 07:32:06.331 1337 TRACE nova.servicegroup.drivers.db   File "/usr/lib/python2.7/site-packages/oslo_messaging/transport.py", line 90, in _send
  2015-04-21 07:32:06.331 1337 TRACE nova.servicegroup.drivers.db     timeout=timeout, retry=retry)
  2015-04-21 07:32:06.331 1337 TRACE nova.servicegroup.drivers.db   File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 349, in send
  2015-04-21 07:32:06.331 1337 TRACE nova.servicegroup.drivers.db     retry=retry)
  2015-04-21 07:32:06.331 1337 TRACE nova.servicegroup.drivers.db   File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 338, in _send
  2015-04-21 07:32:06.331 1337 TRACE nova.servicegroup.drivers.db     result = self._waiter.wait(msg_id, timeout)
  2015-04-21 07:32:06.331 1337 TRACE nova.servicegroup.drivers.db   File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 242, in wait
  2015-04-21 07:32:06.331 1337 TRACE nova.servicegroup.drivers.db     message = self.waiters.get(msg_id, timeout=timeout)
  2015-04-21 07:32:06.331 1337 TRACE nova.servicegroup.drivers.db   File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 148, in get
  2015-04-21 07:32:06.331 1337 TRACE nova.servicegroup.drivers.db     'to message ID %s' % msg_id)
  2015-04-21 07:32:06.331 1337 TRACE nova.servicegroup.drivers.db MessagingTimeout: Timed out waiting for a reply to message ID 2bda87fd61fd445ca3f39081977efda9
  2015-04-21 07:32:06.331 1337 TRACE nova.servicegroup.drivers.db
  2015-04-21 07:32:07.450 1337 INFO oslo_messaging._drivers.amqpdriver [-] No calling threads waiting for msg_id : 2bda87fd61fd445ca3f39081977efda9
  2015-04-21 07:32:07.451 1337 INFO oslo_messaging._drivers.amqpdriver [-] No calling threads waiting for msg_id : 2bda87fd61fd445ca3f39081977efda9
  2015-04-21 07:32:07.453 1337 ERROR nova.servicegroup.drivers.db [req-a53a836a-18b5-4da4-9e1a-3ad4a95de788 - - - - -] Recovered model server connection!

  
  2015-04-21 07:32:10.240 1337 INFO nova.scheduler.client.report [-] Compute_service record updated for ('hbcontrol', u'ccdce9d8-2f6a-4d7f-8c53-f89f289fd0a1')
  2015-04-21 07:32:13.702 1337 INFO nova.scheduler.client.report [-] Compute_service record updated for ('hbcontrol', u'ccdce9d8-2f6a-4d7f-8c53-f89f289fd0a1')
  2015-04-21 07:32:15.364 1337 ERROR nova.compute.manager [-] Instance failed network setup after 1 attempt(s)
  2015-04-21 07:32:15.364 1337 TRACE nova.compute.manager Traceback (most recent call last):
  2015-04-21 07:32:15.364 1337 TRACE nova.compute.manager   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1736, in _allocate_network_async
  2015-04-21 07:32:15.364 1337 TRACE nova.compute.manager     dhcp_options=dhcp_options)
  2015-04-21 07:32:15.364 1337 TRACE nova.compute.manager   File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 576, in allocate_for_instance
  2015-04-21 07:32:15.364 1337 TRACE nova.compute.manager     self._delete_ports(neutron, instance, created_port_ids)
  2015-04-21 07:32:15.364 1337 TRACE nova.compute.manager   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 85, in __exit__
  2015-04-21 07:32:15.364 1337 TRACE nova.compute.manager     six.reraise(self.type_, self.value, self.tb)
  2015-04-21 07:32:15.364 1337 TRACE nova.compute.manager   File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 568, in allocate_for_instance
  2015-04-21 07:32:15.364 1337 TRACE nova.compute.manager     security_group_ids, available_macs, dhcp_opts)
  2015-04-21 07:32:15.364 1337 TRACE nova.compute.manager   File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 298, in _create_port
  2015-04-21 07:32:15.364 1337 TRACE nova.compute.manager     instance=instance.uuid)
  2015-04-21 07:32:15.364 1337 TRACE nova.compute.manager PortNotFree: No free port available for instance 5597c25c-287e-420f-89d3-4f5a211471b8.
  2015-04-21 07:32:15.364 1337 TRACE nova.compute.manager
  2015-04-21 07:32:17.390 1337 ERROR nova.virt.ironic.driver [-] Error preparing deploy for instance 5597c25c-287e-420f-89d3-4f5a211471b8 on baremetal node ccdce9d8-2f6a-4d7f-8c53-f89f289fd0a1.
  2015-04-21 07:32:17.573 1337 ERROR nova.compute.manager [-] [instance: 5597c25c-287e-420f-89d3-4f5a211471b8] Instance failed to spawn
  2015-04-21 07:32:17.573 1337 TRACE nova.compute.manager [instance: 5597c25c-287e-420f-89d3-4f5a211471b8] Traceback (most recent call last):
  2015-04-21 07:32:17.573 1337 TRACE nova.compute.manager [instance: 5597c25c-287e-420f-89d3-4f5a211471b8]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2458, in _build_resources
  2015-04-21 07:32:17.573 1337 TRACE nova.compute.manager [instance: 5597c25c-287e-420f-89d3-4f5a211471b8]     yield resources
  2015-04-21 07:32:17.573 1337 TRACE nova.compute.manager [instance: 5597c25c-287e-420f-89d3-4f5a211471b8]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2322, in _build_and_run_instance
  2015-04-21 07:32:17.573 1337 TRACE nova.compute.manager [instance: 5597c25c-287e-420f-89d3-4f5a211471b8]     block_device_info=block_device_info)
  2015-04-21 07:32:17.573 1337 TRACE nova.compute.manager [instance: 5597c25c-287e-420f-89d3-4f5a211471b8]   File "/usr/lib/python2.7/site-packages/nova/virt/ironic/driver.py", line 695, in spawn
  2015-04-21 07:32:17.573 1337 TRACE nova.compute.manager [instance: 5597c25c-287e-420f-89d3-4f5a211471b8]     flavor=flavor)
  2015-04-21 07:32:17.573 1337 TRACE nova.compute.manager [instance: 5597c25c-287e-420f-89d3-4f5a211471b8]   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 85, in __exit__
  2015-04-21 07:32:17.573 1337 TRACE nova.compute.manager [instance: 5597c25c-287e-420f-89d3-4f5a211471b8]     six.reraise(self.type_, self.value, self.tb)
  2015-04-21 07:32:17.573 1337 TRACE nova.compute.manager [instance: 5597c25c-287e-420f-89d3-4f5a211471b8]   File "/usr/lib/python2.7/site-packages/nova/virt/ironic/driver.py", line 686, in spawn
  2015-04-21 07:32:17.573 1337 TRACE nova.compute.manager [instance: 5597c25c-287e-420f-89d3-4f5a211471b8]     self._plug_vifs(node, instance, network_info)
  2015-04-21 07:32:17.573 1337 TRACE nova.compute.manager [instance: 5597c25c-287e-420f-89d3-4f5a211471b8]   File "/usr/lib/python2.7/site-packages/nova/virt/ironic/driver.py", line 954, in _plug_vifs
  2015-04-21 07:32:17.573 1337 TRACE nova.compute.manager [instance: 5597c25c-287e-420f-89d3-4f5a211471b8]     network_info_str = str(network_info)
  2015-04-21 07:32:17.573 1337 TRACE nova.compute.manager [instance: 5597c25c-287e-420f-89d3-4f5a211471b8]   File "/usr/lib/python2.7/site-packages/nova/network/model.py", line 480, in __str__
  2015-04-21 07:32:17.573 1337 TRACE nova.compute.manager [instance: 5597c25c-287e-420f-89d3-4f5a211471b8]     return self._sync_wrapper(fn, *args, **kwargs)
  2015-04-21 07:32:17.573 1337 TRACE nova.compute.manager [instance: 5597c25c-287e-420f-89d3-4f5a211471b8]   File "/usr/lib/python2.7/site-packages/nova/network/model.py", line 463, in _sync_wrapper
  2015-04-21 07:32:17.573 1337 TRACE nova.compute.manager [instance: 5597c25c-287e-420f-89d3-4f5a211471b8]     self.wait()
  2015-04-21 07:32:17.573 1337 TRACE nova.compute.manager [instance: 5597c25c-287e-420f-89d3-4f5a211471b8]   File "/usr/lib/python2.7/site-packages/nova/network/model.py", line 495, in wait
  2015-04-21 07:32:17.573 1337 TRACE nova.compute.manager [instance: 5597c25c-287e-420f-89d3-4f5a211471b8]     self[:] = self._gt.wait()
  2015-04-21 07:32:17.573 1337 TRACE nova.compute.manager [instance: 5597c25c-287e-420f-89d3-4f5a211471b8]   File "/usr/lib/python2.7/site-packages/eventlet/greenthread.py", line 175, in wait
  2015-04-21 07:32:17.573 1337 TRACE nova.compute.manager [instance: 5597c25c-287e-420f-89d3-4f5a211471b8]     return self._exit_event.wait()
  2015-04-21 07:32:17.573 1337 TRACE nova.compute.manager [instance: 5597c25c-287e-420f-89d3-4f5a211471b8]   File "/usr/lib/python2.7/site-packages/eventlet/event.py", line 125, in wait
  2015-04-21 07:32:17.573 1337 TRACE nova.compute.manager [instance: 5597c25c-287e-420f-89d3-4f5a211471b8]     current.throw(*self._exc)
  2015-04-21 07:32:17.573 1337 TRACE nova.compute.manager [instance: 5597c25c-287e-420f-89d3-4f5a211471b8]   File "/usr/lib/python2.7/site-packages/eventlet/greenthread.py", line 214, in main
  2015-04-21 07:32:17.573 1337 TRACE nova.compute.manager [instance: 5597c25c-287e-420f-89d3-4f5a211471b8]     result = function(*args, **kwargs)
  2015-04-21 07:32:17.573 1337 TRACE nova.compute.manager [instance: 5597c25c-287e-420f-89d3-4f5a211471b8]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1736, in _allocate_network_async
  2015-04-21 07:32:17.573 1337 TRACE nova.compute.manager [instance: 5597c25c-287e-420f-89d3-4f5a211471b8]     dhcp_options=dhcp_options)
  2015-04-21 07:32:17.573 1337 TRACE nova.compute.manager [instance: 5597c25c-287e-420f-89d3-4f5a211471b8]   File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 576, in allocate_for_instance
  2015-04-21 07:32:17.573 1337 TRACE nova.compute.manager [instance: 5597c25c-287e-420f-89d3-4f5a211471b8]     self._delete_ports(neutron, instance, created_port_ids)
  2015-04-21 07:32:17.573 1337 TRACE nova.compute.manager [instance: 5597c25c-287e-420f-89d3-4f5a211471b8]   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 85, in __exit__
  2015-04-21 07:32:17.573 1337 TRACE nova.compute.manager [instance: 5597c25c-287e-420f-89d3-4f5a211471b8]     six.reraise(self.type_, self.value, self.tb)
  2015-04-21 07:32:17.573 1337 TRACE nova.compute.manager [instance: 5597c25c-287e-420f-89d3-4f5a211471b8]   File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 568, in allocate_for_instance
  2015-04-21 07:32:17.573 1337 TRACE nova.compute.manager [instance: 5597c25c-287e-420f-89d3-4f5a211471b8]     security_group_ids, available_macs, dhcp_opts)
  2015-04-21 07:32:17.573 1337 TRACE nova.compute.manager [instance: 5597c25c-287e-420f-89d3-4f5a211471b8]   File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 298, in _create_port
  2015-04-21 07:32:17.573 1337 TRACE nova.compute.manager [instance: 5597c25c-287e-420f-89d3-4f5a211471b8]     instance=instance.uuid)
  2015-04-21 07:32:17.573 1337 TRACE nova.compute.manager [instance: 5597c25c-287e-420f-89d3-4f5a211471b8] PortNotFree: No free port available for instance 5597c25c-287e-420f-89d3-4f5a211471b8.
  2015-04-21 07:32:17.573 1337 TRACE nova.compute.manager [instance: 5597c25c-287e-420f-89d3-4f5a211471b8]
  2015-04-21 07:32:17.594 1337 INFO nova.compute.manager [req-9abbf97b-8bf0-495a-850e-7874dbb87a22 2b0fbc7394cf4459867f2957e268e2d2 db9f9ab6aef84239a0206c2bb810b55a - - -] [instance: 5597c25c-287e-420f-89d3-4f5a211471b8] Terminating instance
  2015-04-21 07:32:17.659 1337 WARNING nova.virt.ironic.driver [-] Destroy called on non-existing instance 5597c25c-287e-420f-89d3-4f5a211471b8.
  2015-04-21 07:32:18.311 1337 ERROR nova.network.neutronv2.api [-] Unable to clear device ID for port 'None'
  2015-04-21 07:32:18.311 1337 TRACE nova.network.neutronv2.api Traceback (most recent call last):
  2015-04-21 07:32:18.311 1337 TRACE nova.network.neutronv2.api   File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 365, in _unbind_ports
  2015-04-21 07:32:18.311 1337 TRACE nova.network.neutronv2.api     port_client.update_port(port_id, port_req_body)
  2015-04-21 07:32:18.311 1337 TRACE nova.network.neutronv2.api   File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 99, in with_params
  2015-04-21 07:32:18.311 1337 TRACE nova.network.neutronv2.api     ret = self.function(instance, *args, **kwargs)
  2015-04-21 07:32:18.311 1337 TRACE nova.network.neutronv2.api   File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 512, in update_port
  2015-04-21 07:32:18.311 1337 TRACE nova.network.neutronv2.api     return self.put(self.port_path % (port), body=body)
  2015-04-21 07:32:18.311 1337 TRACE nova.network.neutronv2.api   File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 299, in put
  2015-04-21 07:32:18.311 1337 TRACE nova.network.neutronv2.api     headers=headers, params=params)
  2015-04-21 07:32:18.311 1337 TRACE nova.network.neutronv2.api   File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 267, in retry_request
  2015-04-21 07:32:18.311 1337 TRACE nova.network.neutronv2.api     headers=headers, params=params)
  2015-04-21 07:32:18.311 1337 TRACE nova.network.neutronv2.api   File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 208, in do_request
  2015-04-21 07:32:18.311 1337 TRACE nova.network.neutronv2.api     self._handle_fault_response(status_code, replybody)
  2015-04-21 07:32:18.311 1337 TRACE nova.network.neutronv2.api   File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 182, in _handle_fault_response
  2015-04-21 07:32:18.311 1337 TRACE nova.network.neutronv2.api     exception_handler_v20(status_code, des_error_body)
  2015-04-21 07:32:18.311 1337 TRACE nova.network.neutronv2.api   File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 80, in exception_handler_v20
  2015-04-21 07:32:18.311 1337 TRACE nova.network.neutronv2.api     message=message)
  2015-04-21 07:32:18.311 1337 TRACE nova.network.neutronv2.api NeutronClientException: 404 Not Found
  2015-04-21 07:32:18.311 1337 TRACE nova.network.neutronv2.api
  2015-04-21 07:32:18.311 1337 TRACE nova.network.neutronv2.api The resource could not be found.
  2015-04-21 07:32:18.311 1337 TRACE nova.network.neutronv2.api
  2015-04-21 07:32:18.311 1337 TRACE nova.network.neutronv2.api
  2015-04-21 07:32:18.311 1337 TRACE nova.network.neutronv2.api
  2015-04-21 07:32:18.485 1337 INFO nova.scheduler.client.report [-] Compute_service record updated for ('hbcontrol', u'ccdce9d8-2f6a-4d7f-8c53-f89f289fd0a1')
  2015-04-21 07:32:18.580 1337 ERROR nova.network.neutronv2.api [-] Unable to clear device ID for port 'None'
  2015-04-21 07:32:18.580 1337 TRACE nova.network.neutronv2.api Traceback (most recent call last):
  2015-04-21 07:32:18.580 1337 TRACE nova.network.neutronv2.api   File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 365, in _unbind_ports
  2015-04-21 07:32:18.580 1337 TRACE nova.network.neutronv2.api     port_client.update_port(port_id, port_req_body)
  2015-04-21 07:32:18.580 1337 TRACE nova.network.neutronv2.api   File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 99, in with_params
  2015-04-21 07:32:18.580 1337 TRACE nova.network.neutronv2.api     ret = self.function(instance, *args, **kwargs)
  2015-04-21 07:32:18.580 1337 TRACE nova.network.neutronv2.api   File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 512, in update_port
  2015-04-21 07:32:18.580 1337 TRACE nova.network.neutronv2.api     return self.put(self.port_path % (port), body=body)
  2015-04-21 07:32:18.580 1337 TRACE nova.network.neutronv2.api   File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 299, in put
  2015-04-21 07:32:18.580 1337 TRACE nova.network.neutronv2.api     headers=headers, params=params)
  2015-04-21 07:32:18.580 1337 TRACE nova.network.neutronv2.api   File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 267, in retry_request
  2015-04-21 07:32:18.580 1337 TRACE nova.network.neutronv2.api     headers=headers, params=params)
  2015-04-21 07:32:18.580 1337 TRACE nova.network.neutronv2.api   File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 208, in do_request
  2015-04-21 07:32:18.580 1337 TRACE nova.network.neutronv2.api     self._handle_fault_response(status_code, replybody)
  2015-04-21 07:32:18.580 1337 TRACE nova.network.neutronv2.api   File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 182, in _handle_fault_response
  2015-04-21 07:32:18.580 1337 TRACE nova.network.neutronv2.api     exception_handler_v20(status_code, des_error_body)
  2015-04-21 07:32:18.580 1337 TRACE nova.network.neutronv2.api   File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 80, in exception_handler_v20
  2015-04-21 07:32:18.580 1337 TRACE nova.network.neutronv2.api     message=message)
  2015-04-21 07:32:18.580 1337 TRACE nova.network.neutronv2.api NeutronClientException: 404 Not Found
  2015-04-21 07:32:18.580 1337 TRACE nova.network.neutronv2.api
  2015-04-21 07:32:18.580 1337 TRACE nova.network.neutronv2.api The resource could not be found.
  2015-04-21 07:32:18.580 1337 TRACE nova.network.neutronv2.api
  2015-04-21 07:32:18.580 1337 TRACE nova.network.neutronv2.api
  2015-04-21 07:32:18.580 1337 TRACE nova.network.neutronv2.api
  ^C
  [root@hbcontrol ~]#

To manage notifications about this bug go to:
https://bugs.launchpad.net/ironic/+bug/1446449/+subscriptions
Follow ups

[Bug 1446449] Re: ironic hypervisor resource should be released for booting failed case
From: Dmitry Tantsur, 2015-08-10
[Bug 1446449] [NEW] ironic hypervisor resource should be released for booting failed case
From: Haomeng,Wang, 2015-04-21
References

[Bug 1446449] [NEW] ironic hypervisor resource should be released for booting failed case
From: Haomeng,Wang, 2015-04-21