← Back to team overview

yahoo-eng-team team mailing list archive

  1. yahoo-eng-team team
  2. Mailing list archive
  3. Message #52908

[Bug 1596922] [NEW] instance uuid not cleared in an ironic node

  Thread Previous • Date Previous • • Thread Next 
Public bug reported:

Description
===========
In nova with ironic environment, "instance UUID" still remains if nova boot
got an error.  As a result, the ironic node cannot delete by API and deploy
baremetal instance.


Steps to reproduce
==================
1. create an ironic-node
2. create an ironic-port
3. nova boot for ironic instance with following illegal instance name.

   $ nova boot --flavor my-baremetal-agent --image $MY_IMAGE_UUID \
               --key-name default test3.141592 --nic net-id=$NET_UUID


Expected result
===============
Nova got an error and state has changed "ERROR".  Then, tenant user deletes
nova's instance($nova delete <instance_uuid>).  After that, "instance UUID"
of an ironic node should be cleared.


Actual result
=============
After $nova delete <instance_uuid>, "instance UUID" still remains into the
ironic node database.

$ ironic node-delete 0ea7c2e3-e5be-4052-85ac-a7b1adf0f30b
Failed to delete node 0ea7c2e3-e5be-4052-85ac-a7b1adf0f30b: Node 0ea7c2e3-e5be-4052-85ac-a7b1adf0f30b is associated with instance 84911c1f-976b-4428-a509-8ab6cf04182a. (HTTP 409)


Environment
===========
* Devstack all-in-one
* Nova's source code is for "May 18".

commit fe8a119e8d80de35d7f99e0c1d9a9e5095840146
Merge: b56d861 6f2a46f
Author: Jenkins <jenkins@xxxxxxxxxxxxxxxxxxxx>
Date:   Wed May 18 23:33:00 2016 +0000

    Merge "Remove unused base_options param from
_get_image_defined_bdms"

2. Which hypervisor did you use?
   ironic

3. Which networking type did you use?
   (For example: nova-network, Neutron with OpenVSwitch, ...)
neutron + ML2 + openvswitch driver + linuxbridge driver

Logs
====
2016-06-28 21:03:04.670 ERROR nova.scheduler.utils [req-7db0e0e5-5d18-4a5a-9293-52dd2e0f4351 admin admin] [instance: 84911c1f-976b-4428-a509-8ab6cf04182a] Error from last host: f
urukawa-dev-ironic (node 0ea7c2e3-e5be-4052-85ac-a7b1adf0f30b): [u'Traceback (most recent call last):\n', u'  File "/opt/stack/nova/nova/compute/manager.py", line 1749, in _do_bu
ild_and_run_instance\n    filter_properties)\n', u'  File "/opt/stack/nova/nova/compute/manager.py", line 1939, in _build_and_run_instance\n    instance_uuid=instance.uuid, reaso
n=six.text_type(e))\n', u"RescheduledException: Build of instance 84911c1f-976b-4428-a509-8ab6cf04182a was re-scheduled: Invalid input for dns_name. Reason: 'test3.141592' not a
valid PQDN or FQDN. Reason: TLD '141592' must not be all numeric.\n"]
2016-06-28 21:03:04.689 DEBUG oslo_messaging._drivers.amqpdriver [req-7db0e0e5-5d18-4a5a-9293-52dd2e0f4351 admin admin] sending reply msg_id: dea4917aad334ffda701e6cd23cf6a4c rep
ly queue: reply_a163275e9821450cb6494257a3b9629f time elapsed: 0.0230762520805s from (pid=5506) _send_reply /usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdri
ver.py:74
2016-06-28 21:03:04.688 DEBUG oslo_messaging._drivers.amqpdriver [req-7db0e0e5-5d18-4a5a-9293-52dd2e0f4351 admin admin] CALL msg_id: 51218c1cd9ad403798f2d830d9d0abb3 exchange 'no
va' topic 'scheduler' from (pid=5507) _send /usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py:450
2016-06-28 21:03:04.741 DEBUG oslo_messaging._drivers.amqpdriver [-] received reply msg_id: 51218c1cd9ad403798f2d830d9d0abb3 from (pid=5507) __call__ /usr/local/lib/python2.7/dis
t-packages/oslo_messaging/_drivers/amqpdriver.py:298
2016-06-28 21:03:04.743 WARNING nova.scheduler.utils [req-7db0e0e5-5d18-4a5a-9293-52dd2e0f4351 admin admin] Failed to compute_task_build_instances: No valid host was found. There
 are not enough hosts available.
Traceback (most recent call last):

  File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/server.py", line 200, in inner
    return func(*args, **kwargs)

  File "/opt/stack/nova/nova/scheduler/manager.py", line 104, in select_destinations
    dests = self.driver.select_destinations(ctxt, spec_obj)

  File "/opt/stack/nova/nova/scheduler/filter_scheduler.py", line 74, in select_destinations
    raise exception.NoValidHost(reason=reason)

NoValidHost: No valid host was found. There are not enough hosts
available.

** Affects: nova
     Importance: Undecided
         Status: New


** Tags: ironic

** Attachment added: "Operation log"
   https://bugs.launchpad.net/bugs/1596922/+attachment/4691573/+files/opelog.txt

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1596922

Title:
  instance uuid not cleared  in an ironic node

Status in OpenStack Compute (nova):
  New

Bug description:
  Description
  ===========
  In nova with ironic environment, "instance UUID" still remains if nova boot
  got an error.  As a result, the ironic node cannot delete by API and deploy
  baremetal instance.

  
  Steps to reproduce
  ==================
  1. create an ironic-node
  2. create an ironic-port
  3. nova boot for ironic instance with following illegal instance name.

     $ nova boot --flavor my-baremetal-agent --image $MY_IMAGE_UUID \
                 --key-name default test3.141592 --nic net-id=$NET_UUID

  
  Expected result
  ===============
  Nova got an error and state has changed "ERROR".  Then, tenant user deletes
  nova's instance($nova delete <instance_uuid>).  After that, "instance UUID"
  of an ironic node should be cleared.

  
  Actual result
  =============
  After $nova delete <instance_uuid>, "instance UUID" still remains into the
  ironic node database.

  $ ironic node-delete 0ea7c2e3-e5be-4052-85ac-a7b1adf0f30b
  Failed to delete node 0ea7c2e3-e5be-4052-85ac-a7b1adf0f30b: Node 0ea7c2e3-e5be-4052-85ac-a7b1adf0f30b is associated with instance 84911c1f-976b-4428-a509-8ab6cf04182a. (HTTP 409)

  
  Environment
  ===========
  * Devstack all-in-one
  * Nova's source code is for "May 18".

  commit fe8a119e8d80de35d7f99e0c1d9a9e5095840146
  Merge: b56d861 6f2a46f
  Author: Jenkins <jenkins@xxxxxxxxxxxxxxxxxxxx>
  Date:   Wed May 18 23:33:00 2016 +0000

      Merge "Remove unused base_options param from
  _get_image_defined_bdms"

  2. Which hypervisor did you use?
     ironic

  3. Which networking type did you use?
     (For example: nova-network, Neutron with OpenVSwitch, ...)
  neutron + ML2 + openvswitch driver + linuxbridge driver

  Logs
  ====
  2016-06-28 21:03:04.670 ERROR nova.scheduler.utils [req-7db0e0e5-5d18-4a5a-9293-52dd2e0f4351 admin admin] [instance: 84911c1f-976b-4428-a509-8ab6cf04182a] Error from last host: f
  urukawa-dev-ironic (node 0ea7c2e3-e5be-4052-85ac-a7b1adf0f30b): [u'Traceback (most recent call last):\n', u'  File "/opt/stack/nova/nova/compute/manager.py", line 1749, in _do_bu
  ild_and_run_instance\n    filter_properties)\n', u'  File "/opt/stack/nova/nova/compute/manager.py", line 1939, in _build_and_run_instance\n    instance_uuid=instance.uuid, reaso
  n=six.text_type(e))\n', u"RescheduledException: Build of instance 84911c1f-976b-4428-a509-8ab6cf04182a was re-scheduled: Invalid input for dns_name. Reason: 'test3.141592' not a
  valid PQDN or FQDN. Reason: TLD '141592' must not be all numeric.\n"]
  2016-06-28 21:03:04.689 DEBUG oslo_messaging._drivers.amqpdriver [req-7db0e0e5-5d18-4a5a-9293-52dd2e0f4351 admin admin] sending reply msg_id: dea4917aad334ffda701e6cd23cf6a4c rep
  ly queue: reply_a163275e9821450cb6494257a3b9629f time elapsed: 0.0230762520805s from (pid=5506) _send_reply /usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdri
  ver.py:74
  2016-06-28 21:03:04.688 DEBUG oslo_messaging._drivers.amqpdriver [req-7db0e0e5-5d18-4a5a-9293-52dd2e0f4351 admin admin] CALL msg_id: 51218c1cd9ad403798f2d830d9d0abb3 exchange 'no
  va' topic 'scheduler' from (pid=5507) _send /usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py:450
  2016-06-28 21:03:04.741 DEBUG oslo_messaging._drivers.amqpdriver [-] received reply msg_id: 51218c1cd9ad403798f2d830d9d0abb3 from (pid=5507) __call__ /usr/local/lib/python2.7/dis
  t-packages/oslo_messaging/_drivers/amqpdriver.py:298
  2016-06-28 21:03:04.743 WARNING nova.scheduler.utils [req-7db0e0e5-5d18-4a5a-9293-52dd2e0f4351 admin admin] Failed to compute_task_build_instances: No valid host was found. There
   are not enough hosts available.
  Traceback (most recent call last):

    File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/server.py", line 200, in inner
      return func(*args, **kwargs)

    File "/opt/stack/nova/nova/scheduler/manager.py", line 104, in select_destinations
      dests = self.driver.select_destinations(ctxt, spec_obj)

    File "/opt/stack/nova/nova/scheduler/filter_scheduler.py", line 74, in select_destinations
      raise exception.NoValidHost(reason=reason)

  NoValidHost: No valid host was found. There are not enough hosts
  available.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1596922/+subscriptions

Follow ups

  Thread Previous • Date Previous • • Thread Next