yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1297853] Re: failed to launch an instance from ISO image: TRACE nova.compute.manager MessagingTimeout: Timed out waiting for a reply to message ID

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Sean Dague <sean@xxxxxxxxx>
Date: Wed, 17 Sep 2014 13:25:59 -0000
Reply-to: Bug 1297853 <1297853@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
** Changed in: nova
       Status: Incomplete => Invalid

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1297853

Title:
  failed to launch an instance from ISO image: TRACE
  nova.compute.manager MessagingTimeout: Timed out waiting for a reply
  to message ID

Status in OpenStack Compute (Nova):
  Invalid

Bug description:
  Description of problem:
  The Openstack is installed as AIO (with nova networking) on fedora 20. The instance was launched on a flavor that has the following parameters:  
  +--------------------------------------+-----------+-----------+------+-----------+------+-------+-------------+-----------+
  | ID                                   | Name      | Memory_MB | Disk | Ephemeral | Swap | VCPUs | RXTX_Factor | Is_Public |
  +--------------------------------------+-----------+-----------+------+-----------+------+-------+-------------+-----------+

  | 8ddee6ea-c7b3-4482-97b3-f4c9ca6a2c19 | m1.medium | 4096      | 40   | 40        |      | 2     | 1.0         | True      |
  +--------------------------------------+-----------+-----------+------+-----------+------+-------+-------------+-----------+

  On the first try the instance status was stuck on 'spawning', even after a time out stopped the process. 
  On the second try the instance status was changed from 'spawning' to 'Error'. 

  The nova compute log:
  2014-03-26 15:33:58.548 15699 DEBUG nova.compute.manager [-] An error occurred _heal_instance_info_cache /usr/lib/python2.7/site-packages/nova/compute/manager.py:4569
  2014-03-26 15:33:58.548 15699 TRACE nova.compute.manager Traceback (most recent call last):
  2014-03-26 15:33:58.548 15699 TRACE nova.compute.manager   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 4565, in _heal_instance_info_cache
  2014-03-26 15:33:58.548 15699 TRACE nova.compute.manager     self._get_instance_nw_info(context, instance, use_slave=True)
  2014-03-26 15:33:58.548 15699 TRACE nova.compute.manager   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 908, in _get_instance_nw_info
  2014-03-26 15:33:58.548 15699 TRACE nova.compute.manager     instance)
  2014-03-26 15:33:58.548 15699 TRACE nova.compute.manager   File "/usr/lib/python2.7/site-packages/nova/network/api.py", line 94, in wrapped
  2014-03-26 15:33:58.548 15699 TRACE nova.compute.manager     return func(self, context, *args, **kwargs)
  2014-03-26 15:33:58.548 15699 TRACE nova.compute.manager   File "/usr/lib/python2.7/site-packages/nova/network/api.py", line 389, in get_instance_nw_info
  2014-03-26 15:33:58.548 15699 TRACE nova.compute.manager     result = self._get_instance_nw_info(context, instance)
  2014-03-26 15:33:58.548 15699 TRACE nova.compute.manager   File "/usr/lib/python2.7/site-packages/nova/network/api.py", line 405, in _get_instance_nw_info
  2014-03-26 15:33:58.548 15699 TRACE nova.compute.manager     nw_info = self.network_rpcapi.get_instance_nw_info(context, **args)
  2014-03-26 15:33:58.548 15699 TRACE nova.compute.manager   File "/usr/lib/python2.7/site-packages/nova/network/rpcapi.py", line 222, in get_instance_nw_info
  2014-03-26 15:33:58.548 15699 TRACE nova.compute.manager     host=host, project_id=project_id)
  2014-03-26 15:33:58.548 15699 TRACE nova.compute.manager   File "/usr/lib/python2.7/site-packages/oslo/messaging/rpc/client.py", line 150, in call
  2014-03-26 15:33:58.548 15699 TRACE nova.compute.manager     wait_for_reply=True, timeout=timeout)
  2014-03-26 15:33:58.548 15699 TRACE nova.compute.manager   File "/usr/lib/python2.7/site-packages/oslo/messaging/transport.py", line 90, in _send
  2014-03-26 15:33:58.548 15699 TRACE nova.compute.manager     timeout=timeout)
  2014-03-26 15:33:58.548 15699 TRACE nova.compute.manager   File "/usr/lib/python2.7/site-packages/oslo/messaging/_drivers/amqpdriver.py", line 409, in send
  2014-03-26 15:33:58.548 15699 TRACE nova.compute.manager     return self._send(target, ctxt, message, wait_for_reply, timeout)
  2014-03-26 15:33:58.548 15699 TRACE nova.compute.manager   File "/usr/lib/python2.7/site-packages/oslo/messaging/_drivers/amqpdriver.py", line 400, in _send
  2014-03-26 15:33:58.548 15699 TRACE nova.compute.manager     result = self._waiter.wait(msg_id, timeout)
  2014-03-26 15:33:58.548 15699 TRACE nova.compute.manager   File "/usr/lib/python2.7/site-packages/oslo/messaging/_drivers/amqpdriver.py", line 280, in wait
  2014-03-26 15:33:58.548 15699 TRACE nova.compute.manager     reply, ending, trylock = self._poll_queue(msg_id, timeout)
  2014-03-26 15:33:58.548 15699 TRACE nova.compute.manager   File "/usr/lib/python2.7/site-packages/oslo/messaging/_drivers/amqpdriver.py", line 220, in _poll_queue
  2014-03-26 15:33:58.548 15699 TRACE nova.compute.manager     message = self.waiters.get(msg_id, timeout)
  2014-03-26 15:33:58.548 15699 TRACE nova.compute.manager   File "/usr/lib/python2.7/site-packages/oslo/messaging/_drivers/amqpdriver.py", line 126, in get
  2014-03-26 15:33:58.548 15699 TRACE nova.compute.manager     'to message ID %s' % msg_id)
  2014-03-26 15:33:58.548 15699 TRACE nova.compute.manager MessagingTimeout: Timed out waiting for a reply to message ID 2b184725ad034655bf5e55c59a643758
  2014-03-26 15:33:58.548 15699 TRACE nova.compute.manager 

  Version-Release number of selected component (if applicable):
  python-novaclient-2.16.0-2.fc21.noarch
  openstack-nova-conductor-2014.1-0.13.b3.fc20.noarch
  openstack-nova-network-2014.1-0.13.b3.fc20.noarch
  openstack-nova-api-2014.1-0.13.b3.fc20.noarch
  openstack-nova-console-2014.1-0.13.b3.fc20.noarch
  openstack-nova-compute-2014.1-0.13.b3.fc20.noarch
  openstack-nova-scheduler-2014.1-0.13.b3.fc20.noarch
  openstack-nova-common-2014.1-0.13.b3.fc20.noarch
  openstack-nova-novncproxy-2014.1-0.13.b3.fc20.noarch
  openstack-nova-cert-2014.1-0.13.b3.fc20.noarch
  python-nova-2014.1-0.13.b3.fc20.noarch
  python-glanceclient-0.12.0-1.fc20.noarch
  python-glance-2014.1-0.4.b3.fc21.noarch
  openstack-glance-2014.1-0.4.b3.fc21.noarch

  
  How reproducible:
  100%

  Steps to Reproduce:
  1. upload an ISO image.
  2. change a flavor to have the same parameters as it shows above.
  3. launch an instance.

  Actual results:
  The launch fails.

  Expected results:
  The instance is up and running.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1297853/+subscriptions
References

[Bug 1297853] [NEW] failed to launch an instance from ISO image: TRACE nova.compute.manager MessagingTimeout: Timed out waiting for a reply to message ID
From: Yogev Rabl, 2014-03-26