← Back to team overview

yahoo-eng-team team mailing list archive

  1. yahoo-eng-team team
  2. Mailing list archive
  3. Message #02256

[Bug 1113120] Re: Nova-Conductor stops processing conductor queue

  Thread Previous • Date Previous • • Thread Next 
This bug lacks the necessary information to effectively reproduce and
fix it, therefore it has been closed. Feel free to reopen the bug by
providing the requested information and set the bug status back to
''New''.

** Changed in: nova
       Status: Incomplete => Invalid

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1113120

Title:
  Nova-Conductor stops processing conductor queue

Status in OpenStack Compute (Nova):
  Invalid

Bug description:
  Running a single rabbit node on the same host as nova-conductor, it
  initially processes calls from nova compute, however aftera a few
  minutes we see the following error in nova-compute. There is no
  corresponding error in nova-conductor, just unread messages in the
  conductor queue.

  Am running latest nova master branch, Although also saw same issue
  with code from 19 December.

  
  2013-02-01 22:36:29 nova.conductor.api: ERROR [req-456fa80a-770b-49b2-b0b7-5add79efd808 None None] Timed out waiting for nova-conductor. Is it running? Or did this service start before nova-conductor?
  2013-02-01 22:36:29.575 14827 TRACE nova.conductor.api Traceback (most recent call last):
  2013-02-01 22:36:29.575 14827 TRACE nova.conductor.api   File "/usr/lib/python2.6/site-packages/nova-2013.1-py2.6.egg/nova/conductor/api.py", line 344, in wait_until_ready
  2013-02-01 22:36:29.575 14827 TRACE nova.conductor.api     self.ping(context, '1.21 GigaWatts', timeout=timeout)
  2013-02-01 22:36:29.575 14827 TRACE nova.conductor.api   File "/usr/lib/python2.6/site-packages/nova-2013.1-py2.6.egg/nova/conductor/api.py", line 352, in ping
  2013-02-01 22:36:29.575 14827 TRACE nova.conductor.api     return self.conductor_rpcapi.ping(context, arg, timeout)
  2013-02-01 22:36:29.575 14827 TRACE nova.conductor.api   File "/usr/lib/python2.6/site-packages/nova-2013.1-py2.6.egg/nova/conductor/rpcapi.py", line 85, in ping
  2013-02-01 22:36:29.575 14827 TRACE nova.conductor.api     return self.call(context, msg, version='1.22', timeout=timeout)
  2013-02-01 22:36:29.575 14827 TRACE nova.conductor.api   File "/usr/lib/python2.6/site-packages/nova-2013.1-py2.6.egg/nova/openstack/common/rpc/proxy.py", line 80, in call
  2013-02-01 22:36:29.575 14827 TRACE nova.conductor.api     return rpc.call(context, self._get_topic(topic), msg, timeout)
  2013-02-01 22:36:29.575 14827 TRACE nova.conductor.api   File "/usr/lib/python2.6/site-packages/nova-2013.1-py2.6.egg/nova/openstack/common/rpc/__init__.py", line 109, in call
  2013-02-01 22:36:29.575 14827 TRACE nova.conductor.api     return _get_impl().call(cfg.CONF, context, topic, msg, timeout)
  2013-02-01 22:36:29.575 14827 TRACE nova.conductor.api   File "/usr/lib/python2.6/site-packages/nova-2013.1-py2.6.egg/nova/openstack/common/rpc/impl_kombu.py", line 765, in call
  2013-02-01 22:36:29.575 14827 TRACE nova.conductor.api     rpc_amqp.get_connection_pool(conf, Connection))
  2013-02-01 22:36:29.575 14827 TRACE nova.conductor.api   File "/usr/lib/python2.6/site-packages/nova-2013.1-py2.6.egg/nova/openstack/common/rpc/amqp.py", line 379, in call
  2013-02-01 22:36:29.575 14827 TRACE nova.conductor.api     rv = list(rv)
  2013-02-01 22:36:29.575 14827 TRACE nova.conductor.api   File "/usr/lib/python2.6/site-packages/nova-2013.1-py2.6.egg/nova/openstack/common/rpc/amqp.py", line 340, in __iter__
  2013-02-01 22:36:29.575 14827 TRACE nova.conductor.api     self.done()
  2013-02-01 22:36:29.575 14827 TRACE nova.conductor.api   File "/usr/lib64/python2.6/contextlib.py", line 23, in __exit__
  2013-02-01 22:36:29.575 14827 TRACE nova.conductor.api     self.gen.next()
  2013-02-01 22:36:29.575 14827 TRACE nova.conductor.api   File "/usr/lib/python2.6/site-packages/nova-2013.1-py2.6.egg/nova/openstack/common/rpc/amqp.py", line 337, in __iter__
  2013-02-01 22:36:29.575 14827 TRACE nova.conductor.api     self._iterator.next()
  2013-02-01 22:36:29.575 14827 TRACE nova.conductor.api   File "/usr/lib/python2.6/site-packages/nova-2013.1-py2.6.egg/nova/openstack/common/rpc/impl_kombu.py", line 639, in iterconsume
  2013-02-01 22:36:29.575 14827 TRACE nova.conductor.api     yield self.ensure(_error_callback, _consume)
  2013-02-01 22:36:29.575 14827 TRACE nova.conductor.api   File "/usr/lib/python2.6/site-packages/nova-2013.1-py2.6.egg/nova/openstack/common/rpc/impl_kombu.py", line 557, in ensure
  2013-02-01 22:36:29.575 14827 TRACE nova.conductor.api     error_callback(e)
  2013-02-01 22:36:29.575 14827 TRACE nova.conductor.api   File "/usr/lib/python2.6/site-packages/nova-2013.1-py2.6.egg/nova/openstack/common/rpc/impl_kombu.py", line 620, in _error_callback
  2013-02-01 22:36:29.575 14827 TRACE nova.conductor.api     raise rpc_common.Timeout()
  2013-02-01 22:36:29.575 14827 TRACE nova.conductor.api Timeout: Timeout while waiting on RPC response.
  2013-02-01 22:36:29.575 14827 TRACE nova.conductor.api

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1113120/+subscriptions
  Thread Previous • Date Previous • • Thread Next