yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1229475] Re: terminate_instance(): RuntimeError: Second simultaneous read on fileno 16 detected

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Thierry Carrez <thierry.carrez+lp@xxxxxxxxx>
Date: Wed, 02 Oct 2013 10:38:35 -0000
Reply-to: Bug 1229475 <1229475@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
No need for a nova patch

** Changed in: nova
       Status: Triaged => In Progress

** Changed in: nova
       Status: In Progress => Invalid

** No longer affects: oslo

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1229475

Title:
  terminate_instance(): RuntimeError: Second simultaneous read on fileno
  16 detected

Status in OpenStack Compute (Nova):
  Invalid
Status in Python client library for Neutron:
  In Progress

Bug description:
  http://logs.openstack.org/59/47659/6/check/gate-tempest-devstack-vm-
  neutron/fc83d44/logs/screen-n-cpu.txt.gz#_2013-09-23_18_25_06_484


  2013-09-23 18:25:06.484 ERROR nova.openstack.common.rpc.amqp [req-3a80ff5a-817b-4eb8-be67-7f180bef8a6e demo demo] Exception during message handling
  2013-09-23 18:25:06.484 3476 TRACE nova.openstack.common.rpc.amqp Traceback (most recent call last):
  2013-09-23 18:25:06.484 3476 TRACE nova.openstack.common.rpc.amqp   File "/opt/stack/new/nova/nova/openstack/common/rpc/amqp.py", line 461, in _process_data
  2013-09-23 18:25:06.484 3476 TRACE nova.openstack.common.rpc.amqp     **args)
  2013-09-23 18:25:06.484 3476 TRACE nova.openstack.common.rpc.amqp   File "/opt/stack/new/nova/nova/openstack/common/rpc/dispatcher.py", line 172, in dispatch
  2013-09-23 18:25:06.484 3476 TRACE nova.openstack.common.rpc.amqp     result = getattr(proxyobj, method)(ctxt, **kwargs)
  2013-09-23 18:25:06.484 3476 TRACE nova.openstack.common.rpc.amqp   File "/opt/stack/new/nova/nova/compute/manager.py", line 354, in decorated_function
  2013-09-23 18:25:06.484 3476 TRACE nova.openstack.common.rpc.amqp     return function(self, context, *args, **kwargs)
  2013-09-23 18:25:06.484 3476 TRACE nova.openstack.common.rpc.amqp   File "/opt/stack/new/nova/nova/exception.py", line 90, in wrapped
  2013-09-23 18:25:06.484 3476 TRACE nova.openstack.common.rpc.amqp     payload)
  2013-09-23 18:25:06.484 3476 TRACE nova.openstack.common.rpc.amqp   File "/opt/stack/new/nova/nova/exception.py", line 73, in wrapped
  2013-09-23 18:25:06.484 3476 TRACE nova.openstack.common.rpc.amqp     return f(self, context, *args, **kw)
  2013-09-23 18:25:06.484 3476 TRACE nova.openstack.common.rpc.amqp   File "/opt/stack/new/nova/nova/compute/manager.py", line 244, in decorated_function
  2013-09-23 18:25:06.484 3476 TRACE nova.openstack.common.rpc.amqp     pass
  2013-09-23 18:25:06.484 3476 TRACE nova.openstack.common.rpc.amqp   File "/opt/stack/new/nova/nova/compute/manager.py", line 230, in decorated_function
  2013-09-23 18:25:06.484 3476 TRACE nova.openstack.common.rpc.amqp     return function(self, context, *args, **kwargs)
  2013-09-23 18:25:06.484 3476 TRACE nova.openstack.common.rpc.amqp   File "/opt/stack/new/nova/nova/compute/manager.py", line 295, in decorated_function
  2013-09-23 18:25:06.484 3476 TRACE nova.openstack.common.rpc.amqp     function(self, context, *args, **kwargs)
  2013-09-23 18:25:06.484 3476 TRACE nova.openstack.common.rpc.amqp   File "/opt/stack/new/nova/nova/compute/manager.py", line 272, in decorated_function
  2013-09-23 18:25:06.484 3476 TRACE nova.openstack.common.rpc.amqp     e, sys.exc_info())
  2013-09-23 18:25:06.484 3476 TRACE nova.openstack.common.rpc.amqp   File "/opt/stack/new/nova/nova/compute/manager.py", line 259, in decorated_function
  2013-09-23 18:25:06.484 3476 TRACE nova.openstack.common.rpc.amqp     return function(self, context, *args, **kwargs)
  2013-09-23 18:25:06.484 3476 TRACE nova.openstack.common.rpc.amqp   File "/opt/stack/new/nova/nova/compute/manager.py", line 1793, in terminate_instance
  2013-09-23 18:25:06.484 3476 TRACE nova.openstack.common.rpc.amqp     do_terminate_instance(instance, bdms)
  2013-09-23 18:25:06.484 3476 TRACE nova.openstack.common.rpc.amqp   File "/opt/stack/new/nova/nova/openstack/common/lockutils.py", line 246, in inner
  2013-09-23 18:25:06.484 3476 TRACE nova.openstack.common.rpc.amqp     return f(*args, **kwargs)
  2013-09-23 18:25:06.484 3476 TRACE nova.openstack.common.rpc.amqp   File "/opt/stack/new/nova/nova/compute/manager.py", line 1785, in do_terminate_instance
  2013-09-23 18:25:06.484 3476 TRACE nova.openstack.common.rpc.amqp     reservations=reservations)
  2013-09-23 18:25:06.484 3476 TRACE nova.openstack.common.rpc.amqp   File "/opt/stack/new/nova/nova/hooks.py", line 105, in inner
  2013-09-23 18:25:06.484 3476 TRACE nova.openstack.common.rpc.amqp     rv = f(*args, **kwargs)
  2013-09-23 18:25:06.484 3476 TRACE nova.openstack.common.rpc.amqp   File "/opt/stack/new/nova/nova/compute/manager.py", line 1758, in _delete_instance
  2013-09-23 18:25:06.484 3476 TRACE nova.openstack.common.rpc.amqp     user_id=user_id)
  2013-09-23 18:25:06.484 3476 TRACE nova.openstack.common.rpc.amqp   File "/opt/stack/new/nova/nova/compute/manager.py", line 1727, in _delete_instance
  2013-09-23 18:25:06.484 3476 TRACE nova.openstack.common.rpc.amqp     self.conductor_api.instance_info_cache_delete(context, db_inst)
  2013-09-23 18:25:06.484 3476 TRACE nova.openstack.common.rpc.amqp   File "/opt/stack/new/nova/nova/conductor/api.py", line 107, in instance_info_cache_delete
  2013-09-23 18:25:06.484 3476 TRACE nova.openstack.common.rpc.amqp     return self._manager.instance_info_cache_delete(context, instance)
  2013-09-23 18:25:06.484 3476 TRACE nova.openstack.common.rpc.amqp   File "/opt/stack/new/nova/nova/conductor/rpcapi.py", line 312, in instance_info_cache_delete
  2013-09-23 18:25:06.484 3476 TRACE nova.openstack.common.rpc.amqp     cctxt.call(context, 'instance_info_cache_delete', instance=instance_p)
  2013-09-23 18:25:06.484 3476 TRACE nova.openstack.common.rpc.amqp   File "/opt/stack/new/nova/nova/rpcclient.py", line 85, in call
  2013-09-23 18:25:06.484 3476 TRACE nova.openstack.common.rpc.amqp     return self._invoke(self.proxy.call, ctxt, method, **kwargs)
  2013-09-23 18:25:06.484 3476 TRACE nova.openstack.common.rpc.amqp   File "/opt/stack/new/nova/nova/rpcclient.py", line 63, in _invoke
  2013-09-23 18:25:06.484 3476 TRACE nova.openstack.common.rpc.amqp     return cast_or_call(ctxt, msg, **self.kwargs)
  2013-09-23 18:25:06.484 3476 TRACE nova.openstack.common.rpc.amqp   File "/opt/stack/new/nova/nova/openstack/common/rpc/proxy.py", line 126, in call
  2013-09-23 18:25:06.484 3476 TRACE nova.openstack.common.rpc.amqp     result = rpc.call(context, real_topic, msg, timeout)
  2013-09-23 18:25:06.484 3476 TRACE nova.openstack.common.rpc.amqp   File "/opt/stack/new/nova/nova/openstack/common/rpc/__init__.py", line 139, in call
  2013-09-23 18:25:06.484 3476 TRACE nova.openstack.common.rpc.amqp     return _get_impl().call(CONF, context, topic, msg, timeout)
  2013-09-23 18:25:06.484 3476 TRACE nova.openstack.common.rpc.amqp   File "/opt/stack/new/nova/nova/openstack/common/rpc/impl_kombu.py", line 816, in call
  2013-09-23 18:25:06.484 3476 TRACE nova.openstack.common.rpc.amqp     rpc_amqp.get_connection_pool(conf, Connection))
  2013-09-23 18:25:06.484 3476 TRACE nova.openstack.common.rpc.amqp   File "/opt/stack/new/nova/nova/openstack/common/rpc/amqp.py", line 572, in call
  2013-09-23 18:25:06.484 3476 TRACE nova.openstack.common.rpc.amqp     rv = multicall(conf, context, topic, msg, timeout, connection_pool)
  2013-09-23 18:25:06.484 3476 TRACE nova.openstack.common.rpc.amqp   File "/opt/stack/new/nova/nova/openstack/common/rpc/amqp.py", line 565, in multicall
  2013-09-23 18:25:06.484 3476 TRACE nova.openstack.common.rpc.amqp     with ConnectionContext(conf, connection_pool) as conn:
  2013-09-23 18:25:06.484 3476 TRACE nova.openstack.common.rpc.amqp   File "/opt/stack/new/nova/nova/openstack/common/rpc/amqp.py", line 122, in __init__
  2013-09-23 18:25:06.484 3476 TRACE nova.openstack.common.rpc.amqp     self.connection = connection_pool.get()
  2013-09-23 18:25:06.484 3476 TRACE nova.openstack.common.rpc.amqp   File "/usr/local/lib/python2.7/dist-packages/eventlet/pools.py", line 119, in get
  2013-09-23 18:25:06.484 3476 TRACE nova.openstack.common.rpc.amqp     created = self.create()
  2013-09-23 18:25:06.484 3476 TRACE nova.openstack.common.rpc.amqp   File "/opt/stack/new/nova/nova/openstack/common/rpc/amqp.py", line 76, in create
  2013-09-23 18:25:06.484 3476 TRACE nova.openstack.common.rpc.amqp     return self.connection_cls(self.conf)
  2013-09-23 18:25:06.484 3476 TRACE nova.openstack.common.rpc.amqp   File "/opt/stack/new/nova/nova/openstack/common/rpc/impl_kombu.py", line 465, in __init__
  2013-09-23 18:25:06.484 3476 TRACE nova.openstack.common.rpc.amqp     self.reconnect()
  2013-09-23 18:25:06.484 3476 TRACE nova.openstack.common.rpc.amqp   File "/opt/stack/new/nova/nova/openstack/common/rpc/impl_kombu.py", line 535, in reconnect
  2013-09-23 18:25:06.484 3476 TRACE nova.openstack.common.rpc.amqp     self._connect(params)
  2013-09-23 18:25:06.484 3476 TRACE nova.openstack.common.rpc.amqp   File "/opt/stack/new/nova/nova/openstack/common/rpc/impl_kombu.py", line 511, in _connect
  2013-09-23 18:25:06.484 3476 TRACE nova.openstack.common.rpc.amqp     self.connection.connect()
  2013-09-23 18:25:06.484 3476 TRACE nova.openstack.common.rpc.amqp   File "/usr/local/lib/python2.7/dist-packages/kombu/connection.py", line 246, in connect
  2013-09-23 18:25:06.484 3476 TRACE nova.openstack.common.rpc.amqp     return self.connection
  2013-09-23 18:25:06.484 3476 TRACE nova.openstack.common.rpc.amqp   File "/usr/local/lib/python2.7/dist-packages/kombu/connection.py", line 761, in connection
  2013-09-23 18:25:06.484 3476 TRACE nova.openstack.common.rpc.amqp     self._connection = self._establish_connection()
  2013-09-23 18:25:06.484 3476 TRACE nova.openstack.common.rpc.amqp   File "/usr/local/lib/python2.7/dist-packages/kombu/connection.py", line 720, in _establish_connection
  2013-09-23 18:25:06.484 3476 TRACE nova.openstack.common.rpc.amqp     conn = self.transport.establish_connection()
  2013-09-23 18:25:06.484 3476 TRACE nova.openstack.common.rpc.amqp   File "/usr/local/lib/python2.7/dist-packages/kombu/transport/pyamqp.py", line 115, in establish_connection
  2013-09-23 18:25:06.484 3476 TRACE nova.openstack.common.rpc.amqp     conn = self.Connection(**opts)
  2013-09-23 18:25:06.484 3476 TRACE nova.openstack.common.rpc.amqp   File "/usr/local/lib/python2.7/dist-packages/amqp/connection.py", line 142, in __init__
  2013-09-23 18:25:06.484 3476 TRACE nova.openstack.common.rpc.amqp     (10, 10),  # start
  2013-09-23 18:25:06.484 3476 TRACE nova.openstack.common.rpc.amqp   File "/usr/local/lib/python2.7/dist-packages/amqp/abstract_channel.py", line 69, in wait
  2013-09-23 18:25:06.484 3476 TRACE nova.openstack.common.rpc.amqp     self.channel_id, allowed_methods)
  2013-09-23 18:25:06.484 3476 TRACE nova.openstack.common.rpc.amqp   File "/usr/local/lib/python2.7/dist-packages/amqp/connection.py", line 204, in _wait_method
  2013-09-23 18:25:06.484 3476 TRACE nova.openstack.common.rpc.amqp     self.method_reader.read_method()
  2013-09-23 18:25:06.484 3476 TRACE nova.openstack.common.rpc.amqp   File "/usr/local/lib/python2.7/dist-packages/amqp/method_framing.py", line 192, in read_method
  2013-09-23 18:25:06.484 3476 TRACE nova.openstack.common.rpc.amqp     raise m
  2013-09-23 18:25:06.484 3476 TRACE nova.openstack.common.rpc.amqp RuntimeError: Second simultaneous read on fileno 16 detected.  Unless you really know what you're doing, make sure that only one greenthread can read any particular socket.  Consider using a pools.Pool. If you do know what you're doing and want to disable this error, call eventlet.debug.hub_prevent_multiple_readers(False)
  2013-09-23 18:25:06.484 3476 TRACE nova.openstack.common.rpc.amqp

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1229475/+subscriptions