yahoo-eng-team team mailing list archive
-
yahoo-eng-team team
-
Mailing list archive
-
Message #06105
[Bug 1252846] Re: instance deletion fails due to neutron client error: "simultaneous read on fileno N detected"
*** This bug is a duplicate of bug 1229475 ***
https://bugs.launchpad.net/bugs/1229475
Please make sure you have the proper neutronclient with this patch
https://github.com/openstack/python-
neutronclient/commit/e6a9f79066943b75e2c60b0165533d5886cc9cd8
** This bug is no longer a duplicate of bug 1251920
Tempest failures due to failure to return console logs from an instance
** This bug has been marked a duplicate of bug 1229475
terminate_instance(): RuntimeError: Second simultaneous read on fileno 16 detected
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1252846
Title:
instance deletion fails due to neutron client error: "simultaneous
read on fileno N detected"
Status in OpenStack Compute (Nova):
New
Bug description:
Doing performance testing with the latest OpenStack code, I regularly
have a problem when deleting instances. When attempting to query
neutron during instance deallocation, nova-compute apparently makes a
greenthread-unsafe use of its neutron client:
{u'message':
"Connection to neutron failed: Second simultaneous read on fileno 25 detected. Unless you really know what you're doing, make sure that only one greenthread can read any particular socket. Consider using a pools.Pool. If you do know what you're doing and",
u'code': 500,
u'details': u'
File "/opt/stack/nova/nova/compute/manager.py", line 259, in decorated_function
return function(self, context, *args, **kwargs)
File "/opt/stack/nova/nova/compute/manager.py", line 1984, in terminate_instance
do_terminate_instance(instance, bdms)
File "/opt/stack/nova/nova/openstack/common/lockutils.py", line 248, in inner
return f(*args, **kwargs)
File "/opt/stack/nova/nova/compute/manager.py", line 1976, in do_terminate_instance
reservations=reservations)
File "/opt/stack/nova/nova/hooks.py", line 105, in inner
rv = f(*args, **kwargs)
File "/opt/stack/nova/nova/compute/manager.py", line 1949, in _delete_instance
user_id=user_id)
File "/opt/stack/nova/nova/compute/manager.py", line 1919, in _delete_instance
self._shutdown_instance(context, db_inst, bdms)
File "/opt/stack/nova/nova/compute/manager.py", line 1854, in _shutdown_instance
self._try_deallocate_network(context, instance, requested_networks)
File "/opt/stack/nova/nova/compute/manager.py", line 1814, in _try_deallocate_network
self._set_instance_error_state(context, instance[\'uuid\'])
File "/opt/stack/nova/nova/compute/manager.py", line 1809, in _try_deallocate_network
self._deallocate_network(context, instance, requested_networks)
File "/opt/stack/nova/nova/compute/manager.py", line 1492, in _deallocate_network
context, instance, requested_networks=requested_networks)
File "/opt/stack/nova/nova/network/neutronv2/api.py", line 404, in deallocate_for_instance
data = neutronv2.get_client(context).list_ports(**search_opts)
File "/opt/stack/python-neutronclient/neutronclient/v2_0/client.py", line 111, in with_params
ret = self.function(instance, *args, **kwargs)
File "/opt/stack/python-neutronclient/neutronclient/v2_0/client.py", line 306, in list_ports
**_params)
File "/opt/stack/python-neutronclient/neutronclient/v2_0/client.py", line 1253, in list
for r in self._pagination(collection, path, **params):
File "/opt/stack/python-neutronclient/neutronclient/v2_0/client.py", line 1266, in _pagination
res = self.get(path, params=params)
File "/opt/stack/python-neutronclient/neutronclient/v2_0/client.py", line 1239, in get
headers=headers, params=params)
File "/opt/stack/python-neutronclient/neutronclient/v2_0/client.py", line 1221, in retry_request
headers=headers, params=params)
File "/opt/stack/python-neutronclient/neutronclient/v2_0/client.py", line 1156, in do_request
resp, replybody = self.httpclient.do_request(action, method, body=body)
File "/opt/stack/python-neutronclient/neutronclient/client.py", line 190, in do_request
**kwargs)
File "/opt/stack/python-neutronclient/neutronclient/client.py", line 152, in _cs_request
raise exceptions.ConnectionFailed(reason=e)
', u'created': u'2013-11-19T19:36:11Z'}
Note that I had to make a minor hack to my python-neutronclient to get
this exception:
diff --git a/neutronclient/v2_0/client.py b/neutronclient/v2_0/client.py
index 733f339..d2ac237 100644
--- a/neutronclient/v2_0/client.py
+++ b/neutronclient/v2_0/client.py
@@ -1224,6 +1224,8 @@ class Client(object):
if i < self.retries:
_logger.debug(_('Retrying connection to Neutron service'))
time.sleep(self.retry_interval)
+ else:
+ raise
raise exceptions.ConnectionFailed(reason=_("Maximum attempts reached"))
Without that patch, you just see the error "Maximum attempts reached,"
which doesn't tell you anything.
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1252846/+subscriptions