yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1260782] [NEW] continuous rpc errors

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Robert Pothier <rpothier@xxxxxxxxx>
Date: Fri, 13 Dec 2013 16:06:42 -0000
Reply-to: Bug 1260782 <1260782@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
Public bug reported:

Seeing continuous RPC errors with Havana.


2013-12-13 15:34:37.621 8619 ERROR neutron.openstack.common.rpc.common [-] Failed to publish message to topic 'reply_e4c0169d68c64218a798b508002f32ee': [Errno 104] Connection reset by peer
2013-12-13 15:34:37.621 8619 TRACE neutron.openstack.common.rpc.common Traceback (most recent call last):
2013-12-13 15:34:37.621 8619 TRACE neutron.openstack.common.rpc.common   File "/usr/lib/python2.7/dist-packages/neutron/openstack/common/rpc/impl_kombu.py", line 565, in ensure
2013-12-13 15:34:37.621 8619 TRACE neutron.openstack.common.rpc.common     return method(*args, **kwargs)
2013-12-13 15:34:37.621 8619 TRACE neutron.openstack.common.rpc.common   File "/usr/lib/python2.7/dist-packages/neutron/openstack/common/rpc/impl_kombu.py", line 676, in _publish
2013-12-13 15:34:37.621 8619 TRACE neutron.openstack.common.rpc.common     publisher = cls(self.conf, self.channel, topic, **kwargs)
2013-12-13 15:34:37.621 8619 TRACE neutron.openstack.common.rpc.common   File "/usr/lib/python2.7/dist-packages/neutron/openstack/common/rpc/impl_kombu.py", line 332, in __init__
2013-12-13 15:34:37.621 8619 TRACE neutron.openstack.common.rpc.common     type='direct', **options)
2013-12-13 15:34:37.621 8619 TRACE neutron.openstack.common.rpc.common   File "/usr/lib/python2.7/dist-packages/neutron/openstack/common/rpc/impl_kombu.py", line 298, in __init__
2013-12-13 15:34:37.621 8619 TRACE neutron.openstack.common.rpc.common     self.reconnect(channel)
2013-12-13 15:34:37.621 8619 TRACE neutron.openstack.common.rpc.common   File "/usr/lib/python2.7/dist-packages/neutron/openstack/common/rpc/impl_kombu.py", line 306, in reconnect
2013-12-13 15:34:37.621 8619 TRACE neutron.openstack.common.rpc.common     routing_key=self.routing_key)
2013-12-13 15:34:37.621 8619 TRACE neutron.openstack.common.rpc.common   File "/usr/local/lib/python2.7/dist-packages/kombu/messaging.py", line 82, in __init__
2013-12-13 15:34:37.621 8619 TRACE neutron.openstack.common.rpc.common     self.revive(self._channel)
2013-12-13 15:34:37.621 8619 TRACE neutron.openstack.common.rpc.common   File "/usr/local/lib/python2.7/dist-packages/kombu/messaging.py", line 216, in revive
2013-12-13 15:34:37.621 8619 TRACE neutron.openstack.common.rpc.common     self.declare()
2013-12-13 15:34:37.621 8619 TRACE neutron.openstack.common.rpc.common   File "/usr/local/lib/python2.7/dist-packages/kombu/messaging.py", line 102, in declare
2013-12-13 15:34:37.621 8619 TRACE neutron.openstack.common.rpc.common     self.exchange.declare()
2013-12-13 15:34:37.621 8619 TRACE neutron.openstack.common.rpc.common   File "/usr/local/lib/python2.7/dist-packages/kombu/entity.py", line 166, in declare
2013-12-13 15:34:37.621 8619 TRACE neutron.openstack.common.rpc.common     nowait=nowait, passive=passive,
2013-12-13 15:34:37.621 8619 TRACE neutron.openstack.common.rpc.common   File "/usr/local/lib/python2.7/dist-packages/amqp/channel.py", line 604, in exchange_declare
2013-12-13 15:34:37.621 8619 TRACE neutron.openstack.common.rpc.common     self._send_method((40, 10), args)
2013-12-13 15:34:37.621 8619 TRACE neutron.openstack.common.rpc.common   File "/usr/local/lib/python2.7/dist-packages/amqp/abstract_channel.py", line 62, in _send_method
2013-12-13 15:34:37.621 8619 TRACE neutron.openstack.common.rpc.common     self.channel_id, method_sig, args, content,
2013-12-13 15:34:37.621 8619 TRACE neutron.openstack.common.rpc.common   File "/usr/local/lib/python2.7/dist-packages/amqp/method_framing.py", line 227, in write_method
2013-12-13 15:34:37.621 8619 TRACE neutron.openstack.common.rpc.common     write_frame(1, channel, payload)
2013-12-13 15:34:37.621 8619 TRACE neutron.openstack.common.rpc.common   File "/usr/local/lib/python2.7/dist-packages/amqp/transport.py", line 183, in write_frame
2013-12-13 15:34:37.621 8619 TRACE neutron.openstack.common.rpc.common     frame_type, channel, size, payload, 0xce,
2013-12-13 15:34:37.621 8619 TRACE neutron.openstack.common.rpc.common   File "/usr/lib/python2.7/dist-packages/eventlet/greenio.py", line 307, in sendall
2013-12-13 15:34:37.621 8619 TRACE neutron.openstack.common.rpc.common     tail = self.send(data, flags)
2013-12-13 15:34:37.621 8619 TRACE neutron.openstack.common.rpc.common   File "/usr/lib/python2.7/dist-packages/eventlet/greenio.py", line 293, in send
2013-12-13 15:34:37.621 8619 TRACE neutron.openstack.common.rpc.common     total_sent += fd.send(data[total_sent:], flags)
2013-12-13 15:34:37.621 8619 TRACE neutron.openstack.common.rpc.common error: [Errno 104] Connection reset by peer
2013-12-13 15:34:37.621 8619 TRACE neutron.openstack.common.rpc.common

** Affects: neutron
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1260782

Title:
  continuous rpc errors

Status in OpenStack Neutron (virtual network service):
  New

Bug description:
  Seeing continuous RPC errors with Havana.


  2013-12-13 15:34:37.621 8619 ERROR neutron.openstack.common.rpc.common [-] Failed to publish message to topic 'reply_e4c0169d68c64218a798b508002f32ee': [Errno 104] Connection reset by peer
  2013-12-13 15:34:37.621 8619 TRACE neutron.openstack.common.rpc.common Traceback (most recent call last):
  2013-12-13 15:34:37.621 8619 TRACE neutron.openstack.common.rpc.common   File "/usr/lib/python2.7/dist-packages/neutron/openstack/common/rpc/impl_kombu.py", line 565, in ensure
  2013-12-13 15:34:37.621 8619 TRACE neutron.openstack.common.rpc.common     return method(*args, **kwargs)
  2013-12-13 15:34:37.621 8619 TRACE neutron.openstack.common.rpc.common   File "/usr/lib/python2.7/dist-packages/neutron/openstack/common/rpc/impl_kombu.py", line 676, in _publish
  2013-12-13 15:34:37.621 8619 TRACE neutron.openstack.common.rpc.common     publisher = cls(self.conf, self.channel, topic, **kwargs)
  2013-12-13 15:34:37.621 8619 TRACE neutron.openstack.common.rpc.common   File "/usr/lib/python2.7/dist-packages/neutron/openstack/common/rpc/impl_kombu.py", line 332, in __init__
  2013-12-13 15:34:37.621 8619 TRACE neutron.openstack.common.rpc.common     type='direct', **options)
  2013-12-13 15:34:37.621 8619 TRACE neutron.openstack.common.rpc.common   File "/usr/lib/python2.7/dist-packages/neutron/openstack/common/rpc/impl_kombu.py", line 298, in __init__
  2013-12-13 15:34:37.621 8619 TRACE neutron.openstack.common.rpc.common     self.reconnect(channel)
  2013-12-13 15:34:37.621 8619 TRACE neutron.openstack.common.rpc.common   File "/usr/lib/python2.7/dist-packages/neutron/openstack/common/rpc/impl_kombu.py", line 306, in reconnect
  2013-12-13 15:34:37.621 8619 TRACE neutron.openstack.common.rpc.common     routing_key=self.routing_key)
  2013-12-13 15:34:37.621 8619 TRACE neutron.openstack.common.rpc.common   File "/usr/local/lib/python2.7/dist-packages/kombu/messaging.py", line 82, in __init__
  2013-12-13 15:34:37.621 8619 TRACE neutron.openstack.common.rpc.common     self.revive(self._channel)
  2013-12-13 15:34:37.621 8619 TRACE neutron.openstack.common.rpc.common   File "/usr/local/lib/python2.7/dist-packages/kombu/messaging.py", line 216, in revive
  2013-12-13 15:34:37.621 8619 TRACE neutron.openstack.common.rpc.common     self.declare()
  2013-12-13 15:34:37.621 8619 TRACE neutron.openstack.common.rpc.common   File "/usr/local/lib/python2.7/dist-packages/kombu/messaging.py", line 102, in declare
  2013-12-13 15:34:37.621 8619 TRACE neutron.openstack.common.rpc.common     self.exchange.declare()
  2013-12-13 15:34:37.621 8619 TRACE neutron.openstack.common.rpc.common   File "/usr/local/lib/python2.7/dist-packages/kombu/entity.py", line 166, in declare
  2013-12-13 15:34:37.621 8619 TRACE neutron.openstack.common.rpc.common     nowait=nowait, passive=passive,
  2013-12-13 15:34:37.621 8619 TRACE neutron.openstack.common.rpc.common   File "/usr/local/lib/python2.7/dist-packages/amqp/channel.py", line 604, in exchange_declare
  2013-12-13 15:34:37.621 8619 TRACE neutron.openstack.common.rpc.common     self._send_method((40, 10), args)
  2013-12-13 15:34:37.621 8619 TRACE neutron.openstack.common.rpc.common   File "/usr/local/lib/python2.7/dist-packages/amqp/abstract_channel.py", line 62, in _send_method
  2013-12-13 15:34:37.621 8619 TRACE neutron.openstack.common.rpc.common     self.channel_id, method_sig, args, content,
  2013-12-13 15:34:37.621 8619 TRACE neutron.openstack.common.rpc.common   File "/usr/local/lib/python2.7/dist-packages/amqp/method_framing.py", line 227, in write_method
  2013-12-13 15:34:37.621 8619 TRACE neutron.openstack.common.rpc.common     write_frame(1, channel, payload)
  2013-12-13 15:34:37.621 8619 TRACE neutron.openstack.common.rpc.common   File "/usr/local/lib/python2.7/dist-packages/amqp/transport.py", line 183, in write_frame
  2013-12-13 15:34:37.621 8619 TRACE neutron.openstack.common.rpc.common     frame_type, channel, size, payload, 0xce,
  2013-12-13 15:34:37.621 8619 TRACE neutron.openstack.common.rpc.common   File "/usr/lib/python2.7/dist-packages/eventlet/greenio.py", line 307, in sendall
  2013-12-13 15:34:37.621 8619 TRACE neutron.openstack.common.rpc.common     tail = self.send(data, flags)
  2013-12-13 15:34:37.621 8619 TRACE neutron.openstack.common.rpc.common   File "/usr/lib/python2.7/dist-packages/eventlet/greenio.py", line 293, in send
  2013-12-13 15:34:37.621 8619 TRACE neutron.openstack.common.rpc.common     total_sent += fd.send(data[total_sent:], flags)
  2013-12-13 15:34:37.621 8619 TRACE neutron.openstack.common.rpc.common error: [Errno 104] Connection reset by peer
  2013-12-13 15:34:37.621 8619 TRACE neutron.openstack.common.rpc.common

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1260782/+subscriptions
Follow ups

[Bug 1260782] Re: continuous rpc errors
From: Launchpad Bug Tracker, 2014-05-12
[Bug 1260782] [NEW] continuous rpc errors
From: Robert Pothier, 2013-12-13
References

[Bug 1260782] [NEW] continuous rpc errors
From: Robert Pothier, 2013-12-13