openstack team mailing list archive
Mailing list archive
On 07/29/2012 03:24 AM, Lars Kellogg-Stedman wrote:
> Our environment has connection-tracking firewalls that drop idle
> connections after an hour. There is a connection between nova-compute
> and our qpidd server that appears to be idle for long periods of time.
> When the firewall drops this connection, the participating hosts are
> unaware of that fact and ultimately stop communicating with each other
> until we restart nova-compute.
> I was hoping that the qpid_heartbeat parameter would avoid this
> problem by keeping the connection active, but despite having
> qpid_heartbeat set explicitly in our configuration...
> # This is supposed to be the default
> qpid_heartbeat = 5
> ...there is no traffic across this connection
> I can deal with this problem by forcing (via libkeepalive,
> http://libkeepalive.sourceforge.net) SO_KEEPALIVE on the AMQ sockets
> (and tuning the net.ipv4.tcp_keepalive_time sysctl to be < the
> firewall connection timeout), but that seems a bit of a hack. It's
> also possible to work around this by disabling idle connection
> timeouts on the firewall, so we're not completely stymied...
> ...but I would like to understand why setting qpid_heartbeat does not,
> in fact, result in the regular transmission of heartbeat packets
> across the connection.
> We're running openstack-nova-2012.1.1-0.20120615.13614 from EPEL (and
> qpid 0.14).
Looks like a typo.
Could you try this.
diff --git a/nova/rpc/impl_qpid.py b/nova/rpc/impl_qpid.py
index 289f21b..e19079e 100644
@@ -317,7 +317,7 @@ class Connection(object):
self.connection.reconnect_interval = FLAGS.qpid_reconnect_interval
- self.connection.hearbeat = FLAGS.qpid_heartbeat
+ self.connection.heartbeat = FLAGS.qpid_heartbeat
self.connection.protocol = FLAGS.qpid_protocol
self.connection.tcp_nodelay = FLAGS.qpid_tcp_nodelay