openstack team mailing list archive

[Lars Kellogg-Stedman <lars@xxxxxxxxxxxxxxxx>]

Our environment has connection-tracking firewalls that drop idle
connections after an hour.  There is a connection between nova-compute
and our qpidd server that appears to be idle for long periods of time.

When the firewall drops this connection, the participating hosts are
unaware of that fact and ultimately stop communicating with each other
until we restart nova-compute.

I was hoping that the qpid_heartbeat parameter would avoid this
problem by keeping the connection active, but despite having
qpid_heartbeat set explicitly in our configuration...

  # This is supposed to be the default
  qpid_heartbeat = 5

...there is no traffic across this connection

I can deal with this problem by forcing (via libkeepalive,
http://libkeepalive.sourceforge.net) SO_KEEPALIVE on the AMQ sockets
(and tuning the net.ipv4.tcp_keepalive_time sysctl to be < the
firewall connection timeout), but that seems a bit of a hack.  It's
also possible to work around this by disabling idle connection
timeouts on the firewall, so we're not completely stymied...

...but I would like to understand why setting qpid_heartbeat does not,
in fact, result in the regular transmission of heartbeat packets
across the connection.

We're running openstack-nova-2012.1.1-0.20120615.13614 from EPEL (and
qpid 0.14).

Thanks,

-- 
Lars Kellogg-Stedman <lars@xxxxxxxxxxxxxxxx>       |
Senior Technologist                                | http://ac.seas.harvard.edu/
Academic Computing                                 | http://code.seas.harvard.edu/
Harvard School of Engineering and Applied Sciences |