openstack team mailing list archive
-
openstack team
-
Mailing list archive
-
Message #06373
Re: reconnecting to RabbitMQ
Thanks, Ask!
amqplib versions after and including 1.0 enables SO_KEEPALIVE by default,
> and Kombu versions after and including 1.2.1 depends on amqplib >= 1.0,
> so maybe you should update your package dependencies?
devstack requied amqplib 0.6.1.
I will research the best solution more detailed.
2011/12/29 Ask Solem <ask@xxxxxxxxxxxx>
>
> On 28 Dec 2011, at 14:38, Roman Sokolkov wrote:
>
> > Hi!
> >
> > I tried to make Rabbit HA for Openstack.
> >
> > My first e-mail was little innacurate.
> >
> > I use two nodes with active/active rabbits with mirrored queues.
> Additionally I use Virtual IP. And all it managed by Pacemaker. Openstack
> uses this remote rabbitmq via VIP.
> >
> > I hardly reboot node with VIP and VIP moves to other node. In this case,
> connections on Openstack host staying forever in ESTABLISHED state(in
> py-amqlib 0.6.1 no tcp keepalive options).
> > Openstack think that all good and doesn't try to reconnect. But I if
> correctly stop RabbitMQ, connections closed correctly and Openstack will
> try to reconnect.
> >
> > I solve this issue by adding tcp_keepalive options in
> py-amqlib(transport.py).
>
> amqplib versions after and including 1.0 enables SO_KEEPALIVE by default,
> and Kombu versions after and including 1.2.1 depends on amqplib >= 1.0,
> so maybe you should update your package dependencies?
>
>
>
> >
> >
> > It's very interesting what implemented in KAMQP. And how heartbeats work?
>
> Heartbeats is functionality part of the amqp 0-8 spec.
> Heartbeats are enabled by specifying a heartbeat frequency, the broker
> will then
> send heartbeat frames at that suggested frequency, and the client will
> also have
> to send heartbeat frames back. If heartbeats are missed by either the
> broker or client
> the connection shall be disconnected.
>
> It's a very simple concept, but the problem with amqplib is that it is
> non-threaded,
> so it can be for the client to miss sending a heartbeat if it is blocking
> elsewhere. This is not a problem when using eventlet/gevent (that is
> unless the process is blocking, but that should never happen when using
> greenlets).
--
Regards, Roman Sokolkov
Follow ups
References