openstack team mailing list archive

[Eric Windisch <eric@xxxxxxxxxxxxxxxx>]

We actually do have ACKs in ZeroMQ, as far as I understand how they work in AMQP, but they're really simple. The send() method is actually synchronous with the message being received on the other end. However, we don't wait for this and spawn an eventlet coroutine, because there is no benefit of blocking the caller.

All calls have a timeout (TTL). The ZeroMQ driver also implements a TTL on the casts, and I'm quite sure we should support this in Kombu/Qpid as well to avoid a thundering-herd.

While there is no message persistence in the ZeroMQ driver, there is some limited benefit of having this on casts. There is limited or no benefit for calls, because the return value won't be received -- the calling stack is no longer going to do anything with the return value. (This would be a good case for a better Actor-driven model, because we could actually handle return values across a relaunched caller)

Anyway, in the ZeroMQ driver, we could have a local queue to track casts and remove them when the send() coroutine completes.  This would provide restart protection for casts. 

-- 
Eric Windisch


On Tuesday, June 12, 2012 at 09:55 AM, Johannes Erdfelt wrote:

> As part of a patch to add idempotency to the xenapi driver, I've
> modified impl_kombu driver to implement delayed ACKs. This is so RPC
> messages are retried after nova-compute restarts.
> 
> https://review.openstack.org/#/c/7323/
> 
> However, this only works with impl_kombu.
> 
> impl_qpid does not ACK anything, although since it's AMQP based there
> should be an ACK somewhere, I just don't see it in impl_qpid or the Qpid
> module so I must be missing something.
> 
> 0MQ does not have any concept of ACKing messages, which makes sense
> since there is no message persistence.
> 
> I'm not aware of any formal discussion on what the RPC layer semantics
> are, but a set of implied semantics have developed. In particular, that
> messages in Openstack cannot reliably be persistent at the RPC layer.
> 
> My changes to impl_kombu are obviously not the path forward since they
> cannot be implemented in the other RPC drivers, but it's not clear what
> the best path forward is.
> 
> Is it a custom layer on top of RPC?
> 
> Is it an orchestration layer?
> 
> Something else?
> 
> Related, should queue persistence be disabled in impl_kombu?
> 
> JE
> 
> 
> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@xxxxxxxxxxxxxxxxxxx (mailto:openstack@xxxxxxxxxxxxxxxxxxx)
> Unsubscribe : https://launchpad.net/~openstack
> More help : https://help.launchpad.net/ListHelp
> 
>