yahoo-eng-team team mailing list archive

[Flavio Percoco <1251757@xxxxxxxxxxxxxxxxxx>]

** Also affects: nova
   Importance: Undecided
       Status: New

** Also affects: heat
   Importance: Undecided
       Status: New

** Also affects: cinder
   Importance: Undecided
       Status: New

** Also affects: ceilometer
   Importance: Undecided
       Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1251757

Title:
  On restart of QPID broker, fanout no longer works

Status in OpenStack Telemetry (Ceilometer):
  New
Status in Cinder:
  New
Status in Orchestration API (Heat):
  New
Status in OpenStack Compute (Nova):
  New
Status in Oslo - a Library of Common OpenStack Code:
  Fix Released
Status in Messaging API for OpenStack:
  Fix Released

Bug description:
  When the QPID broker is restarted, RPC servers attempt to re-connect.
  This re-connection process is not done correctly for fanout
  subscriptions - two subscriptions are established to the same fanout
  address.

  This problem is compounded by the fix to bug#1178375
  https://bugs.launchpad.net/oslo/+bug/1178375

  With this bug fix, when topology version 2 is used, the reconnect
  attempt uses a malformed subscriber address.

  For example, I have a simple RPC server script that attempts to
  service "my-topic".   When it initially connects to the broker using
  topology-version 1, these are the subscriptions that are established:

  (py27)[kgiusti@t530 work (master)]$ ./my-server.py --topology=1 --auto-delete server-02
  Running server, name=server-02 exchange=my-exchange topic=my-topic namespace=my-namespace
  Using QPID topology version 1
  Enable auto-delete
  Recevr openstack/my-topic ; {"node": {"x-declare": {"auto-delete": true, "durable": true}, "type": "topic"}, "create": "always", "link": {"x-declare": {"auto-delete": true, "exclusive": false, "durable": false}, "durable": true, "name": "my-topic"}}
  Recevr openstack/my-topic.server-02 ; {"node": {"x-declare": {"auto-delete": true, "durable": true}, "type": "topic"}, "create": "always", "link": {"x-declare": {"auto-delete": true, "exclusive": false, "durable": false}, "durable": true, "name": "my-topic.server-02"}}
  Recevr my-topic_fanout ; {"node": {"x-declare": {"auto-delete": true, "durable": false, "type": "fanout"}, "type": "topic"}, "create": "always", "link": {"x-declare": {"auto-delete": true, "exclusive": true, "durable": false}, "durable": true, "name": "my-topic_fanout_489a3178fc704123b0e5e2fbee125247"}}

  When I restart the qpid broker, the server reconnects using the
  following subscriptions

  Recevr my-topic_fanout ; {"node": {"x-declare": {"auto-delete": true, "durable": false, "type": "fanout"}, "type": "topic"}, "create": "always", "link": {"x-declare": {"auto-delete": true, "exclusive": true, "durable": false}, "durable": true, "name": "my-topic_fanout_b40001afd9d946a582ead3b7b858b588"}}
  Recevr my-topic_fanout ; {"node": {"x-declare": {"auto-delete": true, "durable": false, "type": "fanout"}, "type": "topic"}, "create": "always", "link": {"x-declare": {"auto-delete": true, "exclusive": true, "durable": false}, "durable": true, "name": "my-topic_fanout_b40001afd9d946a582ead3b7b858b588"}}
  ^^^^--- Note: subscribing twice to the same exclusive address!  (Bad!)
  Recevr openstack/my-topic.server-02 ; {"node": {"x-declare": {"auto-delete": true, "durable": true}, "type": "topic"}, "create": "always", "link": {"x-declare": {"auto-delete": true, "exclusive": false, "durable": false}, "durable": true, "name": "my-topic.server-02"}}
  Recevr openstack/my-topic ; {"node": {"x-declare": {"auto-delete": true, "durable": true}, "type": "topic"}, "create": "always", "link": {"x-declare": {"auto-delete": true, "exclusive": false, "durable": false}, "durable": true, "name": "my-topic"}}

  
  When using topology=2, the failure case is a bit different.  On reconnect, the fanout addresses are lacking proper topic names:

  Recevr amq.topic/topic/openstack/my-topic ; {"link": {"x-declare": {"auto-delete": true, "durable": false}}}
  Recevr amq.topic/fanout/ ; {"link": {"x-declare": {"auto-delete": true, "exclusive": true}}}
  Recevr amq.topic/fanout/ ; {"link": {"x-declare": {"auto-delete": true, "exclusive": true}}}
  Recevr amq.topic/topic/openstack/my-topic.server-02 ; {"link": {"x-declare": {"auto-delete": true, "durable": false}}}

  Note again - two subscriptions to fanout, and 'my-topic' is missing
  (it should be after that trailing /)

  FYI - my test RPC server and client can be accessed here:
  https://github.com/kgiusti/oslo-messaging-clients

To manage notifications about this bug go to:
https://bugs.launchpad.net/ceilometer/+bug/1251757/+subscriptions