yahoo-eng-team team mailing list archive

[Assaf Muller <amuller@xxxxxxxxxx>]

Public bug reported:

Several patches merged as a part of:
https://review.openstack.org/#/q/status:merged+project:openstack/neutron+branch:master+topic:bp/rpc-docs-and-namespaces,n,z

Broke Neutron rolling upgrade (Specifically: Upgrading the server(s)
before the agents or vice versa). This was done knowingly and discussed
in the spec process. While we don't test a rolling upgrade scenario,
there is no reason to break it knowingly. I've spoken to operators that
have successfully performed such an upgrade from I to J and it will be
very surprising to them if the same doesn't work from J to K.

The breakage comes from the introduction of RPC namespaces, a very
useful concept of putting RPC endpoints in separate namespaces. i.e. you
may place the same method name listening in the same process if it
belongs to different namespaces.

Possible solutions:
Have the server listen on both the new namespaces, and in the root namespace. However, this effectively brings all such methods into one big namespace, so I don't see the point. Why not revert and forgo the usage of namespaces altogether? We could delay by making a change in Oslo messaging where if a new and optional backwards_compatibility flag is passed in to a target along with a namespace, then the dispatcher will check against the namespace as well as the root namespace, and we simply stop passing the flag in the L cycle (This means that we only support rolling upgrades from version N to N+1).

Testing:
I've been working on basic RPC tests:
https://review.openstack.org/#/q/status:open+project:openstack/neutron+branch:master+topic:rpc_tests,n,z

But I don't think such a framework will allow us to test rolling
upgrades. I can't think of an alternative to actually performing one and
seeing what happens (A spin on the grenade job).

** Affects: neutron
     Importance: Undecided
     Assignee: Assaf Muller (amuller)
         Status: New

** Changed in: neutron
     Assignee: (unassigned) => Assaf Muller (amuller)

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1430984

Title:
  Recent RPC namespacing breaks rolling upgrades

Status in OpenStack Neutron (virtual network service):
  New

Bug description:
  Several patches merged as a part of:
  https://review.openstack.org/#/q/status:merged+project:openstack/neutron+branch:master+topic:bp/rpc-docs-and-namespaces,n,z

  Broke Neutron rolling upgrade (Specifically: Upgrading the server(s)
  before the agents or vice versa). This was done knowingly and
  discussed in the spec process. While we don't test a rolling upgrade
  scenario, there is no reason to break it knowingly. I've spoken to
  operators that have successfully performed such an upgrade from I to J
  and it will be very surprising to them if the same doesn't work from J
  to K.

  The breakage comes from the introduction of RPC namespaces, a very
  useful concept of putting RPC endpoints in separate namespaces. i.e.
  you may place the same method name listening in the same process if it
  belongs to different namespaces.

  Possible solutions:
  Have the server listen on both the new namespaces, and in the root namespace. However, this effectively brings all such methods into one big namespace, so I don't see the point. Why not revert and forgo the usage of namespaces altogether? We could delay by making a change in Oslo messaging where if a new and optional backwards_compatibility flag is passed in to a target along with a namespace, then the dispatcher will check against the namespace as well as the root namespace, and we simply stop passing the flag in the L cycle (This means that we only support rolling upgrades from version N to N+1).

  Testing:
  I've been working on basic RPC tests:
  https://review.openstack.org/#/q/status:open+project:openstack/neutron+branch:master+topic:rpc_tests,n,z

  But I don't think such a framework will allow us to test rolling
  upgrades. I can't think of an alternative to actually performing one
  and seeing what happens (A spin on the grenade job).

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1430984/+subscriptions