yahoo-eng-team team mailing list archive

[Sam Morrison <1687616@xxxxxxxxxxxxxxxxxx>]

I have just done the N -> O upgrade and have seen this error.

We have done the expand and migrate db syncs.

We have 3 newton keystones and when I added an ocata one I saw this
issue on the ocata one.

Its happening on a POST to /v3/auth/tokens and is affecting about 3% of
requests (we have around 10 requests per second on our keystone)

Happy to provide more information.

Currently I have rolled back but am thinking this might just be an issue
during the transition so could bite the bullet and do it quickly.

** Changed in: keystone
       Status: Expired => New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Identity (keystone).
https://bugs.launchpad.net/bugs/1687616

Title:
  KeyError 'options' while doing zero downtime upgrade from N to O

Status in OpenStack Identity (keystone):
  New

Bug description:
  I am trying to do a zero downtime upgrade from N release to O release
  following [1].

  I have 3 controller nodes running behind a HAProxy. Everytime, when I
  upgraded one of the keystone and bring it back to the cluster, it
  would encounter this error [2] when I tried to update a created user
  for about 5 minutes. After I brought back all the 3 upgraded keystone
  nodes, and 5 or more minutes later, this error will disappear and
  everything works fine.

  I am using the same conf file for both releases as shown in [3].

  [1]. https://docs.openstack.org/keystone/latest/admin/identity-upgrading.html
  [2]. http://paste.openstack.org/show/608557/
  [3]. http://paste.openstack.org/show/608558/

To manage notifications about this bug go to:
https://bugs.launchpad.net/keystone/+bug/1687616/+subscriptions