yahoo-eng-team team mailing list archive

[OpenStack Infra <1630259@xxxxxxxxxxxxxxxxxx>]

Reviewed:  https://review.openstack.org/382032
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=4fd55f230b69d305ceb98e3066766868f63a7e09
Submitter: Jenkins
Branch:    master

commit 4fd55f230b69d305ceb98e3066766868f63a7e09
Author: Lance Bragstad <lbragstad@xxxxxxxxx>
Date:   Tue Oct 4 18:25:44 2016 +0000

    Make returning is_domain conditional
    
    During an upgrade, a node running this code may need to handle a
    persisted token (UUID, PKI, or PKIZ) created without this attribute.
    
    Closes-Bug: 1630259
    Change-Id: I0c5959b6491bb13a02eb1b9b7e7e37d2f2d73f85


** Changed in: keystone
       Status: In Progress => Fix Released

** Tags added: in-stable-newton

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Identity (keystone).
https://bugs.launchpad.net/bugs/1630259

Title:
  KeyError: 'is_domain' during mitaka -> newton rolling upgrade

Status in OpenStack Identity (keystone):
  Fix Released

Bug description:
  I have 3 Controller nodes running HA active/active mode. Using Mysql-server as shared database.
  After upgrade Controller1, I start it to handle the request to make the system no downtime.
  But when a request is handling by Controller1, an error happended: "There is either no auth token in the request or the certificate issuer is not trusted. No auth context will be set". Keystone raise that: KeyError: 'is_domain' 

  How to reproduce:
  Follow this guide: http://docs.openstack.org/developer/keystone/upgrading.html#upgrading-without-downtime

  # Controller1
  $ sudo service apache2 stop

  $ cd /opt/stack/keystone/
  $ git checkout remotes/origin/stable/newton
  $ git checkout -b stable/newton remotes/origin/stable/newton
  $ sudo pip install -r requirements.txt --upgrade

  $ keystone-manage doctor
  $ keystone-manage db_sync --expand
  $ keystone-manage db_sync --migrate
  $ sudo python setup.py install
  $ sudo service apache2 start

  # Controller2 or any openstack clients
  $ for i in {1..10}; do openstack neutron network list; done
  ...
  503 Service Unavailable
  The server is currently unavailable. Please try again at a later time
  ...

  Full log in kestone here: http://paste.openstack.org/show/584107/

  After I upgraded all 3 Controller nodes follow the same above steps
  except upgrading db, the error never occurs again.

  At step 9 in the guideline: "Upgrade all keystone nodes to the next release, and restart them one at a time..."
  I think we will have downtime in this process. So I tried to upgrade controller1 first, then make it online to ensure that the system have not downtime.

To manage notifications about this bug go to:
https://bugs.launchpad.net/keystone/+bug/1630259/+subscriptions