yahoo-eng-team team mailing list archive

[Tomas Stodulka <1895723@xxxxxxxxxxxxxxxxxx>]

Public bug reported:

After restart of keystone's container, it keeps restarting. I found only this error in docker logs keystone:
Running command: '/usr/bin/keystone-startup.sh -DFOREGROUND'
+ exec /usr/bin/keystone-startup.sh -DFOREGROUND
+ set -o errexit
+ set -o pipefail
+ TOKEN_DIR=/etc/keystone/fernet-keys
+ n=0
+ '[' '!' -f /etc/keystone/fernet-keys/0 ']'
++ ls -1 /etc/keystone/fernet-keys
++ sort -hr
++ head -n 1
+ TOKEN_PRIMARY=5
++ date +%s
++ date +%s -r /etc/keystone/fernet-keys/5
+ TOKEN_AGE=589164
+ '[' 589164 -gt 86400 ']'
+ echo 'ERROR: Primary token 5 is stale.'
+ exit 1

Workaround is change expiration from 86400 to 864000 in
/etc/kolla/keystone/keystone-startup.sh:

# Compare if it's older than fernet_token_expiry and run key rotation if needed
if [ "${TOKEN_AGE}" -gt "864000" ]; then
    echo "ERROR: Primary token ${TOKEN_PRIMARY} is stale."
    exit 1
fi

Regarding the comment in code, It should also run rotation of primary
key. But this part is missing, it only throws an exception as mentioned.
Or I would like to ask, why the primary key wasn't rotated automatically
when it was needed.

I am using 2 weeks old deployment of Ussuri, deployd by kolla-ansible on
CentOS8.

** Affects: keystone
     Importance: Undecided
         Status: New

** Attachment added: "keystone.log"
   https://bugs.launchpad.net/bugs/1895723/+attachment/5411121/+files/keystone.log

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Identity (keystone).
https://bugs.launchpad.net/bugs/1895723

Title:
  Keystone is restarting due to stale primary key

Status in OpenStack Identity (keystone):
  New

Bug description:
  After restart of keystone's container, it keeps restarting. I found only this error in docker logs keystone:
  Running command: '/usr/bin/keystone-startup.sh -DFOREGROUND'
  + exec /usr/bin/keystone-startup.sh -DFOREGROUND
  + set -o errexit
  + set -o pipefail
  + TOKEN_DIR=/etc/keystone/fernet-keys
  + n=0
  + '[' '!' -f /etc/keystone/fernet-keys/0 ']'
  ++ ls -1 /etc/keystone/fernet-keys
  ++ sort -hr
  ++ head -n 1
  + TOKEN_PRIMARY=5
  ++ date +%s
  ++ date +%s -r /etc/keystone/fernet-keys/5
  + TOKEN_AGE=589164
  + '[' 589164 -gt 86400 ']'
  + echo 'ERROR: Primary token 5 is stale.'
  + exit 1

  Workaround is change expiration from 86400 to 864000 in
  /etc/kolla/keystone/keystone-startup.sh:

  # Compare if it's older than fernet_token_expiry and run key rotation if needed
  if [ "${TOKEN_AGE}" -gt "864000" ]; then
      echo "ERROR: Primary token ${TOKEN_PRIMARY} is stale."
      exit 1
  fi

  Regarding the comment in code, It should also run rotation of primary
  key. But this part is missing, it only throws an exception as
  mentioned. Or I would like to ask, why the primary key wasn't rotated
  automatically when it was needed.

  I am using 2 weeks old deployment of Ussuri, deployd by kolla-ansible
  on CentOS8.

To manage notifications about this bug go to:
https://bugs.launchpad.net/keystone/+bug/1895723/+subscriptions