← Back to team overview

fuel-dev team mailing list archive

memcache issue

 

Hi guys,

Today we faced with the issue on the lab created using Fuel.

Problem:
On Horizon there were a lot of pop-ups with "Unauthorized" error. When we
tried to send REST-requests only several first attempts were successful.
E.g. http://paste.openstack.org/show/54123/ (successful attempt, the second
one was "Unauthorized"). The problem appeared only with nova. No issue with
cli.

Investigations:
>From the logs:
Nov 13 06:58:49 controller-1461 nova keystoneclient.middleware.auth_token
DEBUG Token validation failure.
Traceback (most recent call last):
  File
"/usr/lib/python2.6/site-packages/keystoneclient/middleware/auth_token.py",
line 684, in _validate_u
ser_token
    cached = self._cache_get(token_id)
  File
"/usr/lib/python2.6/site-packages/keystoneclient/middleware/auth_token.py",
line 898, in _cache_get
    raise InvalidUserToken('Token authorization failed')
InvalidUserToken: Token authorization failed
Nov 13 06:58:49 controller-1461 nova keystoneclient.middleware.auth_token
DEBUG Marking token 211b590c4ba94d62a3981fbf91e934dc as unauthorized in
memcache

Memcache restarting didn't help. Issues've gone when we turned memcache off.

Fix:
As it turned out a cluster with no memcache works slow. So the fix is to
add additional configs to nova.conf:
[keystone_authtoken]
token_cache_time=300
memcache_security_strategy=ENCRYPT
memcache_secret_key=hjhs445

All secrets keys are equal in all controllers. More configs you may find
here
https://github.com/openstack/python-keystoneclient/blob/master/keystoneclient/middleware/auth_token.py#L273.

I decided to let you know about this issue. Maybe it is needed to add
additional configuration to Fuel. If this is our mistake during lab setup -
I'm sorry for inconvenience :)

Thanks for attention,
Nadya

Follow ups