yahoo-eng-team team mailing list archive

[Justinas Balciunas <1883659@xxxxxxxxxxxxxxxxxx>]

Public bug reported:

Using: keystone-17.0.0, Ussuri
 
I've noticed a very odd behaviour of keystone_authtoken with memcached and neutron-server. The connection count to memcached grows over time, ignoring the settings of memcache_pool_maxsize and memcache_pool_unused_timeout. The keystone_authtoken middleware configuration is as follows:

[keystone_authtoken]
www_authenticate_uri = http://keystone_vip:5000
auth_url = http://keystone_vip:35357
auth_type = password
project_domain_id = default
user_domain_id = default
project_name = service
username = neutron
password = neutron_password_here
cafile =
memcache_security_strategy = ENCRYPT
memcache_secret_key = secret_key_here
memcached_servers = memcached_server_1:11211,memcached_server_2:11211,memcached_server_3:11211
memcache_pool_maxsize = 100
memcache_pool_unused_timeout = 600
token_cache_time = 3600

Commenting out memcached settings under [keystone_authtoken] and
restarting neutron-server drops the connection count in memcached to
normal levels, i.e. hundreds, rather than thousands when neutron-server
is using memcached. Neutron team (slaweq) suggested this is a Keystone
issue because quote: "Neutron is just using keystonemiddleware as one of
the middlewares in the pipeline".

Grafana memcached connection graphs: https://ibb.co/p3TCJqC AND
https://ibb.co/nmmvvH4

The drops in the graphs indicate the restart of the neutron-server, so
not sure if this is something to be expected, or there is an issue with
the configuration, or it's a bug?

** Affects: keystone
     Importance: Undecided
         Status: New

** Summary changed:

- keystonemiddleware connections to memcached from neutron-server grows beyond configured values
+ keystonemiddleware connections to memcached from neutron-server grow beyond configured values

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Identity (keystone).
https://bugs.launchpad.net/bugs/1883659

Title:
  keystonemiddleware connections to memcached from neutron-server grow
  beyond configured values

Status in OpenStack Identity (keystone):
  New

Bug description:
  Using: keystone-17.0.0, Ussuri
   
  I've noticed a very odd behaviour of keystone_authtoken with memcached and neutron-server. The connection count to memcached grows over time, ignoring the settings of memcache_pool_maxsize and memcache_pool_unused_timeout. The keystone_authtoken middleware configuration is as follows:

  [keystone_authtoken]
  www_authenticate_uri = http://keystone_vip:5000
  auth_url = http://keystone_vip:35357
  auth_type = password
  project_domain_id = default
  user_domain_id = default
  project_name = service
  username = neutron
  password = neutron_password_here
  cafile =
  memcache_security_strategy = ENCRYPT
  memcache_secret_key = secret_key_here
  memcached_servers = memcached_server_1:11211,memcached_server_2:11211,memcached_server_3:11211
  memcache_pool_maxsize = 100
  memcache_pool_unused_timeout = 600
  token_cache_time = 3600

  Commenting out memcached settings under [keystone_authtoken] and
  restarting neutron-server drops the connection count in memcached to
  normal levels, i.e. hundreds, rather than thousands when neutron-
  server is using memcached. Neutron team (slaweq) suggested this is a
  Keystone issue because quote: "Neutron is just using
  keystonemiddleware as one of the middlewares in the pipeline".

  Grafana memcached connection graphs: https://ibb.co/p3TCJqC AND
  https://ibb.co/nmmvvH4

  The drops in the graphs indicate the restart of the neutron-server, so
  not sure if this is something to be expected, or there is an issue
  with the configuration, or it's a bug?

To manage notifications about this bug go to:
https://bugs.launchpad.net/keystone/+bug/1883659/+subscriptions