yahoo-eng-team team mailing list archive

[Colleen Murphy <colleen@xxxxxxxxxxx>]

** Also affects: devstack
   Importance: Undecided
       Status: New

** Changed in: keystone
       Status: Confirmed => Invalid

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Identity (keystone).
https://bugs.launchpad.net/bugs/1822251

Title:
  grenade job fail due to missing memcache module

Status in devstack:
  New
Status in OpenStack Identity (keystone):
  Invalid

Bug description:
  We saw it quite often recently that grenade jobs are failing due to
  missing memcache module while keystone is starting.

  Error message:

  2019-03-29 00:22:39.361 | + lib/keystone:bootstrap_keystone:564      :   /usr/local/bin/keystone-manage bootstrap --bootstrap-username admin --bootstrap-password secretadmin --bootstrap-project-name admin --bootstrap-role-name admin --bootstrap-service-name keystone --bootstrap-region-id RegionOne --bootstrap-admin-url http://10.209.34.128/identity --bootstrap-public-url http://10.209.34.128/identity
  2019-03-29 00:22:41.201 | CRITICAL keystone [-] Unhandled error: ModuleNotFoundError: No module named 'memcache'
  2019-03-29 00:22:41.201 | ERROR keystone Traceback (most recent call last):
  2019-03-29 00:22:41.201 | ERROR keystone   File "/usr/local/bin/keystone-manage", line 10, in <module>
  2019-03-29 00:22:41.201 | ERROR keystone     sys.exit(main())
  2019-03-29 00:22:41.201 | ERROR keystone   File "/opt/stack/old/keystone/keystone/cmd/manage.py", line 41, in main
  2019-03-29 00:22:41.201 | ERROR keystone     cli.main(argv=sys.argv, developer_config_file=developer_config)
  2019-03-29 00:22:41.201 | ERROR keystone   File "/opt/stack/old/keystone/keystone/cmd/cli.py", line 1216, in main
  2019-03-29 00:22:41.201 | ERROR keystone     CONF.command.cmd_class.main()
  2019-03-29 00:22:41.201 | ERROR keystone   File "/opt/stack/old/keystone/keystone/cmd/cli.py", line 177, in main
  2019-03-29 00:22:41.201 | ERROR keystone     klass = cls()
  2019-03-29 00:22:41.201 | ERROR keystone   File "/opt/stack/old/keystone/keystone/cmd/cli.py", line 66, in __init__
  2019-03-29 00:22:41.201 | ERROR keystone     self.bootstrapper = bootstrap.Bootstrapper()
  2019-03-29 00:22:41.201 | ERROR keystone   File "/opt/stack/old/keystone/keystone/cmd/bootstrap.py", line 31, in __init__
  2019-03-29 00:22:41.201 | ERROR keystone     backends.load_backends()
  2019-03-29 00:22:41.201 | ERROR keystone   File "/opt/stack/old/keystone/keystone/server/backends.py", line 41, in load_backends
  2019-03-29 00:22:41.202 | ERROR keystone     cache.configure_cache()
  2019-03-29 00:22:41.202 | ERROR keystone   File "/opt/stack/old/keystone/keystone/common/cache/core.py", line 124, in configure_cache
  2019-03-29 00:22:41.202 | ERROR keystone     cache.configure_cache_region(CONF, region)
  2019-03-29 00:22:41.202 | ERROR keystone   File "/usr/local/lib/python3.6/dist-packages/oslo_cache/core.py", line 235, in configure_cache_region
  2019-03-29 00:22:41.202 | ERROR keystone     '%s.' % conf.cache.config_prefix)
  2019-03-29 00:22:41.202 | ERROR keystone   File "/usr/local/lib/python3.6/dist-packages/dogpile/cache/region.py", line 591, in configure_from_config
  2019-03-29 00:22:41.202 | ERROR keystone     "%sreplace_existing_backend" % prefix, False),
  2019-03-29 00:22:41.202 | ERROR keystone   File "/usr/local/lib/python3.6/dist-packages/dogpile/cache/region.py", line 423, in configure
  2019-03-29 00:22:41.202 | ERROR keystone     _config_prefix
  2019-03-29 00:22:41.202 | ERROR keystone   File "/usr/local/lib/python3.6/dist-packages/dogpile/cache/api.py", line 89, in from_config_dict
  2019-03-29 00:22:41.202 | ERROR keystone     for key in config_dict
  2019-03-29 00:22:41.202 | ERROR keystone   File "/usr/local/lib/python3.6/dist-packages/dogpile/cache/backends/memcached.py", line 208, in __init__
  2019-03-29 00:22:41.202 | ERROR keystone     super(MemcacheArgs, self).__init__(arguments)
  2019-03-29 00:22:41.202 | ERROR keystone   File "/usr/local/lib/python3.6/dist-packages/dogpile/cache/backends/memcached.py", line 108, in __init__
  2019-03-29 00:22:41.202 | ERROR keystone     self._imports()
  2019-03-29 00:22:41.202 | ERROR keystone   File "/usr/local/lib/python3.6/dist-packages/dogpile/cache/backends/memcached.py", line 287, in _imports
  2019-03-29 00:22:41.202 | ERROR keystone     import memcache  # noqa
  2019-03-29 00:22:41.202 | ERROR keystone ModuleNotFoundError: No module named 'memcache'

  
  Logstash query which I used to find similar examples: http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22ModuleNotFoundError%3A%20No%20module%20named%20'memcache'%5C%22

  Looks that it happens at least for 10 runs in last 7 days.

To manage notifications about this bug go to:
https://bugs.launchpad.net/devstack/+bug/1822251/+subscriptions