yahoo-eng-team team mailing list archive

[OpenStack Infra <1572062@xxxxxxxxxxxxxxxxxx>]

Reviewed:  https://review.openstack.org/307718
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=59192cfbf9482fe3798cdcd4c2044d8c85f66e95
Submitter: Jenkins
Branch:    master

commit 59192cfbf9482fe3798cdcd4c2044d8c85f66e95
Author: Jens Rosenboom <j.rosenboom@xxxxxxxx>
Date:   Tue Apr 19 13:18:52 2016 +0200

    Warn when using null cache backend
    
    The default backend for oslo_cache is dogpile.cache.null, leading to a
    broken setup when following the current release notes or the deprecation
    message verbatim. So we should at least emit a warning when the config
    does not specify a different backend.
    
    Note: I'm not sure whether it is possible to amend the release note like
    this or whether there needs a new note to be added, please advise.
    
    Change-Id: I16647e424d8382dae98f13cb1f73a7e0c0aebaf5
    Closes-Bug: 1572062


** Changed in: nova
       Status: In Progress => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1572062

Title:
  nova-consoleauth doesn't play well with memcached

Status in OpenStack Compute (nova):
  Fix Released

Bug description:
  When running with the Ubuntu Xenial Mitaka packages, I'm seeing the
  following behaviour:

  Following the release notes at
  http://docs.openstack.org/releasenotes/nova/mitaka.html I remove the
  old option "memcached_servers" and set up a cache section with

  [cache]
  enabled = true
  memcache_servers = host1:11211,host2:11211,host3:11211

  The result is that there are errors logged when creating a token:

  2016-04-19 10:17:33.501 15952 WARNING nova.consoleauth.manager [req-ad043de9-616d-4cbb-a93f-fc5e345f1f53 1b0106d4f339406bb7012a83162ba5f2 e3c253d3e8344a8796e70bc4f96b6166 - - -] Token: dd0e7bbc-c593-421f-a56b-52578aaec20e failed to save into memcached.
  2016-04-19 10:17:33.503 15952 WARNING nova.consoleauth.manager [req-ad043de9-616d-4cbb-a93f-fc5e345f1f53 1b0106d4f339406bb7012a83162ba5f2 e3c253d3e8344a8796e70bc4f96b6166 - - -] Instance: d3267abb-258d-4b5d-a22e-8e3b0a39905c failed to save into memcached

  Only after a lot of debugging it turns out that the default backend
  for oslo_cache is dogpile.cache.null, implying that no values get
  cached and token validation always fails. Only after adding

  [cache]
  backend = oslo_cache.memcache_pool

  does console authentication start working. Strangely though, the above
  warning messages are still being logged in the working setup, which
  made debugging this even more difficult.

  So I suggest the following fixes:

  1. Change the text of the warnings from "failed to save into memcached" to "failed to save into cache", as with the change to using oslo_cache, there may be other backends in use instead of memcached.
  2. Either override the default of using the null backend or refuse to run with it or at the very least give a big fat warning that the configuration can not work.
  3. Stop generating the warning messages when the data got in fact saved into cache properly.

  Package versions for reference:
  # dpkg -l | grep nova
  ii  nova-api-metadata                    2:13.0.0-0ubuntu2               all          OpenStack Compute - metadata API frontend
  ii  nova-api-os-compute                  2:13.0.0-0ubuntu2               all          OpenStack Compute - OpenStack Compute API frontend
  ii  nova-cert                            2:13.0.0-0ubuntu2               all          OpenStack Compute - certificate management
  ii  nova-common                          2:13.0.0-0ubuntu2               all          OpenStack Compute - common files
  ii  nova-conductor                       2:13.0.0-0ubuntu2               all          OpenStack Compute - conductor service
  ii  nova-consoleauth                     2:13.0.0-0ubuntu2               all          OpenStack Compute - Console Authenticator
  ii  nova-scheduler                       2:13.0.0-0ubuntu2               all          OpenStack Compute - virtual machine scheduler
  ii  nova-spiceproxy                      2:13.0.0-0ubuntu2               all          OpenStack Compute - spice html5 proxy
  ii  python-nova                          2:13.0.0-0ubuntu2               all          OpenStack Compute Python libraries
  ii  python-novaclient                    2:3.3.1-2                       all          client library for OpenStack Compute API - Python 2.7
  # dpkg -l | grep oslo
  ii  python-oslo.cache                    1.6.0-2                         all          cache storage for Openstack projects - Python 2.7
  ii  python-oslo.concurrency              3.7.0-2                         all          concurrency and locks for OpenStack projects - Python 2.x
  ii  python-oslo.config                   1:3.9.0-3                       all          Common code for Openstack Projects (configuration API) - Python 2.x
  ii  python-oslo.context                  2.2.0-2                         all          WSGI context helpers for OpenStack - Python 2.x
  ii  python-oslo.db                       4.7.0-2ubuntu1                  all          database connectivity to the different backends and helper utils - Python 2.x
  ii  python-oslo.i18n                     3.5.0-2                         all          Oslo Internationalization Utilities - Python 2.x
  ii  python-oslo.log                      3.2.0-2                         all          OpenStack logging configuration library - Python 2.x
  ii  python-oslo.messaging                4.6.1-2ubuntu1                  all          oslo messaging library - Python 2.x
  ii  python-oslo.middleware               3.8.0-2                         all          various WSGI middleware components for OpenStack - Python 2.x
  ii  python-oslo.policy                   1.6.0-2                         all          RBAC policy enforcement library for OpenStack - Python 2.x
  ii  python-oslo.reports                  1.7.0-2                         all          reports serialized in various data types - Python 2.7
  ii  python-oslo.rootwrap                 4.1.0-2                         all          allows fine filtering of shell commands to run as root - Python 2.x
  ii  python-oslo.serialization            2.4.0-2                         all          utilities for serialization , especially JSON - Python 2.x
  ii  python-oslo.service                  1.8.0-1ubuntu1                  all          library for running OpenStack services - Python 2.x
  ii  python-oslo.utils                    3.8.0-2                         all          set of utility functions for OpenStack - Python 2.x
  ii  python-oslo.versionedobjects         1.8.0-1                         all          deals with DB schema versions and code expectations - Python 2.x
  ii  python-oslo.vmware                   2.5.0-2                         all          VMware library for OpenStack projects - Python 2.7

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1572062/+subscriptions