yahoo-eng-team team mailing list archive

[Kui Shi <skuicloud@xxxxxxxxx>]

Public bug reported:

If we choose memcached as the token backend driver, we should set proper
servers in '[memcache]' section of keystone.conf.

For the memcache backend, there are 3  memcached client supported by dogpile, 
---------------------------------------------------
# pydoc dogpile.cache.backends.memcached 
 
   dogpile.cache.api.CacheBackend(__builtin__.object)
        GenericMemcachedBackend
            BMemcachedBackend
            MemcachedBackend(MemcacheArgs, GenericMemcachedBackend)
            PylibmcBackend(MemcacheArgs, GenericMemcachedBackend)

----------------------------------------------------


In keystone, they are listed in keystone/common/kvs/backends/memcached.py:

VALID_DOGPILE_BACKENDS = dict(pylibmc=memcached.PylibmcBackend,
                              bmemcached=memcached.BMemcachedBackend,
                              memcached=memcached.MemcachedBackend)


By default, memcached will be used. The "memcached_backend" option will determine which backend to use.
But there is no simple to pass the "memcached_backend" option.


Here is a solution for this:
-------------------------------------------------------------------------------
diff --git a/doc/source/developing.rst b/doc/source/developing.rst
index 9cc2ee9..3dc66ad 100644
--- a/doc/source/developing.rst
+++ b/doc/source/developing.rst
@@ -531,8 +531,10 @@ memcached servers, lock_timeout, etc).
 
 The memcached backend uses the Keystone manager mechanism to support the use of any of the
 provided memcached backends (``bmemcached``, ``pylibmc``, and basic ``memcached``).
-By default the ``memcached`` backend is used.  Currently the Memcache URLs come from the
-``servers`` option in the ``[memcache]`` configuration section of the Keystone config.
+By default the ``memcached`` backend is used.  An alternative is to specify the
+``memcached_backend`` option in the ``[memcache]`` configuration section. Currently the
+Memcache URLs come from the ``servers`` option in the ``[memcache]`` configuration section
+of the Keystone config.
 
 The following is an example showing how to configure the KVS system to use a
 KeyValueStore object named "TestKVSRegion" and a specific Memcached driver:
diff --git a/etc/keystone.conf.sample b/etc/keystone.conf.sample
index 88a2a73..1353d66 100644
--- a/etc/keystone.conf.sample
+++ b/etc/keystone.conf.sample
@@ -1179,6 +1179,12 @@
 # Memcache servers in the format of "host:port". (list value)
 #servers=localhost:11211
 
+# Memcached backend driver. (string value)
+# Currnetly, three memcached backends are supported:
+# bmemcached, pylibmc, memcached.
+# By default the ``memcached`` backend is used.
+#memcached_backend=memcached
+
 # Number of compare-and-set attempts to make when using
 # compare-and-set in the token memcache back end. (integer
 # value)
diff --git a/keystone/common/config.py b/keystone/common/config.py
index f694c51..d9e9bc6 100644
--- a/keystone/common/config.py
+++ b/keystone/common/config.py
@@ -750,6 +750,9 @@ FILE_OPTIONS = {
                         'the available pipelines.'),
     ],
     'memcache': [
+        cfg.StrOpt('memcached_backend', default='memcached',
+                   help='Name of specific dogpile.cache memcached driver: '
+                        'memcached, bmemcached, pylibmc.'),
         cfg.ListOpt('servers', default=['localhost:11211'],
                     help='Memcache servers in the format of "host:port".'),
         cfg.IntOpt('max_compare_and_set_retry', default=16,
diff --git a/keystone/common/kvs/backends/memcached.py b/keystone/common/kvs/backends/memcached.py
index 7950827..03d70e4 100644
--- a/keystone/common/kvs/backends/memcached.py
+++ b/keystone/common/kvs/backends/memcached.py
@@ -78,7 +78,7 @@ class MemcachedBackend(manager.Manager):
     """Pivot point to leverage the various dogpile.cache memcached backends.
 
     To specify a specific dogpile.cache memcached driver, pass the argument
-    `memcached_driver` set to one of the provided memcached drivers (at this
+    `memcached_backend` set to one of the provided memcached drivers (at this
     time `memcached`, `bmemcached`, `pylibmc` are valid).
     """
     def __init__(self, arguments):
diff --git a/keystone/token/persistence/backends/memcache.py b/keystone/token/persistence/backends/memcache.py
index 01d9f06..f59b58c 100644
--- a/keystone/token/persistence/backends/memcache.py
+++ b/keystone/token/persistence/backends/memcache.py
@@ -27,4 +27,5 @@ class Token(kvs.Token):
         kwargs['no_expiry_keys'] = [self.revocation_key]
         kwargs['memcached_expire_time'] = CONF.token.expiration
         kwargs['url'] = CONF.memcache.servers
+        kwargs['memcached_backend'] = CONF.memcache.memcached_backend
         super(Token, self).__init__(*args, **kwargs)

** Affects: keystone
     Importance: Undecided
     Assignee: Kui Shi (skuicloud)
         Status: New

** Changed in: keystone
     Assignee: (unassigned) => Kui Shi (skuicloud)

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to Keystone.
https://bugs.launchpad.net/bugs/1360553

Title:
  Add memcached_backend option in keystone.conf

Status in OpenStack Identity (Keystone):
  New

Bug description:
  If we choose memcached as the token backend driver, we should set
  proper servers in '[memcache]' section of keystone.conf.

  For the memcache backend, there are 3  memcached client supported by dogpile, 
  ---------------------------------------------------
  # pydoc dogpile.cache.backends.memcached 
   
     dogpile.cache.api.CacheBackend(__builtin__.object)
          GenericMemcachedBackend
              BMemcachedBackend
              MemcachedBackend(MemcacheArgs, GenericMemcachedBackend)
              PylibmcBackend(MemcacheArgs, GenericMemcachedBackend)

  ----------------------------------------------------

  
  In keystone, they are listed in keystone/common/kvs/backends/memcached.py:

  VALID_DOGPILE_BACKENDS = dict(pylibmc=memcached.PylibmcBackend,
                                bmemcached=memcached.BMemcachedBackend,
                                memcached=memcached.MemcachedBackend)

  
  By default, memcached will be used. The "memcached_backend" option will determine which backend to use.
  But there is no simple to pass the "memcached_backend" option.

  
  Here is a solution for this:
  -------------------------------------------------------------------------------
  diff --git a/doc/source/developing.rst b/doc/source/developing.rst
  index 9cc2ee9..3dc66ad 100644
  --- a/doc/source/developing.rst
  +++ b/doc/source/developing.rst
  @@ -531,8 +531,10 @@ memcached servers, lock_timeout, etc).
   
   The memcached backend uses the Keystone manager mechanism to support the use of any of the
   provided memcached backends (``bmemcached``, ``pylibmc``, and basic ``memcached``).
  -By default the ``memcached`` backend is used.  Currently the Memcache URLs come from the
  -``servers`` option in the ``[memcache]`` configuration section of the Keystone config.
  +By default the ``memcached`` backend is used.  An alternative is to specify the
  +``memcached_backend`` option in the ``[memcache]`` configuration section. Currently the
  +Memcache URLs come from the ``servers`` option in the ``[memcache]`` configuration section
  +of the Keystone config.
   
   The following is an example showing how to configure the KVS system to use a
   KeyValueStore object named "TestKVSRegion" and a specific Memcached driver:
  diff --git a/etc/keystone.conf.sample b/etc/keystone.conf.sample
  index 88a2a73..1353d66 100644
  --- a/etc/keystone.conf.sample
  +++ b/etc/keystone.conf.sample
  @@ -1179,6 +1179,12 @@
   # Memcache servers in the format of "host:port". (list value)
   #servers=localhost:11211
   
  +# Memcached backend driver. (string value)
  +# Currnetly, three memcached backends are supported:
  +# bmemcached, pylibmc, memcached.
  +# By default the ``memcached`` backend is used.
  +#memcached_backend=memcached
  +
   # Number of compare-and-set attempts to make when using
   # compare-and-set in the token memcache back end. (integer
   # value)
  diff --git a/keystone/common/config.py b/keystone/common/config.py
  index f694c51..d9e9bc6 100644
  --- a/keystone/common/config.py
  +++ b/keystone/common/config.py
  @@ -750,6 +750,9 @@ FILE_OPTIONS = {
                           'the available pipelines.'),
       ],
       'memcache': [
  +        cfg.StrOpt('memcached_backend', default='memcached',
  +                   help='Name of specific dogpile.cache memcached driver: '
  +                        'memcached, bmemcached, pylibmc.'),
           cfg.ListOpt('servers', default=['localhost:11211'],
                       help='Memcache servers in the format of "host:port".'),
           cfg.IntOpt('max_compare_and_set_retry', default=16,
  diff --git a/keystone/common/kvs/backends/memcached.py b/keystone/common/kvs/backends/memcached.py
  index 7950827..03d70e4 100644
  --- a/keystone/common/kvs/backends/memcached.py
  +++ b/keystone/common/kvs/backends/memcached.py
  @@ -78,7 +78,7 @@ class MemcachedBackend(manager.Manager):
       """Pivot point to leverage the various dogpile.cache memcached backends.
   
       To specify a specific dogpile.cache memcached driver, pass the argument
  -    `memcached_driver` set to one of the provided memcached drivers (at this
  +    `memcached_backend` set to one of the provided memcached drivers (at this
       time `memcached`, `bmemcached`, `pylibmc` are valid).
       """
       def __init__(self, arguments):
  diff --git a/keystone/token/persistence/backends/memcache.py b/keystone/token/persistence/backends/memcache.py
  index 01d9f06..f59b58c 100644
  --- a/keystone/token/persistence/backends/memcache.py
  +++ b/keystone/token/persistence/backends/memcache.py
  @@ -27,4 +27,5 @@ class Token(kvs.Token):
           kwargs['no_expiry_keys'] = [self.revocation_key]
           kwargs['memcached_expire_time'] = CONF.token.expiration
           kwargs['url'] = CONF.memcache.servers
  +        kwargs['memcached_backend'] = CONF.memcache.memcached_backend
           super(Token, self).__init__(*args, **kwargs)

To manage notifications about this bug go to:
https://bugs.launchpad.net/keystone/+bug/1360553/+subscriptions