← Back to team overview

yahoo-eng-team team mailing list archive

  1. yahoo-eng-team team
  2. Mailing list archive
  3. Message #34820

[Bug 1462152] Re: memcache Token persistence backend does not support connecting to IPv6-only hostnames or addresses

  Thread Previous • Date Previous • • Thread Next 
Added to the top of the bug report for openstack-manuals:

OpenStack Manuals (for both Master and Kilo) need to be updated to
eliminate the recommendation to use the memcache token persistence
backend. The memcache token persistence backend is a poor choice due to
performance concerns of the code itself, the fact that it is assumed
that the token backend is stable storage (memcached is not) and can
expose security concerns if restarted in some scenarios (PKI tokens and
revoked tokens becoming valid again), and finally that the python-
memcache library is all around poor (it will not work with IPv6 and is
not Python3 compatible, among other poor design choices).


We should be recommending the SQL backend (or Fernet) with a cron to flush the tokens (if SQL is used).

** Also affects: openstack-manuals
   Importance: Undecided
       Status: New

** Tags added: documentation install-guide

** Description changed:

- As of Kilo the default Token persistence backend is memcached.
+ (morganfainberg):
+ OpenStack Manuals (for both Master and Kilo) need to be updated to eliminate the recommendation to use the memcache token persistence backend. The memcache token persistence backend is a poor choice due to performance concerns of the code itself, the fact that it is assumed that the token backend is stable storage (memcached is not) and can expose security concerns if restarted in some scenarios (PKI tokens and revoked tokens becoming valid again), and finally that the python-memcache library is all around poor (it will not work with IPv6 and is not Python3 compatible, among other poor design choices).
  
- The backend driver does not support IPv6.
+ 
+ ========================================================================
+ The memcache backend driver does not support IPv6.
  
  I have included three scenarios (A, B and C) that will reproduce the bug
  and a control test that succeeds with same configuration using
  IPv4-resolving hostname.
  
- 
  To reproduce scenario A: Bare IPv6 address in config
  1) Configure keystone according to http://docs.openstack.org/kilo/install-guide/install/apt/content/keystone-install.html
  2) In section [memcache] in /etc/keystone/keystone.conf change servers = line:
-  servers = 2001:db8:1000:1:f816:3eff:fe2a:f9c7:11211,2001:db8:1000:1:f816:3eff:fee9:9ce3:11211,2001:db8:1000:1:f816:3eff:fead:8f7f:11211
+  servers = 2001:db8:1000:1:f816:3eff:fe2a:f9c7:11211,2001:db8:1000:1:f816:3eff:fee9:9ce3:11211,2001:db8:1000:1:f816:3eff:fead:8f7f:11211
  3) Restart keystone/apache
  4) Attempt to issue token:
-  openstack --os-auth-url http://192.168.0.15:35357   --os-project-name admin --os-username admin --os-auth-type password   token issue
+  openstack --os-auth-url http://192.168.0.15:35357   --os-project-name admin --os-username admin --os-auth-type password   token issue
  
  ERROR: openstack An unexpected error prevented the server from
  fulfilling your request: Unable to parse connection string:
  "2001:db8:1000:1:f816:3eff:fe2a:f9c7:11211" (Disable debug mode to
  suppress these details.) (HTTP 500) (Request-ID: req-7c2bfd39-4b83-462b-
  92c6-f75f7677c8e5)
  
- 
  To reproduce scenario B: IPv6 address enclosed in brackets
  1) Configure keystone according to http://docs.openstack.org/kilo/install-guide/install/apt/content/keystone-install.html
  2) In section [memcache] in /etc/keystone/keystone.conf change servers = line:
-  servers = [2001:db8:1000:1:f816:3eff:fe2a:f9c7]:11211,[2001:db8:1000:1:f816:3eff:fee9:9ce3]:11211,[2001:db8:1000:1:f816:3eff:fead:8f7f]:11211
+  servers = [2001:db8:1000:1:f816:3eff:fe2a:f9c7]:11211,[2001:db8:1000:1:f816:3eff:fee9:9ce3]:11211,[2001:db8:1000:1:f816:3eff:fead:8f7f]:11211
  3) Restart keystone/apache
  4) Attempt to issue token:
-  openstack --os-auth-url http://192.168.0.15:35357   --os-project-name admin --os-username admin --os-auth-type password   token issue
+  openstack --os-auth-url http://192.168.0.15:35357   --os-project-name admin --os-username admin --os-auth-type password   token issue
  
  ERROR: openstack An unexpected error prevented the server from
  fulfilling your request: Unable to parse connection string:
  "[2001:db8:1000:1:f816:3eff:fe2a:f9c7]:11211" (Disable debug mode to
  suppress these details.) (HTTP 500) (Request-ID: req-
  869eb953-74af-4336-b3e1-dc3a417180f9)
  
- 
  To reproduce scenario C: hostname that resolves to IPv6-only address
  1) Configure keystone according to http://docs.openstack.org/kilo/install-guide/install/apt/content/keystone-install.html
  2) In section [memcache] in /etc/keystone/keystone.conf change servers = line:
-  servers = keystone-1:11211,keystone-2:11211,keystone-3:11211
+  servers = keystone-1:11211,keystone-2:11211,keystone-3:11211
  
  3) Edit /etc/hosts:
  2001:db8:1000:1:f816:3eff:fe2a:f9c7	keystone-1
  2001:db8:1000:1:f816:3eff:fee9:9ce3	keystone-2
  2001:db8:1000:1:f816:3eff:fead:8f7f	keystone-3
  
  3) Restart keystone/apache
  4) Attempt to issue token:
  
  openstack --os-auth-url http://192.168.0.15:35357   --os-project-name admin --os-username admin --os-auth-type password   token issue
- Password: 
+ Password:
  ERROR: openstack Maximum lock attempts on _lockusertokens-30dbbe8174b24174a3a24d1ae554ab17 occurred. (Disable debug mode to suppress these details.) (HTTP 500) (Request-ID: req-efd53eae-4bcf-4fd9-bab2-dd4c86fb9798)
- 
  
  Control test:
  1) Configure keystone according to http://docs.openstack.org/kilo/install-guide/install/apt/content/keystone-install.html
  2) In section [memcache] in /etc/keystone/keystone.conf change servers = line:
-  servers = keystone-1:11211,keystone-2:11211,keystone-3:11211
+  servers = keystone-1:11211,keystone-2:11211,keystone-3:11211
  
  3) Edit /etc/hosts:
  192.168.0.15	keystone-1
  192.168.0.14	keystone-2
  192.168.0.16	keystone-3
  
  3) Restart keystone/apache
  4) Attempt to issue token:
  
  openstack --os-auth-url http://192.168.0.15:35357   --os-project-name admin --os-username admin --os-auth-type password   token issue
- Password: 
+ Password:
  +------------+----------------------------------+
  | Field      | Value                            |
  +------------+----------------------------------+
  | expires    | 2015-06-05T00:31:30Z             |
  | id         | 2a188e9950f44decb78f196b5a3c3f78 |
  | project_id | 91bb6f536fca40a68fb5d4cf72527388 |
  | user_id    | 30dbbe8174b24174a3a24d1ae554ab17 |
  +------------+----------------------------------+

** Changed in: keystone
       Status: New => Triaged

** Description changed:

  (morganfainberg):
  OpenStack Manuals (for both Master and Kilo) need to be updated to eliminate the recommendation to use the memcache token persistence backend. The memcache token persistence backend is a poor choice due to performance concerns of the code itself, the fact that it is assumed that the token backend is stable storage (memcached is not) and can expose security concerns if restarted in some scenarios (PKI tokens and revoked tokens becoming valid again), and finally that the python-memcache library is all around poor (it will not work with IPv6 and is not Python3 compatible, among other poor design choices).
  
- 
  ========================================================================
- The memcache backend driver does not support IPv6.
+ The memcache backend driver that utilizes "python-memcache" does not support IPv6.
  
  I have included three scenarios (A, B and C) that will reproduce the bug
  and a control test that succeeds with same configuration using
  IPv4-resolving hostname.
  
  To reproduce scenario A: Bare IPv6 address in config
  1) Configure keystone according to http://docs.openstack.org/kilo/install-guide/install/apt/content/keystone-install.html
  2) In section [memcache] in /etc/keystone/keystone.conf change servers = line:
   servers = 2001:db8:1000:1:f816:3eff:fe2a:f9c7:11211,2001:db8:1000:1:f816:3eff:fee9:9ce3:11211,2001:db8:1000:1:f816:3eff:fead:8f7f:11211
  3) Restart keystone/apache
  4) Attempt to issue token:
   openstack --os-auth-url http://192.168.0.15:35357   --os-project-name admin --os-username admin --os-auth-type password   token issue
  
  ERROR: openstack An unexpected error prevented the server from
  fulfilling your request: Unable to parse connection string:
  "2001:db8:1000:1:f816:3eff:fe2a:f9c7:11211" (Disable debug mode to
  suppress these details.) (HTTP 500) (Request-ID: req-7c2bfd39-4b83-462b-
  92c6-f75f7677c8e5)
  
  To reproduce scenario B: IPv6 address enclosed in brackets
  1) Configure keystone according to http://docs.openstack.org/kilo/install-guide/install/apt/content/keystone-install.html
  2) In section [memcache] in /etc/keystone/keystone.conf change servers = line:
   servers = [2001:db8:1000:1:f816:3eff:fe2a:f9c7]:11211,[2001:db8:1000:1:f816:3eff:fee9:9ce3]:11211,[2001:db8:1000:1:f816:3eff:fead:8f7f]:11211
  3) Restart keystone/apache
  4) Attempt to issue token:
   openstack --os-auth-url http://192.168.0.15:35357   --os-project-name admin --os-username admin --os-auth-type password   token issue
  
  ERROR: openstack An unexpected error prevented the server from
  fulfilling your request: Unable to parse connection string:
  "[2001:db8:1000:1:f816:3eff:fe2a:f9c7]:11211" (Disable debug mode to
  suppress these details.) (HTTP 500) (Request-ID: req-
  869eb953-74af-4336-b3e1-dc3a417180f9)
  
  To reproduce scenario C: hostname that resolves to IPv6-only address
  1) Configure keystone according to http://docs.openstack.org/kilo/install-guide/install/apt/content/keystone-install.html
  2) In section [memcache] in /etc/keystone/keystone.conf change servers = line:
   servers = keystone-1:11211,keystone-2:11211,keystone-3:11211
  
  3) Edit /etc/hosts:
  2001:db8:1000:1:f816:3eff:fe2a:f9c7	keystone-1
  2001:db8:1000:1:f816:3eff:fee9:9ce3	keystone-2
  2001:db8:1000:1:f816:3eff:fead:8f7f	keystone-3
  
  3) Restart keystone/apache
  4) Attempt to issue token:
  
  openstack --os-auth-url http://192.168.0.15:35357   --os-project-name admin --os-username admin --os-auth-type password   token issue
  Password:
  ERROR: openstack Maximum lock attempts on _lockusertokens-30dbbe8174b24174a3a24d1ae554ab17 occurred. (Disable debug mode to suppress these details.) (HTTP 500) (Request-ID: req-efd53eae-4bcf-4fd9-bab2-dd4c86fb9798)
  
  Control test:
  1) Configure keystone according to http://docs.openstack.org/kilo/install-guide/install/apt/content/keystone-install.html
  2) In section [memcache] in /etc/keystone/keystone.conf change servers = line:
   servers = keystone-1:11211,keystone-2:11211,keystone-3:11211
  
  3) Edit /etc/hosts:
  192.168.0.15	keystone-1
  192.168.0.14	keystone-2
  192.168.0.16	keystone-3
  
  3) Restart keystone/apache
  4) Attempt to issue token:
  
  openstack --os-auth-url http://192.168.0.15:35357   --os-project-name admin --os-username admin --os-auth-type password   token issue
  Password:
  +------------+----------------------------------+
  | Field      | Value                            |
  +------------+----------------------------------+
  | expires    | 2015-06-05T00:31:30Z             |
  | id         | 2a188e9950f44decb78f196b5a3c3f78 |
  | project_id | 91bb6f536fca40a68fb5d4cf72527388 |
  | user_id    | 30dbbe8174b24174a3a24d1ae554ab17 |
  +------------+----------------------------------+

** Summary changed:

- memcache Token persistence backend does not support connecting to IPv6-only hostnames or addresses
+ python-memcache (and therefore) token memcache persistence driver does not support ipv6

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to Keystone.
https://bugs.launchpad.net/bugs/1462152

Title:
  python-memcache (and therefore) token memcache persistence driver does
  not support ipv6

Status in OpenStack Identity (Keystone):
  Triaged
Status in OpenStack Manuals:
  New

Bug description:
  (morganfainberg):
  OpenStack Manuals (for both Master and Kilo) need to be updated to eliminate the recommendation to use the memcache token persistence backend. The memcache token persistence backend is a poor choice due to performance concerns of the code itself, the fact that it is assumed that the token backend is stable storage (memcached is not) and can expose security concerns if restarted in some scenarios (PKI tokens and revoked tokens becoming valid again), and finally that the python-memcache library is all around poor (it will not work with IPv6 and is not Python3 compatible, among other poor design choices).

  ========================================================================
  The memcache backend driver that utilizes "python-memcache" does not support IPv6.

  I have included three scenarios (A, B and C) that will reproduce the
  bug and a control test that succeeds with same configuration using
  IPv4-resolving hostname.

  To reproduce scenario A: Bare IPv6 address in config
  1) Configure keystone according to http://docs.openstack.org/kilo/install-guide/install/apt/content/keystone-install.html
  2) In section [memcache] in /etc/keystone/keystone.conf change servers = line:
   servers = 2001:db8:1000:1:f816:3eff:fe2a:f9c7:11211,2001:db8:1000:1:f816:3eff:fee9:9ce3:11211,2001:db8:1000:1:f816:3eff:fead:8f7f:11211
  3) Restart keystone/apache
  4) Attempt to issue token:
   openstack --os-auth-url http://192.168.0.15:35357   --os-project-name admin --os-username admin --os-auth-type password   token issue

  ERROR: openstack An unexpected error prevented the server from
  fulfilling your request: Unable to parse connection string:
  "2001:db8:1000:1:f816:3eff:fe2a:f9c7:11211" (Disable debug mode to
  suppress these details.) (HTTP 500) (Request-ID: req-7c2bfd39-4b83
  -462b-92c6-f75f7677c8e5)

  To reproduce scenario B: IPv6 address enclosed in brackets
  1) Configure keystone according to http://docs.openstack.org/kilo/install-guide/install/apt/content/keystone-install.html
  2) In section [memcache] in /etc/keystone/keystone.conf change servers = line:
   servers = [2001:db8:1000:1:f816:3eff:fe2a:f9c7]:11211,[2001:db8:1000:1:f816:3eff:fee9:9ce3]:11211,[2001:db8:1000:1:f816:3eff:fead:8f7f]:11211
  3) Restart keystone/apache
  4) Attempt to issue token:
   openstack --os-auth-url http://192.168.0.15:35357   --os-project-name admin --os-username admin --os-auth-type password   token issue

  ERROR: openstack An unexpected error prevented the server from
  fulfilling your request: Unable to parse connection string:
  "[2001:db8:1000:1:f816:3eff:fe2a:f9c7]:11211" (Disable debug mode to
  suppress these details.) (HTTP 500) (Request-ID: req-
  869eb953-74af-4336-b3e1-dc3a417180f9)

  To reproduce scenario C: hostname that resolves to IPv6-only address
  1) Configure keystone according to http://docs.openstack.org/kilo/install-guide/install/apt/content/keystone-install.html
  2) In section [memcache] in /etc/keystone/keystone.conf change servers = line:
   servers = keystone-1:11211,keystone-2:11211,keystone-3:11211

  3) Edit /etc/hosts:
  2001:db8:1000:1:f816:3eff:fe2a:f9c7	keystone-1
  2001:db8:1000:1:f816:3eff:fee9:9ce3	keystone-2
  2001:db8:1000:1:f816:3eff:fead:8f7f	keystone-3

  3) Restart keystone/apache
  4) Attempt to issue token:

  openstack --os-auth-url http://192.168.0.15:35357   --os-project-name admin --os-username admin --os-auth-type password   token issue
  Password:
  ERROR: openstack Maximum lock attempts on _lockusertokens-30dbbe8174b24174a3a24d1ae554ab17 occurred. (Disable debug mode to suppress these details.) (HTTP 500) (Request-ID: req-efd53eae-4bcf-4fd9-bab2-dd4c86fb9798)

  Control test:
  1) Configure keystone according to http://docs.openstack.org/kilo/install-guide/install/apt/content/keystone-install.html
  2) In section [memcache] in /etc/keystone/keystone.conf change servers = line:
   servers = keystone-1:11211,keystone-2:11211,keystone-3:11211

  3) Edit /etc/hosts:
  192.168.0.15	keystone-1
  192.168.0.14	keystone-2
  192.168.0.16	keystone-3

  3) Restart keystone/apache
  4) Attempt to issue token:

  openstack --os-auth-url http://192.168.0.15:35357   --os-project-name admin --os-username admin --os-auth-type password   token issue
  Password:
  +------------+----------------------------------+
  | Field      | Value                            |
  +------------+----------------------------------+
  | expires    | 2015-06-05T00:31:30Z             |
  | id         | 2a188e9950f44decb78f196b5a3c3f78 |
  | project_id | 91bb6f536fca40a68fb5d4cf72527388 |
  | user_id    | 30dbbe8174b24174a3a24d1ae554ab17 |
  +------------+----------------------------------+

To manage notifications about this bug go to:
https://bugs.launchpad.net/keystone/+bug/1462152/+subscriptions

References

  Thread Previous • Date Previous • • Thread Next