yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1362678] Re: multi-domain has problems with LDAP identity on default domain

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Marcus Klein <kleini76@xxxxxxxxx>
Date: Fri, 02 Jan 2015 15:30:19 -0000
Reply-to: Bug 1362678 <1362678@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
I upgraded now completely on the Juno release. A different problem now
appears.

Due to the LDAP identity backend driver for the default domain, user
logins are used for trustor_user_ids. This work without any problems as
long as the login name does not contain the dot character. Unfortunately
are most of our users created using a pattern like givenname.surename.

This leads then to the following error in keystone:

2015-01-02 16:19:45.587 3984 WARNING keystone.common.wsgi [-] Invalid
input for field 'trustor_user_id'. The value is 'marcus.klein'.

And then the heat-engine failes with:

2015-01-02 16:19:45.591 27660 ERROR oslo.messaging.rpc.dispatcher [req-2824ab9b-d70c-4d38-8088-1bc5cdf45096 ] Exception during message handling: Invalid input for field 'trustor_user_id'. The value is 'marcus.klein'. (HTTP 400)
2015-01-02 16:19:45.591 27660 TRACE oslo.messaging.rpc.dispatcher Traceback (most recent call last):
2015-01-02 16:19:45.591 27660 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 134, in _dispatch_and_reply
2015-01-02 16:19:45.591 27660 TRACE oslo.messaging.rpc.dispatcher     incoming.message))
2015-01-02 16:19:45.591 27660 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 177, in _dispatch
2015-01-02 16:19:45.591 27660 TRACE oslo.messaging.rpc.dispatcher     return self._do_dispatch(endpoint, method, ctxt, args)
2015-01-02 16:19:45.591 27660 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 123, in _do_dispatch
2015-01-02 16:19:45.591 27660 TRACE oslo.messaging.rpc.dispatcher     result = getattr(endpoint, method)(ctxt, **new_args)
2015-01-02 16:19:45.591 27660 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/heat/engine/service.py", line 69, in wrapped
2015-01-02 16:19:45.591 27660 TRACE oslo.messaging.rpc.dispatcher     return func(self, ctx, *args, **kwargs)
2015-01-02 16:19:45.591 27660 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/heat/engine/service.py", line 647, in create_stack
2015-01-02 16:19:45.591 27660 TRACE oslo.messaging.rpc.dispatcher     stack.store()
2015-01-02 16:19:45.591 27660 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/heat/engine/stack.py", line 315, in store
2015-01-02 16:19:45.591 27660 TRACE oslo.messaging.rpc.dispatcher     trust_ctx = keystone.create_trust_context()
2015-01-02 16:19:45.591 27660 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/heat/common/heat_keystoneclient.py", line 291, in create_trust_context
2015-01-02 16:19:45.591 27660 TRACE oslo.messaging.rpc.dispatcher     role_names=roles)
2015-01-02 16:19:45.591 27660 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/keystoneclient/v3/contrib/trusts.py", line 74, in create
2015-01-02 16:19:45.591 27660 TRACE oslo.messaging.rpc.dispatcher     **kwargs)
2015-01-02 16:19:45.591 27660 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/keystoneclient/base.py", line 71, in func
2015-01-02 16:19:45.591 27660 TRACE oslo.messaging.rpc.dispatcher     return f(*args, **new_kwargs)
2015-01-02 16:19:45.591 27660 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/keystoneclient/base.py", line 327, in create
2015-01-02 16:19:45.591 27660 TRACE oslo.messaging.rpc.dispatcher     self.key)
2015-01-02 16:19:45.591 27660 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/keystoneclient/base.py", line 150, in _create
2015-01-02 16:19:45.591 27660 TRACE oslo.messaging.rpc.dispatcher     return self._post(url, body, response_key, return_raw, **kwargs)
2015-01-02 16:19:45.591 27660 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/keystoneclient/base.py", line 164, in _post
2015-01-02 16:19:45.591 27660 TRACE oslo.messaging.rpc.dispatcher     resp, body = self.client.post(url, body=body, **kwargs)
2015-01-02 16:19:45.591 27660 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/keystoneclient/httpclient.py", line 617, in post
2015-01-02 16:19:45.591 27660 TRACE oslo.messaging.rpc.dispatcher     return self._cs_request(url, 'POST', **kwargs)
2015-01-02 16:19:45.591 27660 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/keystoneclient/httpclient.py", line 603, in _cs_request
2015-01-02 16:19:45.591 27660 TRACE oslo.messaging.rpc.dispatcher     return self.request(url, method, **kwargs)
2015-01-02 16:19:45.591 27660 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/keystoneclient/httpclient.py", line 578, in request
2015-01-02 16:19:45.591 27660 TRACE oslo.messaging.rpc.dispatcher     resp = super(HTTPClient, self).request(url, method, **kwargs)
2015-01-02 16:19:45.591 27660 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/keystoneclient/baseclient.py", line 21, in request
2015-01-02 16:19:45.591 27660 TRACE oslo.messaging.rpc.dispatcher     return self.session.request(url, method, **kwargs)
2015-01-02 16:19:45.591 27660 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/keystoneclient/utils.py", line 318, in inner
2015-01-02 16:19:45.591 27660 TRACE oslo.messaging.rpc.dispatcher     return func(*args, **kwargs)
2015-01-02 16:19:45.591 27660 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/keystoneclient/session.py", line 339, in request
2015-01-02 16:19:45.591 27660 TRACE oslo.messaging.rpc.dispatcher     raise exceptions.from_response(resp, method, url)
2015-01-02 16:19:45.591 27660 TRACE oslo.messaging.rpc.dispatcher BadRequest: Invalid input for field 'trustor_user_id'. The value is 'marcus.klein'. (HTTP 400)
2015-01-02 16:19:45.591 27660 TRACE oslo.messaging.rpc.dispatcher 
2015-01-02 16:19:45.595 27660 ERROR oslo.messaging._drivers.common [req-2824ab9b-d70c-4d38-8088-1bc5cdf45096 ] Returning exception Invalid input for field 'trustor_user_id'. The value is 'marcus.klein'. (HTTP 400) to caller
2015-01-02 16:19:45.596 27660 ERROR oslo.messaging._drivers.common [req-2824ab9b-d70c-4d38-8088-1bc5cdf45096 ] ['Traceback (most recent call last):\n', '  File "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 134, in _dispatch_and_reply\n    incoming.message))\n', '  File "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 177, in _dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 123, in _do_dispatch\n    result = getattr(endpoint, method)(ctxt, **new_args)\n', '  File "/usr/lib/python2.7/dist-packages/heat/engine/service.py", line 69, in wrapped\n    return func(self, ctx, *args, **kwargs)\n', '  File "/usr/lib/python2.7/dist-packages/heat/engine/service.py", line 647, in create_stack\n    stack.store()\n', '  File "/usr/lib/python2.7/dist-packages/heat/engine/stack.py", line 315, in store\n    trust_ctx = keystone.create_trust_context()\n', '  File "/usr/lib/python2.7/dist-packages/heat/common/heat_keystoneclient.py", line 291, in create_trust_context\n    role_names=roles)\n', '  File "/usr/lib/python2.7/dist-packages/keystoneclient/v3/contrib/trusts.py", line 74, in create\n    **kwargs)\n', '  File "/usr/lib/python2.7/dist-packages/keystoneclient/base.py", line 71, in func\n    return f(*args, **new_kwargs)\n', '  File "/usr/lib/python2.7/dist-packages/keystoneclient/base.py", line 327, in create\n    self.key)\n', '  File "/usr/lib/python2.7/dist-packages/keystoneclient/base.py", line 150, in _create\n    return self._post(url, body, response_key, return_raw, **kwargs)\n', '  File "/usr/lib/python2.7/dist-packages/keystoneclient/base.py", line 164, in _post\n    resp, body = self.client.post(url, body=body, **kwargs)\n', '  File "/usr/lib/python2.7/dist-packages/keystoneclient/httpclient.py", line 617, in post\n    return self._cs_request(url, \'POST\', **kwargs)\n', '  File "/usr/lib/python2.7/dist-packages/keystoneclient/httpclient.py", line 603, in _cs_request\n    return self.request(url, method, **kwargs)\n', '  File "/usr/lib/python2.7/dist-packages/keystoneclient/httpclient.py", line 578, in request\n    resp = super(HTTPClient, self).request(url, method, **kwargs)\n', '  File "/usr/lib/python2.7/dist-packages/keystoneclient/baseclient.py", line 21, in request\n    return self.session.request(url, method, **kwargs)\n', '  File "/usr/lib/python2.7/dist-packages/keystoneclient/utils.py", line 318, in inner\n    return func(*args, **kwargs)\n', '  File "/usr/lib/python2.7/dist-packages/keystoneclient/session.py", line 339, in request\n    raise exceptions.from_response(resp, method, url)\n', "BadRequest: Invalid input for field 'trustor_user_id'. The value is 'marcus.klein'. (HTTP 400)\n"]

** Changed in: keystone
       Status: Invalid => New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to Keystone.
https://bugs.launchpad.net/bugs/1362678

Title:
  multi-domain has problems with LDAP identity on default domain

Status in OpenStack Identity (Keystone):
  New

Bug description:
  What I try to achieve:

  I want to authenticate all users of the default domain against our
  company's central LDAP server. This works pretty good.

  For Heat I need a user storage that is writable. Our central LDAP
  server can not be written from OpenStack. Therefore I configured the
  heat domain with SQL identity.

  This all works up to the point, when the heat domain admin needs to be
  authorized. This authorization request is always processed with the
  LDAP identity. I don't know whether the domain is missing here for the
  keystone V3 API authorization request or keystone does not route the
  request correctly to the SQL identity. To clarify this, I opened this
  bug and Steven Hardy encouraged me to do so.

  /etc/keystone/keystone.conf:

  [identity]
  default_domain_id=default
  domain_specific_drivers_enabled=true
  domain_config_dir=/etc/keystone/domains
  driver = keystone.identity.backends.ldap.Identity

  [ldap]
  url=ldap://ldap2.open-xchange.com:389
  suffix=dc=open-xchange,dc=com
  etc.

  /etc/keystone/domains/keystone.heat.conf:

  [identity]
  driver = keystone.identity.backends.sql.Identity

  [ldap]

  /etc/heat/heat.conf:
  deferred_auth_method=trusts
  trusts_delegated_roles=heat_stack_owner
  heat_stack_user_role=heat_stack_user
  stack_user_domain=a904d890e0de47dc9f2090c20bb1f45c
  stack_domain_admin=heat_domain_admin
  stack_domain_admin_password=********

  openstack --os-token $OS_TOKEN --os-url=http://contorller:5000/v3 --os-identity-api-version=3 domain list
  +----------------------------------+---------+---------+----------------------------------------------------------------------+
  | ID                               | Name    | Enabled | Description                                                          |
  +----------------------------------+---------+---------+----------------------------------------------------------------------+
  | a904d890e0de47dc9f2090c20bb1f45c | heat    | True    | Owns users and projects created by heat                              |
  | default                          | Default | True    | Owns users and tenants (i.e. projects) available on Identity API v2. |
  +----------------------------------+---------+---------+----------------------------------------------------------------------+

  openstack --os-token $OS_TOKEN --os-url=http://controller:5000/v3 --os-identity-api-version=3 user create --password **** --domain a904d890e0de47dc9f2090c20bb1f45c --description "Manages users and projects created by heat" heat_domain_admin
  +-------------+-------------------------------------------------------------------------------------+
  | Field       | Value                                                                               |
  +-------------+-------------------------------------------------------------------------------------+
  | description | Manages users and projects created by heat                                          |
  | domain_id   | a904d890e0de47dc9f2090c20bb1f45c                                                    |
  | enabled     | True                                                                                |
  | id          | 38877ca5daed4c9fbbb6c853d3d88e36                                                    |
  | links       | {u'self': u'http://controller-test:5000/v3/users/38877ca5daed4c9fbbb6c853d3d88e36'} |
  | name        | heat_domain_admin                                                                   |
  +-------------+-------------------------------------------------------------------------------------+

  openstack --os-token $OS_TOKEN --os-url=http://controller:5000/v3
  --os-identity-api-version=3 role add --user
  38877ca5daed4c9fbbb6c853d3d88e36 --domain
  a904d890e0de47dc9f2090c20bb1f45c admin

  Everything set up according to:
  http://hardysteven.blogspot.de/2014/04/heat-auth-model-updates-part-1-trusts.html
  http://hardysteven.blogspot.de/2014/04/heat-auth-model-updates-part-2-stack.html

  I tested this using this example stack: https://github.com/openstack
  /heat-templates/blob/master/hot/software-config/example-templates
  /example-deploy-sequence.yaml

  Then I get the following authentication problem in keystone:
  2014-08-28 13:20:40.172 4915 INFO eventlet.wsgi.server [-] 10.20.31.200 - - [28/Aug/2014 13:20:40] "POST /v3/auth/tokens HTTP/1.1" 201 12110 0.163805
  2014-08-28 13:20:40.326 4915 DEBUG keystone.middleware.core [-] Auth token not in the request header. Will not build auth context. process_request /usr/lib/python2.7/dist-packages/keystone/middleware/core.py:271
  2014-08-28 13:20:40.329 4915 DEBUG keystone.common.wsgi [-] arg_dict: {} __call__ /usr/lib/python2.7/dist-packages/keystone/common/wsgi.py:181
  2014-08-28 13:20:40.355 4915 DEBUG keystone.notifications [-] CADF Event: {'typeURI': 'http://schemas.dmtf.org/cloud/audit/1.0/event', 'initiator': {'typeURI': 'service/security/account/user', 'host': {'agent': 'python-keystoneclient', 'a
  ddress': '10.20.31.200'}, 'id': 'openstack:d7c2f1ec-aae3-4fe5-8721-a82ca842eca3', 'name': u'38877ca5daed4c9fbbb6c853d3d88e36'}, 'target': {'typeURI': 'service/security/account/user', 'id': 'openstack:54c887e0-9820-46d8-9af5-1159960abf5c'}
  , 'observer': {'typeURI': 'service/security', 'id': 'openstack:2186c327-ce39-415a-8dd0-6d99841365bb'}, 'eventType': 'activity', 'eventTime': '2014-08-28T11:20:40.355197+0000', 'action': 'authenticate', 'outcome': 'pending', 'id': 'opensta
  ck:c8133c07-751e-4b52-9a23-33cd111c456e'} _send_audit_notification /usr/lib/python2.7/dist-packages/keystone/notifications.py:289
  2014-08-28 13:20:40.378 4915 INFO passlib.registry [-] registered crypt handler 'sha512_crypt': <class 'passlib.handlers.sha2_crypt.sha512_crypt'>
  2014-08-28 13:20:40.465 4915 DEBUG keystone.notifications [-] CADF Event: {'typeURI': 'http://schemas.dmtf.org/cloud/audit/1.0/event', 'initiator': {'typeURI': 'service/security/account/user', 'host': {'agent': 'python-keystoneclient', 'a
  ddress': '10.20.31.200'}, 'id': 'openstack:d7c2f1ec-aae3-4fe5-8721-a82ca842eca3', 'name': u'38877ca5daed4c9fbbb6c853d3d88e36'}, 'target': {'typeURI': 'service/security/account/user', 'id': 'openstack:9c9ea536-26fc-438e-9a86-a07d541102e8'}
  , 'observer': {'typeURI': 'service/security', 'id': 'openstack:06751111-ab05-4f79-a6ab-82185c519468'}, 'eventType': 'activity', 'eventTime': '2014-08-28T11:20:40.465304+0000', 'action': 'authenticate', 'outcome': 'success', 'id': 'opensta
  ck:65b076a4-8220-4a99-864b-df390c56e28c'} _send_audit_notification /usr/lib/python2.7/dist-packages/keystone/notifications.py:289
  2014-08-28 13:20:40.482 4915 DEBUG keystone.common.ldap.core [-] LDAP init: url=ldap://ldap2.open-xchange.com:389 __init__ /usr/lib/python2.7/dist-packages/keystone/common/ldap/core.py:603
  2014-08-28 13:20:40.486 4915 DEBUG keystone.common.ldap.core [-] LDAP init: use_tls=False
  tls_cacertfile=None
  tls_cacertdir=None
  tls_req_cert=2
  tls_avail=1
   __init__ /usr/lib/python2.7/dist-packages/keystone/common/ldap/core.py:613
  2014-08-28 13:20:40.487 4915 DEBUG keystone.common.ldap.core [-] LDAP search: dn=ou=Users,ou=OxObjects,dc=open-xchange,dc=com, scope=1, query=(&(uid=38877ca5daed4c9fbbb6c853d3d88e36)(objectClass=posixAccount)), attrs=['mail', 'userPasswor
  d', 'uid', 'mailEnabled'] search_s /usr/lib/python2.7/dist-packages/keystone/common/ldap/core.py:704
  2014-08-28 13:20:40.491 4915 DEBUG keystone.common.ldap.core [-] LDAP unbind unbind_s /usr/lib/python2.7/dist-packages/keystone/common/ldap/core.py:677
  2014-08-28 13:20:40.492 4915 WARNING keystone.common.wsgi [-] Could not find user, 38877ca5daed4c9fbbb6c853d3d88e36.

  This results in the following error in Heat engine:
  2014-08-28 13:20:38.539 407 INFO heat.engine.resource [-] creating Server "server_a" Stack "sequence" [c4ab8875-34a5-45b2-a2dc-0dfce18ef0d8]
  2014-08-28 13:20:38.709 407 INFO urllib3.connectionpool [-] Starting new HTTPS connection (1): cloud.open-xchange.com
  2014-08-28 13:20:39.081 407 INFO urllib3.connectionpool [-] Starting new HTTPS connection (1): cloud.open-xchange.com
  2014-08-28 13:20:39.601 407 INFO urllib3.connectionpool [-] Starting new HTTPS connection (1): cloud.open-xchange.com
  2014-08-28 13:20:39.979 407 INFO urllib3.connectionpool [-] Starting new HTTPS connection (1): cloud.open-xchange.com
  2014-08-28 13:20:40.293 407 INFO urllib3.connectionpool [-] Starting new HTTPS connection (1): cloud.open-xchange.com
  2014-08-28 13:20:40.495 407 ERROR heat.engine.resource [-] CREATE : Server "server_a" Stack "sequence" [c4ab8875-34a5-45b2-a2dc-0dfce18ef0d8]
  2014-08-28 13:20:40.495 407 TRACE heat.engine.resource Traceback (most recent call last):
  2014-08-28 13:20:40.495 407 TRACE heat.engine.resource   File "/usr/lib/python2.7/dist-packages/heat/engine/resource.py", line 417, in _do_action
  2014-08-28 13:20:40.495 407 TRACE heat.engine.resource     handle())
  2014-08-28 13:20:40.495 407 TRACE heat.engine.resource   File "/usr/lib/python2.7/dist-packages/heat/engine/resources/server.py", line 480, in handle_create
  2014-08-28 13:20:40.495 407 TRACE heat.engine.resource     self._create_transport_credentials()
  2014-08-28 13:20:40.495 407 TRACE heat.engine.resource   File "/usr/lib/python2.7/dist-packages/heat/engine/resources/server.py", line 397, in _create_transport_credentials
  2014-08-28 13:20:40.495 407 TRACE heat.engine.resource     self._create_user()
  2014-08-28 13:20:40.495 407 TRACE heat.engine.resource   File "/usr/lib/python2.7/dist-packages/heat/engine/stack_user.py", line 44, in _create_user
  2014-08-28 13:20:40.495 407 TRACE heat.engine.resource     self.stack.id)
  2014-08-28 13:20:40.495 407 TRACE heat.engine.resource   File "/usr/lib/python2.7/dist-packages/heat/common/heat_keystoneclient.py", line 390, in create_stack_domain_project
  2014-08-28 13:20:40.495 407 TRACE heat.engine.resource     domain_project = self.domain_admin_client.projects.create(
  2014-08-28 13:20:40.495 407 TRACE heat.engine.resource   File "/usr/lib/python2.7/dist-packages/heat/common/heat_keystoneclient.py", line 133, in domain_admin_client
  2014-08-28 13:20:40.495 407 TRACE heat.engine.resource     if c.authenticate(domain_id=self.stack_domain_id):
  2014-08-28 13:20:40.495 407 TRACE heat.engine.resource   File "/usr/lib/python2.7/dist-packages/keystoneclient/utils.py", line 318, in inner
  2014-08-28 13:20:40.495 407 TRACE heat.engine.resource     return func(*args, **kwargs)
  2014-08-28 13:20:40.495 407 TRACE heat.engine.resource   File "/usr/lib/python2.7/dist-packages/keystoneclient/httpclient.py", line 392, in authenticate
  2014-08-28 13:20:40.495 407 TRACE heat.engine.resource     resp = self.get_raw_token_from_identity_service(**kwargs)
  2014-08-28 13:20:40.495 407 TRACE heat.engine.resource   File "/usr/lib/python2.7/dist-packages/keystoneclient/v3/client.py", line 178, in get_raw_token_from_identity_service
  2014-08-28 13:20:40.495 407 TRACE heat.engine.resource     '%s' % e)
  2014-08-28 13:20:40.495 407 TRACE heat.engine.resource AuthorizationFailure: Authorization failed: Could not find user, 38877ca5daed4c9fbbb6c853d3d88e36. (HTTP 404)
  2014-08-28 13:20:40.495 407 TRACE heat.engine.resource 
  2014-08-28 13:20:40.605 407 INFO urllib3.connectionpool [-] Starting new HTTPS connection (1): cloud.open-xchange.com
  2014-08-28 13:20:40.990 407 INFO urllib3.connectionpool [-] Starting new HTTPS connection (1): cloud.open-xchange.com
  2014-08-28 13:20:41.570 407 WARNING heat.engine.service [-] Stack create failed, status FAILED

To manage notifications about this bug go to:
https://bugs.launchpad.net/keystone/+bug/1362678/+subscriptions
References

[Bug 1362678] [NEW] multi-domain has problems with LDAP identity on default domain
From: Marcus Klein, 2014-08-28