yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1712415] [NEW] Handle ldap.SIZELIMIT_EXCEEDED error

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: prashkre <1712415@xxxxxxxxxxxxxxxxxx>
Date: Tue, 22 Aug 2017 19:37:04 -0000
Reply-to: Bug 1712415 <1712415@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
Public bug reported:

With ldap as identity backend server configured to keystone, GET
users/groups API is abruptly failing in below stacktrace without proper
error handling when ldap search results(users/groups) count is more than
default sizelimit configured on ldap server.

2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi Traceback (most recent call last):
2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi   File "/usr/lib/python2.7/site-packages/keystone/common/wsgi.py", line 228, in __call__
2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi     result = method(req, **params)
2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi   File "/usr/lib/python2.7/site-packages/keystone/common/controller.py", line 138, in wrapper
2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi     return f(self, request, filters, **kwargs)
2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi   File "/usr/lib/python2.7/site-packages/keystone/identity/controllers.py", line 229, in list_users
2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi     refs = self.identity_api.list_users(domain_scope=domain, hints=hints)
2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi   File "/usr/lib/python2.7/site-packages/keystone/common/manager.py", line 110, in wrapped
2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi     __ret_val = __f(*args, **kwargs)
2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi   File "/usr/lib/python2.7/site-packages/keystone/identity/core.py", line 413, in wrapper
2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi     return f(self, *args, **kwargs)
2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi   File "/usr/lib/python2.7/site-packages/keystone/identity/core.py", line 423, in wrapper
2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi     return f(self, *args, **kwargs)
2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi   File "/usr/lib/python2.7/site-packages/keystone/identity/core.py", line 1036, in list_users
2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi     ref_list = self._handle_federated_attributes_in_hints(driver, hints)
2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi   File "/usr/lib/python2.7/site-packages/keystone/identity/core.py", line 1019, in _handle_federated_attributes_in_hints
2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi     return driver.list_users(hints)
2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi   File "/usr/lib/python2.7/site-packages/keystone/identity/backends/ldap/core.py", line 87, in list_users
2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi     return self.user.get_all_filtered(hints)
2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi   File "/usr/lib/python2.7/site-packages/keystone/identity/backends/ldap/core.py", line 327, in get_all_filtered
2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi     for user in self.get_all(query, hints)]
2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi   File "/usr/lib/python2.7/site-packages/keystone/identity/backends/ldap/core.py", line 319, in get_all
2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi     hints=hints)
2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi   File "/usr/lib/python2.7/site-packages/keystone/identity/backends/ldap/common.py", line 1822, in get_all
2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi     return super(EnabledEmuMixIn, self).get_all(ldap_filter, hints)
2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi   File "/usr/lib/python2.7/site-packages/keystone/identity/backends/ldap/common.py", line 1524, in get_all
2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi     for x in self._ldap_get_all(hints, ldap_filter)]
2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi   File "/usr/lib/python2.7/site-packages/keystone/common/driver_hints.py", line 42, in wrapper
2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi     return f(self, hints, *args, **kwargs)
2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi   File "/usr/lib/python2.7/site-packages/keystone/identity/backends/ldap/common.py", line 1474, in _ldap_get_all
2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi     attrs)
2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi   File "/usr/lib/python2.7/site-packages/keystone/identity/backends/ldap/common.py", line 935, in search_s
2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi     attrlist_utf8, attrsonly)
2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi   File "/usr/lib/python2.7/site-packages/keystone/identity/backends/ldap/common.py", line 634, in wrapper
2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi     return func(self, conn, *args, **kwargs)
2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi   File "/usr/lib/python2.7/site-packages/keystone/identity/backends/ldap/common.py", line 764, in search_s
2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi     attrsonly)
2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi   File "/usr/lib64/python2.7/site-packages/ldap/ldapobject.py", line 773, in search_s
2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi     return self.search_ext_s(base,scope,filterstr,attrlist,attrsonly,None,None,timeout=self.timeout)
2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi   File "/usr/lib64/python2.7/site-packages/ldap/ldapobject.py", line 1180, in search_ext_s
2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi     return self._apply_method_s(SimpleLDAPObject.search_ext_s,*args,**kwargs)
2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi   File "/usr/lib64/python2.7/site-packages/ldap/ldapobject.py", line 1118, in _apply_method_s
2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi     return func(self,*args,**kwargs)
2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi   File "/usr/lib64/python2.7/site-packages/ldap/ldapobject.py", line 767, in search_ext_s
2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi     return self.result(msgid,all=1,timeout=timeout)[1]
2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi   File "/usr/lib64/python2.7/site-packages/ldap/ldapobject.py", line 674, in result
2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi     resp_type, resp_data, resp_msgid = self.result2(msgid,all,timeout)
2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi   File "/usr/lib64/python2.7/site-packages/ldap/ldapobject.py", line 678, in result2
2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi     resp_type, resp_data, resp_msgid, resp_ctrls = self.result3(msgid,all,timeout)
2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi   File "/usr/lib64/python2.7/site-packages/ldap/ldapobject.py", line 685, in result3
2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi     resp_ctrl_classes=resp_ctrl_classes
2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi   File "/usr/lib64/python2.7/site-packages/ldap/ldapobject.py", line 692, in result4
2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi     ldap_result = self._ldap_call(self._l.result4,msgid,all,timeout,add_ctrls,add_intermediates,add_extop)
2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi   File "/usr/lib64/python2.7/site-packages/ldap/ldapobject.py", line 265, in _ldap_call
2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi     result = func(*args,**kwargs)
2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi SIZELIMIT_EXCEEDED: {'desc': u'Size limit exceeded'}
2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi

** Affects: keystone
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Identity (keystone).
https://bugs.launchpad.net/bugs/1712415

Title:
  Handle ldap.SIZELIMIT_EXCEEDED error

Status in OpenStack Identity (keystone):
  New

Bug description:
  With ldap as identity backend server configured to keystone, GET
  users/groups API is abruptly failing in below stacktrace without
  proper error handling when ldap search results(users/groups) count is
  more than default sizelimit configured on ldap server.

  2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi Traceback (most recent call last):
  2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi   File "/usr/lib/python2.7/site-packages/keystone/common/wsgi.py", line 228, in __call__
  2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi     result = method(req, **params)
  2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi   File "/usr/lib/python2.7/site-packages/keystone/common/controller.py", line 138, in wrapper
  2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi     return f(self, request, filters, **kwargs)
  2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi   File "/usr/lib/python2.7/site-packages/keystone/identity/controllers.py", line 229, in list_users
  2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi     refs = self.identity_api.list_users(domain_scope=domain, hints=hints)
  2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi   File "/usr/lib/python2.7/site-packages/keystone/common/manager.py", line 110, in wrapped
  2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi     __ret_val = __f(*args, **kwargs)
  2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi   File "/usr/lib/python2.7/site-packages/keystone/identity/core.py", line 413, in wrapper
  2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi     return f(self, *args, **kwargs)
  2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi   File "/usr/lib/python2.7/site-packages/keystone/identity/core.py", line 423, in wrapper
  2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi     return f(self, *args, **kwargs)
  2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi   File "/usr/lib/python2.7/site-packages/keystone/identity/core.py", line 1036, in list_users
  2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi     ref_list = self._handle_federated_attributes_in_hints(driver, hints)
  2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi   File "/usr/lib/python2.7/site-packages/keystone/identity/core.py", line 1019, in _handle_federated_attributes_in_hints
  2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi     return driver.list_users(hints)
  2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi   File "/usr/lib/python2.7/site-packages/keystone/identity/backends/ldap/core.py", line 87, in list_users
  2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi     return self.user.get_all_filtered(hints)
  2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi   File "/usr/lib/python2.7/site-packages/keystone/identity/backends/ldap/core.py", line 327, in get_all_filtered
  2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi     for user in self.get_all(query, hints)]
  2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi   File "/usr/lib/python2.7/site-packages/keystone/identity/backends/ldap/core.py", line 319, in get_all
  2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi     hints=hints)
  2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi   File "/usr/lib/python2.7/site-packages/keystone/identity/backends/ldap/common.py", line 1822, in get_all
  2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi     return super(EnabledEmuMixIn, self).get_all(ldap_filter, hints)
  2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi   File "/usr/lib/python2.7/site-packages/keystone/identity/backends/ldap/common.py", line 1524, in get_all
  2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi     for x in self._ldap_get_all(hints, ldap_filter)]
  2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi   File "/usr/lib/python2.7/site-packages/keystone/common/driver_hints.py", line 42, in wrapper
  2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi     return f(self, hints, *args, **kwargs)
  2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi   File "/usr/lib/python2.7/site-packages/keystone/identity/backends/ldap/common.py", line 1474, in _ldap_get_all
  2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi     attrs)
  2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi   File "/usr/lib/python2.7/site-packages/keystone/identity/backends/ldap/common.py", line 935, in search_s
  2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi     attrlist_utf8, attrsonly)
  2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi   File "/usr/lib/python2.7/site-packages/keystone/identity/backends/ldap/common.py", line 634, in wrapper
  2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi     return func(self, conn, *args, **kwargs)
  2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi   File "/usr/lib/python2.7/site-packages/keystone/identity/backends/ldap/common.py", line 764, in search_s
  2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi     attrsonly)
  2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi   File "/usr/lib64/python2.7/site-packages/ldap/ldapobject.py", line 773, in search_s
  2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi     return self.search_ext_s(base,scope,filterstr,attrlist,attrsonly,None,None,timeout=self.timeout)
  2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi   File "/usr/lib64/python2.7/site-packages/ldap/ldapobject.py", line 1180, in search_ext_s
  2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi     return self._apply_method_s(SimpleLDAPObject.search_ext_s,*args,**kwargs)
  2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi   File "/usr/lib64/python2.7/site-packages/ldap/ldapobject.py", line 1118, in _apply_method_s
  2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi     return func(self,*args,**kwargs)
  2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi   File "/usr/lib64/python2.7/site-packages/ldap/ldapobject.py", line 767, in search_ext_s
  2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi     return self.result(msgid,all=1,timeout=timeout)[1]
  2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi   File "/usr/lib64/python2.7/site-packages/ldap/ldapobject.py", line 674, in result
  2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi     resp_type, resp_data, resp_msgid = self.result2(msgid,all,timeout)
  2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi   File "/usr/lib64/python2.7/site-packages/ldap/ldapobject.py", line 678, in result2
  2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi     resp_type, resp_data, resp_msgid, resp_ctrls = self.result3(msgid,all,timeout)
  2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi   File "/usr/lib64/python2.7/site-packages/ldap/ldapobject.py", line 685, in result3
  2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi     resp_ctrl_classes=resp_ctrl_classes
  2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi   File "/usr/lib64/python2.7/site-packages/ldap/ldapobject.py", line 692, in result4
  2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi     ldap_result = self._ldap_call(self._l.result4,msgid,all,timeout,add_ctrls,add_intermediates,add_extop)
  2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi   File "/usr/lib64/python2.7/site-packages/ldap/ldapobject.py", line 265, in _ldap_call
  2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi     result = func(*args,**kwargs)
  2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi SIZELIMIT_EXCEEDED: {'desc': u'Size limit exceeded'}
  2017-08-04 11:44:46.368 102104 ERROR keystone.common.wsgi

To manage notifications about this bug go to:
https://bugs.launchpad.net/keystone/+bug/1712415/+subscriptions
Follow ups

[Bug 1712415] Re: Keystone should handle ldap.SIZELIMIT_EXCEEDED error
From: Lance Bragstad, 2017-10-26
[Bug 1712415] Re: Keystone should handle ldap.SIZELIMIT_EXCEEDED error
From: OpenStack Infra, 2017-10-24