yahoo-eng-team team mailing list archive

[Grzegorz Grasza <1952458@xxxxxxxxxxxxxxxxxx>]

Public bug reported:

When connecting to some LDAP servers, the LDAP library returns bytes
data instead of strings, resulting in unexpected errors, ex:

a call to

/v3/projects/x/groups/y/roles/z

results in keystone.exception.GroupNotFound: Could not find group: b'Q'.

After adding more debug logs it was determined that get_id_mapping
returns the LDAP group name as binary type. get_id_mapping is memoized
(@MEMOIZE_ID_MAPPING), the cache is filled not only during the
"memoization" but also inside the create_id_mapping method:

    def create_id_mapping(self, local_entity, public_id=None):
        public_id = self.driver.create_id_mapping(local_entity, public_id)
        if MEMOIZE_ID_MAPPING.should_cache(public_id):
            self._get_public_id.set(public_id, self,
                                    local_entity['domain_id'],
                                    local_entity['local_id'],
                                    local_entity['entity_type'])
            self.get_id_mapping.set(local_entity, self, public_id)
        return public_id

What is cached is the input dictionary, which is passed into the function, instead of what the SQL backend returns.
The sql backend transparently converts bytes when inserting data into the database, and always returns strings when the data is read.
The intersection of the above causes the unexpected behavior with transient errors.

The local_id is returned as bytes from the LDAP backend, but it's
difficult to trace exactly where, without access to the environment with
this specific LDAP software.

** Affects: keystone
     Importance: Undecided
         Status: In Progress

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Identity (keystone).
https://bugs.launchpad.net/bugs/1952458

Title:
  create_id_mapping method caches bytes with LDAP backend

Status in OpenStack Identity (keystone):
  In Progress

Bug description:
  When connecting to some LDAP servers, the LDAP library returns bytes
  data instead of strings, resulting in unexpected errors, ex:

  a call to

  /v3/projects/x/groups/y/roles/z

  results in keystone.exception.GroupNotFound: Could not find group:
  b'Q'.

  After adding more debug logs it was determined that get_id_mapping
  returns the LDAP group name as binary type. get_id_mapping is memoized
  (@MEMOIZE_ID_MAPPING), the cache is filled not only during the
  "memoization" but also inside the create_id_mapping method:

      def create_id_mapping(self, local_entity, public_id=None):
          public_id = self.driver.create_id_mapping(local_entity, public_id)
          if MEMOIZE_ID_MAPPING.should_cache(public_id):
              self._get_public_id.set(public_id, self,
                                      local_entity['domain_id'],
                                      local_entity['local_id'],
                                      local_entity['entity_type'])
              self.get_id_mapping.set(local_entity, self, public_id)
          return public_id

  What is cached is the input dictionary, which is passed into the function, instead of what the SQL backend returns.
  The sql backend transparently converts bytes when inserting data into the database, and always returns strings when the data is read.
  The intersection of the above causes the unexpected behavior with transient errors.

  The local_id is returned as bytes from the LDAP backend, but it's
  difficult to trace exactly where, without access to the environment
  with this specific LDAP software.

To manage notifications about this bug go to:
https://bugs.launchpad.net/keystone/+bug/1952458/+subscriptions