yahoo-eng-team team mailing list archive

[Jay Pipes <1265071@xxxxxxxxxxxxxxxxxx>]

Public bug reported:

If a new model does not contain an "extra" column of type JsonBlob, unit
tests fail.

The reason is because keystone.common.sql.core.DictBase's from_dict()
classmethod looks like this:

    @classmethod
    def from_dict(cls, d):
        new_d = d.copy()

        new_d['extra'] = dict((k, new_d.pop(k)) for k in d.iterkeys()
                              if k not in cls.attributes and k != 'extra')

        return cls(**new_d)

and the cls(**new_d) ends up calling the __setitem__ method of
keystone.openstack.common.db.models.ModelBase, which looks like this:

    def __setitem__(self, key, value):
        setattr(self, key, value)

The problem is that because new_d's 'extra' key is set, setattr() tries
to set an attribute of the model called 'extra', which blows up if the
model does not have a column in its table called "extra".

Frankly, the denormalized "extra" JSON blob columns just need to be
killed off. They are poor relational database design, in that they
aren't searchable or indexable and don't properly describe the
attributes of the entity.

** Affects: keystone
     Importance: Undecided
         Status: New


** Tags: sql

** Description changed:

  If a new model does not contain an "extra" column of type JsonBlob, unit
  tests fail.
  
  The reason is because keystone.common.sql.core.DictBase's from_dict()
  classmethod looks like this:
  
-     @classmethod
-     def from_dict(cls, d):
-         new_d = d.copy()
+     @classmethod
+     def from_dict(cls, d):
+         new_d = d.copy()
  
-         new_d['extra'] = dict((k, new_d.pop(k)) for k in d.iterkeys()
-                               if k not in cls.attributes and k != 'extra')
+         new_d['extra'] = dict((k, new_d.pop(k)) for k in d.iterkeys()
+                               if k not in cls.attributes and k != 'extra')
  
-         return cls(**new_d)
+         return cls(**new_d)
  
  and the cls(**new_d) ends up calling the __setitem__ method of
  keystone.openstack.common.db.models.ModelBase, which looks like this:
  
-     def __setitem__(self, key, value):
-         setattr(self, key, value)
+     def __setitem__(self, key, value):
+         setattr(self, key, value)
  
  The problem is that because new_d's 'extra' key is set, setattr() tries
  to set an attribute of the model called 'extra', which blows up if the
  model does not have a column in its table called "extra".
  
  Frankly, the denormalized "extra" JSON blob columns just need to be
- killed off. They poor relational database design, in that they aren't
- searchable or indexable and don't properly describe the attributes of
- the entity.
+ killed off. They are poor relational database design, in that they
+ aren't searchable or indexable and don't properly describe the
+ attributes of the entity.

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to Keystone.
https://bugs.launchpad.net/bugs/1265071

Title:
  extra column is required for new models, otherwise unit tests fail

Status in OpenStack Identity (Keystone):
  New

Bug description:
  If a new model does not contain an "extra" column of type JsonBlob,
  unit tests fail.

  The reason is because keystone.common.sql.core.DictBase's from_dict()
  classmethod looks like this:

      @classmethod
      def from_dict(cls, d):
          new_d = d.copy()

          new_d['extra'] = dict((k, new_d.pop(k)) for k in d.iterkeys()
                                if k not in cls.attributes and k != 'extra')

          return cls(**new_d)

  and the cls(**new_d) ends up calling the __setitem__ method of
  keystone.openstack.common.db.models.ModelBase, which looks like this:

      def __setitem__(self, key, value):
          setattr(self, key, value)

  The problem is that because new_d's 'extra' key is set, setattr()
  tries to set an attribute of the model called 'extra', which blows up
  if the model does not have a column in its table called "extra".

  Frankly, the denormalized "extra" JSON blob columns just need to be
  killed off. They are poor relational database design, in that they
  aren't searchable or indexable and don't properly describe the
  attributes of the entity.

To manage notifications about this bug go to:
https://bugs.launchpad.net/keystone/+bug/1265071/+subscriptions