yahoo-eng-team team mailing list archive

[Chris Friesen <chris.friesen@xxxxxxxxxxxxx>]

Public bug reported:

Sorry for the complicated write-up below, but the issue is complicated.


I'm running into a problem between Mitaka and Kilo, but I *think* it'll also hit Mitaka/Liberty.  The problem scenario is when we have older and newer services talking to each other.  The problem occurs when nova-conductor writes to an object field that is removed in obj_make_compatible().  In particular, I'm hitting this with 'parent_addr' in the PciDevice class since it gets written in PciDevice._from_db_object().

In oslo_versionedobjects/base.py the remotable() function has the following line:
	self._changed_fields = set(updates.get('obj_what_changed', []))

This blindly sets the local self._changed_fields to be whatever the
remote end sent as updates['obj_what_changed'].

This is a problem because the far end can include fields that don't
actually exist in the older object version.  On the far end (which may
be newer) in nova.conductor.manager.ConductorManager.object_action(), we
will call the following (where 'objinst' is the current version of the
object):

        updates['obj_what_changed'] = objinst.obj_what_changed()

Since this is called against the newer object code, it can specify
fields that do not exist in the older version of the object if nova-
conductor has written those fields.

The only workaround I've been able to come up with for this is to modify
oslo_versionedobjects.base.remotable() to only include a field in
self._changed_fields if it's in self.fields.  This requires updating the
older code prior to an upgrade, however.


I think there's another related issue.  In VersionedObject.obj_to_primitive() we set the changes in the primitive like this:

        if self.obj_what_changed():
            obj[self._obj_primitive_key('changes')] = list(
                self.obj_what_changed())

Since we call self.obj_what_changed() on the newer version of the
object, I think we will include changes to fields that were removed by
obj_make_compatible_from_manifest().

It seems to me that in obj_to_primitive() we should not allow fields to
be included in obj[self._obj_primitive_key('changes')] unless they're
also listed in obj[self._obj_primitive_key('data')].

** Affects: nova
     Importance: Undecided
         Status: New


** Tags: compute oslo

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1613488

Title:
  changed fields of versionedobjects not tracked properly when down-
  versioning object

Status in OpenStack Compute (nova):
  New

Bug description:
  Sorry for the complicated write-up below, but the issue is
  complicated.

  
  I'm running into a problem between Mitaka and Kilo, but I *think* it'll also hit Mitaka/Liberty.  The problem scenario is when we have older and newer services talking to each other.  The problem occurs when nova-conductor writes to an object field that is removed in obj_make_compatible().  In particular, I'm hitting this with 'parent_addr' in the PciDevice class since it gets written in PciDevice._from_db_object().

  In oslo_versionedobjects/base.py the remotable() function has the following line:
  	self._changed_fields = set(updates.get('obj_what_changed', []))

  This blindly sets the local self._changed_fields to be whatever the
  remote end sent as updates['obj_what_changed'].

  This is a problem because the far end can include fields that don't
  actually exist in the older object version.  On the far end (which may
  be newer) in nova.conductor.manager.ConductorManager.object_action(),
  we will call the following (where 'objinst' is the current version of
  the object):

          updates['obj_what_changed'] = objinst.obj_what_changed()

  Since this is called against the newer object code, it can specify
  fields that do not exist in the older version of the object if nova-
  conductor has written those fields.

  The only workaround I've been able to come up with for this is to
  modify oslo_versionedobjects.base.remotable() to only include a field
  in self._changed_fields if it's in self.fields.  This requires
  updating the older code prior to an upgrade, however.

  
  I think there's another related issue.  In VersionedObject.obj_to_primitive() we set the changes in the primitive like this:

          if self.obj_what_changed():
              obj[self._obj_primitive_key('changes')] = list(
                  self.obj_what_changed())

  Since we call self.obj_what_changed() on the newer version of the
  object, I think we will include changes to fields that were removed by
  obj_make_compatible_from_manifest().

  It seems to me that in obj_to_primitive() we should not allow fields
  to be included in obj[self._obj_primitive_key('changes')] unless
  they're also listed in obj[self._obj_primitive_key('data')].

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1613488/+subscriptions