yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1611627] Re: revision plugin throwing objectdeletederror

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: OpenStack Infra <1611627@xxxxxxxxxxxxxxxxxx>
Date: Fri, 12 Aug 2016 09:46:00 -0000
Reply-to: Bug 1611627 <1611627@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
Reviewed:  https://review.openstack.org/353277
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=9e8bcf45a53718da588e70785c77dd7215e12017
Submitter: Jenkins
Branch:    master

commit 9e8bcf45a53718da588e70785c77dd7215e12017
Author: Kevin Benton <kevin@xxxxxxxxxx>
Date:   Fri Aug 5 16:25:29 2016 -0700

    Make revision bump robust to concurrent removals
    
    If there expired objects in the session that are concurrently
    deleted by another sesssion, the revision code will get an
    ObjectDeletedError when it goes to check if the expired object
    is a related object to the object it is handling. So we need
    to catch this error an continue on since there is nothing it
    needs to do.
    
    Change-Id: Ie6034f36c34099f2646840cbdd569a4398534df5
    Closes-Bug: #1611627


** Changed in: neutron
       Status: In Progress => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1611627

Title:
  revision plugin throwing objectdeletederror

Status in neutron:
  Fix Released

Bug description:
  If there is an expired object in the session that got concurrently
  deleted by another session, the revision plugin may throw an
  ObjectDeletedError when it goes to bump the revision and references
  expired attributes (causing a DB lookup that returns nothing).

  
  2016-08-09 23:57:02.093 17791 ERROR neutron.api.v2.resource [req-c1b008ba-a6a7-4ac0-bb71-baf27253e098 tempest-NetworksTestDHCPv6-1396555541 -] create failed: No details.
  2016-08-09 23:57:02.093 17791 ERROR neutron.api.v2.resource Traceback (most recent call last):
  2016-08-09 23:57:02.093 17791 ERROR neutron.api.v2.resource   File "/opt/stack/new/neutron/neutron/api/v2/resource.py", line 79, in resource
  2016-08-09 23:57:02.093 17791 ERROR neutron.api.v2.resource     result = method(request=request, **args)
  2016-08-09 23:57:02.093 17791 ERROR neutron.api.v2.resource   File "/opt/stack/new/neutron/neutron/api/v2/base.py", line 397, in create
  2016-08-09 23:57:02.093 17791 ERROR neutron.api.v2.resource     return self._create(request, body, **kwargs)
  2016-08-09 23:57:02.093 17791 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/oslo_db/api.py", line 151, in wrapper
  2016-08-09 23:57:02.093 17791 ERROR neutron.api.v2.resource     ectxt.value = e.inner_exc
  2016-08-09 23:57:02.093 17791 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
  2016-08-09 23:57:02.093 17791 ERROR neutron.api.v2.resource     self.force_reraise()
  2016-08-09 23:57:02.093 17791 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
  2016-08-09 23:57:02.093 17791 ERROR neutron.api.v2.resource     six.reraise(self.type_, self.value, self.tb)
  2016-08-09 23:57:02.093 17791 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/oslo_db/api.py", line 139, in wrapper
  2016-08-09 23:57:02.093 17791 ERROR neutron.api.v2.resource     return f(*args, **kwargs)
  2016-08-09 23:57:02.093 17791 ERROR neutron.api.v2.resource   File "/opt/stack/new/neutron/neutron/api/v2/base.py", line 510, in _create
  2016-08-09 23:57:02.093 17791 ERROR neutron.api.v2.resource     obj = do_create(body)
  2016-08-09 23:57:02.093 17791 ERROR neutron.api.v2.resource   File "/opt/stack/new/neutron/neutron/api/v2/base.py", line 492, in do_create
  2016-08-09 23:57:02.093 17791 ERROR neutron.api.v2.resource     request.context, reservation.reservation_id)
  2016-08-09 23:57:02.093 17791 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
  2016-08-09 23:57:02.093 17791 ERROR neutron.api.v2.resource     self.force_reraise()
  2016-08-09 23:57:02.093 17791 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
  2016-08-09 23:57:02.093 17791 ERROR neutron.api.v2.resource     six.reraise(self.type_, self.value, self.tb)
  2016-08-09 23:57:02.093 17791 ERROR neutron.api.v2.resource   File "/opt/stack/new/neutron/neutron/api/v2/base.py", line 485, in do_create
  2016-08-09 23:57:02.093 17791 ERROR neutron.api.v2.resource     return obj_creator(request.context, **kwargs)
  2016-08-09 23:57:02.093 17791 ERROR neutron.api.v2.resource   File "/opt/stack/new/neutron/neutron/common/utils.py", line 613, in inner
  2016-08-09 23:57:02.093 17791 ERROR neutron.api.v2.resource     return f(self, context, *args, **kwargs)
  2016-08-09 23:57:02.093 17791 ERROR neutron.api.v2.resource   File "/opt/stack/new/neutron/neutron/plugins/ml2/plugin.py", line 977, in create_subnet
  2016-08-09 23:57:02.093 17791 ERROR neutron.api.v2.resource     result, mech_context = self._create_subnet_db(context, subnet)
  2016-08-09 23:57:02.093 17791 ERROR neutron.api.v2.resource   File "/opt/stack/new/neutron/neutron/plugins/ml2/plugin.py", line 965, in _create_subnet_db
  2016-08-09 23:57:02.093 17791 ERROR neutron.api.v2.resource     result = super(Ml2Plugin, self).create_subnet(context, subnet)
  2016-08-09 23:57:02.093 17791 ERROR neutron.api.v2.resource   File "/opt/stack/new/neutron/neutron/db/db_base_plugin_v2.py", line 723, in create_subnet
  2016-08-09 23:57:02.093 17791 ERROR neutron.api.v2.resource     return self._create_subnet(context, subnet, subnetpool_id)
  2016-08-09 23:57:02.093 17791 ERROR neutron.api.v2.resource   File "/opt/stack/new/neutron/neutron/db/db_base_plugin_v2.py", line 623, in _create_subnet
  2016-08-09 23:57:02.093 17791 ERROR neutron.api.v2.resource     subnet, ipam_subnet)
  2016-08-09 23:57:02.093 17791 ERROR neutron.api.v2.resource   File "/opt/stack/new/neutron/neutron/db/ipam_non_pluggable_backend.py", line 350, in add_auto_addrs_on_network_ports
  2016-08-09 23:57:02.093 17791 ERROR neutron.api.v2.resource     context.session.add(allocated)
  2016-08-09 23:57:02.093 17791 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 490, in __exit__
  2016-08-09 23:57:02.093 17791 ERROR neutron.api.v2.resource     self.rollback()
  2016-08-09 23:57:02.093 17791 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/util/langhelpers.py", line 60, in __exit__
  2016-08-09 23:57:02.093 17791 ERROR neutron.api.v2.resource     compat.reraise(exc_type, exc_value, exc_tb)
  2016-08-09 23:57:02.093 17791 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 487, in __exit__
  2016-08-09 23:57:02.093 17791 ERROR neutron.api.v2.resource     self.commit()
  2016-08-09 23:57:02.093 17791 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 392, in commit
  2016-08-09 23:57:02.093 17791 ERROR neutron.api.v2.resource     self._prepare_impl()
  2016-08-09 23:57:02.093 17791 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 372, in _prepare_impl
  2016-08-09 23:57:02.093 17791 ERROR neutron.api.v2.resource     self.session.flush()
  2016-08-09 23:57:02.093 17791 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 2019, in flush
  2016-08-09 23:57:02.093 17791 ERROR neutron.api.v2.resource     self._flush(objects)
  2016-08-09 23:57:02.093 17791 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 2046, in _flush
  2016-08-09 23:57:02.093 17791 ERROR neutron.api.v2.resource     self.dispatch.before_flush(self, flush_context, objects)
  2016-08-09 23:57:02.093 17791 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/event/attr.py", line 218, in __call__
  2016-08-09 23:57:02.093 17791 ERROR neutron.api.v2.resource     fn(*args, **kw)
  2016-08-09 23:57:02.093 17791 ERROR neutron.api.v2.resource   File "/opt/stack/new/neutron/neutron/services/revisions/revision_plugin.py", line 52, in bump_revisions
  2016-08-09 23:57:02.093 17791 ERROR neutron.api.v2.resource     self._bump_related_revisions(session, obj)
  2016-08-09 23:57:02.093 17791 ERROR neutron.api.v2.resource   File "/opt/stack/new/neutron/neutron/services/revisions/revision_plugin.py", line 56, in _bump_related_revisions
  2016-08-09 23:57:02.093 17791 ERROR neutron.api.v2.resource     related_obj = self._find_related_obj(session, obj, revises_col)
  2016-08-09 23:57:02.093 17791 ERROR neutron.api.v2.resource   File "/opt/stack/new/neutron/neutron/services/revisions/revision_plugin.py", line 100, in _find_related_obj
  2016-08-09 23:57:02.093 17791 ERROR neutron.api.v2.resource     if getattr(session_obj, pk.name) == rel_id:
  2016-08-09 23:57:02.093 17791 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/attributes.py", line 237, in __get__
  2016-08-09 23:57:02.093 17791 ERROR neutron.api.v2.resource     return self.impl.get(instance_state(instance), dict_)
  2016-08-09 23:57:02.093 17791 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/attributes.py", line 578, in get
  2016-08-09 23:57:02.093 17791 ERROR neutron.api.v2.resource     value = state._load_expired(state, passive)
  2016-08-09 23:57:02.093 17791 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/state.py", line 474, in _load_expired
  2016-08-09 23:57:02.093 17791 ERROR neutron.api.v2.resource     self.manager.deferred_scalar_loader(self, toload)
  2016-08-09 23:57:02.093 17791 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/loading.py", line 669, in load_scalar_attributes
  2016-08-09 23:57:02.093 17791 ERROR neutron.api.v2.resource     raise orm_exc.ObjectDeletedError(state)
  2016-08-09 23:57:02.093 17791 ERROR neutron.api.v2.resource ObjectDeletedError: Instance '<Port at 0x7f994e799d50>' has been deleted, or its row is otherwise not present.

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1611627/+subscriptions
References

[Bug 1611627] [NEW] revision plugin throwing objectdeletederror
From: Kevin Benton, 2016-08-10