yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1657358] [NEW] Create fw_policy with used fw_rule hit internal error

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: zhaobo <zhaobo6@xxxxxxxxxx>
Date: Wed, 18 Jan 2017 08:02:35 -0000
Reply-to: Bug 1657358 <1657358@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
Public bug reported:

Hit internal error when create fw_policy with exist fw_rule.

If there is a policy with this fw_rule already, new request create an
new policy with the fw_rule, neutron-server will hit internal error.


repo
-----
1. create a fw_ruleA
2. create a fw_policyX with fw_ruleA
3. create another fw_policyY with fw_ruleA    ==== ERROR


2017-01-16 16:01:20.220 ERROR neutron.api.v2.resource [req-d4fa6f6b-852d-4233-a39b-9ae73dbd8c45 admin a4d7fafd8b6e42f5aa1de690e360d285] create failed: No details.
2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource Traceback (most recent call last):
2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource   File "/opt/stack/neutron/neutron/api/v2/resource.py", line 79, in resource
2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource     result = method(request=request, **args)
2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource   File "/opt/stack/neutron/neutron/api/v2/base.py", line 438, in create
2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource     return self._create(request, body, **kwargs)
2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource   File "/opt/stack/neutron/neutron/db/api.py", line 92, in wrapped
2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource     setattr(e, '_RETRY_EXCEEDED', True)
2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource     self.force_reraise()
2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource     six.reraise(self.type_, self.value, self.tb)
2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource   File "/opt/stack/neutron/neutron/db/api.py", line 88, in wrapped
2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource     return f(*args, **kwargs)
2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/oslo_db/api.py", line 151, in wrapper
2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource     ectxt.value = e.inner_exc
2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource     self.force_reraise()
2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource     six.reraise(self.type_, self.value, self.tb)
2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/oslo_db/api.py", line 139, in wrapper
2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource     return f(*args, **kwargs)
2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource   File "/opt/stack/neutron/neutron/db/api.py", line 128, in wrapped
2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource     traceback.format_exc())
2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource     self.force_reraise()
2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource     six.reraise(self.type_, self.value, self.tb)
2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource   File "/opt/stack/neutron/neutron/db/api.py", line 123, in wrapped
2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource     return f(*dup_args, **dup_kwargs)
2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource   File "/opt/stack/neutron/neutron/api/v2/base.py", line 551, in _create
2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource     obj = do_create(body)
2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource   File "/opt/stack/neutron/neutron/api/v2/base.py", line 533, in do_create
2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource     request.context, reservation.reservation_id)
2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource     self.force_reraise()
2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource     six.reraise(self.type_, self.value, self.tb)
2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource   File "/opt/stack/neutron/neutron/api/v2/base.py", line 526, in do_create
2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource     return obj_creator(request.context, **kwargs)
2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource   File "/opt/stack/neutron-fwaas/neutron_fwaas/db/firewall/v2/firewall_db_v2.py", line 641, in create_firewall_policy
2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource     self._set_rules_for_policy(context, fwp_db, fwp)
2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource   File "/opt/stack/neutron-fwaas/neutron_fwaas/db/firewall/v2/firewall_db_v2.py", line 627, in _set_rules_for_policy
2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource     fwp_db.rule_associations.reorder()
2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 553, in __exit__
2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource     self.rollback()
2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/util/langhelpers.py", line 60, in __exit__
2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource     compat.reraise(exc_type, exc_value, exc_tb)
2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 550, in __exit__
2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource     self.commit()
2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 455, in commit
2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource     self._prepare_impl()
2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 435, in _prepare_impl
2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource     self.session.flush()
2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 2080, in flush
2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource     self._flush(objects)
2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 2198, in _flush
2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource     transaction.rollback(_capture_exception=True)
2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/util/langhelpers.py", line 60, in __exit__
2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource     compat.reraise(exc_type, exc_value, exc_tb)
2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 2162, in _flush
2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource     flush_context.execute()
2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/unitofwork.py", line 373, in execute
2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource     rec.execute(self)
2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/unitofwork.py", line 487, in execute
2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource     self.dependency_processor.process_saves(uow, states)
2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/dependency.py", line 558, in process_saves
2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource     uowcommit, False)
2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/dependency.py", line 575, in _synchronize
2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource     sync.clear(dest, self.mapper, self.prop.synchronize_pairs)
2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/sync.py", line 74, in clear
2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource     (r, orm_util.state_str(dest))
2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource AssertionError: Dependency rule tried to blank-out primary key column 'firewall_policy_rule_associations_v2.firewall_policy_id' on instance '<FirewallPolicyRuleAssociation at 0x7fb4729b7590>'

** Affects: neutron
     Importance: Undecided
     Assignee: zhaobo (zhaobo6)
         Status: New


** Tags: fwaas

** Changed in: neutron
     Assignee: (unassigned) => zhaobo (zhaobo6)

** Tags added: fwaas

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1657358

Title:
  Create fw_policy with used fw_rule hit internal error

Status in neutron:
  New

Bug description:
  Hit internal error when create fw_policy with exist fw_rule.

  If there is a policy with this fw_rule already, new request create an
  new policy with the fw_rule, neutron-server will hit internal error.

  
  repo
  -----
  1. create a fw_ruleA
  2. create a fw_policyX with fw_ruleA
  3. create another fw_policyY with fw_ruleA    ==== ERROR

  
  2017-01-16 16:01:20.220 ERROR neutron.api.v2.resource [req-d4fa6f6b-852d-4233-a39b-9ae73dbd8c45 admin a4d7fafd8b6e42f5aa1de690e360d285] create failed: No details.
  2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource Traceback (most recent call last):
  2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource   File "/opt/stack/neutron/neutron/api/v2/resource.py", line 79, in resource
  2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource     result = method(request=request, **args)
  2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource   File "/opt/stack/neutron/neutron/api/v2/base.py", line 438, in create
  2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource     return self._create(request, body, **kwargs)
  2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource   File "/opt/stack/neutron/neutron/db/api.py", line 92, in wrapped
  2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource     setattr(e, '_RETRY_EXCEEDED', True)
  2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
  2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource     self.force_reraise()
  2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
  2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource     six.reraise(self.type_, self.value, self.tb)
  2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource   File "/opt/stack/neutron/neutron/db/api.py", line 88, in wrapped
  2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource     return f(*args, **kwargs)
  2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/oslo_db/api.py", line 151, in wrapper
  2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource     ectxt.value = e.inner_exc
  2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
  2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource     self.force_reraise()
  2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
  2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource     six.reraise(self.type_, self.value, self.tb)
  2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/oslo_db/api.py", line 139, in wrapper
  2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource     return f(*args, **kwargs)
  2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource   File "/opt/stack/neutron/neutron/db/api.py", line 128, in wrapped
  2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource     traceback.format_exc())
  2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
  2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource     self.force_reraise()
  2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
  2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource     six.reraise(self.type_, self.value, self.tb)
  2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource   File "/opt/stack/neutron/neutron/db/api.py", line 123, in wrapped
  2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource     return f(*dup_args, **dup_kwargs)
  2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource   File "/opt/stack/neutron/neutron/api/v2/base.py", line 551, in _create
  2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource     obj = do_create(body)
  2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource   File "/opt/stack/neutron/neutron/api/v2/base.py", line 533, in do_create
  2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource     request.context, reservation.reservation_id)
  2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
  2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource     self.force_reraise()
  2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
  2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource     six.reraise(self.type_, self.value, self.tb)
  2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource   File "/opt/stack/neutron/neutron/api/v2/base.py", line 526, in do_create
  2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource     return obj_creator(request.context, **kwargs)
  2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource   File "/opt/stack/neutron-fwaas/neutron_fwaas/db/firewall/v2/firewall_db_v2.py", line 641, in create_firewall_policy
  2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource     self._set_rules_for_policy(context, fwp_db, fwp)
  2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource   File "/opt/stack/neutron-fwaas/neutron_fwaas/db/firewall/v2/firewall_db_v2.py", line 627, in _set_rules_for_policy
  2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource     fwp_db.rule_associations.reorder()
  2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 553, in __exit__
  2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource     self.rollback()
  2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/util/langhelpers.py", line 60, in __exit__
  2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource     compat.reraise(exc_type, exc_value, exc_tb)
  2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 550, in __exit__
  2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource     self.commit()
  2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 455, in commit
  2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource     self._prepare_impl()
  2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 435, in _prepare_impl
  2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource     self.session.flush()
  2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 2080, in flush
  2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource     self._flush(objects)
  2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 2198, in _flush
  2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource     transaction.rollback(_capture_exception=True)
  2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/util/langhelpers.py", line 60, in __exit__
  2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource     compat.reraise(exc_type, exc_value, exc_tb)
  2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 2162, in _flush
  2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource     flush_context.execute()
  2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/unitofwork.py", line 373, in execute
  2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource     rec.execute(self)
  2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/unitofwork.py", line 487, in execute
  2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource     self.dependency_processor.process_saves(uow, states)
  2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/dependency.py", line 558, in process_saves
  2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource     uowcommit, False)
  2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/dependency.py", line 575, in _synchronize
  2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource     sync.clear(dest, self.mapper, self.prop.synchronize_pairs)
  2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/sync.py", line 74, in clear
  2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource     (r, orm_util.state_str(dest))
  2017-01-16 16:01:20.220 TRACE neutron.api.v2.resource AssertionError: Dependency rule tried to blank-out primary key column 'firewall_policy_rule_associations_v2.firewall_policy_id' on instance '<FirewallPolicyRuleAssociation at 0x7fb4729b7590>'

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1657358/+subscriptions
Follow ups

[Bug 1657358] Re: Create fw_policy with used fw_rule hit internal error
From: OpenStack Infra, 2017-01-22