yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1906500] [NEW] [ovn] Tempest tests failing while creating security group driver with KeyError: 'remote_address_group_id'

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: yatin <1906500@xxxxxxxxxxxxxxxxxx>
Date: Wed, 02 Dec 2020 10:22:45 -0000
Reply-to: Bug 1906500 <1906500@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
Public bug reported:

This is failing post
https://review.opendev.org/c/openstack/neutron/+/751110, detected in a
packstack job, it fails with below Traceback:-

2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource Traceback (most recent call last):
2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource   File "/usr/lib/python3.6/site-packages/neutron/api/v2/resource.py", line 98, in resource
2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource     result = method(request=request, **args)
2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource   File "/usr/lib/python3.6/site-packages/neutron/api/v2/base.py", line 437, in create
2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource     return self._create(request, body, **kwargs)
2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource   File "/usr/lib/python3.6/site-packages/neutron_lib/db/api.py", line 139, in wrapped
2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource     setattr(e, '_RETRY_EXCEEDED', True)
2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource   File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource     self.force_reraise()
2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource   File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource     six.reraise(self.type_, self.value, self.tb)
2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource   File "/usr/lib/python3.6/site-packages/six.py", line 703, in reraise
2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource     raise value
2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource   File "/usr/lib/python3.6/site-packages/neutron_lib/db/api.py", line 135, in wrapped
2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource     return f(*args, **kwargs)
2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource   File "/usr/lib/python3.6/site-packages/oslo_db/api.py", line 154, in wrapper
2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource     ectxt.value = e.inner_exc
2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource   File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource     self.force_reraise()
2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource   File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource     six.reraise(self.type_, self.value, self.tb)
2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource   File "/usr/lib/python3.6/site-packages/six.py", line 703, in reraise
2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource     raise value
2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource   File "/usr/lib/python3.6/site-packages/oslo_db/api.py", line 142, in wrapper
2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource     return f(*args, **kwargs)
2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource   File "/usr/lib/python3.6/site-packages/neutron_lib/db/api.py", line 183, in wrapped
2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource     LOG.debug("Retry wrapper got retriable exception: %s", e)
2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource   File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource     self.force_reraise()
2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource   File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource     six.reraise(self.type_, self.value, self.tb)
2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource   File "/usr/lib/python3.6/site-packages/six.py", line 703, in reraise
2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource     raise value
2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource   File "/usr/lib/python3.6/site-packages/neutron_lib/db/api.py", line 179, in wrapped
2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource     return f(*dup_args, **dup_kwargs)
2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource   File "/usr/lib/python3.6/site-packages/neutron/api/v2/base.py", line 547, in _create
2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource     objs = do_create(body, bulk=True)
2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource   File "/usr/lib/python3.6/site-packages/neutron/api/v2/base.py", line 543, in do_create
2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource     request.context, reservation.reservation_id)
2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource   File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource     self.force_reraise()
2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource   File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource     six.reraise(self.type_, self.value, self.tb)
2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource   File "/usr/lib/python3.6/site-packages/six.py", line 703, in reraise
2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource     raise value
2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource   File "/usr/lib/python3.6/site-packages/neutron/api/v2/base.py", line 536, in do_create
2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource     return obj_creator(request.context, **kwargs)
2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource   File "/usr/lib/python3.6/site-packages/neutron/db/securitygroups_rpc_base.py", line 67, in create_security_group_rule_bulk
2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource     context, security_group_rules)
2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource   File "/usr/lib/python3.6/site-packages/neutron_lib/db/api.py", line 233, in wrapped
2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource     return method(*args, **kwargs)
2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource   File "/usr/lib/python3.6/site-packages/neutron_lib/db/api.py", line 139, in wrapped
2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource     setattr(e, '_RETRY_EXCEEDED', True)
2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource   File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource     self.force_reraise()
2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource   File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource     six.reraise(self.type_, self.value, self.tb)
2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource   File "/usr/lib/python3.6/site-packages/six.py", line 703, in reraise
2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource     raise value
2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource   File "/usr/lib/python3.6/site-packages/neutron_lib/db/api.py", line 135, in wrapped
2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource     return f(*args, **kwargs)
2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource   File "/usr/lib/python3.6/site-packages/oslo_db/api.py", line 154, in wrapper
2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource     ectxt.value = e.inner_exc
2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource   File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource     self.force_reraise()
2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource   File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource     six.reraise(self.type_, self.value, self.tb)
2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource   File "/usr/lib/python3.6/site-packages/six.py", line 703, in reraise
2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource     raise value
2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource   File "/usr/lib/python3.6/site-packages/oslo_db/api.py", line 142, in wrapper
2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource     return f(*args, **kwargs)
2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource   File "/usr/lib/python3.6/site-packages/neutron_lib/db/api.py", line 183, in wrapped
2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource     LOG.debug("Retry wrapper got retriable exception: %s", e)
2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource   File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource     self.force_reraise()
2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource   File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource     six.reraise(self.type_, self.value, self.tb)
2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource   File "/usr/lib/python3.6/site-packages/six.py", line 703, in reraise
2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource     raise value
2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource   File "/usr/lib/python3.6/site-packages/neutron_lib/db/api.py", line 179, in wrapped
2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource     return f(*dup_args, **dup_kwargs)
2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource   File "/usr/lib/python3.6/site-packages/neutron/db/securitygroups_db.py", line 387, in create_security_group_rule_bulk_native
2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource     context, security_group_rules)
2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource   File "/usr/lib/python3.6/site-packages/neutron/db/securitygroups_db.py", line 666, in _validate_security_group_rules
2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource     self._validate_security_group_rule(context, rule)
2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource   File "/usr/lib/python3.6/site-packages/neutron/db/securitygroups_db.py", line 640, in _validate_security_group_rule
2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource     remote_address_group_id = rule['remote_address_group_id']
2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource KeyError: 'remote_address_group_id'
2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource 

Example logs:-
Example:- https://logserver.rdoproject.org/ci.centos.org/weirdo-generic-packstack-scenario001/11909/weirdo-project/logs/latest/tempest.log.txt.gz
https://logserver.rdoproject.org/ci.centos.org/weirdo-generic-packstack-scenario001/11909/weirdo-project/logs/neutron/server.log.txt.gz


The issue didn't got detected in neutron devstack jobs running with ovn
as extension security-groups-remote-address-group got loaded via
'logger' mechanism driver. security-groups-remote-address-group needs to
be loaded with ovn driver too to get security group creation working

** Affects: neutron
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1906500

Title:
  [ovn] Tempest tests failing while creating security group driver with
  KeyError: 'remote_address_group_id'

Status in neutron:
  New

Bug description:
  This is failing post
  https://review.opendev.org/c/openstack/neutron/+/751110, detected in a
  packstack job, it fails with below Traceback:-

  2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource Traceback (most recent call last):
  2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource   File "/usr/lib/python3.6/site-packages/neutron/api/v2/resource.py", line 98, in resource
  2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource     result = method(request=request, **args)
  2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource   File "/usr/lib/python3.6/site-packages/neutron/api/v2/base.py", line 437, in create
  2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource     return self._create(request, body, **kwargs)
  2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource   File "/usr/lib/python3.6/site-packages/neutron_lib/db/api.py", line 139, in wrapped
  2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource     setattr(e, '_RETRY_EXCEEDED', True)
  2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource   File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__
  2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource     self.force_reraise()
  2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource   File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
  2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource     six.reraise(self.type_, self.value, self.tb)
  2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource   File "/usr/lib/python3.6/site-packages/six.py", line 703, in reraise
  2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource     raise value
  2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource   File "/usr/lib/python3.6/site-packages/neutron_lib/db/api.py", line 135, in wrapped
  2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource     return f(*args, **kwargs)
  2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource   File "/usr/lib/python3.6/site-packages/oslo_db/api.py", line 154, in wrapper
  2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource     ectxt.value = e.inner_exc
  2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource   File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__
  2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource     self.force_reraise()
  2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource   File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
  2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource     six.reraise(self.type_, self.value, self.tb)
  2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource   File "/usr/lib/python3.6/site-packages/six.py", line 703, in reraise
  2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource     raise value
  2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource   File "/usr/lib/python3.6/site-packages/oslo_db/api.py", line 142, in wrapper
  2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource     return f(*args, **kwargs)
  2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource   File "/usr/lib/python3.6/site-packages/neutron_lib/db/api.py", line 183, in wrapped
  2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource     LOG.debug("Retry wrapper got retriable exception: %s", e)
  2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource   File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__
  2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource     self.force_reraise()
  2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource   File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
  2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource     six.reraise(self.type_, self.value, self.tb)
  2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource   File "/usr/lib/python3.6/site-packages/six.py", line 703, in reraise
  2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource     raise value
  2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource   File "/usr/lib/python3.6/site-packages/neutron_lib/db/api.py", line 179, in wrapped
  2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource     return f(*dup_args, **dup_kwargs)
  2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource   File "/usr/lib/python3.6/site-packages/neutron/api/v2/base.py", line 547, in _create
  2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource     objs = do_create(body, bulk=True)
  2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource   File "/usr/lib/python3.6/site-packages/neutron/api/v2/base.py", line 543, in do_create
  2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource     request.context, reservation.reservation_id)
  2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource   File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__
  2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource     self.force_reraise()
  2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource   File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
  2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource     six.reraise(self.type_, self.value, self.tb)
  2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource   File "/usr/lib/python3.6/site-packages/six.py", line 703, in reraise
  2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource     raise value
  2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource   File "/usr/lib/python3.6/site-packages/neutron/api/v2/base.py", line 536, in do_create
  2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource     return obj_creator(request.context, **kwargs)
  2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource   File "/usr/lib/python3.6/site-packages/neutron/db/securitygroups_rpc_base.py", line 67, in create_security_group_rule_bulk
  2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource     context, security_group_rules)
  2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource   File "/usr/lib/python3.6/site-packages/neutron_lib/db/api.py", line 233, in wrapped
  2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource     return method(*args, **kwargs)
  2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource   File "/usr/lib/python3.6/site-packages/neutron_lib/db/api.py", line 139, in wrapped
  2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource     setattr(e, '_RETRY_EXCEEDED', True)
  2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource   File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__
  2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource     self.force_reraise()
  2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource   File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
  2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource     six.reraise(self.type_, self.value, self.tb)
  2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource   File "/usr/lib/python3.6/site-packages/six.py", line 703, in reraise
  2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource     raise value
  2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource   File "/usr/lib/python3.6/site-packages/neutron_lib/db/api.py", line 135, in wrapped
  2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource     return f(*args, **kwargs)
  2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource   File "/usr/lib/python3.6/site-packages/oslo_db/api.py", line 154, in wrapper
  2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource     ectxt.value = e.inner_exc
  2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource   File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__
  2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource     self.force_reraise()
  2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource   File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
  2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource     six.reraise(self.type_, self.value, self.tb)
  2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource   File "/usr/lib/python3.6/site-packages/six.py", line 703, in reraise
  2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource     raise value
  2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource   File "/usr/lib/python3.6/site-packages/oslo_db/api.py", line 142, in wrapper
  2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource     return f(*args, **kwargs)
  2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource   File "/usr/lib/python3.6/site-packages/neutron_lib/db/api.py", line 183, in wrapped
  2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource     LOG.debug("Retry wrapper got retriable exception: %s", e)
  2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource   File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__
  2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource     self.force_reraise()
  2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource   File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
  2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource     six.reraise(self.type_, self.value, self.tb)
  2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource   File "/usr/lib/python3.6/site-packages/six.py", line 703, in reraise
  2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource     raise value
  2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource   File "/usr/lib/python3.6/site-packages/neutron_lib/db/api.py", line 179, in wrapped
  2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource     return f(*dup_args, **dup_kwargs)
  2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource   File "/usr/lib/python3.6/site-packages/neutron/db/securitygroups_db.py", line 387, in create_security_group_rule_bulk_native
  2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource     context, security_group_rules)
  2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource   File "/usr/lib/python3.6/site-packages/neutron/db/securitygroups_db.py", line 666, in _validate_security_group_rules
  2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource     self._validate_security_group_rule(context, rule)
  2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource   File "/usr/lib/python3.6/site-packages/neutron/db/securitygroups_db.py", line 640, in _validate_security_group_rule
  2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource     remote_address_group_id = rule['remote_address_group_id']
  2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource KeyError: 'remote_address_group_id'
  2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource 

  Example logs:-
  Example:- https://logserver.rdoproject.org/ci.centos.org/weirdo-generic-packstack-scenario001/11909/weirdo-project/logs/latest/tempest.log.txt.gz
  https://logserver.rdoproject.org/ci.centos.org/weirdo-generic-packstack-scenario001/11909/weirdo-project/logs/neutron/server.log.txt.gz


  The issue didn't got detected in neutron devstack jobs running with
  ovn as extension security-groups-remote-address-group got loaded via
  'logger' mechanism driver. security-groups-remote-address-group needs
  to be loaded with ovn driver too to get security group creation
  working

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1906500/+subscriptions
Follow ups

[Bug 1906500] Re: [ovn] Tempest tests failing while creating security group driver with KeyError: 'remote_address_group_id'
From: Brian Haley, 2022-12-06