yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1928794] Re: [OVN] DHCP agent fails to update network

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: OpenStack Infra <1928794@xxxxxxxxxxxxxxxxxx>
Date: Mon, 19 Jul 2021 13:49:05 -0000
Reply-to: Bug 1928794 <1928794@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
Reviewed:  https://review.opendev.org/c/openstack/neutron/+/781555
Committed: https://opendev.org/openstack/neutron/commit/90980f496cfa3cc5df1c93cf834a44f33d3f1f6f
Submitter: "Zuul (22348)"
Branch:    master

commit 90980f496cfa3cc5df1c93cf834a44f33d3f1f6f
Author: Terry Wilson <twilson@xxxxxxxxxx>
Date:   Thu Mar 18 15:01:07 2021 -0500

    Add wait for the post-fork event to nb/sb objects
    
    It is possible for events from the nb/sb to fire before the opposite
    db connection is made. These events can call back into driver code
    which tries to access the other db before it is connected.
    
    Closes-Bug: #1929197
    Closes-Bug: #1928794
    Closes-Bug: #1929633
    Change-Id: If947581b90ced42981c4611c32de8f428a052c69


** Changed in: neutron
       Status: In Progress => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1928794

Title:
  [OVN] DHCP agent fails to update network

Status in neutron:
  Fix Released

Bug description:
  In latest Victoria build, DHCP agent is unable to update networks with
  OVN driver:

  2021-05-18 11:42:32.508 7 ERROR neutron.agent.dhcp.agent [req-e8c3cd96-2a53-4ea8-b2eb-4de64af3600c - - - - -] Unable to enable dhcp for c9caf247-edb4-4282-a253-b948de5f9da7.: oslo_messaging.rpc.client.RemoteError: Remote error: MechanismDriverError 
  ['Traceback (most recent call last):\n', '  File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", line 241, in inner\n    return func(*args, **kwargs)\n', '  File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", line 241, in inner\n    return func(*args, **kwargs)\n', '  File "/var/lib/kolla/venv/lib/python3.6/site-packages/neutron_lib/db/api.py", line 139, in wrapped\n    setattr(e, \'_RETRY_EXCEEDED\', True)\n', '  File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n    self.force_reraise()\n', '  File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n    six.reraise(self.type_, self.value, self.tb)\n', '  File "/usr/lib/python3.6/site-packages/six.py", line 703, in reraise\n    raise value\n', '  File "/var/lib/kolla/venv/lib/python3.6/site-packages/neutron_lib/db/api.py", line 135, in wrapped\n    return f(*args, **kwargs)\n', '  File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_db/api.py", line 154, in wrapper\n    ectxt.value = e.inner_exc\n', '  File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n    self.force_reraise()\n', '  File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n    six.reraise(self.type_, self.value, self.tb)\n', '  File "/usr/lib/python3.6/site-packages/six.py", line 703, in reraise\n    raise value\n', '  File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_db/api.py", line 142, in wrapper\n    return f(*args, **kwargs)\n', '  File "/var/lib/kolla/venv/lib/python3.6/site-packages/neutron_lib/db/api.py", line 183, in wrapped\n    LOG.debug("Retry wrapper got retriable exception: %s", e)\n', '  File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n    self.force_reraise()\n', '  File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n    six.reraise(self.type_, self.value, self.tb)\n', '  File "/usr/lib/python3.6/site-packages/six.py", line 703, in reraise\n    raise value\n', '  File "/var/lib/kolla/venv/lib/python3.6/site-packages/neutron_lib/db/api.py", line 179, in wrapped\n    return f(*dup_args, **dup_kwargs)\n', '  File "/var/lib/kolla/venv/lib/python3.6/site-packages/neutron/api/rpc/handlers/dhcp_rpc.py", line 311, in update_dhcp_port\n    return self._port_action(plugin, context, port, \'update_port\')\n', '  File "/var/lib/kolla/venv/lib/python3.6/site-packages/neutron/api/rpc/handlers/dhcp_rpc.py", line 101, in _port_action\n    return plugin.update_port(context, port[\'id\'], port)\n', '  File "/var/lib/kolla/venv/lib/python3.6/site-packages/neutron/common/utils.py", line 681, in inner\n    return f(self, context, *args, **kwargs)\n', '  File "/var/lib/kolla/venv/lib/python3.6/site-packages/neutron_lib/db/api.py", line 233, in wrapped\n    return method(*args, **kwargs)\n', '  File "/var/lib/kolla/venv/lib/python3.6/site-packages/neutron_lib/db/api.py", line 139, in wrapped\n    setattr(e, \'_RETRY_EXCEEDED\', True)\n', '  File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n    self.force_reraise()\n', '  File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n    six.reraise(self.type_, self.value, self.tb)\n', '  File "/usr/lib/python3.6/site-packages/six.py", line 703, in reraise\n    raise value\n', '  File "/var/lib/kolla/venv/lib/python3.6/site-packages/neutron_lib/db/api.py", line 135, in wrapped\n    return f(*args, **kwargs)\n', '  File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_db/api.py", line 154, in wrapper\n    ectxt.value = e.inner_exc\n', '  File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n    self.force_reraise()\n', '  File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n    six.reraise(self.type_, self.value, self.tb)\n', '  File "/usr/lib/python3.6/site-packages/six.py", line 703, in reraise\n    raise value\n', '  File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_db/api.py", line 142, in wrapper\n    return f(*args, **kwargs)\n', '  File "/var/lib/kolla/venv/lib/python3.6/site-packages/neutron_lib/db/api.py", line 183, in wrapped\n    LOG.debug("Retry wrapper got retriable exception: %s", e)\n', '  File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n    self.force_reraise()\n', '  File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n    six.reraise(self.type_, self.value, self.tb)\n', '  File "/usr/lib/python3.6/site-packages/six.py", line 703, in reraise\n    raise value\n', '  File "/var/lib/kolla/venv/lib/python3.6/site-packages/neutron_lib/db/api.py", line 179, in wrapped\n    return f(*dup_args, **dup_kwargs)\n', '  File "/var/lib/kolla/venv/lib/python3.6/site-packages/neutron/plugins/ml2/plugin.py", line 1728, in update_port\n    self.mechanism_manager.update_port_precommit(mech_context)\n', '  File "/var/lib/kolla/venv/lib/python3.6/site-packages/neutron/plugins/ml2/managers.py", line 731, in update_port_precommit\n    raise_db_retriable=True)\n', '  File "/var/lib/kolla/venv/lib/python3.6/site-packages/neutron/plugins/ml2/managers.py", line 495, in _call_on_drivers\n    errors=errors\n', 'neutron.plugins.ml2.common.exceptions.MechanismDriverError\n'].
  2021-05-18 11:42:32.508 7 ERROR neutron.agent.dhcp.agent Traceback (most recent call last):
  2021-05-18 11:42:32.508 7 ERROR neutron.agent.dhcp.agent   File "/var/lib/kolla/venv/lib/python3.6/site-packages/neutron/agent/dhcp/agent.py", line 227, in call_driver
  2021-05-18 11:42:32.508 7 ERROR neutron.agent.dhcp.agent     rv = getattr(driver, action)(**action_kwargs)
  2021-05-18 11:42:32.508 7 ERROR neutron.agent.dhcp.agent   File "/var/lib/kolla/venv/lib/python3.6/site-packages/neutron/agent/linux/dhcp.py", line 266, in enable
  2021-05-18 11:42:32.508 7 ERROR neutron.agent.dhcp.agent     common_utils.wait_until_true(self._enable, timeout=300)
  2021-05-18 11:42:32.508 7 ERROR neutron.agent.dhcp.agent   File "/var/lib/kolla/venv/lib/python3.6/site-packages/neutron/common/utils.py", line 697, in wait_until_true
  2021-05-18 11:42:32.508 7 ERROR neutron.agent.dhcp.agent     while not predicate():
  2021-05-18 11:42:32.508 7 ERROR neutron.agent.dhcp.agent   File "/var/lib/kolla/venv/lib/python3.6/site-packages/neutron/agent/linux/dhcp.py", line 278, in _enable
  2021-05-18 11:42:32.508 7 ERROR neutron.agent.dhcp.agent     interface_name = self.device_manager.setup(self.network)
  2021-05-18 11:42:32.508 7 ERROR neutron.agent.dhcp.agent   File "/var/lib/kolla/venv/lib/python3.6/site-packages/neutron/agent/linux/dhcp.py", line 1676, in setup
  2021-05-18 11:42:32.508 7 ERROR neutron.agent.dhcp.agent     self._cleanup_stale_devices(network, dhcp_port=None)
  2021-05-18 11:42:32.508 7 ERROR neutron.agent.dhcp.agent   File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__
  2021-05-18 11:42:32.508 7 ERROR neutron.agent.dhcp.agent     self.force_reraise()
  2021-05-18 11:42:32.508 7 ERROR neutron.agent.dhcp.agent   File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
  2021-05-18 11:42:32.508 7 ERROR neutron.agent.dhcp.agent     six.reraise(self.type_, self.value, self.tb)
  2021-05-18 11:42:32.508 7 ERROR neutron.agent.dhcp.agent   File "/usr/lib/python3.6/site-packages/six.py", line 703, in reraise
  2021-05-18 11:42:32.508 7 ERROR neutron.agent.dhcp.agent     raise value
  2021-05-18 11:42:32.508 7 ERROR neutron.agent.dhcp.agent   File "/var/lib/kolla/venv/lib/python3.6/site-packages/neutron/agent/linux/dhcp.py", line 1671, in setup
  2021-05-18 11:42:32.508 7 ERROR neutron.agent.dhcp.agent     port = self.setup_dhcp_port(network)
  2021-05-18 11:42:32.508 7 ERROR neutron.agent.dhcp.agent   File "/var/lib/kolla/venv/lib/python3.6/site-packages/neutron/agent/linux/dhcp.py", line 1610, in setup_dhcp_port
  2021-05-18 11:42:32.508 7 ERROR neutron.agent.dhcp.agent     dhcp_port = setup_method(network, device_id, dhcp_subnets)
  2021-05-18 11:42:32.508 7 ERROR neutron.agent.dhcp.agent   File "/var/lib/kolla/venv/lib/python3.6/site-packages/neutron/agent/linux/dhcp.py", line 1534, in _setup_reserved_dhcp_port
  2021-05-18 11:42:32.508 7 ERROR neutron.agent.dhcp.agent     'device_id': device_id}})
  2021-05-18 11:42:32.508 7 ERROR neutron.agent.dhcp.agent   File "/var/lib/kolla/venv/lib/python3.6/site-packages/neutron/agent/dhcp/agent.py", line 894, in update_dhcp_port
  2021-05-18 11:42:32.508 7 ERROR neutron.agent.dhcp.agent     port_id=port_id, port=port, host=self.host)
  2021-05-18 11:42:32.508 7 ERROR neutron.agent.dhcp.agent   File "/var/lib/kolla/venv/lib/python3.6/site-packages/neutron_lib/rpc.py", line 157, in call
  2021-05-18 11:42:32.508 7 ERROR neutron.agent.dhcp.agent     return self._original_context.call(ctxt, method, **kwargs)
  2021-05-18 11:42:32.508 7 ERROR neutron.agent.dhcp.agent   File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/client.py", line 179, in call
  2021-05-18 11:42:32.508 7 ERROR neutron.agent.dhcp.agent     transport_options=self.transport_options)
  2021-05-18 11:42:32.508 7 ERROR neutron.agent.dhcp.agent   File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/transport.py", line 128, in _send
  2021-05-18 11:42:32.508 7 ERROR neutron.agent.dhcp.agent     transport_options=transport_options)
  2021-05-18 11:42:32.508 7 ERROR neutron.agent.dhcp.agent   File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 682, in send
  2021-05-18 11:42:32.508 7 ERROR neutron.agent.dhcp.agent     transport_options=transport_options)
  2021-05-18 11:42:32.508 7 ERROR neutron.agent.dhcp.agent   File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 672, in _send
  2021-05-18 11:42:32.508 7 ERROR neutron.agent.dhcp.agent     raise result
  2021-05-18 11:42:32.508 7 ERROR neutron.agent.dhcp.agent oslo_messaging.rpc.client.RemoteError: Remote error: MechanismDriverError 
  2021-05-18 11:42:32.508 7 ERROR neutron.agent.dhcp.agent ['Traceback (most recent call last):\n', '  File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", line 241, in inner\n    return func(*args, **kwargs)\n', '  File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", line 241, in inner\n    return func(*args, **kwargs)\n', '  File "/var/lib/kolla/venv/lib/python3.6/site-packages/neutron_lib/db/api.py", line 139, in wrapped\n    setattr(e, \'_RETRY_EXCEEDED\', True)\n', '  File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n    self.force_reraise()\n', '  File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n    six.reraise(self.type_, self.value, self.tb)\n', '  File "/usr/lib/python3.6/site-packages/six.py", line 703, in reraise\n    raise value\n', '  File "/var/lib/kolla/venv/lib/python3.6/site-packages/neutron_lib/db/api.py", line 135, in wrapped\n    return f(*args, **kwargs)\n', '  File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_db/api.py", line 154, in wrapper\n    ectxt.value = e.inner_exc\n', '  File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n    self.force_reraise()\n', '  File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n    six.reraise(self.type_, self.value, self.tb)\n', '  File "/usr/lib/python3.6/site-packages/six.py", line 703, in reraise\n    raise value\n', '  File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_db/api.py", line 142, in wrapper\n    return f(*args, **kwargs)\n', '  File "/var/lib/kolla/venv/lib/python3.6/site-packages/neutron_lib/db/api.py", line 183, in wrapped\n    LOG.debug("Retry wrapper got retriable exception: %s", e)\n', '  File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n    self.force_reraise()\n', '  File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n    six.reraise(self.type_, self.value, self.tb)\n', '  File "/usr/lib/python3.6/site-packages/six.py", line 703, in reraise\n    raise value\n', '  File "/var/lib/kolla/venv/lib/python3.6/site-packages/neutron_lib/db/api.py", line 179, in wrapped\n    return f(*dup_args, **dup_kwargs)\n', '  File "/var/lib/kolla/venv/lib/python3.6/site-packages/neutron/api/rpc/handlers/dhcp_rpc.py", line 311, in update_dhcp_port\n    return self._port_action(plugin, context, port, \'update_port\')\n', '  File "/var/lib/kolla/venv/lib/python3.6/site-packages/neutron/api/rpc/handlers/dhcp_rpc.py", line 101, in _port_action\n    return plugin.update_port(context, port[\'id\'], port)\n', '  File "/var/lib/kolla/venv/lib/python3.6/site-packages/neutron/common/utils.py", line 681, in inner\n    return f(self, context, *args, **kwargs)\n', '  File "/var/lib/kolla/venv/lib/python3.6/site-packages/neutron_lib/db/api.py", line 233, in wrapped\n    return method(*args, **kwargs)\n', '  File "/var/lib/kolla/venv/lib/python3.6/site-packages/neutron_lib/db/api.py", line 139, in wrapped\n    setattr(e, \'_RETRY_EXCEEDED\', True)\n', '  File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n    self.force_reraise()\n', '  File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n    six.reraise(self.type_, self.value, self.tb)\n', '  File "/usr/lib/python3.6/site-packages/six.py", line 703, in reraise\n    raise value\n', '  File "/var/lib/kolla/venv/lib/python3.6/site-packages/neutron_lib/db/api.py", line 135, in wrapped\n    return f(*args, **kwargs)\n', '  File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_db/api.py", line 154, in wrapper\n    ectxt.value = e.inner_exc\n', '  File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n    self.force_reraise()\n', '  File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n    six.reraise(self.type_, self.value, self.tb)\n', '  File "/usr/lib/python3.6/site-packages/six.py", line 703, in reraise\n    raise value\n', '  File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_db/api.py", line 142, in wrapper\n    return f(*args, **kwargs)\n', '  File "/var/lib/kolla/venv/lib/python3.6/site-packages/neutron_lib/db/api.py", line 183, in wrapped\n    LOG.debug("Retry wrapper got retriable exception: %s", e)\n', '  File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n    self.force_reraise()\n', '  File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n    six.reraise(self.type_, self.value, self.tb)\n', '  File "/usr/lib/python3.6/site-packages/six.py", line 703, in reraise\n    raise value\n', '  File "/var/lib/kolla/venv/lib/python3.6/site-packages/neutron_lib/db/api.py", line 179, in wrapped\n    return f(*dup_args, **dup_kwargs)\n', '  File "/var/lib/kolla/venv/lib/python3.6/site-packages/neutron/plugins/ml2/plugin.py", line 1728, in update_port\n    self.mechanism_manager.update_port_precommit(mech_context)\n', '  File "/var/lib/kolla/venv/lib/python3.6/site-packages/neutron/plugins/ml2/managers.py", line 731, in update_port_precommit\n    raise_db_retriable=True)\n', '  File "/var/lib/kolla/venv/lib/python3.6/site-packages/neutron/plugins/ml2/managers.py", line 495, in _call_on_drivers\n    errors=errors\n', 'neutron.plugins.ml2.common.exceptions.MechanismDriverError\n'].
  2021-05-18 11:42:32.508 7 ERROR neutron.agent.dhcp.agent

  Neutron:

  2021-05-18 11:42:32.500 29 ERROR neutron.plugins.ml2.managers [req-e8c3cd96-2a53-4ea8-b2eb-4de64af3600c - - - - -] Mechanism driver 'ovn' failed in update_port_precommit: AttributeError: 'NoneType' object has no attribute 'chassis_exists'
  2021-05-18 11:42:32.500 29 ERROR neutron.plugins.ml2.managers Traceback (most recent call last):
  2021-05-18 11:42:32.500 29 ERROR neutron.plugins.ml2.managers   File "/var/lib/kolla/venv/lib/python3.6/site-packages/neutron/plugins/ml2/managers.py", line 477, in _call_on_drivers
  2021-05-18 11:42:32.500 29 ERROR neutron.plugins.ml2.managers     getattr(driver.obj, method_name)(context)
  2021-05-18 11:42:32.500 29 ERROR neutron.plugins.ml2.managers   File "/var/lib/kolla/venv/lib/python3.6/site-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/mech_driver.py", line 695, in update_port_precommit
  2021-05-18 11:42:32.500 29 ERROR neutron.plugins.ml2.managers     context.original_host):
  2021-05-18 11:42:32.500 29 ERROR neutron.plugins.ml2.managers   File "/var/lib/kolla/venv/lib/python3.6/site-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/mech_driver.py", line 616, in _is_port_provisioning_required
  2021-05-18 11:42:32.500 29 ERROR neutron.plugins.ml2.managers     if not self._sb_ovn.chassis_exists(host):
  2021-05-18 11:42:32.500 29 ERROR neutron.plugins.ml2.managers AttributeError: 'NoneType' object has no attribute 'chassis_exists'
  2021-05-18 11:42:32.500 29 ERROR neutron.plugins.ml2.managers 
  2021-05-18 11:42:32.503 29 ERROR oslo_messaging.rpc.server [req-e8c3cd96-2a53-4ea8-b2eb-4de64af3600c - - - - -] Exception during message handling: neutron.plugins.ml2.common.exceptions.MechanismDriverError
  2021-05-18 11:42:32.503 29 ERROR oslo_messaging.rpc.server Traceback (most recent call last):
  2021-05-18 11:42:32.503 29 ERROR oslo_messaging.rpc.server   File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
  2021-05-18 11:42:32.503 29 ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
  2021-05-18 11:42:32.503 29 ERROR oslo_messaging.rpc.server   File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
  2021-05-18 11:42:32.503 29 ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
  2021-05-18 11:42:32.503 29 ERROR oslo_messaging.rpc.server   File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
  2021-05-18 11:42:32.503 29 ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
  2021-05-18 11:42:32.503 29 ERROR oslo_messaging.rpc.server   File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", line 241, in inner
  2021-05-18 11:42:32.503 29 ERROR oslo_messaging.rpc.server     return func(*args, **kwargs)
  2021-05-18 11:42:32.503 29 ERROR oslo_messaging.rpc.server   File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", line 241, in inner
  2021-05-18 11:42:32.503 29 ERROR oslo_messaging.rpc.server     return func(*args, **kwargs)
  2021-05-18 11:42:32.503 29 ERROR oslo_messaging.rpc.server   File "/var/lib/kolla/venv/lib/python3.6/site-packages/neutron_lib/db/api.py", line 139, in wrapped
  2021-05-18 11:42:32.503 29 ERROR oslo_messaging.rpc.server     setattr(e, '_RETRY_EXCEEDED', True)
  2021-05-18 11:42:32.503 29 ERROR oslo_messaging.rpc.server   File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__
  2021-05-18 11:42:32.503 29 ERROR oslo_messaging.rpc.server     self.force_reraise()
  2021-05-18 11:42:32.503 29 ERROR oslo_messaging.rpc.server   File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
  2021-05-18 11:42:32.503 29 ERROR oslo_messaging.rpc.server     six.reraise(self.type_, self.value, self.tb)
  2021-05-18 11:42:32.503 29 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/six.py", line 703, in reraise
  2021-05-18 11:42:32.503 29 ERROR oslo_messaging.rpc.server     raise value
  2021-05-18 11:42:32.503 29 ERROR oslo_messaging.rpc.server   File "/var/lib/kolla/venv/lib/python3.6/site-packages/neutron_lib/db/api.py", line 135, in wrapped
  2021-05-18 11:42:32.503 29 ERROR oslo_messaging.rpc.server     return f(*args, **kwargs)
  2021-05-18 11:42:32.503 29 ERROR oslo_messaging.rpc.server   File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_db/api.py", line 154, in wrapper
  2021-05-18 11:42:32.503 29 ERROR oslo_messaging.rpc.server     ectxt.value = e.inner_exc
  2021-05-18 11:42:32.503 29 ERROR oslo_messaging.rpc.server   File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__
  2021-05-18 11:42:32.503 29 ERROR oslo_messaging.rpc.server     self.force_reraise()
  2021-05-18 11:42:32.503 29 ERROR oslo_messaging.rpc.server   File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
  2021-05-18 11:42:32.503 29 ERROR oslo_messaging.rpc.server     six.reraise(self.type_, self.value, self.tb)
  2021-05-18 11:42:32.503 29 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/six.py", line 703, in reraise
  2021-05-18 11:42:32.503 29 ERROR oslo_messaging.rpc.server     raise value
  2021-05-18 11:42:32.503 29 ERROR oslo_messaging.rpc.server   File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_db/api.py", line 142, in wrapper
  2021-05-18 11:42:32.503 29 ERROR oslo_messaging.rpc.server     return f(*args, **kwargs)
  2021-05-18 11:42:32.503 29 ERROR oslo_messaging.rpc.server   File "/var/lib/kolla/venv/lib/python3.6/site-packages/neutron_lib/db/api.py", line 183, in wrapped
  2021-05-18 11:42:32.503 29 ERROR oslo_messaging.rpc.server     LOG.debug("Retry wrapper got retriable exception: %s", e)
  2021-05-18 11:42:32.503 29 ERROR oslo_messaging.rpc.server   File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__
  2021-05-18 11:42:32.503 29 ERROR oslo_messaging.rpc.server     self.force_reraise()
  2021-05-18 11:42:32.503 29 ERROR oslo_messaging.rpc.server   File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
  2021-05-18 11:42:32.503 29 ERROR oslo_messaging.rpc.server     six.reraise(self.type_, self.value, self.tb)
  2021-05-18 11:42:32.503 29 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/six.py", line 703, in reraise
  2021-05-18 11:42:32.503 29 ERROR oslo_messaging.rpc.server     raise value
  2021-05-18 11:42:32.503 29 ERROR oslo_messaging.rpc.server   File "/var/lib/kolla/venv/lib/python3.6/site-packages/neutron_lib/db/api.py", line 179, in wrapped
  2021-05-18 11:42:32.503 29 ERROR oslo_messaging.rpc.server     return f(*dup_args, **dup_kwargs)
  2021-05-18 11:42:32.503 29 ERROR oslo_messaging.rpc.server   File "/var/lib/kolla/venv/lib/python3.6/site-packages/neutron/api/rpc/handlers/dhcp_rpc.py", line 311, in update_dhcp_port
  2021-05-18 11:42:32.503 29 ERROR oslo_messaging.rpc.server     return self._port_action(plugin, context, port, 'update_port')
  2021-05-18 11:42:32.503 29 ERROR oslo_messaging.rpc.server   File "/var/lib/kolla/venv/lib/python3.6/site-packages/neutron/api/rpc/handlers/dhcp_rpc.py", line 101, in _port_action
  2021-05-18 11:42:32.503 29 ERROR oslo_messaging.rpc.server     return plugin.update_port(context, port['id'], port)
  2021-05-18 11:42:32.503 29 ERROR oslo_messaging.rpc.server   File "/var/lib/kolla/venv/lib/python3.6/site-packages/neutron/common/utils.py", line 681, in inner
  2021-05-18 11:42:32.503 29 ERROR oslo_messaging.rpc.server     return f(self, context, *args, **kwargs)
  2021-05-18 11:42:32.503 29 ERROR oslo_messaging.rpc.server   File "/var/lib/kolla/venv/lib/python3.6/site-packages/neutron_lib/db/api.py", line 233, in wrapped
  2021-05-18 11:42:32.503 29 ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
  2021-05-18 11:42:32.503 29 ERROR oslo_messaging.rpc.server   File "/var/lib/kolla/venv/lib/python3.6/site-packages/neutron_lib/db/api.py", line 139, in wrapped
  2021-05-18 11:42:32.503 29 ERROR oslo_messaging.rpc.server     setattr(e, '_RETRY_EXCEEDED', True)
  2021-05-18 11:42:32.503 29 ERROR oslo_messaging.rpc.server   File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__
  2021-05-18 11:42:32.503 29 ERROR oslo_messaging.rpc.server     self.force_reraise()
  2021-05-18 11:42:32.503 29 ERROR oslo_messaging.rpc.server   File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
  2021-05-18 11:42:32.503 29 ERROR oslo_messaging.rpc.server     six.reraise(self.type_, self.value, self.tb)
  2021-05-18 11:42:32.503 29 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/six.py", line 703, in reraise
  2021-05-18 11:42:32.503 29 ERROR oslo_messaging.rpc.server     raise value
  2021-05-18 11:42:32.503 29 ERROR oslo_messaging.rpc.server   File "/var/lib/kolla/venv/lib/python3.6/site-packages/neutron_lib/db/api.py", line 135, in wrapped
  2021-05-18 11:42:32.503 29 ERROR oslo_messaging.rpc.server     return f(*args, **kwargs)
  2021-05-18 11:42:32.503 29 ERROR oslo_messaging.rpc.server   File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_db/api.py", line 154, in wrapper
  2021-05-18 11:42:32.503 29 ERROR oslo_messaging.rpc.server     ectxt.value = e.inner_exc
  2021-05-18 11:42:32.503 29 ERROR oslo_messaging.rpc.server   File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__
  2021-05-18 11:42:32.503 29 ERROR oslo_messaging.rpc.server     self.force_reraise()
  2021-05-18 11:42:32.503 29 ERROR oslo_messaging.rpc.server   File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
  2021-05-18 11:42:32.503 29 ERROR oslo_messaging.rpc.server     six.reraise(self.type_, self.value, self.tb)
  2021-05-18 11:42:32.503 29 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/six.py", line 703, in reraise
  2021-05-18 11:42:32.503 29 ERROR oslo_messaging.rpc.server     raise value
  2021-05-18 11:42:32.503 29 ERROR oslo_messaging.rpc.server   File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_db/api.py", line 142, in wrapper
  2021-05-18 11:42:32.503 29 ERROR oslo_messaging.rpc.server     return f(*args, **kwargs)
  2021-05-18 11:42:32.503 29 ERROR oslo_messaging.rpc.server   File "/var/lib/kolla/venv/lib/python3.6/site-packages/neutron_lib/db/api.py", line 183, in wrapped
  2021-05-18 11:42:32.503 29 ERROR oslo_messaging.rpc.server     LOG.debug("Retry wrapper got retriable exception: %s", e)
  2021-05-18 11:42:32.503 29 ERROR oslo_messaging.rpc.server   File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__
  2021-05-18 11:42:32.503 29 ERROR oslo_messaging.rpc.server     self.force_reraise()
  2021-05-18 11:42:32.503 29 ERROR oslo_messaging.rpc.server   File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
  2021-05-18 11:42:32.503 29 ERROR oslo_messaging.rpc.server     six.reraise(self.type_, self.value, self.tb)
  2021-05-18 11:42:32.503 29 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/six.py", line 703, in reraise
  2021-05-18 11:42:32.503 29 ERROR oslo_messaging.rpc.server     raise value
  2021-05-18 11:42:32.503 29 ERROR oslo_messaging.rpc.server   File "/var/lib/kolla/venv/lib/python3.6/site-packages/neutron_lib/db/api.py", line 179, in wrapped
  2021-05-18 11:42:32.503 29 ERROR oslo_messaging.rpc.server     return f(*dup_args, **dup_kwargs)
  2021-05-18 11:42:32.503 29 ERROR oslo_messaging.rpc.server   File "/var/lib/kolla/venv/lib/python3.6/site-packages/neutron/plugins/ml2/plugin.py", line 1728, in update_port
  2021-05-18 11:42:32.503 29 ERROR oslo_messaging.rpc.server     self.mechanism_manager.update_port_precommit(mech_context)
  2021-05-18 11:42:32.503 29 ERROR oslo_messaging.rpc.server   File "/var/lib/kolla/venv/lib/python3.6/site-packages/neutron/plugins/ml2/managers.py", line 731, in update_port_precommit
  2021-05-18 11:42:32.503 29 ERROR oslo_messaging.rpc.server     raise_db_retriable=True)
  2021-05-18 11:42:32.503 29 ERROR oslo_messaging.rpc.server   File "/var/lib/kolla/venv/lib/python3.6/site-packages/neutron/plugins/ml2/managers.py", line 495, in _call_on_drivers
  2021-05-18 11:42:32.503 29 ERROR oslo_messaging.rpc.server     errors=errors
  2021-05-18 11:42:32.503 29 ERROR oslo_messaging.rpc.server neutron.plugins.ml2.common.exceptions.MechanismDriverError
  2021-05-18 11:42:32.503 29 ERROR oslo_messaging.rpc.server 

  This problem does not exist in earlier builds (>1 month) of Neutron
  server.

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1928794/+subscriptions
References

[Bug 1928794] [NEW] [OVN] DHCP agent fails to update network
From: Piotr Parczewski, 2021-05-18