yahoo-eng-team team mailing list archive

[Rodolfo Alonso <1885547@xxxxxxxxxxxxxxxxxx>]

Public bug reported:

In fullstack test, unlike in a real deployment, more than one OVS agent is spawned in the same host, reading from the same OVS database. That could lead to isolation errors when receiving notifications. For example, OVS has a thread monitoring the new ports added to the host, using the shell CLI:
  "ovsdb-client monitor Interface name,ofport,external_ids"

In a fullstack test, two OVS agents running in the same host will have
independent integration bridges (randomly generated names). But when a
port is added to one of those integration bridges, both agents will
receive the event of a new port (interface) added. The OVS agent where
the port is not added, won't find it in its own integration bridge and
will treat it as dead port [1], sending a "update_device_down" message
to the server.

This behavior could clash with the correct update sent from the other
OVS agent, setting the port up. E.g.: in log [2] we can see how the
first OVS agent sets the port UP and one second later, the second OVS
agent sets the port down [3]. That will make this test fail.


[1]https://opendev.org/openstack/neutron/src/branch/master/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py#L1833-L1834
[2]https://4317fd6ef51d22a4668e-8daa7927094e2ef08ab8f0f95a49fc40.ssl.cf5.rackcdn.com/733250/18/check/neutron-fullstack-with-uwsgi/79ec86c/controller/logs/dsvm-fullstack-logs/TestLegacyL3Agent.test_external_subnet_changed/neutron-server--2020-06-26--12-24-50-479143_log.txt
[3]http://paste.openstack.org/show/795317/

** Affects: neutron
     Importance: Undecided
     Assignee: Rodolfo Alonso (rodolfo-alonso-hernandez)
         Status: New

** Changed in: neutron
     Assignee: (unassigned) => Rodolfo Alonso (rodolfo-alonso-hernandez)

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1885547

Title:
  [fullstack] OVS interface events isolation error with more than one
  OVS agent

Status in neutron:
  New

Bug description:
  In fullstack test, unlike in a real deployment, more than one OVS agent is spawned in the same host, reading from the same OVS database. That could lead to isolation errors when receiving notifications. For example, OVS has a thread monitoring the new ports added to the host, using the shell CLI:
    "ovsdb-client monitor Interface name,ofport,external_ids"

  In a fullstack test, two OVS agents running in the same host will have
  independent integration bridges (randomly generated names). But when a
  port is added to one of those integration bridges, both agents will
  receive the event of a new port (interface) added. The OVS agent where
  the port is not added, won't find it in its own integration bridge and
  will treat it as dead port [1], sending a "update_device_down" message
  to the server.

  This behavior could clash with the correct update sent from the other
  OVS agent, setting the port up. E.g.: in log [2] we can see how the
  first OVS agent sets the port UP and one second later, the second OVS
  agent sets the port down [3]. That will make this test fail.

  
  [1]https://opendev.org/openstack/neutron/src/branch/master/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py#L1833-L1834
  [2]https://4317fd6ef51d22a4668e-8daa7927094e2ef08ab8f0f95a49fc40.ssl.cf5.rackcdn.com/733250/18/check/neutron-fullstack-with-uwsgi/79ec86c/controller/logs/dsvm-fullstack-logs/TestLegacyL3Agent.test_external_subnet_changed/neutron-server--2020-06-26--12-24-50-479143_log.txt
  [3]http://paste.openstack.org/show/795317/

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1885547/+subscriptions