yahoo-eng-team team mailing list archive

[Rodolfo Alonso <2094840@xxxxxxxxxxxxxxxxxx>]

*** This bug is a duplicate of bug 2095590 ***
    https://bugs.launchpad.net/bugs/2095590

** This bug is no longer a duplicate of bug 2094736
   [OVN] Missing ``LogicalSwitchPortUpdateUpEvent`` when a port is bound
** This bug has been marked a duplicate of bug 2095590
   [OVN] Neutron API stops refreshing the hash ring node

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/2094840

Title:
  OVN driver doesn't receive port-binding UP update from OVN controller;
  Nova eventually times out while building VM

Status in neutron:
  New

Bug description:
  I'm seeing this in gate in this run:
  https://zuul.opendev.org/t/openstack/build/8987272b4be843ea9dffb266ca559006/logs

  The symptom of the test failure is that VM is not built. This happens
  because port is not UP (nova hasn't received vif event for the state
  change). When we check ovn-controller logs, we see the relevant port
  is set to up:

  ```
  2025-01-13T22:30:10.704Z|00067|binding|INFO|Claiming lport 75ffc908-b9c5-421f-b1b9-117b158e860d for this chassis.
  2025-01-13T22:30:10.704Z|00068|binding|INFO|75ffc908-b9c5-421f-b1b9-117b158e860d: Claiming fa:16:3e:11:92:5d 10.1.0.13
  2025-01-13T22:30:10.704Z|00069|binding|INFO|Claiming lport c469b2c4-6961-4953-a2c6-0c106c80b5c8 for this chassis.
  2025-01-13T22:30:10.704Z|00070|binding|INFO|c469b2c4-6961-4953-a2c6-0c106c80b5c8: Claiming fa:16:3e:d5:d9:4e 10.1.0.7
  2025-01-13T22:30:10.730Z|00071|binding|INFO|Setting lport 75ffc908-b9c5-421f-b1b9-117b158e860d ovn-installed in OVS
  2025-01-13T22:30:10.730Z|00072|binding|INFO|Setting lport 75ffc908-b9c5-421f-b1b9-117b158e860d up in Southbound
  2025-01-13T22:30:10.730Z|00073|binding|INFO|Setting lport c469b2c4-6961-4953-a2c6-0c106c80b5c8 ovn-installed in OVS
  2025-01-13T22:30:10.730Z|00074|binding|INFO|Setting lport c469b2c4-6961-4953-a2c6-0c106c80b5c8 up in Southbound

  ```

  The port in question is 75ffc908-b9c5-421f-b1b9-117b158e860d.

  In neutron-api log, ovsdb-monitor receives the updates to SB port-
  binding table:

  Jan 13 22:30:10.766618 np0039557494 devstack@neutron-api.service[62570]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovsdb_monitor [None req-0d900a20-58b1-4662-9217-856061d78672 None None] Hash Ring: Node d6133697-439f-4059-b5e7-152917a32dd1 (host: np0039557494) handling event "update" for row cfeb99a7-95f6-4f8c-bb64-cd628734b141 (table: Port_Binding) {{(pid=62570) notify /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovsdb_monitor.py:852}}
  Jan 13 22:30:10.832940 np0039557494 devstack@neutron-api.service[62570]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovsdb_monitor [None req-0d900a20-58b1-4662-9217-856061d78672 None None] Hash Ring: Node d6133697-439f-4059-b5e7-152917a32dd1 (host: np0039557494) handling event "update" for row cfeb99a7-95f6-4f8c-bb64-cd628734b141 (table: Port_Binding) {{(pid=62570) notify /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovsdb_monitor.py:852}}

  But note that both events refer to the same row id. Note that in ovn-
  controller log snippet, two different ports are being claimed / set to
  up at the same time.

  Also note that no LSP events are seen at all in the timeframe. Is it
  northd that failed to translate to NB status=up for LSP (which is what
  we watch for in ovsdb monitor)? Or is it NB ovsdb-server failing to
  notify? Or is it neutron monitor that somehow misses the event? Hard
  to say, nothing of interest is seen in ovsdb-server and northd logs,
  as far as I can see. (It would help if we could get more debug logs
  from ovs and ovn services.)

  ---

  In gate, OVN is quite old: 22.03; OVS components are at 2.17.9 (also
  very old). Wonder if there were some race conditions now fixed in
  later python-ovs or ovsdb-server or ovn-controller...

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/2094840/+subscriptions