yahoo-eng-team team mailing list archive

[OpenStack Infra <1948891@xxxxxxxxxxxxxxxxxx>]

Reviewed:  https://review.opendev.org/c/openstack/neutron/+/816698
Committed: https://opendev.org/openstack/neutron/commit/7874c576013928c036dca4d9c0a38e5b8ae06bb4
Submitter: "Zuul (22348)"
Branch:    master

commit 7874c576013928c036dca4d9c0a38e5b8ae06bb4
Author: Daniel Alvarez Sanchez <dalvarez@xxxxxxxxxx>
Date:   Thu Nov 4 14:34:32 2021 +0100

    [ovn] Add timeout option to ovsdb-client command
    
    Today, we invoke ovsdb-client to cleanup the MAC_Binding entries
    without specifying any timeout. This can lead to workers blocking
    forever if there's an issue with the connection to the server.
    
    This patch is adding a timeout parameter to the command line to
    prevent this condition.
    
    Closes-Bug: #1948891
    Related-Bug: #1946318
    
    Signed-off-by: Daniel Alvarez Sanchez <dalvarez@xxxxxxxxxx>
    Change-Id: Id393cbec31dd64a795e85d756b7b843c9dfc59f3


** Changed in: neutron
       Status: In Progress => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1948891

Title:
  [ovn] Using ovsdb-client for MAC_Binding could theoretically block
  indefinitely

Status in neutron:
  Fix Released

Bug description:
  As a workaround for saving massive amounts of memory by not monitoring
  the MAC_Binding table, we recently switched to using ovsdb-client for
  deleting FIP MAC_Binding entries. ovsdb-client will open a new
  connection every time it is called. There are some cases where it
  might block the worker longer than we would expect.

  1) If ovsdb-server is busy and the connection takes a long time,
  ovsdb-client will wait up to 2 minutes before giving up. (tested by
  just setting iptables to drop packets to ovsdb-server port)

  2) If ovsdb-server connects, but just doesn't respond (maybe a cable
  is cut), ovsdb-client will wait forever for the response. (tested by
  just having nc listen on the ovsdb-server port)

  A quick workaround would be to pass --timeout to ovsdb-client,
  possibly with the ovsdb_connection_timeout. There is also a daemon
  mode for ovsdb-client.

  I also have a patch [1] waiting in python-ovs that will allow us to
  use the existing Idl connection to craft arbitrary transact operations
  like we pass to ovsdb-client. This will lose the overhead of making a
  new connection with each FIP delete.

  [1]
  https://patchwork.ozlabs.org/project/openvswitch/patch/20211013224023.69938-1-twilson@xxxxxxxxxx/

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1948891/+subscriptions