yahoo-eng-team team mailing list archive

[OpenStack Infra <1834213@xxxxxxxxxxxxxxxxxx>]

Reviewed:  https://review.opendev.org/c/openstack/neutron/+/871659
Committed: https://opendev.org/openstack/neutron/commit/c609084b59c68f003153a58c2063f99b52f169e0
Submitter: "Zuul (22348)"
Branch:    master

commit c609084b59c68f003153a58c2063f99b52f169e0
Author: Brian Haley <haleyb.dev@xxxxxxxxx>
Date:   Tue Jan 24 15:16:47 2023 -0500

    Add doc note on nf_conntrack module requirement
    
    The OVS firewall driver requires nf_conntrack module(s)
    to be loaded to function properly. While they are typically
    loaded automatically, add a note to the admin guide about
    the requirement to make it explicit.
    
    Closes-bug: #1834213
    
    Change-Id: I55871eff1e37d4155b8d2b5ae8c182d160c4af9f


** Changed in: neutron
       Status: In Progress => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1834213

Title:
  After kernel upgrade, nf_conntrack_ipv4 module unloaded, no IP traffic
  to instances

Status in OpenStack Neutron Open vSwitch Charm:
  Fix Released
Status in neutron:
  Fix Released
Status in linux package in Ubuntu:
  Confirmed

Bug description:
  With an environment running Xenial-Queens, and having just upgraded
  the linux-image-generic kernel for MDS patching, a few of our
  hypervisor hosts that were rebooted (3 out of 100) ended up dropping
  IP (tcp/udp) ingress traffic.

  It turns out that nf_conntrack module was loaded, but
  nf_conntrack_ipv4 was not loading, and the traffic was being dropped
  by this rule:

   table=72, n_packets=214989, priority=50,ct_state=+inv+trk
  actions=resubmit(,93)

  The ct_state "inv" means invalid conntrack state, which the manpage
  describes as:

                       The state is invalid, meaning that the connection tracker
                       couldn’t identify the connection. This flag is a catch-
                       all for problems in the connection or the connection
                       tracker, such as:

                       • L3/L4 protocol handler is not loaded/unavailable.
                              With the Linux kernel datapath, this may mean that
                              the nf_conntrack_ipv4 or nf_conntrack_ipv6 modules
                              are not loaded.

                       • L3/L4 protocol handler determines that the packet
                              is malformed.

                       • Packets are unexpected length for protocol.

  It appears that there may be an issue when patching the OS of a
  hypervisor not running instances may fail to update initrd to load
  nf_conntrack_ipv4 (and/or _ipv6).

  I couldn't find anywhere in the charm code that this would be loaded
  unless the charm's "harden" option is used on nova-compute charm (see
  charmhelpers contrib/host templates).  It is unset in our environment,
  so we are not using any special module probing.

  Did nf_conntrack_ipv4 get split out from nf_conntrack in recent kernel
  upgrades or is it possible that the charm should define a modprobe
  file if we have the OVS firewall driver configured?

To manage notifications about this bug go to:
https://bugs.launchpad.net/charm-neutron-openvswitch/+bug/1834213/+subscriptions