yahoo-eng-team team mailing list archive

[David Ames <david.ames@xxxxxxxxxxxxx>]

** Changed in: charm-neutron-openvswitch
       Status: Fix Committed => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1834213

Title:
  After kernel upgrade, nf_conntrack_ipv4 module unloaded, no IP traffic
  to instances

Status in OpenStack neutron-openvswitch charm:
  Fix Released
Status in neutron:
  New
Status in linux package in Ubuntu:
  Confirmed

Bug description:
  With an environment running Xenial-Queens, and having just upgraded
  the linux-image-generic kernel for MDS patching, a few of our
  hypervisor hosts that were rebooted (3 out of 100) ended up dropping
  IP (tcp/udp) ingress traffic.

  It turns out that nf_conntrack module was loaded, but
  nf_conntrack_ipv4 was not loading, and the traffic was being dropped
  by this rule:

   table=72, n_packets=214989, priority=50,ct_state=+inv+trk
  actions=resubmit(,93)

  The ct_state "inv" means invalid conntrack state, which the manpage
  describes as:

                       The state is invalid, meaning that the connection tracker
                       couldn’t identify the connection. This flag is a catch-
                       all for problems in the connection or the connection
                       tracker, such as:

                       • L3/L4 protocol handler is not loaded/unavailable.
                              With the Linux kernel datapath, this may mean that
                              the nf_conntrack_ipv4 or nf_conntrack_ipv6 modules
                              are not loaded.

                       • L3/L4 protocol handler determines that the packet
                              is malformed.

                       • Packets are unexpected length for protocol.

  It appears that there may be an issue when patching the OS of a
  hypervisor not running instances may fail to update initrd to load
  nf_conntrack_ipv4 (and/or _ipv6).

  I couldn't find anywhere in the charm code that this would be loaded
  unless the charm's "harden" option is used on nova-compute charm (see
  charmhelpers contrib/host templates).  It is unset in our environment,
  so we are not using any special module probing.

  Did nf_conntrack_ipv4 get split out from nf_conntrack in recent kernel
  upgrades or is it possible that the charm should define a modprobe
  file if we have the OVS firewall driver configured?

To manage notifications about this bug go to:
https://bugs.launchpad.net/charm-neutron-openvswitch/+bug/1834213/+subscriptions