yahoo-eng-team team mailing list archive

[OpenStack Infra <1874211@xxxxxxxxxxxxxxxxxx>]

Reviewed:  https://review.opendev.org/721799
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=dc9084a8ec9db08d7ea947e0b73581b748be5819
Submitter: Zuul
Branch:    master

commit dc9084a8ec9db08d7ea947e0b73581b748be5819
Author: Slawek Kaplonski <skaplons@xxxxxxxxxx>
Date:   Wed Apr 22 09:56:40 2020 +0000

    [L3 HA] Add "no_track" option to VIPs in keepalived config
    
    Patch [1] introduced new mechanism which only brings UP interfaces
    on master node of HA router. It works fine with keepalived 1.x
    but it is broken when keepalived 2.x was used (e.g. on Centos 8) as
    in this new version of keepalived by default all interfaces of VIPs
    and routes are tracked, and if one of them is DOWN, keepalived is
    going to FAULT state. Because of that router will never be
    transitioned to MASTER on any node.
    
    This patch fixes it by adding "no_track" option to all VIPs
    and routes in keepalived's config file.
    
    This "no_track" option isn't added to ha interface so this one
    is still tracked by keepalived.
    
    [1] https://review.opendev.org/#/c/707406/
    
    Closes-bug: #1874211
    
    Change-Id: Ic16cf83fe1d1576d91047adb2d4f9e07d57185b6


** Changed in: neutron
       Status: In Progress => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1874211

Title:
  [L3HA] Keepalived 2.x.x tracks state of virtual_ipaddresses interfaces
  and router now

Status in neutron:
  Fix Released

Bug description:
  Patch https://review.opendev.org/#/c/707406/ introduced new mechanism which brings all HA router's interfaces to be DOWN if router is in backup mode.
  And that works fine with keepalived 1.4.x but in keepalived 2.x.x it changed and keepalived now tracks by default interfaces of virtual_ipaddresses and routes. And will go to FAULT state if such interface is DOWN.
  That cause problem with router which will never be transitioned to master state.

  We should add "no_track" option to qg- and qr- interfaces in
  keepalived config file now.

  Errors can be seen e.g. in result of tripleo job:
  https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_2b3/721574/1/check
  /tripleo-ci-
  centos-8-scenario007-standalone/2b3f794/logs/undercloud/var/log/journal.txt

  Apr 21 12:21:45 standalone.localdomain Keepalived[147861]: Starting Keepalived v2.0.10 (11/12,2018)
  Apr 21 12:21:45 standalone.localdomain Keepalived[147861]: Running on Linux 4.18.0-147.8.1.el8_1.x86_64 #1 SMP Thu Apr 9 13:49:54 UTC 2020 (built for Linux 4.18.0)
  Apr 21 12:21:45 standalone.localdomain Keepalived[147861]: Command line: '/usr/sbin/keepalived' '-n' '-l' '-D' '-P' '-f'
  Apr 21 12:21:45 standalone.localdomain Keepalived[147861]:               '/var/lib/neutron/ha_confs/24a8996a-5d64-446d-afcd-e08c3d72d64c/keepalived.conf' '-p'
  Apr 21 12:21:45 standalone.localdomain Keepalived[147861]:               '/var/lib/neutron/ha_confs/24a8996a-5d64-446d-afcd-e08c3d72d64c.pid.keepalived' '-r'
  Apr 21 12:21:45 standalone.localdomain Keepalived[147861]:               '/var/lib/neutron/ha_confs/24a8996a-5d64-446d-afcd-e08c3d72d64c.pid.keepalived-vrrp'
  Apr 21 12:21:45 standalone.localdomain Keepalived[147861]:               '-D'
  Apr 21 12:21:45 standalone.localdomain Keepalived[147861]: Opening file '/var/lib/neutron/ha_confs/24a8996a-5d64-446d-afcd-e08c3d72d64c/keepalived.conf'.
  Apr 21 12:21:45 standalone.localdomain Keepalived[147861]: Starting VRRP child process, pid=147864
  Apr 21 12:21:45 standalone.localdomain Keepalived_vrrp[147864]: Registering Kernel netlink reflector
  Apr 21 12:21:45 standalone.localdomain Keepalived_vrrp[147864]: Registering Kernel netlink command channel
  Apr 21 12:21:45 standalone.localdomain Keepalived_vrrp[147864]: Opening file '/var/lib/neutron/ha_confs/24a8996a-5d64-446d-afcd-e08c3d72d64c/keepalived.conf'.
  Apr 21 12:21:45 standalone.localdomain Keepalived_vrrp[147864]: (Line 22) Cannot specify scope for IPv6 addresses (fe80::f816:3eff:fe0a:3675/64) - ignoring scope
  Apr 21 12:21:45 standalone.localdomain Keepalived_vrrp[147864]: (Line 23) Cannot specify scope for IPv6 addresses (fe80::f816:3eff:fea8:ae56/64) - ignoring scope
  Apr 21 12:21:45 standalone.localdomain Keepalived_vrrp[147864]: (VR_99) Ignoring track_interface ha-dcefeeaa-6d since own interface
  Apr 21 12:21:45 standalone.localdomain Keepalived_vrrp[147864]: Assigned address 169.254.195.203 for interface ha-dcefeeaa-6d
  Apr 21 12:21:45 standalone.localdomain Keepalived_vrrp[147864]: Assigned address fe80::f816:3eff:fec7:3b26 for interface ha-dcefeeaa-6d
  Apr 21 12:21:45 standalone.localdomain Keepalived_vrrp[147864]: (VR_99) entering FAULT state
  Apr 21 12:21:45 standalone.localdomain Keepalived_vrrp[147864]: Registering gratuitous ARP shared channel
  Apr 21 12:21:45 standalone.localdomain Keepalived_vrrp[147864]: Registering gratuitous NDISC shared channel
  Apr 21 12:21:45 standalone.localdomain Keepalived_vrrp[147864]: (VR_99) removing Virtual Routes
  Apr 21 12:21:45 standalone.localdomain Keepalived_vrrp[147864]: (VR_99) removing VIPs.
  Apr 21 12:21:45 standalone.localdomain Keepalived_vrrp[147864]: (VR_99) removing E-VIPs.
  Apr 21 12:21:45 standalone.localdomain Keepalived_vrrp[147864]: (VR_99) removing Virtual Routes

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1874211/+subscriptions