yahoo-eng-team team mailing list archive

[OpenStack Infra <1696537@xxxxxxxxxxxxxxxxxx>]

Reviewed:  https://review.openstack.org/472454
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=3c1a25d9683263505b00cb0d6b04add1831f0ca0
Submitter: Jenkins
Branch:    master

commit 3c1a25d9683263505b00cb0d6b04add1831f0ca0
Author: Kevin Benton <kevin@xxxxxxxxxx>
Date:   Thu Jun 8 14:56:19 2017 -0700

    Make HA deletion attempt on RouterNotFound race
    
    The L3 HA RPC code that creates HA interfaces can race
    with an HA router deletion on the server side. The L3 HA
    code ends up creating a port on the HA network while the
    server side is deleting the router and the HA network.
    
    This stops the L3 HA network from being deleted because
    it has a new port without a bound segment, which leaves the
    HA network in a segmentless condition and no ports after
    the L3 RPC code cleans up its port.
    
    This adjusts the L3 RPC logic to attempt an HA network cleanup
    whenever it encounters the concurrent router deletion case
    to ensure that the HA network gets cleaned up.
    
    To make this more robust in the future, we may need the L3
    HA code to recognize when an HA network has no segments and
    automatically create a new one.
    
    Change-Id: Idd301f6df92e9bc37187e8ed8ec00004e67da928
    Closes-Bug: #1696537


** Changed in: neutron
       Status: In Progress => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1696537

Title:
  test_keepalived_multiple_sighups_does_not_forfeit_mastership fails
  when neutron-server tries to bind with Linuxbridge driver (agent not
  enabled)

Status in neutron:
  Fix Released

Bug description:
  This happens locally and in gate. Gate example:
  http://logs.openstack.org/59/471059/2/check/gate-neutron-dsvm-
  fullstack-ubuntu-xenial/df11b90/testr_results.html.gz

      Traceback (most recent call last):
        File "neutron/tests/base.py", line 118, in func
          return f(self, *args, **kwargs)
        File "neutron/tests/fullstack/test_l3_agent.py", line 252, in test_keepalived_multiple_sighups_does_not_forfeit_mastership
          tenant_id, '13.37.0.0/24', network['id'], router['id'])
        File "neutron/tests/fullstack/test_l3_agent.py", line 61, in _create_and_attach_subnet
          router_interface_info['port_id'])
        File "neutron/tests/fullstack/test_l3_agent.py", line 51, in block_until_port_status_active
          common_utils.wait_until_true(lambda: is_port_status_active(), sleep=1)
        File "neutron/common/utils.py", line 685, in wait_until_true
          raise WaitTimeout("Timed out after %d seconds" % timeout)
      neutron.common.utils.WaitTimeout: Timed out after 60 seconds

  This is not 100% failure rate, depends on which driver server picks to
  bind ports: ovs or linuxbridge. If the latter, it just spins
  attempting to bind with it over and over, until bails out. It never
  tries to switch to ovs.

  In server log, we see this:
  http://logs.openstack.org/59/471059/2/check/gate-neutron-dsvm-
  fullstack-ubuntu-xenial/df11b90/logs/dsvm-fullstack-
  logs/TestHAL3Agent.test_keepalived_multiple_sighups_does_not_forfeit_mastership
  /neutron-server--2017-06-05--
  21-41-34-957535.txt.gz#_2017-06-05_21_42_13_400

  2017-06-05 21:42:13.400 12566 DEBUG neutron.plugins.ml2.drivers.mech_agent [req-6618e950-5260-404d-a511-e314408542f5 - - - - -] Port 4f8dcf10-6f91-4860-b239-6b04460244a3 on network 155ebfd5-20cf-44bc-9cb5-bc885b8d2eae not bound, no agent of type Linux bridge agent registered on host host-745fd526 bind_port /opt/stack/new/neutron/neutron/plugins/ml2/drivers/mech_agent.py:103
  2017-06-05 21:42:13.401 12566 ERROR neutron.plugins.ml2.managers [req-6618e950-5260-404d-a511-e314408542f5 - - - - -] Failed to bind port 4f8dcf10-6f91-4860-b239-6b04460244a3 on host host-745fd526 for vnic_type normal using segments []
  2017-06-05 21:42:13.401 12566 INFO neutron.plugins.ml2.plugin [req-6618e950-5260-404d-a511-e314408542f5 - - - - -] Attempt 2 to bind port 4f8dcf10-6f91-4860-b239-6b04460244a3
  ...
  2017-06-05 21:42:13.822 12566 ERROR neutron.plugins.ml2.managers [req-6618e950-5260-404d-a511-e314408542f5 - - - - -] Failed to bind port 4f8dcf10-6f91-4860-b239-6b04460244a3 on host host-745fd526 for vnic_type normal using segments []

  The fullstack test case configures both ml2 drivers.

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1696537/+subscriptions