yahoo-eng-team team mailing list archive

[OpenStack Infra <1493414@xxxxxxxxxxxxxxxxxx>]

Reviewed:  https://review.openstack.org/248908
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=5289d9494984b7c95407ad2f9b761b2e647953b2
Submitter: Jenkins
Branch:    master

commit 5289d9494984b7c95407ad2f9b761b2e647953b2
Author: Ramu Ramamurthy <ramu.ramamurthy@xxxxxxxxxx>
Date:   Mon Nov 23 15:21:46 2015 -0500

    Remove stale ofport drop-rule upon port-delete
    
    When a port is deleted, that port is set to a dead-vlan, and
    an ofport drop-flow is added in port_dead().
    
    The ofport drop-flow gets removed only in some cases
    in _bind_devices() - depending on the timing of the
    concurrent port-deletion. In other cases, the drop-flow
    never gets removed, and such garbage drop-flow rules
    accumulate forever until the ovs-agent restarts.
    
    The fix is to use the function update_stale_ofport_rules which
    solves this problem of tracking stale ofport flows
    in deleted ports, but currently only applies only to
    prevent_arp_spoofing.
    
    Change-Id: I0d1dbe3918cc7d7b3d0cdc49d7b6ff85f9b02a17
    Closes-Bug: #1493414


** Changed in: neutron
       Status: Fix Committed => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1493414

Title:
  OVS Neutron agent is marking port as dead before they are deleted

Status in neutron:
  Fix Released

Bug description:
  The situation is happening on Liberty-3.

  When trying to clear the gateway port and tenant network interface
  delete in router, the OVS agent is marking the port as dead instead of
  treat them as removed: security group removed and port_unbound

  This is causing to left stale OVS flows in br-int, and it may affect
  the port_unbound() logic in ovs_neutron_agent.py.

  The ovs_neutron_agent is in one iteration of rpc_loop processing the
  deleted port via process_deleted_ports() method, marking the qg- port
  as dead (ovs flow rule to drop the traffic) and in another iteration,
  the ovs_neutron_agent is processing the removed port by
  treat_devices_removed() method.

  In first iteration, the port deleting is triggered by port_delete() method:
  2015-09-04 14:16:20.337 DEBUG neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [req-e43234b1-633b-404d-92d0-0f844dadb586 admin 0f6c0469ea6e4d95a27782c46021243a] port_delete message processed for port 1c749258-74fb-498b-9a08-1fec6725a1cf from (pid=136030) port_delete /opt/openstack/neutron/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py:410

  and in second iteration, the device removed is triggered by ovsdb:
  2015-09-04 14:16:20.848 DEBUG neutron.agent.linux.ovsdb_monitor [-] Output received from ovsdb monitor: {"data":[["bab86f35-d004-4df6-95c2-0f7432338edb","delete","qg-1c749258-74",49,["map",[["attached-mac","fa:16:3e:99:37:68"],["iface-id","1c749258-74fb-498b-9a08-1fec6725a1cf"],["iface-status","active"]]]]],"headings":["row","action","name","ofport","external_ids"]}
   from (pid=136030) _read_stdout /opt/openstack/neutron/neutron/agent/linux/ovsdb_monitor.py:50

  Log from ovs neutron agent:
  http://paste.openstack.org/show/445479/

  Steps to reproduce:
  1. Create router
  2. Add tenant network interface to the router
  3. Launch a VM
  4. Add external network gateway to created router
  5. Check the br-int for current port numbers
  6. Remove external network gateway
  7. Check the br-int for dead port flows (removed port qg-)
  8. Remove the network interface from tenant network
  9. Check the br-int for dead port flows.

  Repeat the steps 4-9 few times to see if dead port flows will appear
  in br-int.

  This is affecting the legacy, dvr and HA router.

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1493414/+subscriptions