yahoo-eng-team team mailing list archive

[OpenStack Infra <1959564@xxxxxxxxxxxxxxxxxx>]

Reviewed:  https://review.opendev.org/c/openstack/neutron/+/827315
Committed: https://opendev.org/openstack/neutron/commit/0ddca284542aed89df4a22607a2da03f193f083c
Submitter: "Zuul (22348)"
Branch:    master

commit 0ddca284542aed89df4a22607a2da03f193f083c
Author: Oleg Bondarev <oleg.bondarev@xxxxxxxxxx>
Date:   Tue Feb 1 18:56:02 2022 +0300

    Make sure "dead vlan" ports cannot transmit packets
    
    https://review.opendev.org/c/openstack/neutron/+/820897 added
    a dead vlan flow that pushes the dead vlan tag onto frames
    belonging to dead ports before these ports are reassigned to
    their proper vlans. However add_flow and delete_flows race and
    delete_flows may run before add_flow, in this case deleting 0 flows
    but not giving us a chance to detect this: neither does it throw
    an error nor does it return the number of deleted flows.
    This leads to port staying inaccessible forever and hence
    breaks corresponding DHCP or router.
    
    Current patch suggests another approach to make sure no packets are
    leaked from newly plugged ports: setting their "vlan_mode" attribute
    to "trunk" and "trunks"=[4095] (along with assigning dead VLAN tag).
    With this OVS normal pipeline will allow only packets tagged with 4095
    from such ports [1], which normally not happens, but even if it does -
    default rule in br-int will drop them anyway.
    Thus untagged packets from such ports will also be dropped until
    ovs agent sets proper VLAN tag and clears vlan_mode to default
    ("access").
    
    This approach avoids the race between dhcp/l3 and ovs agents because
    dhcp/l3 agents no longer modify flow table.
    
    This partially reverts commit 7aae31c9f9ed938760ca0be3c461826b598c7004
    
    [1] https://docs.openvswitch.org/en/latest/ref/ovs-actions.7/?highlight=ovs-actions#the-ovs-normal-pipeline
    
    Closes-Bug: #1930414
    Closes-Bug: #1959564
    Change-Id: I0391dd24224f8656a09ddb002e7dae8783ba37a4


** Changed in: neutron
       Status: In Progress => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1959564

Title:
  Random Tempest test failures(SSH failure) in openvswitch jobs

Status in neutron:
  Fix Released

Bug description:
  Seen multiple similar occurences in Stable/wallaby patches, where tempest tests fails with ssh to VM Timeouts, some examples:-
    - https://cfaa2d1e4f6a936642aa-ae5561c9d080274a217713c4553af257.ssl.cf5.rackcdn.com/824022/2/check/neutron-tempest-plugin-scenario-openvswitch-wallaby/a7c128e/testr_results.html
    - https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_803/824022/2/check/neutron-tempest-plugin-scenario-openvswitch-iptables_hybrid-wallaby/803c276/testr_results.html
    - https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_e19/824022/2/check/neutron-tempest-plugin-scenario-openvswitch-iptables_hybrid-wallaby/e19d9a7/testr_results.html
    - https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_6b0/826830/1/check/neutron-tempest-plugin-scenario-openvswitch-wallaby/6b05e5b/testr_results.html
    - https://f529caf5e8a0adc3d959-479aba3a0d5645603ea5f6db22bcd24f.ssl.cf5.rackcdn.com/826830/1/check/neutron-tempest-plugin-scenario-openvswitch-iptables_hybrid-wallaby/9ad9f43/testr_results.html

  In some failures seeing metadata request failed and thus ssh failed,
  and in some metadata requests passed, but ssh failed, so may be
  multiple issues are there

  
  Builds:- https://zuul.openstack.org/builds?job_name=neutron-tempest-plugin-scenario-openvswitch-iptables_hybrid-wallaby&job_name=neutron-tempest-plugin-scenario-openvswitch-wallaby&project=openstack%2Fneutron&branch=stable%2Fwallaby&skip=0

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1959564/+subscriptions