yahoo-eng-team team mailing list archive

[OpenStack Infra <1740885@xxxxxxxxxxxxxxxxxx>]

Reviewed:  https://review.openstack.org/531414
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=ed57c3de4241f94ead2bc7c7761bab99dd61894e
Submitter: Zuul
Branch:    master

commit ed57c3de4241f94ead2bc7c7761bab99dd61894e
Author: Jakub Libosvar <libosvar@xxxxxxxxxx>
Date:   Fri Jan 5 16:04:44 2018 +0100

    ovsfw: Don't create rules if updated port doesn't exist
    
    The firewall won't attempt on update to initialize port in case
    port hasn't been initialized by sg_agent yet. This fixes a race where update
    rpc call arrives between wiring tap device with integration bridge and
    firewall initialization.
    
    Change-Id: Ice0667df606ae23061acebceea23ab6e49dadbcf
    Closes-bug: #1740885


** Changed in: neutron
       Status: In Progress => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1740885

Title:
  Security group updates fail when port hasn't been initialized yet

Status in neutron:
  Fix Released

Bug description:
  It happens that tpi patch ports between trunk bridge and integration
  bridge don't have internal vlan tag in other_config row in ovsdb. It
  looks like a race between trunk handler and ovs agent.

  Example of failure: http://logs.openstack.org/92/527992/2/check
  /neutron-tempest-plugin-dvr-multinode-
  scenario/166eee3/logs/subnode-2/screen-q-agt.txt.gz#_Dec_14_18_31_25_801432

  Trace example: 
  Dec 14 18:31:25.801432 ubuntu-xenial-rax-ord-0001444835 neutron-openvswitch-agent[17015]: ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [None req-412981e2-ac8a-4fe8-8ec4-288bbb63e2a7 None None] Error while processing VIF ports: OVSFWTagNotFound: Cannot get tag for port tpi-6457d45d-b6 from its other_config: {}
  Dec 14 18:31:25.801580 ubuntu-xenial-rax-ord-0001444835 neutron-openvswitch-agent[17015]: ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent Traceback (most recent call last):
  Dec 14 18:31:25.801708 ubuntu-xenial-rax-ord-0001444835 neutron-openvswitch-agent[17015]: ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent   File "/opt/stack/new/neutron/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py", line 2080, in rpc_loop
  Dec 14 18:31:25.801838 ubuntu-xenial-rax-ord-0001444835 neutron-openvswitch-agent[17015]: ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent     port_info, ovs_restarted)
  Dec 14 18:31:25.801965 ubuntu-xenial-rax-ord-0001444835 neutron-openvswitch-agent[17015]: ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent   File "/usr/local/lib/python2.7/dist-packages/osprofiler/profiler.py", line 157, in wrapper
  Dec 14 18:31:25.802089 ubuntu-xenial-rax-ord-0001444835 neutron-openvswitch-agent[17015]: ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent     result = f(*args, **kwargs)
  Dec 14 18:31:25.802214 ubuntu-xenial-rax-ord-0001444835 neutron-openvswitch-agent[17015]: ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent   File "/opt/stack/new/neutron/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py", line 1676, in process_network_ports
  Dec 14 18:31:25.802345 ubuntu-xenial-rax-ord-0001444835 neutron-openvswitch-agent[17015]: ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent     port_info.get('updated', set()))
  Dec 14 18:31:25.802476 ubuntu-xenial-rax-ord-0001444835 neutron-openvswitch-agent[17015]: ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent   File "/opt/stack/new/neutron/neutron/agent/securitygroups_rpc.py", line 256, in setup_port_filters
  Dec 14 18:31:25.802600 ubuntu-xenial-rax-ord-0001444835 neutron-openvswitch-agent[17015]: ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent     self.refresh_firewall(updated_devices)
  Dec 14 18:31:25.802725 ubuntu-xenial-rax-ord-0001444835 neutron-openvswitch-agent[17015]: ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent   File "/opt/stack/new/neutron/neutron/agent/securitygroups_rpc.py", line 110, in decorated_function
  Dec 14 18:31:25.802850 ubuntu-xenial-rax-ord-0001444835 neutron-openvswitch-agent[17015]: ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent     *args, **kwargs)
  Dec 14 18:31:25.802983 ubuntu-xenial-rax-ord-0001444835 neutron-openvswitch-agent[17015]: ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent   File "/opt/stack/new/neutron/neutron/agent/securitygroups_rpc.py", line 209, in refresh_firewall
  Dec 14 18:31:25.803103 ubuntu-xenial-rax-ord-0001444835 neutron-openvswitch-agent[17015]: ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent     self._apply_port_filter(device_ids, update_filter=True)
  Dec 14 18:31:25.803237 ubuntu-xenial-rax-ord-0001444835 neutron-openvswitch-agent[17015]: ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent   File "/opt/stack/new/neutron/neutron/agent/securitygroups_rpc.py", line 141, in _apply_port_filter
  Dec 14 18:31:25.803366 ubuntu-xenial-rax-ord-0001444835 neutron-openvswitch-agent[17015]: ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent     self.firewall.update_port_filter(device)
  Dec 14 18:31:25.803492 ubuntu-xenial-rax-ord-0001444835 neutron-openvswitch-agent[17015]: ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent   File "/opt/stack/new/neutron/neutron/agent/linux/openvswitch_firewall/firewall.py", line 509, in update_port_filter
  Dec 14 18:31:25.803612 ubuntu-xenial-rax-ord-0001444835 neutron-openvswitch-agent[17015]: ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent     self.prepare_port_filter(port)
  Dec 14 18:31:25.803763 ubuntu-xenial-rax-ord-0001444835 neutron-openvswitch-agent[17015]: ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent   File "/opt/stack/new/neutron/neutron/agent/linux/openvswitch_firewall/firewall.py", line 492, in prepare_port_filter
  Dec 14 18:31:25.803894 ubuntu-xenial-rax-ord-0001444835 neutron-openvswitch-agent[17015]: ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent     of_port = self.get_or_create_ofport(port)
  Dec 14 18:31:25.804021 ubuntu-xenial-rax-ord-0001444835 neutron-openvswitch-agent[17015]: ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent   File "/opt/stack/new/neutron/neutron/agent/linux/openvswitch_firewall/firewall.py", line 464, in get_or_create_ofport
  Dec 14 18:31:25.804152 ubuntu-xenial-rax-ord-0001444835 neutron-openvswitch-agent[17015]: ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent     port_vlan_id = self._get_port_vlan_tag(ovs_port.port_name)
  Dec 14 18:31:25.804275 ubuntu-xenial-rax-ord-0001444835 neutron-openvswitch-agent[17015]: ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent   File "/opt/stack/new/neutron/neutron/agent/linux/openvswitch_firewall/firewall.py", line 448, in _get_port_vlan_tag
  Dec 14 18:31:25.804414 ubuntu-xenial-rax-ord-0001444835 neutron-openvswitch-agent[17015]: ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent     return get_tag_from_other_config(self.int_br.br, port_name)
  Dec 14 18:31:25.804548 ubuntu-xenial-rax-ord-0001444835 neutron-openvswitch-agent[17015]: ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent   File "/opt/stack/new/neutron/neutron/agent/linux/openvswitch_firewall/firewall.py", line 74, in get_tag_from_other_config
  Dec 14 18:31:25.804670 ubuntu-xenial-rax-ord-0001444835 neutron-openvswitch-agent[17015]: ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent     port_name=port_name, other_config=other_config)
  Dec 14 18:31:25.804794 ubuntu-xenial-rax-ord-0001444835 neutron-openvswitch-agent[17015]: ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent OVSFWTagNotFound: Cannot get tag for port tpi-6457d45d-b6 from its other_config: {}
  Dec 14 18:31:25.804923 ubuntu-xenial-rax-ord-0001444835 neutron-openvswitch-agent[17015]: ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1740885/+subscriptions