yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1473413] Re: py34 fails in neutron.tests.unit.agent.test_securitygroups_rpc

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Doug Hellmann <doug@xxxxxxxxxxxxxxxx>
Date: Wed, 29 Jul 2015 18:58:32 -0000
Reply-to: Bug 1473413 <1473413@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
** Changed in: neutron
       Status: Fix Committed => Fix Released

** Changed in: neutron
    Milestone: None => liberty-2

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1473413

Title:
  py34 fails in neutron.tests.unit.agent.test_securitygroups_rpc

Status in neutron:
  Fix Released

Bug description:
  The job sometimes fails in gate for multiple security groups unit
  tests:

  2015-07-10 10:43:53.260 | FAIL: neutron.tests.unit.agent.test_securitygroups_rpc.TestSecurityGroupAgentWithOVSIptables.test_security_group_rule_updated
  2015-07-10 10:43:53.261 | ----------------------------------------------------------------------
  2015-07-10 10:43:53.261 | Empty attachments:
  2015-07-10 10:43:53.261 |   pythonlogging:'neutron.api.extensions'
  2015-07-10 10:43:53.261 | 
  2015-07-10 10:43:53.262 | pythonlogging:'': {{{
  2015-07-10 10:43:53.262 | 2015-07-10 10:43:06,008     INFO [neutron.agent.securitygroups_rpc] Preparing filters for devices ['tap_port1', 'tap_port3']
  2015-07-10 10:43:53.262 | 2015-07-10 10:43:06,008  WARNING [neutron.agent.securitygroups_rpc] security_group_info_for_devices rpc call not supported by the server, falling back to old security_group_rules_for_devices which scales worse.
  2015-07-10 10:43:53.263 | 2015-07-10 10:43:06,012     INFO [neutron.agent.securitygroups_rpc] Security group rule updated ['security_group1']
  2015-07-10 10:43:53.263 | 2015-07-10 10:43:06,013     INFO [neutron.agent.securitygroups_rpc] Refresh firewall rules
  2015-07-10 10:43:53.263 | }}}
  2015-07-10 10:43:53.264 | 
  2015-07-10 10:43:53.264 | Traceback (most recent call last):
  2015-07-10 10:43:53.264 |   File "/home/jenkins/workspace/gate-neutron-python34/neutron/tests/unit/agent/test_securitygroups_rpc.py", line 2996, in test_security_group_rule_updated
  2015-07-10 10:43:53.265 |     self._verify_mock_calls()
  2015-07-10 10:43:53.265 |   File "/home/jenkins/workspace/gate-neutron-python34/neutron/tests/unit/agent/test_securitygroups_rpc.py", line 2624, in _verify_mock_calls
  2015-07-10 10:43:53.265 |     matchers.MatchesRegex(expected_regex))
  2015-07-10 10:43:53.265 |   File "/home/jenkins/workspace/gate-neutron-python34/.tox/py34/lib/python3.4/site-packages/testtools/testcase.py", line 435, in assertThat
  2015-07-10 10:43:53.266 |     raise mismatch_error
  2015-07-10 10:43:53.268 | testtools.matchers._impl.MismatchError: '# Generated by iptables_manager\n*raw\n:run.py-OUTPUT - [0:0]\n:run.py-PREROUTING - [0:0]\n[0:0] -A PREROUTING -j run.py-PREROUTING\n[0:0] -A OUTPUT -j run.py-OUTPUT\n[0:0] -A run.py-PREROUTING -m physdev --physdev-in qvbtap_port1 -j CT --zone 1\n[0:0] -A run.py-PREROUTING -m physdev --physdev-in taptap_port1 -j CT --zone 1\n[0:0] -A run.py-PREROUTING -m physdev --physdev-in qvbtap_port2 -j CT --zone 1\n[0:0] -A run.py-PREROUTING -m physdev --physdev-in taptap_port2 -j CT --zone 1\nCOMMIT\n# Completed by iptables_manager\n# Generated by iptables_manager\n*nat\n:neutron-postrouting-bottom - [0:0]\n:run.py-OUTPUT - [0:0]\n:run.py-POSTROUTING - [0:0]\n:run.py-PREROUTING - [0:0]\n:run.py-float-snat - [0:0]\n:run.py-snat - [0:0]\n[0:0] -A PREROUTING -j run.py-PREROUTING\n[0:0] -A OUTPUT -j run.py-OUTPUT\n[0:0] -A POSTROUTING -j run.py-POSTROUTING\n[0:0] -A POSTROUTING -j neutron-postrouting-bottom\n[0:0] -A neutron-postrouting-bottom -j run.py-snat\n[0:0] -A run.py-snat -j run.py-float-snat\nCOMMIT\n# Completed by iptables_manager\n# Generated by iptables_manager\n*mangle\n:run.py-FORWARD - [0:0]\n:run.py-INPUT - [0:0]\n:run.py-OUTPUT - [0:0]\n:run.py-POSTROUTING - [0:0]\n:run.py-PREROUTING - [0:0]\n:run.py-mark - [0:0]\n[0:0] -A PREROUTING -j run.py-PREROUTING\n[0:0] -A INPUT -j run.py-INPUT\n[0:0] -A FORWARD -j run.py-FORWARD\n[0:0] -A OUTPUT -j run.py-OUTPUT\n[0:0] -A POSTROUTING -j run.py-POSTROUTING\n[0:0] -A run.py-PREROUTING -j run.py-mark\nCOMMIT\n# Completed by iptables_manager\n# Generated by iptables_manager\n*filter\n:neutron-filter-top - [0:0]\n:run.py-FORWARD - [0:0]\n:run.py-INPUT - [0:0]\n:run.py-OUTPUT - [0:0]\n:run.py-itap_port1 - [0:0]\n:run.py-itap_port2 - [0:0]\n:run.py-local - [0:0]\n:run.py-otap_port1 - [0:0]\n:run.py-otap_port2 - [0:0]\n:run.py-sg-chain - [0:0]\n:run.py-sg-fallback - [0:0]\n:run.py-stap_port1 - [0:0]\n:run.py-stap_port2 - [0:0]\n[0:0] -A FORWARD -j neutron-filter-top\n[0:0] -A OUTPUT -j neutron-filter-top\n[0:0] -A neutron-filter-top -j run.py-local\n[0:0] -A INPUT -j run.py-INPUT\n[0:0] -A OUTPUT -j run.py-OUTPUT\n[0:0] -A FORWARD -j run.py-FORWARD\n[0:0] -A run.py-sg-fallback -j DROP\n[0:0] -A run.py-FORWARD -m physdev --physdev-out taptap_port1 --physdev-is-bridged -j run.py-sg-chain\n[0:0] -A run.py-sg-chain -m physdev --physdev-out taptap_port1 --physdev-is-bridged -j run.py-itap_port1\n[0:0] -A run.py-itap_port1 -m state --state INVALID -j DROP\n[0:0] -A run.py-itap_port1 -m state --state RELATED,ESTABLISHED -j RETURN\n[0:0] -A run.py-itap_port1 -s 10.0.0.2/32 -p udp -m udp --sport 67 --dport 68 -j RETURN\n[0:0] -A run.py-itap_port1 -p tcp -m tcp --dport 22 -j RETURN\n[0:0] -A run.py-itap_port1 -s 10.0.0.4/32 -j RETURN\n[0:0] -A run.py-itap_port1 -j run.py-sg-fallback\n[0:0] -A run.py-FORWARD -m physdev --physdev-in taptap_port1 --physdev-is-bridged -j run.py-sg-chain\n[0:0] -A run.py-sg-chain -m physdev --physdev-in taptap_port1 --physdev-is-bridged -j run.py-otap_port1\n[0:0] -A run.py-INPUT -m physdev --physdev-in taptap_port1 --physdev-is-bridged -j run.py-otap_port1\n[0:0] -A run.py-stap_port1 -s 10.0.0.3/32 -m mac --mac-source 12:34:56:78:9A:BC -j RETURN\n[0:0] -A run.py-stap_port1 -j DROP\n[0:0] -A run.py-otap_port1 -p udp -m udp --sport 68 --dport 67 -j RETURN\n[0:0] -A run.py-otap_port1 -j run.py-stap_port1\n[0:0] -A run.py-otap_port1 -p udp -m udp --sport 67 --dport 68 -j DROP\n[0:0] -A run.py-otap_port1 -m state --state INVALID -j DROP\n[0:0] -A run.py-otap_port1 -m state --state RELATED,ESTABLISHED -j RETURN\n[0:0] -A run.py-otap_port1 -j RETURN\n[0:0] -A run.py-otap_port1 -j run.py-sg-fallback\n[0:0] -A run.py-FORWARD -m physdev --physdev-out taptap_port2 --physdev-is-bridged -j run.py-sg-chain\n[0:0] -A run.py-sg-chain -m physdev --physdev-out taptap_port2 --physdev-is-bridged -j run.py-itap_port2\n[0:0] -A run.py-itap_port2 -m state --state INVALID -j DROP\n[0:0] -A run.py-itap_port2 -m state --state RELATED,ESTABLISHED -j RETURN\n[0:0] -A run.py-itap_port2 -s 10.0.0.2/32 -p udp -m udp --sport 67 --dport 68 -j RETURN\n[0:0] -A run.py-itap_port2 -p tcp -m tcp --dport 22 -j RETURN\n[0:0] -A run.py-itap_port2 -s 10.0.0.3/32 -j RETURN\n[0:0] -A run.py-itap_port2 -j run.py-sg-fallback\n[0:0] -A run.py-FORWARD -m physdev --physdev-in taptap_port2 --physdev-is-bridged -j run.py-sg-chain\n[0:0] -A run.py-sg-chain -m physdev --physdev-in taptap_port2 --physdev-is-bridged -j run.py-otap_port2\n[0:0] -A run.py-INPUT -m physdev --physdev-in taptap_port2 --physdev-is-bridged -j run.py-otap_port2\n[0:0] -A run.py-stap_port2 -s 10.0.0.4/32 -m mac --mac-source 12:34:56:78:9A:BD -j RETURN\n[0:0] -A run.py-stap_port2 -j DROP\n[0:0] -A run.py-otap_port2 -p udp -m udp --sport 68 --dport 67 -j RETURN\n[0:0] -A run.py-otap_port2 -j run.py-stap_port2\n[0:0] -A run.py-otap_port2 -p udp -m udp --sport 67 --dport 68 -j DROP\n[0:0] -A run.py-otap_port2 -m state --state INVALID -j DROP\n[0:0] -A run.py-otap_port2 -m state --state RELATED,ESTABLISHED -j RETURN\n[0:0] -A run.py-otap_port2 -j RETURN\n[0:0] -A run.py-otap_port2 -j run.py-sg-fallback\n[0:0] -A run.py-sg-chain -j ACCEPT\nCOMMIT\n# Completed by iptables_manager\n' does not match /# Generated by iptables_manager\n\*raw\n:run.py-OUTPUT - \[0:0\]\n:run.py-PREROUTING - \[0:0\]\n\[0:0\] -A PREROUTING -j run.py-PREROUTING\n\[0:0\] -A OUTPUT -j run.py-OUTPUT\n\[0:0\] -A run.py-PREROUTING -m physdev --physdev-in qvbtap_port2 -j CT --zone 1\n\[0:0\] -A run.py-PREROUTING -m physdev --physdev-in taptap_port2 -j CT --zone 1\n\[0:0\] -A run.py-PREROUTING -m physdev --physdev-in qvbtap_port1 -j CT --zone 1\n\[0:0\] -A run.py-PREROUTING -m physdev --physdev-in taptap_port1 -j CT --zone 1\nCOMMIT\n# Completed by iptables_manager\n# Generated by iptables_manager\n\*nat\n:neutron-postrouting-bottom - \[0:0\]\n:run.py-(OUTPUT|POSTROUTING|PREROUTING|float-snat|snat) - \[0:0\]\n:run.py-(OUTPUT|POSTROUTING|PREROUTING|float-snat|snat) - \[0:0\]\n:run.py-(OUTPUT|POSTROUTING|PREROUTING|float-snat|snat) - \[0:0\]\n:run.py-(OUTPUT|POSTROUTING|PREROUTING|float-snat|snat) - \[0:0\]\n:run.py-(OUTPUT|POSTROUTING|PREROUTING|float-snat|snat) - \[0:0\]\n\[0:0\] -A PREROUTING -j run.py-PREROUTING\n\[0:0\] -A OUTPUT -j run.py-OUTPUT\n\[0:0\] -A POSTROUTING -j run.py-POSTROUTING\n\[0:0\] -A POSTROUTING -j neutron-postrouting-bottom\n\[0:0\] -A neutron-postrouting-bottom -j run.py-snat\n\[0:0\] -A run.py-snat -j run.py-float-snat\nCOMMIT\n# Completed by iptables_manager\n# Generated by iptables_manager\n\*mangle\n:run.py-(FORWARD|INPUT|OUTPUT|POSTROUTING|PREROUTING|mark) - \[0:0\]\n:run.py-(FORWARD|INPUT|OUTPUT|POSTROUTING|PREROUTING|mark) - \[0:0\]\n:run.py-(FORWARD|INPUT|OUTPUT|POSTROUTING|PREROUTING|mark) - \[0:0\]\n:run.py-(FORWARD|INPUT|OUTPUT|POSTROUTING|PREROUTING|mark) - \[0:0\]\n:run.py-(FORWARD|INPUT|OUTPUT|POSTROUTING|PREROUTING|mark) - \[0:0\]\n:run.py-(FORWARD|INPUT|OUTPUT|POSTROUTING|PREROUTING|mark) - \[0:0\]\n\[0:0\] -A PREROUTING -j run.py-PREROUTING\n\[0:0\] -A INPUT -j run.py-INPUT\n\[0:0\] -A FORWARD -j run.py-FORWARD\n\[0:0\] -A OUTPUT -j run.py-OUTPUT\n\[0:0\] -A POSTROUTING -j run.py-POSTROUTING\n\[0:0\] -A run.py-PREROUTING -j run.py-mark\nCOMMIT\n# Completed by iptables_manager\n# Generated by iptables_manager\n\*filter\n:neutron-filter-top - \[0:0\]\n:run.py-(FORWARD|INPUT|OUTPUT|local|sg-chain|sg-fallback|itap_port1|otap_port1|stap_port1|itap_port2|otap_port2|stap_port2) - \[0:0\]\n:run.py-(FORWARD|INPUT|OUTPUT|local|sg-chain|sg-fallback|itap_port1|otap_port1|stap_port1|itap_port2|otap_port2|stap_port2) - \[0:0\]\n:run.py-(FORWARD|INPUT|OUTPUT|local|sg-chain|sg-fallback|itap_port1|otap_port1|stap_port1|itap_port2|otap_port2|stap_port2) - \[0:0\]\n:run.py-(FORWARD|INPUT|OUTPUT|local|sg-chain|sg-fallback|itap_port1|otap_port1|stap_port1|itap_port2|otap_port2|stap_port2) - \[0:0\]\n:run.py-(FORWARD|INPUT|OUTPUT|local|sg-chain|sg-fallback|itap_port1|otap_port1|stap_port1|itap_port2|otap_port2|stap_port2) - \[0:0\]\n:run.py-(FORWARD|INPUT|OUTPUT|local|sg-chain|sg-fallback|itap_port1|otap_port1|stap_port1|itap_port2|otap_port2|stap_port2) - \[0:0\]\n:run.py-(FORWARD|INPUT|OUTPUT|local|sg-chain|sg-fallback|itap_port1|otap_port1|stap_port1|itap_port2|otap_port2|stap_port2) - \[0:0\]\n:run.py-(FORWARD|INPUT|OUTPUT|local|sg-chain|sg-fallback|itap_port1|otap_port1|stap_port1|itap_port2|otap_port2|stap_port2) - \[0:0\]\n:run.py-(FORWARD|INPUT|OUTPUT|local|sg-chain|sg-fallback|itap_port1|otap_port1|stap_port1|itap_port2|otap_port2|stap_port2) - \[0:0\]\n:run.py-(FORWARD|INPUT|OUTPUT|local|sg-chain|sg-fallback|itap_port1|otap_port1|stap_port1|itap_port2|otap_port2|stap_port2) - \[0:0\]\n:run.py-(FORWARD|INPUT|OUTPUT|local|sg-chain|sg-fallback|itap_port1|otap_port1|stap_port1|itap_port2|otap_port2|stap_port2) - \[0:0\]\n:run.py-(FORWARD|INPUT|OUTPUT|local|sg-chain|sg-fallback|itap_port1|otap_port1|stap_port1|itap_port2|otap_port2|stap_port2) - \[0:0\]\n\[0:0\] -A FORWARD -j neutron-filter-top\n\[0:0\] -A OUTPUT -j neutron-filter-top\n\[0:0\] -A neutron-filter-top -j run.py-local\n\[0:0\] -A INPUT -j run.py-INPUT\n\[0:0\] -A OUTPUT -j run.py-OUTPUT\n\[0:0\] -A FORWARD -j run.py-FORWARD\n\[0:0\] -A run.py-sg-fallback -j DROP\n\[0:0\] -A run.py-FORWARD -m physdev --physdev-out taptap_port2 --physdev-is-bridged -j run.py-sg-chain\n\[0:0\] -A run.py-sg-chain -m physdev --physdev-out taptap_port2 --physdev-is-bridged -j run.py-itap_port2\n\[0:0\] -A run.py-itap_port2 -m state --state INVALID -j DROP\n\[0:0\] -A run.py-itap_port2 -m state --state RELATED,ESTABLISHED -j RETURN\n\[0:0\] -A run.py-itap_port2 -s 10.0.0.2/32 -p udp -m udp --sport 67 --dport 68 -j RETURN\n\[0:0\] -A run.py-itap_port2 -p tcp -m tcp --dport 22 -j RETURN\n\[0:0\] -A run.py-itap_port2 -s 10.0.0.3/32 -j RETURN\n\[0:0\] -A run.py-itap_port2 -j run.py-sg-fallback\n\[0:0\] -A run.py-FORWARD -m physdev --physdev-in taptap_port2 --physdev-is-bridged -j run.py-sg-chain\n\[0:0\] -A run.py-sg-chain -m physdev --physdev-in taptap_port2 --physdev-is-bridged -j run.py-otap_port2\n\[0:0\] -A run.py-INPUT -m physdev --physdev-in taptap_port2 --physdev-is-bridged -j run.py-otap_port2\n\[0:0\] -A run.py-stap_port2 -s 10.0.0.4/32 -m mac --mac-source 12:34:56:78:9A:BD -j RETURN\n\[0:0\] -A run.py-stap_port2 -j DROP\n\[0:0\] -A run.py-otap_port2 -p udp -m udp --sport 68 --dport 67 -j RETURN\n\[0:0\] -A run.py-otap_port2 -j run.py-stap_port2\n\[0:0\] -A run.py-otap_port2 -p udp -m udp --sport 67 --dport 68 -j DROP\n\[0:0\] -A run.py-otap_port2 -m state --state INVALID -j DROP\n\[0:0\] -A run.py-otap_port2 -m state --state RELATED,ESTABLISHED -j RETURN\n\[0:0\] -A run.py-otap_port2 -j RETURN\n\[0:0\] -A run.py-otap_port2 -j run.py-sg-fallback\n\[0:0\] -A run.py-FORWARD -m physdev --physdev-out taptap_port1 --physdev-is-bridged -j run.py-sg-chain\n\[0:0\] -A run.py-sg-chain -m physdev --physdev-out taptap_port1 --physdev-is-bridged -j run.py-itap_port1\n\[0:0\] -A run.py-itap_port1 -m state --state INVALID -j DROP\n\[0:0\] -A run.py-itap_port1 -m state --state RELATED,ESTABLISHED -j RETURN\n\[0:0\] -A run.py-itap_port1 -s 10.0.0.2/32 -p udp -m udp --sport 67 --dport 68 -j RETURN\n\[0:0\] -A run.py-itap_port1 -p tcp -m tcp --dport 22 -j RETURN\n\[0:0\] -A run.py-itap_port1 -s 10.0.0.4/32 -j RETURN\n\[0:0\] -A run.py-itap_port1 -j run.py-sg-fallback\n\[0:0\] -A run.py-FORWARD -m physdev --physdev-in taptap_port1 --physdev-is-bridged -j run.py-sg-chain\n\[0:0\] -A run.py-sg-chain -m physdev --physdev-in taptap_port1 --physdev-is-bridged -j run.py-otap_port1\n\[0:0\] -A run.py-INPUT -m physdev --physdev-in taptap_port1 --physdev-is-bridged -j run.py-otap_port1\n\[0:0\] -A run.py-stap_port1 -s 10.0.0.3/32 -m mac --mac-source 12:34:56:78:9A:BC -j RETURN\n\[0:0\] -A run.py-stap_port1 -j DROP\n\[0:0\] -A run.py-otap_port1 -p udp -m udp --sport 68 --dport 67 -j RETURN\n\[0:0\] -A run.py-otap_port1 -j run.py-stap_port1\n\[0:0\] -A run.py-otap_port1 -p udp -m udp --sport 67 --dport 68 -j DROP\n\[0:0\] -A run.py-otap_port1 -m state --state INVALID -j DROP\n\[0:0\] -A run.py-otap_port1 -m state --state RELATED,ESTABLISHED -j RETURN\n\[0:0\] -A run.py-otap_port1 -j RETURN\n\[0:0\] -A run.py-otap_port1 -j run.py-sg-fallback\n\[0:0\] -A run.py-sg-chain -j ACCEPT\nCOMMIT\n# Completed by iptables_manager\n/

  It does not happen for py27. It may be either py34 specific issue or a
  general issue triggered by the fact that py34 does not run full test
  suite.

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1473413/+subscriptions
References

[Bug 1473413] [NEW] py34 fails in neutron.tests.unit.agent.test_securitygroups_rpc
From: Ihar Hrachyshka, 2015-07-10