yahoo-eng-team team mailing list archive

[Chuck Short <chuck.short@xxxxxxxxxxxxx>]

** Also affects: neutron/juno
   Importance: Undecided
       Status: New

** Changed in: neutron/juno
    Milestone: None => 2014.2.2

** Changed in: neutron/juno
       Status: New => Fix Committed

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1381238

Title:
  Race condition on processing DVR floating IPs

Status in OpenStack Neutron (virtual network service):
  Fix Released
Status in neutron juno series:
  Fix Committed

Bug description:
  A race condition can sometimes occur in l-3 agent when a dvr based
  floatingip is being deleted from one router and another dvr based
  floatingip is being configured on another router in the same node.
  Especially if the floatingip being deleted was the last floatingip on
  the node.  Although fix for Bug # 1373100 [1] eliminated frequent
  observation of this behavior in upstream tests, it still shows up.
  Couple of recent examples:

  http://logs.openstack.org/88/128288/1/check/check-tempest-dsvm-
  neutron-dvr/8fdd1de/

  http://logs.openstack.org/03/123403/7/check/check-tempest-dsvm-
  neutron-dvr/859534a/

  Relevant log messages:

  2014-10-14 16:06:15.803 22303 DEBUG neutron.agent.linux.utils [-] Running command: ['sudo', '/usr/local/bin/neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip', 'netns', 'exec', 'fip-82fb2751-30ba-4015-a5da-6c8563064db9', 'ip', 'link', 'del', 'fpr-7ed86ca6-b'] create_process /opt/stack/new/neutron/neutron/agent/linux/utils.py:46
  2014-10-14 16:06:15.838 22303 DEBUG neutron.agent.linux.utils [-] 
  Command: ['sudo', '/usr/local/bin/neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip', 'netns', 'exec', 'qrouter-7ed86ca6-b42d-4ba9-8899-447ff0509174', 'ip', 'addr', 'show', 'rfp-7ed86ca6-b']
  Exit code: 0
  Stdout: '2: rfp-7ed86ca6-b: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000\n    link/ether c6:88:ee:71:a7:51 brd ff:ff:ff:ff:ff:ff\n    inet 169.254.30.212/31 scope global rfp-7ed86ca6-b\n       valid_lft forever preferred_lft forever\n    inet6 fe80::c488:eeff:fe71:a751/64 scope link \n       valid_lft forever preferred_lft forever\n'
  Stderr: '' execute /opt/stack/new/neutron/neutron/agent/linux/utils.py:81
  2014-10-14 16:06:15.839 22303 DEBUG neutron.agent.linux.utils [-] Running command: ['sudo', '/usr/local/bin/neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip', 'netns', 'exec', 'qrouter-7ed86ca6-b42d-4ba9-8899-447ff0509174', 'ip', '-4', 'addr', 'add', '172.24.4.91/32', 'brd', '172.24.4.91', 'scope', 'global', 'dev', 'rfp-7ed86ca6-b'] create_process /opt/stack/new/neutron/neutron/agent/linux/utils.py:46
  2014-10-14 16:06:16.221 22303 DEBUG neutron.agent.linux.utils [-] 
  Command: ['sudo', '/usr/local/bin/neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip', 'netns', 'exec', 'fip-82fb2751-30ba-4015-a5da-6c8563064db9', 'ip', 'link', 'del', 'fpr-7ed86ca6-b']
  Exit code: 0
  Stdout: ''
  Stderr: '' execute /opt/stack/new/neutron/neutron/agent/linux/utils.py:81
  2014-10-14 16:06:16.222 22303 DEBUG neutron.agent.l3_agent [-] DVR: unplug: fg-f04e25ef-e3 _destroy_fip_namespace /opt/stack/new/neutron/neutron/agent/l3_agent.py:679
  2014-10-14 16:06:16.222 22303 DEBUG neutron.agent.linux.utils [-] Running command: ['ip', '-o', 'link', 'show', 'br-ex'] create_process /opt/stack/new/neutron/neutron/agent/linux/utils.py:46
  2014-10-14 16:06:16.251 22303 ERROR neutron.agent.linux.utils [-] 
  Command: ['sudo', '/usr/local/bin/neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip', 'netns', 'exec', 'qrouter-7ed86ca6-b42d-4ba9-8899-447ff0509174', 'ip', '-4', 'addr', 'add', '172.24.4.91/32', 'brd', '172.24.4.91', 'scope', 'global', 'dev', 'rfp-7ed86ca6-b']
  Exit code: 1
  Stdout: ''
  Stderr: 'Cannot find device "rfp-7ed86ca6-b"\n'  


  [1] https://bugs.launchpad.net/neutron/+bug/1373100

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1381238/+subscriptions