yahoo-eng-team team mailing list archive

[OpenStack Infra <1316621@xxxxxxxxxxxxxxxxxx>]

Reviewed:  https://review.openstack.org/431773
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=486e2f4eb5a02c98958582e366a4d6081ea897e0
Submitter: Jenkins
Branch:    master

commit 486e2f4eb5a02c98958582e366a4d6081ea897e0
Author: Kevin Benton <kevin@xxxxxxxxxx>
Date:   Thu Feb 9 15:10:20 2017 -0800

    Pass --concurrent flag to ebtables calls
    
    This flag will force ebtables to acquire a lock so we don't
    have to worry about ebtables errors occuring if something else
    on the system is trying to use ebtables as well.
    
    Closes-Bug: #1316621
    Change-Id: I695c01e015fdc201df8f23d9b48f9d3678240266


** Changed in: neutron
       Status: In Progress => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1316621

Title:
  ebtables calls can race with libvirt

Status in neutron:
  Fix Released
Status in OpenStack Compute (nova):
  Fix Released

Bug description:
  Sometimes request to associate floating IP may fail, when using nova
  network with libvirt like:

  > http://192.168.1.12:8774/v2/258a4b20c77240bf9b386411430683fa/servers/a9e734e4-5310-4191-a7f0-78fca4b367e7/action
  > 
  > BadRequest: Bad request
  > Details: {'message': 'Error. Unable to associate floating ip', 'code': '400'}

  Real issue is that ebtables rootwrap call fails:
  Command: sudo nova-rootwrap /etc/nova/rootwrap.conf ebtables -t nat -I PREROUTING --logical-in br100 -p ipv4 --ip-src 192.168.32.10 ! --ip-dst 192.168.32.0/22 -j redirect --redirect-target ACCEPT
  Exit code: 255
  Stdout: ''
  Stderr: "Unable to update the kernel. Two possible causes:\n1. Multiple ebtables programs were executing simultaneously. The ebtables\n   userspace tool doesn't by default support multiple ebtables programs running\n   concurrently. The ebtables option --concurrent or a tool like flock can be\n   used to support concurrent scripts that update the ebtables kernel tables.\n2. The kernel doesn't support a certain ebtables extension, consider\n   recompiling your kernel or insmod the extension.\n.\n"

  It happens like once in whole tempest run, and also not always, so kernel support and other reasons should not apply here.
  Probably already mentioned in https://www.mail-archive.com/openstack-dev@xxxxxxxxxxxxxxxxxxx/msg23422.html.

  As that call in nova is synchronized, locked, it could be that nova
  can actually race with libvirt itself calling ebtables?

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1316621/+subscriptions