yahoo-eng-team team mailing list archive

[OpenStack Infra <1501366@xxxxxxxxxxxxxxxxxx>]

Reviewed:  https://review.openstack.org/246581
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=17264ee6a66dd60f9af1aa3a737b17f290fc7e19
Submitter: Jenkins
Branch:    master

commit 17264ee6a66dd60f9af1aa3a737b17f290fc7e19
Author: Chet Burgess <cfb@xxxxxxxxxxxxx>
Date:   Tue Nov 17 11:55:55 2015 -0800

    ebtables/libvirt workaround
    
    Idealy nova is run with libvirt 1.2.11 or later to guarantee that
    libvirt is calling ebtables with --concurrent. Since we can't
    always guarantee this we have created this workaround.
    
    The workaround is extremely hacky and not recommend but for those
    who simply have no other way to address this bug the following
    should be done.
    
     * Copy /sbin/ebtables to /sbin/ebtables.real
     * Copy the ebtables.workaround script to /sbin/ebtables
    
    Caution: Future OS level updates and packages way overwrite the
    above changes. Its recommend users upgrade to libvirt 1.2.11.
    
    The work around script was copied from devstack and originally
    written by sdague.
    
    Change-Id: Icdffc59d68b73a6df22ce138558d6e23e1c96336
    Closes-Bug: #1501366


** Changed in: nova
       Status: In Progress => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1501366

Title:
  libvirtError: Error while building firewall: Some rules could not be
  created for interface

Status in OpenStack Compute (nova):
  Fix Released

Bug description:
  http://logs.openstack.org/20/223320/3/check/gate-tempest-dsvm-
  nova-v20-api/ce04943/logs/screen-n-cpu.txt.gz?level=TRACE#_2015-09-29_21_38_32_631

  2015-09-29 21:38:32.631 ERROR nova.compute.manager [req-f57dc3ad-e960-4a18-8290-b01ab46b256b tempest-SecurityGroupsTestJSON-816336435 tempest-SecurityGroupsTestJSON-717971163] [instance: 00a32e3b-4fb9-4c95-951e-febc02c1ba91] Instance failed to spawn
  2015-09-29 21:38:32.631 24733 ERROR nova.compute.manager [instance: 00a32e3b-4fb9-4c95-951e-febc02c1ba91] Traceback (most recent call last):
  2015-09-29 21:38:32.631 24733 ERROR nova.compute.manager [instance: 00a32e3b-4fb9-4c95-951e-febc02c1ba91]   File "/opt/stack/new/nova/nova/compute/manager.py", line 2155, in _build_resources
  2015-09-29 21:38:32.631 24733 ERROR nova.compute.manager [instance: 00a32e3b-4fb9-4c95-951e-febc02c1ba91]     yield resources
  2015-09-29 21:38:32.631 24733 ERROR nova.compute.manager [instance: 00a32e3b-4fb9-4c95-951e-febc02c1ba91]   File "/opt/stack/new/nova/nova/compute/manager.py", line 2009, in _build_and_run_instance
  2015-09-29 21:38:32.631 24733 ERROR nova.compute.manager [instance: 00a32e3b-4fb9-4c95-951e-febc02c1ba91]     block_device_info=block_device_info)
  2015-09-29 21:38:32.631 24733 ERROR nova.compute.manager [instance: 00a32e3b-4fb9-4c95-951e-febc02c1ba91]   File "/opt/stack/new/nova/nova/virt/libvirt/driver.py", line 2444, in spawn
  2015-09-29 21:38:32.631 24733 ERROR nova.compute.manager [instance: 00a32e3b-4fb9-4c95-951e-febc02c1ba91]     block_device_info=block_device_info)
  2015-09-29 21:38:32.631 24733 ERROR nova.compute.manager [instance: 00a32e3b-4fb9-4c95-951e-febc02c1ba91]   File "/opt/stack/new/nova/nova/virt/libvirt/driver.py", line 4516, in _create_domain_and_network
  2015-09-29 21:38:32.631 24733 ERROR nova.compute.manager [instance: 00a32e3b-4fb9-4c95-951e-febc02c1ba91]     xml, pause=pause, power_on=power_on)
  2015-09-29 21:38:32.631 24733 ERROR nova.compute.manager [instance: 00a32e3b-4fb9-4c95-951e-febc02c1ba91]   File "/opt/stack/new/nova/nova/virt/libvirt/driver.py", line 4446, in _create_domain
  2015-09-29 21:38:32.631 24733 ERROR nova.compute.manager [instance: 00a32e3b-4fb9-4c95-951e-febc02c1ba91]     guest.launch(pause=pause)
  2015-09-29 21:38:32.631 24733 ERROR nova.compute.manager [instance: 00a32e3b-4fb9-4c95-951e-febc02c1ba91]   File "/opt/stack/new/nova/nova/virt/libvirt/guest.py", line 141, in launch
  2015-09-29 21:38:32.631 24733 ERROR nova.compute.manager [instance: 00a32e3b-4fb9-4c95-951e-febc02c1ba91]     self._encoded_xml, errors='ignore')
  2015-09-29 21:38:32.631 24733 ERROR nova.compute.manager [instance: 00a32e3b-4fb9-4c95-951e-febc02c1ba91]   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 195, in __exit__
  2015-09-29 21:38:32.631 24733 ERROR nova.compute.manager [instance: 00a32e3b-4fb9-4c95-951e-febc02c1ba91]     six.reraise(self.type_, self.value, self.tb)
  2015-09-29 21:38:32.631 24733 ERROR nova.compute.manager [instance: 00a32e3b-4fb9-4c95-951e-febc02c1ba91]   File "/opt/stack/new/nova/nova/virt/libvirt/guest.py", line 136, in launch
  2015-09-29 21:38:32.631 24733 ERROR nova.compute.manager [instance: 00a32e3b-4fb9-4c95-951e-febc02c1ba91]     return self._domain.createWithFlags(flags)
  2015-09-29 21:38:32.631 24733 ERROR nova.compute.manager [instance: 00a32e3b-4fb9-4c95-951e-febc02c1ba91]   File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 183, in doit
  2015-09-29 21:38:32.631 24733 ERROR nova.compute.manager [instance: 00a32e3b-4fb9-4c95-951e-febc02c1ba91]     result = proxy_call(self._autowrap, f, *args, **kwargs)
  2015-09-29 21:38:32.631 24733 ERROR nova.compute.manager [instance: 00a32e3b-4fb9-4c95-951e-febc02c1ba91]   File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 141, in proxy_call
  2015-09-29 21:38:32.631 24733 ERROR nova.compute.manager [instance: 00a32e3b-4fb9-4c95-951e-febc02c1ba91]     rv = execute(f, *args, **kwargs)
  2015-09-29 21:38:32.631 24733 ERROR nova.compute.manager [instance: 00a32e3b-4fb9-4c95-951e-febc02c1ba91]   File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 122, in execute
  2015-09-29 21:38:32.631 24733 ERROR nova.compute.manager [instance: 00a32e3b-4fb9-4c95-951e-febc02c1ba91]     six.reraise(c, e, tb)
  2015-09-29 21:38:32.631 24733 ERROR nova.compute.manager [instance: 00a32e3b-4fb9-4c95-951e-febc02c1ba91]   File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 80, in tworker
  2015-09-29 21:38:32.631 24733 ERROR nova.compute.manager [instance: 00a32e3b-4fb9-4c95-951e-febc02c1ba91]     rv = meth(*args, **kwargs)
  2015-09-29 21:38:32.631 24733 ERROR nova.compute.manager [instance: 00a32e3b-4fb9-4c95-951e-febc02c1ba91]   File "/usr/local/lib/python2.7/dist-packages/libvirt.py", line 900, in createWithFlags
  2015-09-29 21:38:32.631 24733 ERROR nova.compute.manager [instance: 00a32e3b-4fb9-4c95-951e-febc02c1ba91]     if ret == -1: raise libvirtError ('virDomainCreateWithFlags() failed', dom=self)
  2015-09-29 21:38:32.631 24733 ERROR nova.compute.manager [instance: 00a32e3b-4fb9-4c95-951e-febc02c1ba91] libvirtError: Error while building firewall: Some rules could not be created for interface vnet3: Failure to execute command '$EBT -t nat -A libvirt-J-vnet3  -p 0x0800 -j J-vnet3-ipv4' : 'Illegal target name 'J-vnet3-ipv4'.'.
  2015-09-29 21:38:32.631 24733 ERROR nova.compute.manager [instance: 00a32e3b-4fb9-4c95-951e-febc02c1ba91] 
  2015-09-29 21:38:32.631 24733 ERROR nova.compute.manager [instance: 00a32e3b-4fb9-4c95-951e-febc02c1ba91] 


  http://logstash.openstack.org/#eyJzZWFyY2giOiJtZXNzYWdlOlwibGlidmlydEVycm9yOiBFcnJvciB3aGlsZSBidWlsZGluZyBmaXJld2FsbDogU29tZSBydWxlcyBjb3VsZCBub3QgYmUgY3JlYXRlZCBmb3IgaW50ZXJmYWNlXCIgQU5EIG1lc3NhZ2U6XCJJbGxlZ2FsIHRhcmdldCBuYW1lXCIgQU5EIHRhZ3M6XCJzY3JlZW4tbi1jcHUudHh0XCIiLCJmaWVsZHMiOltdLCJvZmZzZXQiOjAsInRpbWVmcmFtZSI6IjYwNDgwMCIsImdyYXBobW9kZSI6ImNvdW50IiwidGltZSI6eyJ1c2VyX2ludGVydmFsIjowfSwic3RhbXAiOjE0NDM2MjM3MjY1MDV9

  10 hits in 7 days, check and gate, all failures.  Looks like this is
  starting to spike.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1501366/+subscriptions