yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1533572] [NEW] Failed to create vm on kvm while boot with multiple nics

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Saurabh <1533572@xxxxxxxxxxxxxxxxxx>
Date: Wed, 13 Jan 2016 09:25:10 -0000
Reply-to: Bug 1533572 <1533572@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
Public bug reported:

On devsatck master branch, tried to boot vm with 4 vnic and it failed
with error:- Unable to create tap device tap791b4be3-3c: Device or
resource busy, 2 out of 20 times


ERROR trace:-
2016-01-13 21:41:04.550 ^[[01;31mERROR nova.compute.manager [^[[01;36mreq-5bcd4a69-13e3-4ec4-9a24-f665e61759e9 ^[[00;36mctx_rally_06cccf92bf84487293a6cf350e10721c_user_0 ctx_rally_d2f4c584-4152-4e5a-97ab-d2e9e019b22b_tenant_12^[[01;31m] ^[[01;35m[instance: a89c8f38-618e-4307-8f42-b243c900a674] ^[[01;31mInstance failed to spawn^[[00m
^[[01;31m2016-01-13 21:41:04.550 TRACE nova.compute.manager ^[[01;35m[instance: a89c8f38-618e-4307-8f42-b243c900a674] ^[[00mTraceback (most recent call last):
^[[01;31m2016-01-13 21:41:04.550 TRACE nova.compute.manager ^[[01;35m[instance: a89c8f38-618e-4307-8f42-b243c900a674] ^[[00m  File "/opt/stack/nova/nova/compute/manager.py", line 2182, in _build_resources
^[[01;31m2016-01-13 21:41:04.550 TRACE nova.compute.manager ^[[01;35m[instance: a89c8f38-618e-4307-8f42-b243c900a674] ^[[00m    yield resources
^[[01;31m2016-01-13 21:41:04.550 TRACE nova.compute.manager ^[[01;35m[instance: a89c8f38-618e-4307-8f42-b243c900a674] ^[[00m  File "/opt/stack/nova/nova/compute/manager.py", line 2029, in _build_and_run_instance
^[[01;31m2016-01-13 21:41:04.550 TRACE nova.compute.manager ^[[01;35m[instance: a89c8f38-618e-4307-8f42-b243c900a674] ^[[00m    block_device_info=block_device_info)
^[[01;31m2016-01-13 21:41:04.550 TRACE nova.compute.manager ^[[01;35m[instance: a89c8f38-618e-4307-8f42-b243c900a674] ^[[00m  File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 2517, in spawn
^[[01;31m2016-01-13 21:41:04.550 TRACE nova.compute.manager ^[[01;35m[instance: a89c8f38-618e-4307-8f42-b243c900a674] ^[[00m    block_device_info=block_device_info)
^[[01;31m2016-01-13 21:41:04.550 TRACE nova.compute.manager ^[[01;35m[instance: a89c8f38-618e-4307-8f42-b243c900a674] ^[[00m  File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 4631, in _create_domain_and_network
^[[01;31m2016-01-13 21:41:04.550 TRACE nova.compute.manager ^[[01;35m[instance: a89c8f38-618e-4307-8f42-b243c900a674] ^[[00m    xml, pause=pause, power_on=power_on)
^[[01;31m2016-01-13 21:41:04.550 TRACE nova.compute.manager ^[[01;35m[instance: a89c8f38-618e-4307-8f42-b243c900a674] ^[[00m  File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 4561, in _create_domain
^[[01;31m2016-01-13 21:41:04.550 TRACE nova.compute.manager ^[[01;35m[instance: a89c8f38-618e-4307-8f42-b243c900a674] ^[[00m    guest.launch(pause=pause)
^[[01;31m2016-01-13 21:41:04.550 TRACE nova.compute.manager ^[[01;35m[instance: a89c8f38-618e-4307-8f42-b243c900a674] ^[[00m  File "/opt/stack/nova/nova/virt/libvirt/guest.py", line 142, in launch
^[[01;31m2016-01-13 21:41:04.550 TRACE nova.compute.manager ^[[01;35m[instance: a89c8f38-618e-4307-8f42-b243c900a674] ^[[00m    self._encoded_xml, errors='ignore')
^[[01;31m2016-01-13 21:41:04.550 TRACE nova.compute.manager ^[[01;35m[instance: a89c8f38-618e-4307-8f42-b243c900a674] ^[[00m  File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 204, in __exit__
^[[01;31m2016-01-13 21:41:04.550 TRACE nova.compute.manager ^[[01;35m[instance: a89c8f38-618e-4307-8f42-b243c900a674] ^[[00m    six.reraise(self.type_, self.value, self.tb)
^[[01;31m2016-01-13 21:41:04.550 TRACE nova.compute.manager ^[[01;35m[instance: a89c8f38-618e-4307-8f42-b243c900a674] ^[[00m  File "/opt/stack/nova/nova/virt/libvirt/guest.py", line 137, in launch
^[[01;31m2016-01-13 21:41:04.550 TRACE nova.compute.manager ^[[01;35m[instance: a89c8f38-618e-4307-8f42-b243c900a674] ^[[00m    return self._domain.createWithFlags(flags)
^[[01;31m2016-01-13 21:41:04.550 TRACE nova.compute.manager ^[[01;35m[instance: a89c8f38-618e-4307-8f42-b243c900a674] ^[[00m  File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 183, in doit
^[[01;31m2016-01-13 21:41:04.550 TRACE nova.compute.manager ^[[01;35m[instance: a89c8f38-618e-4307-8f42-b243c900a674] ^[[00m    result = proxy_call(self._autowrap, f, *args, **kwargs)
^[[01;31m2016-01-13 21:41:04.550 TRACE nova.compute.manager ^[[01;35m[instance: a89c8f38-618e-4307-8f42-b243c900a674] ^[[00m  File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 141, in proxy_call
^[[01;31m2016-01-13 21:41:04.550 TRACE nova.compute.manager ^[[01;35m[instance: a89c8f38-618e-4307-8f42-b243c900a674] ^[[00m    rv = execute(f, *args, **kwargs)
^[[01;31m2016-01-13 21:41:04.550 TRACE nova.compute.manager ^[[01;35m[instance: a89c8f38-618e-4307-8f42-b243c900a674] ^[[00m  File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 122, in execute
^[[01;31m2016-01-13 21:41:04.550 TRACE nova.compute.manager ^[[01;35m[instance: a89c8f38-618e-4307-8f42-b243c900a674] ^[[00m    six.reraise(c, e, tb)
^[[01;31m2016-01-13 21:41:04.550 TRACE nova.compute.manager ^[[01;35m[instance: a89c8f38-618e-4307-8f42-b243c900a674] ^[[00m  File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 80, in tworker
^[[01;31m2016-01-13 21:41:04.550 TRACE nova.compute.manager ^[[01;35m[instance: a89c8f38-618e-4307-8f42-b243c900a674] ^[[00m    rv = meth(*args, **kwargs)
^[[01;31m2016-01-13 21:41:04.550 TRACE nova.compute.manager ^[[01;35m[instance: a89c8f38-618e-4307-8f42-b243c900a674] ^[[00m  File "/usr/local/lib/python2.7/dist-packages/libvirt.py", line 900, in createWithFlags
^[[01;31m2016-01-13 21:41:04.550 TRACE nova.compute.manager ^[[01;35m[instance: a89c8f38-618e-4307-8f42-b243c900a674] ^[[00m    if ret == -1: raise libvirtError ('virDomainCreateWithFlags() failed', dom=self)
^[[01;31m2016-01-13 21:41:04.550 TRACE nova.compute.manager ^[[01;35m[instance: a89c8f38-618e-4307-8f42-b243c900a674] ^[[00mlibvirtError: Unable to create tap device tap791b4be3-3c: Device or resource busy


XLM copied from logs has 7 tap devices while it should have 4 only
97ab-d2e9e019b22b_tenant_12^[[01;31m] ^[[01;35m^[[01;31mError launching a defined domain with XML: <domain type='kvm'>
  <name>instance-00000072</name>
  <uuid>a89c8f38-618e-4307-8f42-b243c900a674</uuid>
  <metadata>
    <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.0";>
      <nova:package version="13.0.0"/>
      <nova:name>rally_novaserver_j9L1M5GDhn3EgUq1_0</nova:name>
      <nova:creationTime>2016-01-13 16:11:03</nova:creationTime>
      <nova:flavor name="m1.tiny">
        <nova:memory>512</nova:memory>
        <nova:disk>1</nova:disk>
        <nova:swap>0</nova:swap>
        <nova:ephemeral>0</nova:ephemeral>
        <nova:vcpus>1</nova:vcpus>
      </nova:flavor>
      <nova:owner>
        <nova:user uuid="0e158a8b454b42488e451e585ccc1366">ctx_rally_06cccf92bf84487293a6cf350e10721c_user_0</nova:user>
        <nova:project uuid="06cccf92bf84487293a6cf350e10721c">ctx_rally_d2f4c584-4152-4e5a-97ab-d2e9e019b22b_tenant_12</nova:project>
      </nova:owner>
      <nova:root type="image" uuid="cd8e551c-288e-4585-a0da-fc3e43852ed4"/>
    </nova:instance>
  </metadata>
  <memory unit='KiB'>524288</memory>
  <currentMemory unit='KiB'>524288</currentMemory>
  <vcpu placement='static'>1</vcpu>
  <cputune>
    <shares>1024</shares>
  </cputune>
  <sysinfo type='smbios'>
    <system>
      <entry name='manufacturer'>OpenStack Foundation</entry>
      <entry name='product'>OpenStack Nova</entry>
      <entry name='version'>13.0.0</entry>
      <entry name='serial'>4237a912-c226-57d7-55d8-484a848739d1</entry>
      <entry name='uuid'>a89c8f38-618e-4307-8f42-b243c900a674</entry>
      <entry name='family'>Virtual Machine</entry>
    </system>
  </sysinfo>
  <os>
    <type arch='x86_64' machine='pc-i440fx-trusty'>hvm</type>
    <kernel>/opt/stack/data/nova/instances/a89c8f38-618e-4307-8f42-b243c900a674/kernel</kernel>
    <initrd>/opt/stack/data/nova/instances/a89c8f38-618e-4307-8f42-b243c900a674/ramdisk</initrd>
    <cmdline>root=/dev/vda console=tty0 console=ttyS0</cmdline>
    <boot dev='hd'/>
    <smbios mode='sysinfo'/>
  </os>
  <features>
    <acpi/>
    <apic/>
  </features>
  <cpu>
    <topology sockets='1' cores='1' threads='1'/>
  </cpu>
  <clock offset='utc'>
    <timer name='pit' tickpolicy='delay'/>
    <timer name='rtc' tickpolicy='catchup'/>
    <timer name='hpet' present='no'/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>destroy</on_crash>
  <devices>
    <emulator>/usr/bin/kvm-spice</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2' cache='none'/>
      <source file='/opt/stack/data/nova/instances/a89c8f38-618e-4307-8f42-b243c900a674/disk'/>
      <target dev='vda' bus='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x0a' function='0x0'/>
    </disk>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw' cache='none'/>
      <source file='/opt/stack/data/nova/instances/a89c8f38-618e-4307-8f42-b243c900a674/disk.config'/>
      <target dev='hdd' bus='ide'/>
      <readonly/>
      <address type='drive' controller='0' bus='1' target='0' unit='1'/>
    </disk>
    <controller type='usb' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/>
    </controller>
    <controller type='pci' index='0' model='pci-root'/>
    <controller type='ide' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
    </controller>
    <interface type='bridge'>
      <mac address='fa:16:3e:36:5a:f7'/>
      <source bridge='nsx-managed'/>
      <virtualport type='openvswitch'>
        <parameters interfaceid='068a041a-eded-4452-bfb8-931012f4f213'/>
      </virtualport>
      <target dev='tap068a041a-ed'/>
      <model type='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </interface>
    <interface type='bridge'>
      <mac address='fa:16:3e:0f:96:8b'/>
      <source bridge='nsx-managed'/>
      <virtualport type='openvswitch'>
        <parameters interfaceid='791b4be3-3c03-4cba-bb0e-e9744aac862b'/>
      </virtualport>
      <target dev='tap791b4be3-3c'/>
      <model type='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
    </interface>
    <interface type='bridge'>
      <mac address='fa:16:3e:2d:46:45'/>
      <source bridge='nsx-managed'/>
      <virtualport type='openvswitch'>
        <parameters interfaceid='ef9f910d-683a-4297-bcca-497026cd92dc'/>
      </virtualport>
      <target dev='tapef9f910d-68'/>
      <model type='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </interface>
    <interface type='bridge'>
      <mac address='fa:16:3e:0f:96:8b'/>
      <source bridge='nsx-managed'/>
      <virtualport type='openvswitch'>
        <parameters interfaceid='791b4be3-3c03-4cba-bb0e-e9744aac862b'/>
      </virtualport>
      <target dev='tap791b4be3-3c'/>
      <model type='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
    </interface>
    <interface type='bridge'>
      <mac address='fa:16:3e:36:5a:f7'/>
      <source bridge='nsx-managed'/>
      <virtualport type='openvswitch'>
        <parameters interfaceid='068a041a-eded-4452-bfb8-931012f4f213'/>
      </virtualport>
      <target dev='tap068a041a-ed'/>
      <model type='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/>
    </interface>
    <interface type='bridge'>
      <mac address='fa:16:3e:2d:46:45'/>
      <source bridge='nsx-managed'/>
      <virtualport type='openvswitch'>
        <parameters interfaceid='ef9f910d-683a-4297-bcca-497026cd92dc'/>
      </virtualport>
      <target dev='tapef9f910d-68'/>
      <model type='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/>
    </interface>
    <interface type='bridge'>
      <mac address='fa:16:3e:0f:60:ef'/>
      <source bridge='nsx-managed'/>
      <virtualport type='openvswitch'>
        <parameters interfaceid='cdedc99a-8380-4908-9569-ad8c3e6550b4'/>
      </virtualport>
      <target dev='tapcdedc99a-83'/>
      <model type='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/>
    </interface>
    <serial type='file'>
      <source path='/opt/stack/data/nova/instances/a89c8f38-618e-4307-8f42-b243c900a674/console.log'/>
      <target port='0'/>
    </serial>
    <serial type='pty'>
      <target port='1'/>
    </serial>
    <console type='file'>
      <source path='/opt/stack/data/nova/instances/a89c8f38-618e-4307-8f42-b243c900a674/console.log'/>
      <target type='serial' port='0'/>
    </console>
    <input type='mouse' bus='ps2'/>
    <input type='keyboard' bus='ps2'/>
    <graphics type='vnc' port='-1' autoport='yes' listen='10.112.203.208' keymap='en-us'>
      <listen type='address' address='10.112.203.208'/>
    </graphics>
    <video>
      <model type='cirrus' vram='9216' heads='1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </video>
    <memballoon model='virtio'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x0b' function='0x0'/>
      <stats period='10'/>
    </memballoon>
  </devices>
</domain>

** Affects: nova
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1533572

Title:
  Failed to create vm on kvm while boot with multiple nics

Status in OpenStack Compute (nova):
  New

Bug description:
  On devsatck master branch, tried to boot vm with 4 vnic and it failed
  with error:- Unable to create tap device tap791b4be3-3c: Device or
  resource busy, 2 out of 20 times

  
  ERROR trace:-
  2016-01-13 21:41:04.550 ^[[01;31mERROR nova.compute.manager [^[[01;36mreq-5bcd4a69-13e3-4ec4-9a24-f665e61759e9 ^[[00;36mctx_rally_06cccf92bf84487293a6cf350e10721c_user_0 ctx_rally_d2f4c584-4152-4e5a-97ab-d2e9e019b22b_tenant_12^[[01;31m] ^[[01;35m[instance: a89c8f38-618e-4307-8f42-b243c900a674] ^[[01;31mInstance failed to spawn^[[00m
  ^[[01;31m2016-01-13 21:41:04.550 TRACE nova.compute.manager ^[[01;35m[instance: a89c8f38-618e-4307-8f42-b243c900a674] ^[[00mTraceback (most recent call last):
  ^[[01;31m2016-01-13 21:41:04.550 TRACE nova.compute.manager ^[[01;35m[instance: a89c8f38-618e-4307-8f42-b243c900a674] ^[[00m  File "/opt/stack/nova/nova/compute/manager.py", line 2182, in _build_resources
  ^[[01;31m2016-01-13 21:41:04.550 TRACE nova.compute.manager ^[[01;35m[instance: a89c8f38-618e-4307-8f42-b243c900a674] ^[[00m    yield resources
  ^[[01;31m2016-01-13 21:41:04.550 TRACE nova.compute.manager ^[[01;35m[instance: a89c8f38-618e-4307-8f42-b243c900a674] ^[[00m  File "/opt/stack/nova/nova/compute/manager.py", line 2029, in _build_and_run_instance
  ^[[01;31m2016-01-13 21:41:04.550 TRACE nova.compute.manager ^[[01;35m[instance: a89c8f38-618e-4307-8f42-b243c900a674] ^[[00m    block_device_info=block_device_info)
  ^[[01;31m2016-01-13 21:41:04.550 TRACE nova.compute.manager ^[[01;35m[instance: a89c8f38-618e-4307-8f42-b243c900a674] ^[[00m  File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 2517, in spawn
  ^[[01;31m2016-01-13 21:41:04.550 TRACE nova.compute.manager ^[[01;35m[instance: a89c8f38-618e-4307-8f42-b243c900a674] ^[[00m    block_device_info=block_device_info)
  ^[[01;31m2016-01-13 21:41:04.550 TRACE nova.compute.manager ^[[01;35m[instance: a89c8f38-618e-4307-8f42-b243c900a674] ^[[00m  File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 4631, in _create_domain_and_network
  ^[[01;31m2016-01-13 21:41:04.550 TRACE nova.compute.manager ^[[01;35m[instance: a89c8f38-618e-4307-8f42-b243c900a674] ^[[00m    xml, pause=pause, power_on=power_on)
  ^[[01;31m2016-01-13 21:41:04.550 TRACE nova.compute.manager ^[[01;35m[instance: a89c8f38-618e-4307-8f42-b243c900a674] ^[[00m  File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 4561, in _create_domain
  ^[[01;31m2016-01-13 21:41:04.550 TRACE nova.compute.manager ^[[01;35m[instance: a89c8f38-618e-4307-8f42-b243c900a674] ^[[00m    guest.launch(pause=pause)
  ^[[01;31m2016-01-13 21:41:04.550 TRACE nova.compute.manager ^[[01;35m[instance: a89c8f38-618e-4307-8f42-b243c900a674] ^[[00m  File "/opt/stack/nova/nova/virt/libvirt/guest.py", line 142, in launch
  ^[[01;31m2016-01-13 21:41:04.550 TRACE nova.compute.manager ^[[01;35m[instance: a89c8f38-618e-4307-8f42-b243c900a674] ^[[00m    self._encoded_xml, errors='ignore')
  ^[[01;31m2016-01-13 21:41:04.550 TRACE nova.compute.manager ^[[01;35m[instance: a89c8f38-618e-4307-8f42-b243c900a674] ^[[00m  File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 204, in __exit__
  ^[[01;31m2016-01-13 21:41:04.550 TRACE nova.compute.manager ^[[01;35m[instance: a89c8f38-618e-4307-8f42-b243c900a674] ^[[00m    six.reraise(self.type_, self.value, self.tb)
  ^[[01;31m2016-01-13 21:41:04.550 TRACE nova.compute.manager ^[[01;35m[instance: a89c8f38-618e-4307-8f42-b243c900a674] ^[[00m  File "/opt/stack/nova/nova/virt/libvirt/guest.py", line 137, in launch
  ^[[01;31m2016-01-13 21:41:04.550 TRACE nova.compute.manager ^[[01;35m[instance: a89c8f38-618e-4307-8f42-b243c900a674] ^[[00m    return self._domain.createWithFlags(flags)
  ^[[01;31m2016-01-13 21:41:04.550 TRACE nova.compute.manager ^[[01;35m[instance: a89c8f38-618e-4307-8f42-b243c900a674] ^[[00m  File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 183, in doit
  ^[[01;31m2016-01-13 21:41:04.550 TRACE nova.compute.manager ^[[01;35m[instance: a89c8f38-618e-4307-8f42-b243c900a674] ^[[00m    result = proxy_call(self._autowrap, f, *args, **kwargs)
  ^[[01;31m2016-01-13 21:41:04.550 TRACE nova.compute.manager ^[[01;35m[instance: a89c8f38-618e-4307-8f42-b243c900a674] ^[[00m  File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 141, in proxy_call
  ^[[01;31m2016-01-13 21:41:04.550 TRACE nova.compute.manager ^[[01;35m[instance: a89c8f38-618e-4307-8f42-b243c900a674] ^[[00m    rv = execute(f, *args, **kwargs)
  ^[[01;31m2016-01-13 21:41:04.550 TRACE nova.compute.manager ^[[01;35m[instance: a89c8f38-618e-4307-8f42-b243c900a674] ^[[00m  File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 122, in execute
  ^[[01;31m2016-01-13 21:41:04.550 TRACE nova.compute.manager ^[[01;35m[instance: a89c8f38-618e-4307-8f42-b243c900a674] ^[[00m    six.reraise(c, e, tb)
  ^[[01;31m2016-01-13 21:41:04.550 TRACE nova.compute.manager ^[[01;35m[instance: a89c8f38-618e-4307-8f42-b243c900a674] ^[[00m  File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 80, in tworker
  ^[[01;31m2016-01-13 21:41:04.550 TRACE nova.compute.manager ^[[01;35m[instance: a89c8f38-618e-4307-8f42-b243c900a674] ^[[00m    rv = meth(*args, **kwargs)
  ^[[01;31m2016-01-13 21:41:04.550 TRACE nova.compute.manager ^[[01;35m[instance: a89c8f38-618e-4307-8f42-b243c900a674] ^[[00m  File "/usr/local/lib/python2.7/dist-packages/libvirt.py", line 900, in createWithFlags
  ^[[01;31m2016-01-13 21:41:04.550 TRACE nova.compute.manager ^[[01;35m[instance: a89c8f38-618e-4307-8f42-b243c900a674] ^[[00m    if ret == -1: raise libvirtError ('virDomainCreateWithFlags() failed', dom=self)
  ^[[01;31m2016-01-13 21:41:04.550 TRACE nova.compute.manager ^[[01;35m[instance: a89c8f38-618e-4307-8f42-b243c900a674] ^[[00mlibvirtError: Unable to create tap device tap791b4be3-3c: Device or resource busy


  XLM copied from logs has 7 tap devices while it should have 4 only
  97ab-d2e9e019b22b_tenant_12^[[01;31m] ^[[01;35m^[[01;31mError launching a defined domain with XML: <domain type='kvm'>
    <name>instance-00000072</name>
    <uuid>a89c8f38-618e-4307-8f42-b243c900a674</uuid>
    <metadata>
      <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.0";>
        <nova:package version="13.0.0"/>
        <nova:name>rally_novaserver_j9L1M5GDhn3EgUq1_0</nova:name>
        <nova:creationTime>2016-01-13 16:11:03</nova:creationTime>
        <nova:flavor name="m1.tiny">
          <nova:memory>512</nova:memory>
          <nova:disk>1</nova:disk>
          <nova:swap>0</nova:swap>
          <nova:ephemeral>0</nova:ephemeral>
          <nova:vcpus>1</nova:vcpus>
        </nova:flavor>
        <nova:owner>
          <nova:user uuid="0e158a8b454b42488e451e585ccc1366">ctx_rally_06cccf92bf84487293a6cf350e10721c_user_0</nova:user>
          <nova:project uuid="06cccf92bf84487293a6cf350e10721c">ctx_rally_d2f4c584-4152-4e5a-97ab-d2e9e019b22b_tenant_12</nova:project>
        </nova:owner>
        <nova:root type="image" uuid="cd8e551c-288e-4585-a0da-fc3e43852ed4"/>
      </nova:instance>
    </metadata>
    <memory unit='KiB'>524288</memory>
    <currentMemory unit='KiB'>524288</currentMemory>
    <vcpu placement='static'>1</vcpu>
    <cputune>
      <shares>1024</shares>
    </cputune>
    <sysinfo type='smbios'>
      <system>
        <entry name='manufacturer'>OpenStack Foundation</entry>
        <entry name='product'>OpenStack Nova</entry>
        <entry name='version'>13.0.0</entry>
        <entry name='serial'>4237a912-c226-57d7-55d8-484a848739d1</entry>
        <entry name='uuid'>a89c8f38-618e-4307-8f42-b243c900a674</entry>
        <entry name='family'>Virtual Machine</entry>
      </system>
    </sysinfo>
    <os>
      <type arch='x86_64' machine='pc-i440fx-trusty'>hvm</type>
      <kernel>/opt/stack/data/nova/instances/a89c8f38-618e-4307-8f42-b243c900a674/kernel</kernel>
      <initrd>/opt/stack/data/nova/instances/a89c8f38-618e-4307-8f42-b243c900a674/ramdisk</initrd>
      <cmdline>root=/dev/vda console=tty0 console=ttyS0</cmdline>
      <boot dev='hd'/>
      <smbios mode='sysinfo'/>
    </os>
    <features>
      <acpi/>
      <apic/>
    </features>
    <cpu>
      <topology sockets='1' cores='1' threads='1'/>
    </cpu>
    <clock offset='utc'>
      <timer name='pit' tickpolicy='delay'/>
      <timer name='rtc' tickpolicy='catchup'/>
      <timer name='hpet' present='no'/>
    </clock>
    <on_poweroff>destroy</on_poweroff>
    <on_reboot>restart</on_reboot>
    <on_crash>destroy</on_crash>
    <devices>
      <emulator>/usr/bin/kvm-spice</emulator>
      <disk type='file' device='disk'>
        <driver name='qemu' type='qcow2' cache='none'/>
        <source file='/opt/stack/data/nova/instances/a89c8f38-618e-4307-8f42-b243c900a674/disk'/>
        <target dev='vda' bus='virtio'/>
        <address type='pci' domain='0x0000' bus='0x00' slot='0x0a' function='0x0'/>
      </disk>
      <disk type='file' device='cdrom'>
        <driver name='qemu' type='raw' cache='none'/>
        <source file='/opt/stack/data/nova/instances/a89c8f38-618e-4307-8f42-b243c900a674/disk.config'/>
        <target dev='hdd' bus='ide'/>
        <readonly/>
        <address type='drive' controller='0' bus='1' target='0' unit='1'/>
      </disk>
      <controller type='usb' index='0'>
        <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/>
      </controller>
      <controller type='pci' index='0' model='pci-root'/>
      <controller type='ide' index='0'>
        <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
      </controller>
      <interface type='bridge'>
        <mac address='fa:16:3e:36:5a:f7'/>
        <source bridge='nsx-managed'/>
        <virtualport type='openvswitch'>
          <parameters interfaceid='068a041a-eded-4452-bfb8-931012f4f213'/>
        </virtualport>
        <target dev='tap068a041a-ed'/>
        <model type='virtio'/>
        <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
      </interface>
      <interface type='bridge'>
        <mac address='fa:16:3e:0f:96:8b'/>
        <source bridge='nsx-managed'/>
        <virtualport type='openvswitch'>
          <parameters interfaceid='791b4be3-3c03-4cba-bb0e-e9744aac862b'/>
        </virtualport>
        <target dev='tap791b4be3-3c'/>
        <model type='virtio'/>
        <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
      </interface>
      <interface type='bridge'>
        <mac address='fa:16:3e:2d:46:45'/>
        <source bridge='nsx-managed'/>
        <virtualport type='openvswitch'>
          <parameters interfaceid='ef9f910d-683a-4297-bcca-497026cd92dc'/>
        </virtualport>
        <target dev='tapef9f910d-68'/>
        <model type='virtio'/>
        <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
      </interface>
      <interface type='bridge'>
        <mac address='fa:16:3e:0f:96:8b'/>
        <source bridge='nsx-managed'/>
        <virtualport type='openvswitch'>
          <parameters interfaceid='791b4be3-3c03-4cba-bb0e-e9744aac862b'/>
        </virtualport>
        <target dev='tap791b4be3-3c'/>
        <model type='virtio'/>
        <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
      </interface>
      <interface type='bridge'>
        <mac address='fa:16:3e:36:5a:f7'/>
        <source bridge='nsx-managed'/>
        <virtualport type='openvswitch'>
          <parameters interfaceid='068a041a-eded-4452-bfb8-931012f4f213'/>
        </virtualport>
        <target dev='tap068a041a-ed'/>
        <model type='virtio'/>
        <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/>
      </interface>
      <interface type='bridge'>
        <mac address='fa:16:3e:2d:46:45'/>
        <source bridge='nsx-managed'/>
        <virtualport type='openvswitch'>
          <parameters interfaceid='ef9f910d-683a-4297-bcca-497026cd92dc'/>
        </virtualport>
        <target dev='tapef9f910d-68'/>
        <model type='virtio'/>
        <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/>
      </interface>
      <interface type='bridge'>
        <mac address='fa:16:3e:0f:60:ef'/>
        <source bridge='nsx-managed'/>
        <virtualport type='openvswitch'>
          <parameters interfaceid='cdedc99a-8380-4908-9569-ad8c3e6550b4'/>
        </virtualport>
        <target dev='tapcdedc99a-83'/>
        <model type='virtio'/>
        <address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/>
      </interface>
      <serial type='file'>
        <source path='/opt/stack/data/nova/instances/a89c8f38-618e-4307-8f42-b243c900a674/console.log'/>
        <target port='0'/>
      </serial>
      <serial type='pty'>
        <target port='1'/>
      </serial>
      <console type='file'>
        <source path='/opt/stack/data/nova/instances/a89c8f38-618e-4307-8f42-b243c900a674/console.log'/>
        <target type='serial' port='0'/>
      </console>
      <input type='mouse' bus='ps2'/>
      <input type='keyboard' bus='ps2'/>
      <graphics type='vnc' port='-1' autoport='yes' listen='10.112.203.208' keymap='en-us'>
        <listen type='address' address='10.112.203.208'/>
      </graphics>
      <video>
        <model type='cirrus' vram='9216' heads='1'/>
        <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
      </video>
      <memballoon model='virtio'>
        <address type='pci' domain='0x0000' bus='0x00' slot='0x0b' function='0x0'/>
        <stats period='10'/>
      </memballoon>
    </devices>
  </domain>

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1533572/+subscriptions
Follow ups

[Bug 1533572] Re: Failed to create vm on kvm while boot with multiple nics
From: Saurabh, 2016-07-18
[Bug 1533572] Re: Failed to create vm on kvm while boot with multiple nics
From: Launchpad Bug Tracker, 2016-03-15