← Back to team overview

openstack team mailing list archive

Openstack with ESXi - NetworkNotFoundForBridge: Network could not be found for bridge

 

Hi,

I have a two node OpenStack Folsom Setup. One node runs all services with
KVM. The other nodes is a Compute Node for ESXi 5.1. I have set the
availabality_zones for each of this so that I can schedule instances as per
my need. I am able to operate the instances on the KVM just fine. But when
I launch an instance for ESXi, it goes to ERROR state with the error
'NetworkNotFoundForBridge: Network could not be found for bridge'.

After the vanilla Folsom installation, I was getting an AttribueError which
got fixed after applying the patch at
https://bugs.launchpad.net/nova/+bug/1063885

But then after that, I am seeing this error.

root@openstack-kvm:~# nova show 8110c50c-ed26-48dd-886b-71b44ee382dd
+-------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------+
| Property                            | Value

                                    |
+-------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------+
| OS-DCF:diskConfig                   | MANUAL

                                     |
| OS-EXT-SRV-ATTR:host                | openstack-esxi

                                     |
| OS-EXT-SRV-ATTR:hypervisor_hostname | None

                                     |
| OS-EXT-SRV-ATTR:instance_name       | instance-00000016

                                    |
| OS-EXT-STS:power_state              | 0

                                    |
| OS-EXT-STS:task_state               | None

                                     |
| OS-EXT-STS:vm_state                 | error

                                    |
| accessIPv4                          |

                                    |
| accessIPv6                          |

                                    |
| config_drive                        |

                                    |
| created                             | 2013-01-23T07:13:11Z

                                     |
| fault                               | {u'message':
u'NetworkNotFoundForBridge', u'code': 404, u'details': u'Network could not
be found for bridge ', u'created': u'2013-01-23T07:13:14Z'} |
| flavor                              | m1.tiny (6)

                                    |
| hostId                              |
a9edbd6bb1136ff3bcf5225d1f6a34e0d18cc21eb9837acd67dec105
                                                                         |
| id                                  |
8110c50c-ed26-48dd-886b-71b44ee382dd
                                                                         |
| image                               | Ubuntu1204
(4d70a998-a51e-4557-81fa-23747db12305)
                                                              |
| key_name                            | None

                                     |
| metadata                            | {}

                                     |
| name                                | VM2

                                    |
| security_groups                     | [{u'name': u'default'}]

                                    |
| status                              | ERROR

                                    |
| tenant_id                           | b439758af3814f739fe7ab09b98a0120

                                     |
| updated                             | 2013-01-23T07:13:14Z

                                     |
| user_id                             | 9ac7dbe62fb349ce84927598a0f54d68

                                     |
+-------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------+
root@openstack-kvm:~#

My /etc/nova/nova.conf on the ESXi compute nodes is:

[DEFAULT]
logdir=/var/log/nova
state_path=/var/lib/nova
lock_path=/run/lock/nova
verbose=True
api_paste_config=/etc/nova/api-paste.ini
scheduler_driver=nova.scheduler.simple.SimpleScheduler
s3_host=10.5.3.230
ec2_host=10.5.3.230
ec2_dmz_host=10.5.3.230
rabbit_host=10.5.3.230
cc_host=10.5.3.230
dmz_cidr=169.254.169.254/32
metadata_host=10.5.3.230
metadata_listen=0.0.0.0
nova_url=http://10.5.3.230:8774/v1.1/
sql_connection=mysql://novaUser:novaPass@10.5.3.230/nova
ec2_url=http://10.5.3.230:8773/services/Cloud
root_helper=sudo nova-rootwrap /etc/nova/rootwrap.conf
my_ip=10.5.3.231

# Auth
use_deprecated_auth=false
auth_strategy=keystone
keystone_ec2_url=http://10.5.3.230:5000/v2.0/ec2tokens
# Imaging service
glance_api_servers=10.5.3.230:9292
image_service=nova.image.glance.GlanceImageService

# Vnc configuration
novnc_enabled=true
novncproxy_base_url=http://10.5.12.20:6080/vnc_auto.html
novncproxy_port=6080
vncserver_proxyclient_address=10.5.12.20
vncserver_listen=0.0.0.0

# Network settings
network_api_class=nova.network.quantumv2.api.API
quantum_url=http://10.5.3.230:9696
quantum_auth_strategy=keystone
quantum_admin_tenant_name=service
quantum_admin_username=quantum
quantum_admin_password=service_pass
quantum_admin_auth_url=http://10.5.3.230:35357/v2.0
libvirt_vif_driver=nova.virt.libvirt.vif.LibvirtHybridOVSBridgeDriver
linuxnet_interface_driver=nova.network.linux_net.LinuxOVSInterfaceDriver
firewall_driver=nova.virt.libvirt.firewall.IptablesFirewallDriver

# Compute #
compute_driver= vmwareapi.VMWareESXDriver
vmwareapi_host_ip=10.5.3.99
vmwareapi_host_username=root
vmwareapi_host_password=S0nicwall
vmwareapi_wsdl_loc=
http://127.0.0.1:8080/vmware/SDK/vsphere-ws/wsdl/vim25/vimService.wsdl
connection_type = vmwareapi
vmware_vif_driver = nova.virt.vmwareapi.vif.VMWareVlanBridgeDriver
console_manager = nova.console.vmrc_manager.ConsoleVMRCManager
vmwareapi_vlan_interface= vmnic4
node_availability_zone=zone2

# Cinder #
volume_api_class=nova.volume.cinder.API
osapi_volume_listen_port=5900


The log snippet from nova-compute.log is below:


2013-01-22 23:13:14 TRACE nova.compute.manager [instance:
8110c50c-ed26-48dd-886b-71b44ee382dd] Traceback (most recent call last):
2013-01-22 23:13:14 TRACE nova.compute.manager [instance:
8110c50c-ed26-48dd-886b-71b44ee382dd]   File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 743, in
_spawn
2013-01-22 23:13:14 TRACE nova.compute.manager [instance:
8110c50c-ed26-48dd-886b-71b44ee382dd]     block_device_info)
2013-01-22 23:13:14 TRACE nova.compute.manager [instance:
8110c50c-ed26-48dd-886b-71b44ee382dd]   File
"/usr/lib/python2.7/dist-packages/nova/virt/vmwareapi/driver.py", line 132,
in spawn
2013-01-22 23:13:14 TRACE nova.compute.manager [instance:
8110c50c-ed26-48dd-886b-71b44ee382dd]     self._vmops.spawn(context,
instance, image_meta, network_info)
2013-01-22 23:13:14 TRACE nova.compute.manager [instance:
8110c50c-ed26-48dd-886b-71b44ee382dd]   File
"/usr/lib/python2.7/dist-packages/nova/virt/vmwareapi/vmops.py", line 187,
in spawn
2013-01-22 23:13:14 TRACE nova.compute.manager [instance:
8110c50c-ed26-48dd-886b-71b44ee382dd]     vif_infos = _get_vif_infos()
2013-01-22 23:13:14 TRACE nova.compute.manager [instance:
8110c50c-ed26-48dd-886b-71b44ee382dd]   File
"/usr/lib/python2.7/dist-packages/nova/virt/vmwareapi/vmops.py", line 180,
in _get_vif_infos
2013-01-22 23:13:14 TRACE nova.compute.manager [instance:
8110c50c-ed26-48dd-886b-71b44ee382dd]     network_ref =
_check_if_network_bridge_exists(network_name)
2013-01-22 23:13:14 TRACE nova.compute.manager [instance:
8110c50c-ed26-48dd-886b-71b44ee382dd]   File
"/usr/lib/python2.7/dist-packages/nova/virt/vmwareapi/vmops.py", line 168,
in _check_if_network_bridge_exists
2013-01-22 23:13:14 TRACE nova.compute.manager [instance:
8110c50c-ed26-48dd-886b-71b44ee382dd]     raise
exception.NetworkNotFoundForBridge(bridge=network_name)
2013-01-22 23:13:14 TRACE nova.compute.manager [instance:
8110c50c-ed26-48dd-886b-71b44ee382dd] NetworkNotFoundForBridge: Network
could not be found for bridge
2013-01-22 23:13:14 TRACE nova.compute.manager [instance:
8110c50c-ed26-48dd-886b-71b44ee382dd]
2013-01-22 23:13:14 DEBUG nova.compute.manager
[req-16f199b5-f891-48b6-9020-9b8149535b9a 9ac7dbe62fb349ce84927598a0f54d68
b439758af3814f739fe7ab09b98a0120] [instance:
8110c50c-ed26-48dd-886b-71b44ee382dd] Deallocating network for instance
from (pid=2686) _deallocate_network
/usr/lib/python2.7/dist-packages/nova/compute/manager.py:769
2013-01-22 23:13:14 DEBUG nova.network.quantumv2.api
[req-16f199b5-f891-48b6-9020-9b8149535b9a 9ac7dbe62fb349ce84927598a0f54d68
b439758af3814f739fe7ab09b98a0120] deallocate_for_instance() for VM2 from
(pid=2686) deallocate_for_instance
/usr/lib/python2.7/dist-packages/nova/network/quantumv2/api.py:160
2013-01-22 23:13:14 DEBUG nova.compute.manager
[req-16f199b5-f891-48b6-9020-9b8149535b9a 9ac7dbe62fb349ce84927598a0f54d68
b439758af3814f739fe7ab09b98a0120] [instance:
8110c50c-ed26-48dd-886b-71b44ee382dd] Retry info not present, will not
reschedule from (pid=2686) _reschedule
/usr/lib/python2.7/dist-packages/nova/compute/manager.py:564
2013-01-22 23:13:14 DEBUG nova.utils
[req-16f199b5-f891-48b6-9020-9b8149535b9a 9ac7dbe62fb349ce84927598a0f54d68
b439758af3814f739fe7ab09b98a0120] Got semaphore "compute_resources" for
method "update_usage"... from (pid=2686) inner
/usr/lib/python2.7/dist-packages/nova/utils.py:713
2013-01-22 23:13:14 DEBUG nova.utils
[req-16f199b5-f891-48b6-9020-9b8149535b9a 9ac7dbe62fb349ce84927598a0f54d68
b439758af3814f739fe7ab09b98a0120] Got semaphore "compute_resources" for
method "update_usage"... from (pid=2686) inner
/usr/lib/python2.7/dist-packages/nova/utils.py:713
2013-01-22 23:13:14 ERROR nova.openstack.common.rpc.amqp [-] Exception
during message handling
2013-01-22 23:13:14 TRACE nova.openstack.common.rpc.amqp Traceback (most
recent call last):
2013-01-22 23:13:14 TRACE nova.openstack.common.rpc.amqp   File
"/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/amqp.py", line
275, in _process_data
2013-01-22 23:13:14 TRACE nova.openstack.common.rpc.amqp     rval =
self.proxy.dispatch(ctxt, version, method, **args)
2013-01-22 23:13:14 TRACE nova.openstack.common.rpc.amqp   File
"/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/dispatcher.py",
line 145, in dispatch
2013-01-22 23:13:14 TRACE nova.openstack.common.rpc.amqp     return
getattr(proxyobj, method)(ctxt, **kwargs)
2013-01-22 23:13:14 TRACE nova.openstack.common.rpc.amqp   File
"/usr/lib/python2.7/dist-packages/nova/exception.py", line 117, in wrapped
2013-01-22 23:13:14 TRACE nova.openstack.common.rpc.amqp     temp_level,
payload)
2013-01-22 23:13:14 TRACE nova.openstack.common.rpc.amqp   File
"/usr/lib/python2.7/contextlib.py", line 24, in __exit__
2013-01-22 23:13:14 TRACE nova.openstack.common.rpc.amqp     self.gen.next()
2013-01-22 23:13:14 TRACE nova.openstack.common.rpc.amqp   File
"/usr/lib/python2.7/dist-packages/nova/exception.py", line 92, in wrapped
2013-01-22 23:13:14 TRACE nova.openstack.common.rpc.amqp     return
f(*args, **kw)
2013-01-22 23:13:14 TRACE nova.openstack.common.rpc.amqp   File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 181, in
decorated_function
2013-01-22 23:13:14 TRACE nova.openstack.common.rpc.amqp     pass
2013-01-22 23:13:14 TRACE nova.openstack.common.rpc.amqp   File
"/usr/lib/python2.7/contextlib.py", line 24, in __exit__
2013-01-22 23:13:14 TRACE nova.openstack.common.rpc.amqp     self.gen.next()
2013-01-22 23:13:14 TRACE nova.openstack.common.rpc.amqp   File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 167, in
decorated_function
2013-01-22 23:13:14 TRACE nova.openstack.common.rpc.amqp     return
function(self, context, *args, **kwargs)
2013-01-22 23:13:14 TRACE nova.openstack.common.rpc.amqp   File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 202, in
decorated_function
2013-01-22 23:13:14 TRACE nova.openstack.common.rpc.amqp
kwargs['instance']['uuid'], e, sys.exc_info())
2013-01-22 23:13:14 TRACE nova.openstack.common.rpc.amqp   File
"/usr/lib/python2.7/contextlib.py", line 24, in __exit__
2013-01-22 23:13:14 TRACE nova.openstack.common.rpc.amqp     self.gen.next()
2013-01-22 23:13:14 TRACE nova.openstack.common.rpc.amqp   File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 196, in
decorated_function
2013-01-22 23:13:14 TRACE nova.openstack.common.rpc.amqp     return
function(self, context, *args, **kwargs)
2013-01-22 23:13:14 TRACE nova.openstack.common.rpc.amqp   File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 824, in
run_instance
2013-01-22 23:13:14 TRACE nova.openstack.common.rpc.amqp
do_run_instance()
2013-01-22 23:13:14 TRACE nova.openstack.common.rpc.amqp   File
"/usr/lib/python2.7/dist-packages/nova/utils.py", line 752, in inner
2013-01-22 23:13:14 TRACE nova.openstack.common.rpc.amqp     retval =
f(*args, **kwargs)
2013-01-22 23:13:14 TRACE nova.openstack.common.rpc.amqp   File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 823, in
do_run_instance
2013-01-22 23:13:14 TRACE nova.openstack.common.rpc.amqp
admin_password, is_first_time, instance)
2013-01-22 23:13:14 TRACE nova.openstack.common.rpc.amqp   File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 516, in
_run_instance
2013-01-22 23:13:14 TRACE nova.openstack.common.rpc.amqp
self._set_instance_error_state(context, instance['uuid'])
2013-01-22 23:13:14 TRACE nova.openstack.common.rpc.amqp   File
"/usr/lib/python2.7/contextlib.py", line 24, in __exit__
2013-01-22 23:13:14 TRACE nova.openstack.common.rpc.amqp     self.gen.next()
2013-01-22 23:13:14 TRACE nova.openstack.common.rpc.amqp   File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 504, in
_run_instance
2013-01-22 23:13:14 TRACE nova.openstack.common.rpc.amqp     is_first_time,
request_spec, filter_properties)
2013-01-22 23:13:14 TRACE nova.openstack.common.rpc.amqp   File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 496, in
_run_instance
2013-01-22 23:13:14 TRACE nova.openstack.common.rpc.amqp
injected_files, admin_password)
2013-01-22 23:13:14 TRACE nova.openstack.common.rpc.amqp   File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 743, in
_spawn
2013-01-22 23:13:14 TRACE nova.openstack.common.rpc.amqp
block_device_info)
2013-01-22 23:13:14 TRACE nova.openstack.common.rpc.amqp   File
"/usr/lib/python2.7/dist-packages/nova/virt/vmwareapi/driver.py", line 132,
in spawn
2013-01-22 23:13:14 TRACE nova.openstack.common.rpc.amqp
self._vmops.spawn(context, instance, image_meta, network_info)
2013-01-22 23:13:14 TRACE nova.openstack.common.rpc.amqp   File
"/usr/lib/python2.7/dist-packages/nova/virt/vmwareapi/vmops.py", line 187,
in spawn
2013-01-22 23:13:14 TRACE nova.openstack.common.rpc.amqp     vif_infos =
_get_vif_infos()
2013-01-22 23:13:14 TRACE nova.openstack.common.rpc.amqp   File
"/usr/lib/python2.7/dist-packages/nova/virt/vmwareapi/vmops.py", line 180,
in _get_vif_infos
2013-01-22 23:13:14 TRACE nova.openstack.common.rpc.amqp     network_ref =
_check_if_network_bridge_exists(network_name)
2013-01-22 23:13:14 TRACE nova.openstack.common.rpc.amqp   File
"/usr/lib/python2.7/dist-packages/nova/virt/vmwareapi/vmops.py", line 168,
in _check_if_network_bridge_exists
2013-01-22 23:13:14 TRACE nova.openstack.common.rpc.amqp     raise
exception.NetworkNotFoundForBridge(bridge=network_name)
2013-01-22 23:13:14 TRACE nova.openstack.common.rpc.amqp
NetworkNotFoundForBridge: Network could not be found for bridge
2013-01-22 23:13:14 TRACE nova.openstack.common.rpc.amqp

Any help will be greatly appreciated.

Thanks,
Balu

Follow ups