openstack team mailing list archive

[Lillie Ross-CDSR11 <Ross.Lillie@xxxxxxxxxxxxxxxxxxxxx>]

I'm setting up a modest cloud for internal use.  Currently I have the following configuration:

Server1 (nova) is running all nova services EXCEPT nova compute.
Server 2 (nova1) is running only nova-compute.
Servers3-12 are idle, but will eventually run nova-compute

Server 1 is configured with public_interface eth0 and vlan_interface eth1.  The nova.conf file is as follows

--dhcpbridge_flagfile=/etc/nova/nova.conf
--dhcpbridge=/usr/bin/nova-dhcpbridge
--logdir=/var/log/nova
--state_path=/var/lib/nova
--lock_path=/var/lock/nova
--verbose
--s3_host=173.23.181.1
--rabbit_host=173.23.181.1
--cc_host=173.23.181.1
--ec2_url=http://173.23.181.1:8773/services/Cloud
--nova_url=http://173.23.181.1:8774/v1.1/
--fixed_range=192.168.64.0/18
--network_size=8
--FAKE_subdomain=cloud
--routing_source_ip=173.23.181.1
--sql_connection=mysql://nova:nova@173.23.181.1/nova
--glance_api_servers=192.168.10.1:9292
--public_interface=eth0
--vlan_interface=eth1
--use_deprecated_auth
--iscsi_ip_prefix=192.168.10.

Server 2 is configured with vlan_interface eth3.  The nova.conf file is practically identical to Server1 and is as follows

--dhcpbridge_flagfile=/etc/nova/nova.conf
--dhcpbridge=/usr/bin/nova-dhcpbridge
--logdir=/var/log/nova
--state_path=/var/lib/nova
--lock_path=/var/lock/nova
--verbose
--s3_host=173.23.181.1
--rabbit_host=173.23.181.1
--cc_host=173.23.181.1
--ec2_url=http://173.23.181.1:8773/services/Cloud
--nova_url=http://173.23.181.1:8774/v1.1/
--fixed_range=192.168.64.0/18
--network_size=8
--FAKE_subdomain=cloud
--routing_source_ip=173.23.181.1
--sql_connection=mysql://nova:nova@173.23.181.1/nova
--glance_api_servers=192.168.10.1:9292
--vlan_interface=eth3
--use_deprecated_auth
--iscsi_ip_prefix=192.168.10.

I've setup 1 private network 192.168.64.0/19.

I'm seeing the following behavior and wonder if this is a bug.  When starting an instance, if the bridge interface does not exist on Server2 (br100), the instance fails to start and the nova-compute.log file shows the following error:

2012-01-25 15:59:26,644 ERROR nova.exception [-] Uncaught exception
(nova.exception): TRACE: Traceback (most recent call last):
(nova.exception): TRACE:   File "/usr/lib/python2.7/dist-packages/nova/exception.py", line 98, in w
rapped
(nova.exception): TRACE:     return f(*args, **kw)
(nova.exception): TRACE:   File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/connection.py",
 line 672, in spawn
(nova.exception): TRACE:     block_device_info=block_device_info)
(nova.exception): TRACE:   File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/connection.py",
 line 1272, in to_xml
(nova.exception): TRACE:     block_device_info)
(nova.exception): TRACE:   File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/connection.py",
 line 1170, in _prepare_xml_info
(nova.exception): TRACE:     nics.append(self.vif_driver.plug(instance, network, mapping))
(nova.exception): TRACE:   File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/vif.py", line 8
6, in plug
(nova.exception): TRACE:     network['bridge_interface'])
(nova.exception): TRACE:   File "/usr/lib/python2.7/dist-packages/nova/network/linux_net.py", line
889, in ensure_vlan_bridge
(nova.exception): TRACE:     bridge_interface, mac_address)
(nova.exception): TRACE:   File "/usr/lib/python2.7/dist-packages/nova/utils.py", line 688, in inne
r
(nova.exception): TRACE:     retval = f(*args, **kwargs)
(nova.exception): TRACE:   File "/usr/lib/python2.7/dist-packages/nova/network/linux_net.py", line
909, in ensure_vlan
(nova.exception): TRACE:     _execute('ip', 'link', 'set', interface, 'up', run_as_root=True)
(nova.exception): TRACE:   File "/usr/lib/python2.7/dist-packages/nova/network/linux_net.py", line
745, in _execute
(nova.exception): TRACE:     return utils.execute(*cmd, **kwargs)
(nova.exception): TRACE:   File "/usr/lib/python2.7/dist-packages/nova/utils.py", line 191, in exec
ute
(nova.exception): TRACE:     cmd=' '.join(cmd))
(nova.exception): TRACE: ProcessExecutionError: Unexpected error while running command.
(nova.exception): TRACE: Command: sudo ip link set vlan100 up
(nova.exception): TRACE: Exit code: 2
(nova.exception): TRACE: Stdout: ''
(nova.exception): TRACE: Stderr: 'RTNETLINK answers: Network is down\n'
(nova.exception): TRACE:
2012-01-25 15:59:26,645 ERROR nova.compute.manager [-] Instance '11' failed to spawn. Is virtualiza
tion enabled in the BIOS? Details: Unexpected error while running command.
Command: sudo ip link set vlan100 up
Exit code: 2
Stdout: ''
Stderr: 'RTNETLINK answers: Network is down\n'
(nova.compute.manager): TRACE: Traceback (most recent call last):
(nova.compute.manager): TRACE:   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", l
ine 424, in _run_instance
(nova.compute.manager): TRACE:     network_info, block_device_info)
(nova.compute.manager): TRACE:   File "/usr/lib/python2.7/dist-packages/nova/exception.py", line 12
9, in wrapped
(nova.compute.manager): TRACE:     raise Error(str(e))
(nova.compute.manager): TRACE: Error: Unexpected error while running command.
(nova.compute.manager): TRACE: Command: sudo ip link set vlan100 up
(nova.compute.manager): TRACE: Exit code: 2
(nova.compute.manager): TRACE: Stdout: ''
(nova.compute.manager): TRACE: Stderr: 'RTNETLINK answers: Network is down\n'
(nova.compute.manager): TRACE:

if I kill the pending instance on the controller node and re-issue the command to start and instance, I see the following VLAN configuration (vlan100)

vlan100  VID: 100 REORDER_HDR: 1  dev->priv_flags: 4001
         total frames received            0
          total bytes received            0
      Broadcast/Multicast Rcvd            0

      total frames transmitted            0
       total bytes transmitted            0
Device: eth1
INGRESS priority mappings: 0:0  1:0  2:0  3:0  4:0  5:0  6:0 7:0
 EGRESS priority mappings:

Note that the VLAN is using eth1 when (on the compute node) it should be using eth3.

What am I missing?

Regards,
Ross