openstack team mailing list archive
-
openstack team
-
Mailing list archive
-
Message #04817
Re: Duplicate ICMP due to public interface bridge being placed in promiscus mode
Vish,
We are not sure if this particular issue may cause any problem, but just
wanted to understand what is wrong.
To provide a bit more data about this particular environment:
- Dedicated servers config at RAX
- FlatDHCP mode with primary NIC (eth0) bridged and used for
fixed_range as well
- Secondary nic (eth1) used for RabbitMQ/Glance/etc
- Nova-network running on one node only
If we disable promiscuous mode, everything works fine – no DUPs. But in this
case VMs running on other nodes are unable to go outside (what is an
expected behavior in this case).
Here are some details of this config:
Nova.conf:
--s3_host=10.240.107.128
--rabbit_host=10.240.107.128
--ec2_host=10.240.107.128
--glance_api_servers=10.240.107.128:9292
--sql_connection=mysql://root:123@10.240.107.128/nova
--routing_source_ip=172.31.252.152
--fixed_range=172.31.252.0/22
--network_manager=nova.network.manager.FlatDHCPManager
--public_interface=br100
--multi_host=False
…
root@web1:/# ip addr show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet 169.254.169.254/32 scope link lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen
1000
link/ether 84:2b:2b:5a:49:a0 brd ff:ff:ff:ff:ff:ff
inet6 fe80::862b:2bff:fe5a:49a0/64 scope link
valid_lft forever preferred_lft forever
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen
1000
link/ether 84:2b:2b:5a:49:a2 brd ff:ff:ff:ff:ff:ff
inet 10.240.107.128/24 brd 10.240.107.255 scope global eth1
inet6 fe80::862b:2bff:fe5a:49a2/64 scope link
valid_lft forever preferred_lft forever
4: eth2: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000
link/ether 84:2b:2b:5a:49:a4 brd ff:ff:ff:ff:ff:ff
5: eth3: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000
link/ether 84:2b:2b:5a:49:a6 brd ff:ff:ff:ff:ff:ff
6: virbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state
UNKNOWN
link/ether 02:bb:dd:bd:e6:ed brd ff:ff:ff:ff:ff:ff
inet 192.168.122.1/24 brd 192.168.122.255 scope global virbr0
*8: br100: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc noqueue
state UNKNOWN *
* link/ether 84:2b:2b:5a:49:a0 brd ff:ff:ff:ff:ff:ff*
* inet 172.31.252.152/22 brd 172.31.255.255 scope global br100*
* inet 172.31.252.1/22 brd 172.31.255.255 scope global secondary br100*
* inet6 fe80::90db:ceff:fe33:450c/64 scope link *
* valid_lft forever preferred_lft forever*
9: vnet0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state
UNKNOWN qlen 500
link/ether fe:16:3e:0e:11:6e brd ff:ff:ff:ff:ff:ff
inet6 fe80::fc16:3eff:fe0e:116e/64 scope link
valid_lft forever preferred_lft forever
root@web1:/# brctl show
bridge name bridge id STP enabled interfaces
br100 8000.842b2b5a49a0 no eth0
vnet0
virbr0 8000.000000000000 yes
root@web1:/# ifconfig -a
br100 Link encap:Ethernet HWaddr 84:2b:2b:5a:49:a0
inet addr:172.31.252.152 Bcast:172.31.255.255 Mask:255.255.252.0
inet6 addr: fe80::90db:ceff:fe33:450c/64 Scope:Link
UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1
RX packets:6909 errors:0 dropped:621 overruns:0 frame:0
TX packets:2634 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:533738 (533.7 KB) TX bytes:419886 (419.8 KB)
eth0 Link encap:Ethernet HWaddr 84:2b:2b:5a:49:a0
inet6 addr: fe80::862b:2bff:fe5a:49a0/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:6705 errors:0 dropped:0 overruns:0 frame:0
TX packets:2667 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:598182 (598.1 KB) TX bytes:447933 (447.9 KB)
Interrupt:36 Memory:d6000000-d6012800
eth1 Link encap:Ethernet HWaddr 84:2b:2b:5a:49:a2
inet addr:10.240.107.128 Bcast:10.240.107.255 Mask:255.255.255.0
inet6 addr: fe80::862b:2bff:fe5a:49a2/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:557 errors:0 dropped:0 overruns:0 frame:0
TX packets:491 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:214973 (214.9 KB) TX bytes:267663 (267.6 KB)
Interrupt:48 Memory:d8000000-d8012800
eth2 Link encap:Ethernet HWaddr 84:2b:2b:5a:49:a4
BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
Interrupt:32 Memory:da000000-da012800
eth3 Link encap:Ethernet HWaddr 84:2b:2b:5a:49:a6
BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
Interrupt:42 Memory:dc000000-dc012800
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:62789 errors:0 dropped:0 overruns:0 frame:0
TX packets:62789 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:46351343 (46.3 MB) TX bytes:46351343 (46.3 MB)
virbr0 Link encap:Ethernet HWaddr 02:bb:dd:bd:e6:ed
inet addr:192.168.122.1 Bcast:192.168.122.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
vnet0 Link encap:Ethernet HWaddr fe:16:3e:0e:11:6e
inet6 addr: fe80::fc16:3eff:fe0e:116e/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:646 errors:0 dropped:0 overruns:0 frame:0
TX packets:3970 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:500
RX bytes:119572 (119.5 KB) TX bytes:317664 (317.6 KB)
Thanks,
-Vladimir
*From:* openstack-bounces+vladimir=zadarastorage.com@xxxxxxxxxxxxxxxxxxx[mailto:
openstack-bounces+vladimir=zadarastorage.com@xxxxxxxxxxxxxxxxxxx] *On Behalf
Of *Vishvananda Ishaya
*Sent:* Friday, October 14, 2011 9:02 AM
*To:* Shyam Kaushik
*Cc:* openstack@xxxxxxxxxxxxxxxxxxx
*Subject:* Re: [Openstack] Duplicate ICMP due to public interface bridge
being placed in promiscus mode
Strange, I haven't seen this. It sounds like maybe you have a mac address
conflict somehow. The only time I've seen duplicate icmp responses is when
I was running inside of virtualbox and due to the way vagrant was setting up
vms, I had multiple vms with same mac.
Vish
On Oct 14, 2011, at 5:59 AM, Shyam Kaushik wrote:
*Hi Vish,*
In our openstack deployment we observe this:
Since linux_net.py/initialize_gateway_device() does this
# NOTE(vish): If the public interface is the same as the
# bridge, then the bridge has to be in promiscuous
# to forward packets properly.
if(FLAGS.public_interface == dev):
_execute('ip', 'link', 'set',
'dev', dev, 'promisc', 'on', run_as_root=True)
Any VM spawned on the cloud controller node if it sends an ICMP ping to an
external network gets duplicate replies (i.e. there are 2 replies for the
same ICMP request). For VM’s spawned on any other non-cloud controller this
doesn’t happen.
If we turn of promiscus mode on the bridge, the VM on cloud controller
doesn’t see the duplicate replies, but VM’s on non-cloud controller cannot
reach external network.
Question to you is, is this duplicate ICMP replies expected for VM’s running
on cloud controller due to above logic?
--Shyam
_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to : openstack@xxxxxxxxxxxxxxxxxxx
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
References
