openstack team mailing list archive
-
openstack team
-
Mailing list archive
-
Message #24580
Re: Issues with openvswitch agent on Quantum/network node
Thanks much Balu.
For me currently even the GRE tunnels are amiss from the br-tun. So would just add-port be ok to add them back?
Do you mind sharing the guide that you may have followed to setup your multi node Openstack setup?
(is it using VLAN mode?)
Thanks,
Best Regards,
Rekha
-----Original Message-----
From: Balamurugan V G [mailto:balamuruganvg@xxxxxxxxx]
Sent: Thursday, June 20, 2013 6:50 PM
To: Sreerekha Shenoy (sresheno)
Cc: openstack@xxxxxxxxxxxxxxxxxxx
Subject: Re: [Openstack] Issues with openvswitch agent on Quantum/network node
Run a ping in your VM to its network gateway and do the following:
root@openstack-network:~# quantum router-list
+--------------------------------------+----------------------+--------------------------------------------------------+
| id | name |
external_gateway_info |
+--------------------------------------+----------------------+--------------------------------------------------------+
| 2abf6d6d-b2ac-4fad-99e5-ebf96e1ce167 | auto_router_1 |
{"network_id": "d7d40dba-6d14-4ba5-89dd-972fde4361ab"} |
+--------------------------------------+----------------------+--------------------------------------------------------+
root@openstack-network:~# ip netns exec
qrouter-2abf6d6d-b2ac-4fad-99e5-ebf96e1ce167 route -n Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 1.1.1.1 0.0.0.0 UG 0 0 0
qg-ec4cd5f1-e6
1.1.1.0 0.0.0.0 255.255.255.0 U 0 0 0
qg-ec4cd5f1-e6
2.2.2.0 0.0.0.0 255.255.255.0 U 0 0 0
qr-272f8aec-37
root@openstack-network:~# ip netns exec
qrouter-2abf6d6d-b2ac-4fad-99e5-ebf96e1ce167 tcpdump -i qr-272f8aec-37
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on qr-272f8aec-37, link-type EN10MB (Ethernet), capture size
65535 bytes
^C18:44:26.406660 IP 2.2.2.4 > 2.2.2.1: ICMP echo request, id 38008, seq 56, length 64
18:44:26.406692 IP 2.2.2.1 > 2.2.2.4: ICMP echo reply, id 38008, seq 56, length 64
18:44:27.406690 IP 2.2.2.4 > 2.2.2.1: ICMP echo request, id 38008, seq 57, length 64
18:44:27.406724 IP 2.2.2.1 > 2.2.2.4: ICMP echo reply, id 38008, seq 57, length 64
18:44:28.406695 IP 2.2.2.4 > 2.2.2.1: ICMP echo request, id 38008, seq 58, length 64
18:44:28.406721 IP 2.2.2.1 > 2.2.2.4: ICMP echo reply, id 38008, seq 58, length 64
18:44:29.406683 IP 2.2.2.4 > 2.2.2.1: ICMP echo request, id 38008, seq 59, length 64
18:44:29.406716 IP 2.2.2.1 > 2.2.2.4: ICMP echo reply, id 38008, seq 59, length 64
18:44:30.406668 IP 2.2.2.4 > 2.2.2.1: ICMP echo request, id 38008, seq 60, length 64
18:44:30.406690 IP 2.2.2.1 > 2.2.2.4: ICMP echo reply, id 38008, seq 60, length 64
18:44:31.406666 IP 2.2.2.4 > 2.2.2.1: ICMP echo request, id 38008, seq 61, length 64
18:44:31.406701 IP 2.2.2.1 > 2.2.2.4: ICMP echo reply, id 38008, seq 61, length 64
18:44:32.406688 IP 2.2.2.4 > 2.2.2.1: ICMP echo request, id 38008, seq 62, length 64
18:44:32.406709 IP 2.2.2.1 > 2.2.2.4: ICMP echo reply, id 38008, seq 62, length 64
18:44:32.461101 IP 2.2.2.4.627 > 10.203.15.14.31995: UDP, length 100
18:44:33.406671 IP 2.2.2.4 > 2.2.2.1: ICMP echo request, id 38008, seq 63, length 64
18:44:33.406704 IP 2.2.2.1 > 2.2.2.4: ICMP echo reply, id 38008, seq 63, length 64
18:44:34.406666 IP 2.2.2.4 > 2.2.2.1: ICMP echo request, id 38008, seq 64, length 64
18:44:34.406692 IP 2.2.2.1 > 2.2.2.4: ICMP echo reply, id 38008, seq 64, length 64
18:44:35.406674 IP 2.2.2.4 > 2.2.2.1: ICMP echo request, id 38008, seq 65, length 64
18:44:35.406708 IP 2.2.2.1 > 2.2.2.4: ICMP echo reply, id 38008, seq 65, length 64
18:44:36.406687 IP 2.2.2.4 > 2.2.2.1: ICMP echo request, id 38008, seq 66, length 64
18:44:36.406719 IP 2.2.2.1 > 2.2.2.4: ICMP echo reply, id 38008, seq 66, length 64
18:44:37.406672 IP 2.2.2.4 > 2.2.2.1: ICMP echo request, id 38008, seq 67, length 64
18:44:37.406698 IP 2.2.2.1 > 2.2.2.4: ICMP echo reply, id 38008, seq 67, length 64
18:44:38.406669 IP 2.2.2.4 > 2.2.2.1: ICMP echo request, id 38008, seq 68, length 64
18:44:38.406695 IP 2.2.2.1 > 2.2.2.4: ICMP echo reply, id 38008, seq 68, length 64
18:44:39.406712 IP 2.2.2.4 > 2.2.2.1: ICMP echo request, id 38008, seq 69, length 64
18:44:39.406747 IP 2.2.2.1 > 2.2.2.4: ICMP echo reply, id 38008, seq 69, length 64
29 packets captured
29 packets received by filter
0 packets dropped by kernel
root@openstack-network:~#
2.2.2.0/24 is my VM's network.
You should see that the packets are reaching the the network node from the compute node where your VM is running. This might help to narrow down.
Hope this helps. I do not know of any tools as such except general linux commands to debug the issues.
Regards,
Balu
On Thu, Jun 20, 2013 at 6:36 PM, Sreerekha Shenoy (sresheno) <sresheno@xxxxxxxxx> wrote:
> Hi,
>
> It's been a week since we are stuck in getting this setup working
>
> Any pointers on tools / document to move further would be really
> appreciated.
>
>
>
> (currently we are lagging in getting our hands onto tools that can
> help tracing the packets inside OVS to and from VMs..)
>
>
>
> Thanks,
>
> Best Regards,
>
> Rekha
>
>
>
> From: Sreerekha Shenoy (sresheno)
> Sent: Thursday, June 20, 2013 6:19 PM
> To: openstack@xxxxxxxxxxxxxxxxxxx
> Subject: RE: Issues with openvswitch agent on Quantum/network node
>
>
>
> Hi,
>
> What are the good to know tools and networking concepts for me to be
> able to debug connectivity issues inside Openstack VLAN mode of OVS?
>
>
>
> Thanks,
>
> Best Regards,
>
> Rekha
>
>
>
> From: Sreerekha Shenoy (sresheno)
> Sent: Friday, June 14, 2013 2:23 PM
> To: openstack@xxxxxxxxxxxxxxxxxxx
> Subject: Issues with openvswitch agent on Quantum/network node
>
>
>
> Hi,
>
> I have a multi node grizzly setup using the following guide:
>
> https://github.com/mseknibilel/OpenStack-Grizzly-Install-Guide/blob/OV
> S_MultiNode/OpenStack_Grizzly_Install_Guide.rst
>
> My network topology is as attached and is using GRE tunnel based VLAN.
>
>
>
> I have VMs spawning on compute for cirros images.
>
> However, inter-VM ping works (e.g. 50.50.1.4 can ping 50.50.1.5), but
> the following DON'T work:
>
> a) VM cannot ping 50.50.1.1
>
> b) VM cannot ping Compute ETH1 (data n/w) , 10.20.20.53
>
> c) VM cannot be reachable from IP Namespace either (ip netns exec
> qrouter-xxxx ping 50.50.1.4 does NOT work)
>
>
>
> Apart from the same, my network node has the following issue AFTER a reboot:
>
> · On network node the following agent is not running i.e.
> "quantum-plugin-openvswitch-agent"
>
> · Logs of this agent indicates it needs the plugin upgrade....dont
> have the precise error log..
>
> o ovs-vswitchd.log indicates the following:
>
> § Apr 16 22:08:39|00482|netdev|WARN|Dropped 233 log messages in last
> 12 seconds (most recently, 1 seconds ago) due to excessive rate
>
> § Apr 16 22:08:39|00483|netdev|WARN|failed to get flags for network
> device
> qr-12091c58-8a: No such device
>
> § Apr 16 22:08:43|00484|netdev_linux|WARN|Dropped 35 log messages in
> last
> 15 seconds (most recently, 5 seconds ago) due to excessive rate
>
> · The version of OSK-plugin-ovs-agent is 2013.1.1-0ubuntu1
>
> · The result of the above being that "br-tun DOES NOT have gre-2 /
> gre-1 port on it" for both compute and network..
>
>
>
> Please let me know what could be done to:
>
> 1. get the ovs-agent running on compute
>
> 2. have VMs pingable
>
> ======================================================================
> ==========================================
>
> compute Node output:
>
> root@computenode:~# ovs-vsctl show
>
> 4aad5da2-861d-4190-a50b-37dfc5ca237a
>
> Bridge br-tun
>
> Port patch-int
>
> Interface patch-int
>
> type: patch
>
> options: {peer=patch-tun}
>
> Port br-tun
>
> Interface br-tun
>
> type: internal
>
> Bridge br-int
>
> Port "qvo47a58f32-be"
>
> Interface "qvo47a58f32-be"
>
> Port "qvo19e7cd4c-c0"
>
> tag: 1
>
> Interface "qvo19e7cd4c-c0"
>
> Port "qvobaa01ac1-69"
>
> tag: 1
>
> Interface "qvobaa01ac1-69"
>
> Port patch-tun
>
> Interface patch-tun
>
> type: patch
>
> options: {peer=patch-int}
>
> Port br-int
>
> Interface br-int
>
> type: internal
>
> ovs_version: "1.4.0+build0"
>
>
>
> ======================================================================
> ==========================================
>
>
>
> Network node output:
>
>
>
> root@networknode:~# ovs-vsctl show
>
> 9e17bb8c-d69d-48ab-9333-b22447c0c268
>
> Bridge br-tun
>
> Port patch-int
>
> Interface patch-int
>
> type: patch
>
> options: {peer=patch-tun}
>
> Port br-tun
>
> Interface br-tun
>
> type: internal
>
> Bridge br-int
>
> Port patch-tun
>
> Interface patch-tun
>
> type: patch
>
> options: {peer=patch-int}
>
> Port br-int
>
> Interface br-int
>
> type: internal
>
> Port "qr-a7ab3f7c-8e"
>
> Interface "qr-a7ab3f7c-8e"
>
> type: internal
>
> Port "tap6a322215-f0"
>
> tag: 1
>
> Interface "tap6a322215-f0"
>
> type: internal
>
> Bridge br-ex
>
> Port br-ex
>
> Interface br-ex
>
> type: internal
>
> Port "qg-1d552cbd-a0"
>
> Interface "qg-1d552cbd-a0"
>
> type: internal
>
> Port "eth2"
>
> Interface "eth2"
>
> ovs_version: "1.4.0+build0"
>
>
>
>
>
> Thanks and Regards,
>
> Rashmi S N
>
>
>
>
> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~openstack
> More help : https://help.launchpad.net/ListHelp
>
References
