yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1780407] [NEW] There are some errors in neutron_l3_agent and neutron_dhcp_agent after restarting open vswitch with dpdk

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Nanfei Chen <chennanfei@xxxxxxxx>
Date: Fri, 06 Jul 2018 09:32:01 -0000
Reply-to: Bug 1780407 <1780407@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
Public bug reported:

Environment: all in one.
OpenStack: Queens
Open vSwitch: 2.9.0
DPDK: 17.11.2
QEMU: 2.11
Private network: client-net, 10.10.10.0/24, gateway 10.10.10.1, dhcp server 10.10.10.2.

After restart the Open vSwitch with dpdk, there are some errors in
neutron-l3-agent.log and neutron-dhcp-agent.log. The detailed content is
as follows.

neutron-l3-agent.log:
2018-07-06 16:43:35.212 34 WARNING ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: send error: Connection refused
2018-07-06 16:43:35.214 34 WARNING ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: connection dropped (Connection refused)
2018-07-06 16:43:37.216 34 WARNING ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: send error: Connection refused
2018-07-06 16:43:37.217 34 WARNING ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: connection dropped (Connection refused)

neutron-dhcp-agent.log:
2018-07-06 16:43:35.211 7 WARNING ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: send error: Connection refused
2018-07-06 16:43:35.212 7 WARNING ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: connection dropped (Connection refused)
2018-07-06 16:43:37.215 7 WARNING ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: send error: Connection refused
2018-07-06 16:43:37.216 7 WARNING ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: connection dropped (Connection refused)

In the qrouter namespace, the status of the gateway interface for a private network is down. So the gateway interface can not communicate with the dhcp server interface and virtual machines any more.
root@B-6-7:~# ip netns
qrouter-45a13539-c98c-45e9-aab6-2db6dfd687bf
qdhcp-c9b5cf2d-4df9-47fb-865c-cdb6e7bacd07
root@B-6-7:~#
root@B-6-7:~# ip netns exec qrouter-45a13539-c98c-45e9-aab6-2db6dfd687bf ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
20: qr-ac3740b8-b4: <NO-CARRIER,BROADCAST,PROMISC,UP> mtu 1450 qdisc pfifo_fast state DOWN group default qlen 1000
    link/ether fa:16:3e:88:0b:cf brd ff:ff:ff:ff:ff:ff
    inet 10.10.10.1/24 brd 10.10.10.255 scope global qr-ac3740b8-b4
       valid_lft forever preferred_lft forever
    inet6 fe80::f816:3eff:fe88:bcf/64 scope link
       valid_lft forever preferred_lft forever
root@B-6-7:~#
root@B-6-7:~# ip netns exec qrouter-45a13539-c98c-45e9-aab6-2db6dfd687bf ping 10.10.10.2
PING 10.10.10.2 (10.10.10.2) 56(84) bytes of data.
>From 10.10.10.1 icmp_seq=1 Destination Host Unreachable
>From 10.10.10.1 icmp_seq=2 Destination Host Unreachable
>From 10.10.10.1 icmp_seq=3 Destination Host Unreachable
^C
--- 10.10.10.2 ping statistics ---
6 packets transmitted, 0 received, +3 errors, 100% packet loss, time 4999ms
pipe 4

As the same, in the qdhcp namespace, the status of the dhcp server
interface is down too. The dhcp server can not work any more.

** Affects: neutron
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1780407

Title:
  There are some errors in neutron_l3_agent and neutron_dhcp_agent after
  restarting open vswitch with dpdk

Status in neutron:
  New

Bug description:
  Environment: all in one.
  OpenStack: Queens
  Open vSwitch: 2.9.0
  DPDK: 17.11.2
  QEMU: 2.11
  Private network: client-net, 10.10.10.0/24, gateway 10.10.10.1, dhcp server 10.10.10.2.

  After restart the Open vSwitch with dpdk, there are some errors in
  neutron-l3-agent.log and neutron-dhcp-agent.log. The detailed content
  is as follows.

  neutron-l3-agent.log:
  2018-07-06 16:43:35.212 34 WARNING ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: send error: Connection refused
  2018-07-06 16:43:35.214 34 WARNING ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: connection dropped (Connection refused)
  2018-07-06 16:43:37.216 34 WARNING ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: send error: Connection refused
  2018-07-06 16:43:37.217 34 WARNING ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: connection dropped (Connection refused)

  neutron-dhcp-agent.log:
  2018-07-06 16:43:35.211 7 WARNING ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: send error: Connection refused
  2018-07-06 16:43:35.212 7 WARNING ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: connection dropped (Connection refused)
  2018-07-06 16:43:37.215 7 WARNING ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: send error: Connection refused
  2018-07-06 16:43:37.216 7 WARNING ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: connection dropped (Connection refused)

  In the qrouter namespace, the status of the gateway interface for a private network is down. So the gateway interface can not communicate with the dhcp server interface and virtual machines any more.
  root@B-6-7:~# ip netns
  qrouter-45a13539-c98c-45e9-aab6-2db6dfd687bf
  qdhcp-c9b5cf2d-4df9-47fb-865c-cdb6e7bacd07
  root@B-6-7:~#
  root@B-6-7:~# ip netns exec qrouter-45a13539-c98c-45e9-aab6-2db6dfd687bf ip a
  1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1
      link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
      inet 127.0.0.1/8 scope host lo
         valid_lft forever preferred_lft forever
      inet6 ::1/128 scope host
         valid_lft forever preferred_lft forever
  20: qr-ac3740b8-b4: <NO-CARRIER,BROADCAST,PROMISC,UP> mtu 1450 qdisc pfifo_fast state DOWN group default qlen 1000
      link/ether fa:16:3e:88:0b:cf brd ff:ff:ff:ff:ff:ff
      inet 10.10.10.1/24 brd 10.10.10.255 scope global qr-ac3740b8-b4
         valid_lft forever preferred_lft forever
      inet6 fe80::f816:3eff:fe88:bcf/64 scope link
         valid_lft forever preferred_lft forever
  root@B-6-7:~#
  root@B-6-7:~# ip netns exec qrouter-45a13539-c98c-45e9-aab6-2db6dfd687bf ping 10.10.10.2
  PING 10.10.10.2 (10.10.10.2) 56(84) bytes of data.
  From 10.10.10.1 icmp_seq=1 Destination Host Unreachable
  From 10.10.10.1 icmp_seq=2 Destination Host Unreachable
  From 10.10.10.1 icmp_seq=3 Destination Host Unreachable
  ^C
  --- 10.10.10.2 ping statistics ---
  6 packets transmitted, 0 received, +3 errors, 100% packet loss, time 4999ms
  pipe 4

  As the same, in the qdhcp namespace, the status of the dhcp server
  interface is down too. The dhcp server can not work any more.

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1780407/+subscriptions
Follow ups

[Bug 1780407] Re: There are some errors in neutron_l3_agent and neutron_dhcp_agent after restarting open vswitch with dpdk
From: Launchpad Bug Tracker, 2018-09-28