← Back to team overview

yahoo-eng-team team mailing list archive

[Bug 1494866] Re: router allocation doesn't match the record in neutron db when l3_ha is true

 

Assaf, this is a legitimate bug. L3 HA should be updating the port
binding details to correctly reflect where the port is being used. The
OVS mech driver just happens to work with an incorrect host_id.

** Changed in: neutron
       Status: Invalid => Confirmed

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1494866

Title:
  router allocation doesn't match the record in neutron db when l3_ha is
  true

Status in neutron:
  Confirmed

Bug description:
  We are using kilo. In our setup, we have 3 neutron controllers, l3
  agents are running on all the 3 neutron controllers. We make l3_ha =
  true in all the 3 neutron.conf.

  We notice that when we attach a network to a router, the gateway
  namespace is allocated to a controller node which doesn't match the
  record in neutron db. Following is one example.

  Create a router, a network, attach the network to the router.

  1. neutron tells that the gateway ip 1.1.1.1 is at controller-1
  [stack@c5220-01 ~]$ neutron port-show 3306c360-5a3d-4a08-aa92-017498758963
  +-----------------------+--------------------------------------------------------------------------------+
  | Field                 | Value                                                                          |
  +-----------------------+--------------------------------------------------------------------------------+
  | admin_state_up        | True                                                                           |
  | allowed_address_pairs |                                                                                |
  | binding:host_id       | overcloud-controller-1.localdomain                                             |
  | binding:profile       | {}                                                                             |
  | binding:vif_details   | {"port_filter": true, "ovs_hybrid_plug": true}                                 |
  | binding:vif_type      | ovs                                                                            |
  | binding:vnic_type     | normal                                                                         |
  | device_id             | 934f0b90-2d98-4d54-b9ca-5222aac2199d                                           |
  | device_owner          | network:router_interface                                                       |
  | extra_dhcp_opts       |                                                                                |
  | fixed_ips             | {"subnet_id": "463c2f0c-5d56-4abb-8b30-8450d8306f46", "ip_address": "1.1.1.1"} |
  | id                    | 3306c360-5a3d-4a08-aa92-017498758963                                           |
  | mac_address           | fa:16:3e:72:34:4c                                                              |
  | name                  |                                                                                |
  | network_id            | 98f125b6-6d4d-4417-a0b3-e8d9ff530d6f                                           |
  | security_groups       |                                                                                |
  | status                | ACTIVE                                                                         |
  | tenant_id             | 4ef11838925940eb9d177ae9345711ee                                               |
  +-----------------------+--------------------------------------------------------------------------------+

  
  2. However, the gateway ip is at controller-2
  [heat-admin@overcloud-controller-2 ~]$ sudo ip netns exec qrouter-934f0b90-2d98-4d54-b9ca-5222aac2199d ifconfig
  ha-6d47f13a-b7: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
          inet 169.254.192.6  netmask 255.255.192.0  broadcast 169.254.255.255
          inet6 fe80::f816:3eff:fe43:9b80  prefixlen 64  scopeid 0x20<link>
          ether fa:16:3e:43:9b:80  txqueuelen 1000  (Ethernet)
          RX packets 20  bytes 1638 (1.5 KiB)
          RX errors 0  dropped 0  overruns 0  frame 0
          TX packets 309  bytes 16926 (16.5 KiB)
          TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

  lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
          inet 127.0.0.1  netmask 255.0.0.0
          inet6 ::1  prefixlen 128  scopeid 0x10<host>
          loop  txqueuelen 0  (Local Loopback)
          RX packets 0  bytes 0 (0.0 B)
          RX errors 0  dropped 0  overruns 0  frame 0
          TX packets 0  bytes 0 (0.0 B)
          TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

  qg-22431202-eb: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
          inet 10.8.87.25  netmask 255.255.255.0  broadcast 0.0.0.0
          inet6 fe80::f816:3eff:febd:56ad  prefixlen 64  scopeid 0x20<link>
          ether fa:16:3e:bd:56:ad  txqueuelen 1000  (Ethernet)
          RX packets 36  bytes 2746 (2.6 KiB)
          RX errors 0  dropped 0  overruns 0  frame 0
          TX packets 43  bytes 2890 (2.8 KiB)
          TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

  qr-3306c360-5a: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
          inet 1.1.1.1  netmask 255.255.255.0  broadcast 0.0.0.0
          inet6 fe80::f816:3eff:fe72:344c  prefixlen 64  scopeid 0x20<link>
          ether fa:16:3e:72:34:4c  txqueuelen 1000  (Ethernet)
          RX packets 95  bytes 5856 (5.7 KiB)
          RX errors 0  dropped 0  overruns 0  frame 0
          TX packets 90  bytes 4200 (4.1 KiB)
          TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

  
  3. On controller-1, there is no such ip
  [heat-admin@overcloud-controller-1 ~]$ sudo ip netns exec qrouter-934f0b90-2d98-4d54-b9ca-5222aac2199d ifconfig
  ha-7ff9abd2-bd: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
          inet 169.254.192.5  netmask 255.255.192.0  broadcast 169.254.255.255
          inet6 fe80::f816:3eff:fe9d:275c  prefixlen 64  scopeid 0x20<link>
          ether fa:16:3e:9d:27:5c  txqueuelen 1000  (Ethernet)
          RX packets 321  bytes 19678 (19.2 KiB)
          RX errors 0  dropped 0  overruns 0  frame 0
          TX packets 12  bytes 1008 (1008.0 B)
          TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

  lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
          inet 127.0.0.1  netmask 255.0.0.0
          inet6 ::1  prefixlen 128  scopeid 0x10<host>
          loop  txqueuelen 0  (Local Loopback)
          RX packets 0  bytes 0 (0.0 B)
          RX errors 0  dropped 0  overruns 0  frame 0
          TX packets 0  bytes 0 (0.0 B)
          TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

  qg-22431202-eb: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
          ether fa:16:3e:bd:56:ad  txqueuelen 1000  (Ethernet)
          RX packets 42  bytes 3360 (3.2 KiB)
          RX errors 0  dropped 0  overruns 0  frame 0
          TX packets 1  bytes 110 (110.0 B)
          TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

  qr-3306c360-5a: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
          ether fa:16:3e:72:34:4c  txqueuelen 1000  (Ethernet)
          RX packets 105  bytes 6456 (6.3 KiB)
          RX errors 0  dropped 0  overruns 0  frame 0
          TX packets 1  bytes 110 (110.0 B)
          TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

  
  4. On controller-0, there is no such ip
  [heat-admin@overcloud-controller-0 ~]$ sudo ip netns exec qrouter-934f0b90-2d98-4d54-b9ca-5222aac2199d ifconfig
  ha-8dccf24a-2e: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
          inet 169.254.192.4  netmask 255.255.192.0  broadcast 169.254.255.255
          inet6 fe80::f816:3eff:fe98:83dd  prefixlen 64  scopeid 0x20<link>
          ether fa:16:3e:98:83:dd  txqueuelen 1000  (Ethernet)
          RX packets 1140  bytes 68618 (67.0 KiB)
          RX errors 0  dropped 0  overruns 0  frame 0
          TX packets 12  bytes 1008 (1008.0 B)
          TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

  lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
          inet 127.0.0.1  netmask 255.0.0.0
          inet6 ::1  prefixlen 128  scopeid 0x10<host>
          loop  txqueuelen 0  (Local Loopback)
          RX packets 0  bytes 0 (0.0 B)
          RX errors 0  dropped 0  overruns 0  frame 0
          TX packets 0  bytes 0 (0.0 B)
          TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

  qg-22431202-eb: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
          ether fa:16:3e:bd:56:ad  txqueuelen 1000  (Ethernet)
          RX packets 42  bytes 3244 (3.1 KiB)
          RX errors 0  dropped 0  overruns 0  frame 0
          TX packets 1  bytes 110 (110.0 B)
          TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

  qr-3306c360-5a: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
          ether fa:16:3e:72:34:4c  txqueuelen 1000  (Ethernet)
          RX packets 1753  bytes 105336 (102.8 KiB)
          RX errors 0  dropped 0  overruns 0  frame 0
          TX packets 1  bytes 110 (110.0 B)
          TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1494866/+subscriptions


References