yahoo-eng-team team mailing list archive
-
yahoo-eng-team team
-
Mailing list archive
-
Message #38907
[Bug 1499170] [NEW] dnsmasq doesn't listen for DNS queries to its global IPv6 address
Public bug reported:
On Kilo, when setting up an IPv6 subnet with DHCP enabled, the qdhcp
namespace on the l3 agents will acquire global IPv6 addresses from the
subnet prefix. These addresses are also advertised to instances
requesting DNS servers via DHCPv6. However, dnsmasq does not listen for
incoming DNS queries to these addresses (it only listens queries sent to
its link-local IPv4 and IPv6 addresses). The end result is that the
instances are provisioned with a set of DNS servers which simply does
not work, and therefore cannot resolve any hostnames. The console output
below should help explain the issue:
Creating the network/subnet and attaching it to Internet router:
----------------------------------------------------------------
tore@ctrl01:~$ neutron net-create foobarnet
Created a new network:
+-----------------+--------------------------------------+
| Field | Value |
+-----------------+--------------------------------------+
| admin_state_up | True |
| id | a125a5a5-3e16-4f4d-9b77-012c160424f4 |
| mtu | 0 |
| name | foobarnet |
| router:external | False |
| shared | False |
| status | ACTIVE |
| subnets | |
| tenant_id | ebeb8cab76e94d189b7a1803c8bb1752 |
+-----------------+--------------------------------------+
tore@ctrl01:~$ neutron subnet-create --ip-version 6 --ipv6-ra-mode dhcpv6-stateless --ipv6-address-mode dhcpv6-stateless foobarnet 2a02:c0:200:f02f::/64
Created a new subnet:
+-------------------+---------------------------------------------------------------------------------+
| Field | Value |
+-------------------+---------------------------------------------------------------------------------+
| allocation_pools | {"start": "2a02:c0:200:f02f::2", "end": "2a02:c0:200:f02f:ffff:ffff:ffff:fffe"} |
| cidr | 2a02:c0:200:f02f::/64 |
| dns_nameservers | |
| enable_dhcp | True |
| gateway_ip | 2a02:c0:200:f02f::1 |
| host_routes | |
| id | 9f8d4190-5f83-414a-bb0c-0d5c8d31a8e1 |
| ip_version | 6 |
| ipv6_address_mode | dhcpv6-stateless |
| ipv6_ra_mode | dhcpv6-stateless |
| name | |
| network_id | a125a5a5-3e16-4f4d-9b77-012c160424f4 |
| subnetpool_id | |
| tenant_id | ebeb8cab76e94d189b7a1803c8bb1752 |
+-------------------+---------------------------------------------------------------------------------+
tore@ctrl01:~$ neutron router-interface-add 08ec7303-8040-45ca-b391-f5599d265eb9 9f8d4190-5f83-414a-bb0c-0d5c8d31a8e1
Added interface 40544ba9-ddab-417b-9b78-b1e6e80ca644 to router 08ec7303-8040-45ca-b391-f5599d265eb9.
Network node 01 status:
-----------------------
tore@net01:~$ sudo ip netns exec qdhcp-a125a5a5-3e16-4f4d-9b77-012c160424f4 ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
94: tapd21c1dc1-bd: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default
link/ether fa:16:3e:8a:82:5e brd ff:ff:ff:ff:ff:ff
inet 169.254.169.254/16 brd 169.254.255.255 scope global tapd21c1dc1-bd
valid_lft forever preferred_lft forever
inet6 2a02:c0:200:f02f:f816:3eff:fe8a:825e/64 scope global dynamic
valid_lft 86398sec preferred_lft 14398sec
inet6 fe80::f816:3eff:fe8a:825e/64 scope link
valid_lft forever preferred_lft forever
tore@net01:~$ sudo ip netns exec qdhcp-a125a5a5-3e16-4f4d-9b77-012c160424f4 lsof -i :53
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
dnsmasq 12789 nobody 7u IPv4 88487703 0t0 UDP 169.254.169.254:domain
dnsmasq 12789 nobody 8u IPv4 88487704 0t0 TCP 169.254.169.254:domain (LISTEN)
dnsmasq 12789 nobody 12u IPv6 88590343 0t0 UDP [fe80::f816:3eff:fe8a:825e]:domain
dnsmasq 12789 nobody 13u IPv6 88590344 0t0 TCP [fe80::f816:3eff:fe8a:825e]:domain (LISTEN)
Network node 02 status:
-----------------------
tore@net02:~$ sudo ip netns exec qdhcp-a125a5a5-3e16-4f4d-9b77-012c160424f4 ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
94: tap0d726e47-80: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default
link/ether fa:16:3e:7a:29:4d brd ff:ff:ff:ff:ff:ff
inet 169.254.169.254/16 brd 169.254.255.255 scope global tap0d726e47-80
valid_lft forever preferred_lft forever
inet6 2a02:c0:200:f02f:f816:3eff:fe7a:294d/64 scope global dynamic
valid_lft 86400sec preferred_lft 14400sec
inet6 fe80::f816:3eff:fe7a:294d/64 scope link
valid_lft forever preferred_lft forever
tore@net02:~$ sudo ip netns exec qdhcp-a125a5a5-3e16-4f4d-9b77-012c160424f4 lsof -i :53
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
dnsmasq 17217 nobody 7u IPv4 65048317 0t0 UDP 169.254.169.254:domain
dnsmasq 17217 nobody 8u IPv4 65048318 0t0 TCP 169.254.169.254:domain (LISTEN)
dnsmasq 17217 nobody 12u IPv6 65047373 0t0 UDP [fe80::f816:3eff:fe7a:294d]:domain
dnsmasq 17217 nobody 13u IPv6 65047374 0t0 TCP [fe80::f816:3eff:fe7a:294d]:domain (LISTEN)
Testing resolving hostnames on network node 02:
-----------------------------------------------
tore@net02:~$ sudo ip netns exec qdhcp-a125a5a5-3e16-4f4d-9b77-012c160424f4 host launchpad.net. 169.254.169.254
Using domain server:
Name: 169.254.169.254
Address: 169.254.169.254#53
Aliases:
launchpad.net has address 91.189.89.222
launchpad.net has address 91.189.89.223
launchpad.net mail is handled by 10 mx.launchpad.net.
tore@net02:~$ sudo ip netns exec qdhcp-a125a5a5-3e16-4f4d-9b77-012c160424f4 host launchpad.net. fe80::f816:3eff:fe7a:294d%tap0d726e47-80
Using domain server:
Name: fe80::f816:3eff:fe7a:294d%tap0d726e47-80
Address: fe80::f816:3eff:fe7a:294d%94#53
Aliases:
launchpad.net has address 91.189.89.223
launchpad.net has address 91.189.89.222
launchpad.net mail is handled by 10 mx.launchpad.net.
tore@net02:~$ sudo ip netns exec qdhcp-a125a5a5-3e16-4f4d-9b77-012c160424f4 host launchpad.net. 2a02:c0:200:f02f:f816:3eff:fe7a:294d
;; connection timed out; no servers could be reached
** Affects: neutron
Importance: Undecided
Status: New
** Tags: ipv6
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1499170
Title:
dnsmasq doesn't listen for DNS queries to its global IPv6 address
Status in neutron:
New
Bug description:
On Kilo, when setting up an IPv6 subnet with DHCP enabled, the qdhcp
namespace on the l3 agents will acquire global IPv6 addresses from the
subnet prefix. These addresses are also advertised to instances
requesting DNS servers via DHCPv6. However, dnsmasq does not listen
for incoming DNS queries to these addresses (it only listens queries
sent to its link-local IPv4 and IPv6 addresses). The end result is
that the instances are provisioned with a set of DNS servers which
simply does not work, and therefore cannot resolve any hostnames. The
console output below should help explain the issue:
Creating the network/subnet and attaching it to Internet router:
----------------------------------------------------------------
tore@ctrl01:~$ neutron net-create foobarnet
Created a new network:
+-----------------+--------------------------------------+
| Field | Value |
+-----------------+--------------------------------------+
| admin_state_up | True |
| id | a125a5a5-3e16-4f4d-9b77-012c160424f4 |
| mtu | 0 |
| name | foobarnet |
| router:external | False |
| shared | False |
| status | ACTIVE |
| subnets | |
| tenant_id | ebeb8cab76e94d189b7a1803c8bb1752 |
+-----------------+--------------------------------------+
tore@ctrl01:~$ neutron subnet-create --ip-version 6 --ipv6-ra-mode dhcpv6-stateless --ipv6-address-mode dhcpv6-stateless foobarnet 2a02:c0:200:f02f::/64
Created a new subnet:
+-------------------+---------------------------------------------------------------------------------+
| Field | Value |
+-------------------+---------------------------------------------------------------------------------+
| allocation_pools | {"start": "2a02:c0:200:f02f::2", "end": "2a02:c0:200:f02f:ffff:ffff:ffff:fffe"} |
| cidr | 2a02:c0:200:f02f::/64 |
| dns_nameservers | |
| enable_dhcp | True |
| gateway_ip | 2a02:c0:200:f02f::1 |
| host_routes | |
| id | 9f8d4190-5f83-414a-bb0c-0d5c8d31a8e1 |
| ip_version | 6 |
| ipv6_address_mode | dhcpv6-stateless |
| ipv6_ra_mode | dhcpv6-stateless |
| name | |
| network_id | a125a5a5-3e16-4f4d-9b77-012c160424f4 |
| subnetpool_id | |
| tenant_id | ebeb8cab76e94d189b7a1803c8bb1752 |
+-------------------+---------------------------------------------------------------------------------+
tore@ctrl01:~$ neutron router-interface-add 08ec7303-8040-45ca-b391-f5599d265eb9 9f8d4190-5f83-414a-bb0c-0d5c8d31a8e1
Added interface 40544ba9-ddab-417b-9b78-b1e6e80ca644 to router 08ec7303-8040-45ca-b391-f5599d265eb9.
Network node 01 status:
-----------------------
tore@net01:~$ sudo ip netns exec qdhcp-a125a5a5-3e16-4f4d-9b77-012c160424f4 ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
94: tapd21c1dc1-bd: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default
link/ether fa:16:3e:8a:82:5e brd ff:ff:ff:ff:ff:ff
inet 169.254.169.254/16 brd 169.254.255.255 scope global tapd21c1dc1-bd
valid_lft forever preferred_lft forever
inet6 2a02:c0:200:f02f:f816:3eff:fe8a:825e/64 scope global dynamic
valid_lft 86398sec preferred_lft 14398sec
inet6 fe80::f816:3eff:fe8a:825e/64 scope link
valid_lft forever preferred_lft forever
tore@net01:~$ sudo ip netns exec qdhcp-a125a5a5-3e16-4f4d-9b77-012c160424f4 lsof -i :53
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
dnsmasq 12789 nobody 7u IPv4 88487703 0t0 UDP 169.254.169.254:domain
dnsmasq 12789 nobody 8u IPv4 88487704 0t0 TCP 169.254.169.254:domain (LISTEN)
dnsmasq 12789 nobody 12u IPv6 88590343 0t0 UDP [fe80::f816:3eff:fe8a:825e]:domain
dnsmasq 12789 nobody 13u IPv6 88590344 0t0 TCP [fe80::f816:3eff:fe8a:825e]:domain (LISTEN)
Network node 02 status:
-----------------------
tore@net02:~$ sudo ip netns exec qdhcp-a125a5a5-3e16-4f4d-9b77-012c160424f4 ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
94: tap0d726e47-80: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default
link/ether fa:16:3e:7a:29:4d brd ff:ff:ff:ff:ff:ff
inet 169.254.169.254/16 brd 169.254.255.255 scope global tap0d726e47-80
valid_lft forever preferred_lft forever
inet6 2a02:c0:200:f02f:f816:3eff:fe7a:294d/64 scope global dynamic
valid_lft 86400sec preferred_lft 14400sec
inet6 fe80::f816:3eff:fe7a:294d/64 scope link
valid_lft forever preferred_lft forever
tore@net02:~$ sudo ip netns exec qdhcp-a125a5a5-3e16-4f4d-9b77-012c160424f4 lsof -i :53
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
dnsmasq 17217 nobody 7u IPv4 65048317 0t0 UDP 169.254.169.254:domain
dnsmasq 17217 nobody 8u IPv4 65048318 0t0 TCP 169.254.169.254:domain (LISTEN)
dnsmasq 17217 nobody 12u IPv6 65047373 0t0 UDP [fe80::f816:3eff:fe7a:294d]:domain
dnsmasq 17217 nobody 13u IPv6 65047374 0t0 TCP [fe80::f816:3eff:fe7a:294d]:domain (LISTEN)
Testing resolving hostnames on network node 02:
-----------------------------------------------
tore@net02:~$ sudo ip netns exec qdhcp-a125a5a5-3e16-4f4d-9b77-012c160424f4 host launchpad.net. 169.254.169.254
Using domain server:
Name: 169.254.169.254
Address: 169.254.169.254#53
Aliases:
launchpad.net has address 91.189.89.222
launchpad.net has address 91.189.89.223
launchpad.net mail is handled by 10 mx.launchpad.net.
tore@net02:~$ sudo ip netns exec qdhcp-a125a5a5-3e16-4f4d-9b77-012c160424f4 host launchpad.net. fe80::f816:3eff:fe7a:294d%tap0d726e47-80
Using domain server:
Name: fe80::f816:3eff:fe7a:294d%tap0d726e47-80
Address: fe80::f816:3eff:fe7a:294d%94#53
Aliases:
launchpad.net has address 91.189.89.223
launchpad.net has address 91.189.89.222
launchpad.net mail is handled by 10 mx.launchpad.net.
tore@net02:~$ sudo ip netns exec qdhcp-a125a5a5-3e16-4f4d-9b77-012c160424f4 host launchpad.net. 2a02:c0:200:f02f:f816:3eff:fe7a:294d
;; connection timed out; no servers could be reached
To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1499170/+subscriptions
Follow ups
