yahoo-eng-team team mailing list archive
-
yahoo-eng-team team
-
Mailing list archive
-
Message #72043
[Bug 1759694] [NEW] DHCP agent doesn't respawn metadata when enable_isolated_metadata and gateway removed
Public bug reported:
Hi,
We are running Neutron Pike with OVS and DVR.
When enable_isolated_metadata is True and we remove the gateway port for
a network from a router, a metadata process is not respawned to start
serving metadata.
How to replicate :
[root@5c1fced0888e /]# openstack network create test_nw
+---------------------------+--------------------------------------+
| Field | Value |
+---------------------------+--------------------------------------+
| admin_state_up | UP |
| availability_zone_hints | |
| availability_zones | |
| created_at | 2018-03-28T21:18:29Z |
| description | |
| dns_domain | |
| id | d19dabb2-f8c8-4608-8387-1f356a9f0f14 |
| ipv4_address_scope | None |
| ipv6_address_scope | None |
| is_default | False |
| is_vlan_transparent | None |
| mtu | 1500 |
| name | test_nw |
| port_security_enabled | True |
| project_id | c053ae2460e741008fa0ea908ae7da8c |
| provider:network_type | vxlan |
| provider:physical_network | None |
| provider:segmentation_id | 65035 |
| qos_policy_id | None |
| revision_number | 2 |
| router:external | Internal |
| segments | None |
| shared | False |
| status | ACTIVE |
| subnets | |
| tags | |
| updated_at | 2018-03-28T21:18:30Z |
+---------------------------+--------------------------------------+
[root@5c1fced0888e /]# openstack subnet create --network d19dabb2-f8c8-4608-8387-1f356a9f0f14 --subnet-range 10.10.10.0/24 --gateway 10.10.10.254 test_sn
+-------------------------+--------------------------------------+
| Field | Value |
+-------------------------+--------------------------------------+
| allocation_pools | 10.10.10.1-10.10.10.253 |
| cidr | 10.10.10.0/24 |
| created_at | 2018-03-28T21:20:03Z |
| description | |
| dns_nameservers | |
| enable_dhcp | True |
| gateway_ip | 10.10.10.254 |
| host_routes | |
| id | 1cd9d1f4-8c43-411b-85db-9514fe7b5e06 |
| ip_version | 4 |
| ipv6_address_mode | None |
| ipv6_ra_mode | None |
| name | test_sn |
| network_id | d19dabb2-f8c8-4608-8387-1f356a9f0f14 |
| project_id | c053ae2460e741008fa0ea908ae7da8c |
| revision_number | 0 |
| segment_id | None |
| service_types | |
| subnetpool_id | None |
| tags | |
| updated_at | 2018-03-28T21:20:03Z |
| use_default_subnet_pool | None |
+-------------------------+--------------------------------------+
[root@5c1fced0888e /]#
If I look on the neutron edge router, I can see the haproxy process
spawned and that the options in DHCP are going to get 169.254.169.254
routed to this server (10.10.10.1).
[root@<neutron server> neutron]# ps -ef | grep d19dabb2-f8c8-4608-8387-1f356a9f0f14 | grep haproxy
neutron 14768 1 0 21:20 ? 00:00:00 haproxy -f /var/lib/neutron/ns-metadata-proxy/d19dabb2-f8c8-4608-8387-1f356a9f0f14.conf
[root@<neutron server> neutron]# cat /var/lib/neutron/dhcp/d19dabb2-f8c8-4608-8387-1f356a9f0f14/opts
tag:tag0,option:classless-static-route,169.254.169.254/32,10.10.10.1,0.0.0.0/0,10.10.10.254
tag:tag0,249,169.254.169.254/32,10.10.10.1,0.0.0.0/0,10.10.10.254
tag:tag0,option:router,10.10.10.254
tag:tag0,option:dns-server,10.10.10.2,10.10.10.1
[root@<neutron server> neutron]#
I then create a router and plug this network in :
[root@5c1fced0888e /]# openstack router create test_rtr
+-------------------------+--------------------------------------+
| Field | Value |
+-------------------------+--------------------------------------+
| admin_state_up | UP |
| availability_zone_hints | |
| availability_zones | |
| created_at | 2018-03-28T21:23:34Z |
| description | |
| distributed | True |
| external_gateway_info | None |
| flavor_id | None |
| ha | True |
| id | 98d5603a-e476-4144-84ae-fca8da837d70 |
| name | test_rtr |
| project_id | c053ae2460e741008fa0ea908ae7da8c |
| revision_number | None |
| routes | |
| status | ACTIVE |
| tags | |
| updated_at | 2018-03-28T21:23:34Z |
+-------------------------+--------------------------------------+
[root@5c1fced0888e /]# openstack router add subnet 98d5603a-e476-4144-84ae-fca8da837d70 1cd9d1f4-8c43-411b-85db-9514fe7b5e06
[root@5c1fced0888e /]#
If I look on the neutron edge router, I can see the haproxy process is
still running and that the options in DHCP are going to get
169.254.169.254 routed to it's gateway (10.10.10.254). This works but
haproxy on the neutron server is unused.
[root@<neutron server> neutron]# ps -ef | grep d19dabb2-f8c8-4608-8387-1f356a9f0f14 | grep haproxy
neutron 14768 1 0 21:20 ? 00:00:00 haproxy -f /var/lib/neutron/ns-metadata-proxy/d19dabb2-f8c8-4608-8387-1f356a9f0f14.conf
[root@<neutron server> neutron]# cat /var/lib/neutron/dhcp/d19dabb2-f8c8-4608-8387-1f356a9f0f14/opts
tag:tag0,option:classless-static-route,169.254.169.254/32,10.10.10.254,0.0.0.0/0,10.10.10.254
tag:tag0,249,169.254.169.254/32,10.10.10.254,0.0.0.0/0,10.10.10.254
tag:tag0,option:router,10.10.10.254
tag:tag0,option:dns-server,10.10.10.2,10.10.10.1
[root@<neutron server> neutron]#
If I then restart the dhcp agent and wait a couple of minutes the
haproxy process is killed
[root@<neutron server> neutron]# systemctl restart neutron-dhcp-agent
[root@<neutron server> neutron]# ps -ef | grep d19dabb2-f8c8-4608-8387-1f356a9f0f14 | grep haproxy
[root@<neutron server> neutron]#
This all seems normal to me. If I now remove the subnet port from the
router though :
[root@5c1fced0888e /]# openstack router remove subnet 98d5603a-e476-4144-84ae-fca8da837d70 1cd9d1f4-8c43-411b-85db-9514fe7b5e06
[root@5c1fced0888e /]#
If I look on the neutron edge router, I can see the haproxy process is
NOT running and that the options in DHCP are going to get
169.254.169.254 routed to this server (10.10.10.1).
[root@<neutron server> neutron]# ps -ef | grep d19dabb2-f8c8-4608-8387-1f356a9f0f14 | grep haproxy
[root@<neutron server> neutron]# cat /var/lib/neutron/dhcp/d19dabb2-f8c8-4608-8387-1f356a9f0f14/opts
tag:tag0,option:classless-static-route,169.254.169.254/32,10.10.10.1,0.0.0.0/0,10.10.10.254
tag:tag0,249,169.254.169.254/32,10.10.10.1,0.0.0.0/0,10.10.10.254
tag:tag0,option:router,10.10.10.254
tag:tag0,option:dns-server,10.10.10.2,10.10.10.1
[root@<neutron server> neutron]#
At this point, metadata won't work until I restart the dhcp agent and it
does a full resync.
[root@<neutron server> neutron]# systemctl restart neutron-dhcp-agent
[root@<neutron server> neutron]# ps -ef | grep d19dabb2-f8c8-4608-8387-1f356a9f0f14 | grep haproxy
neutron 15376 1 0 21:36 ? 00:00:00 haproxy -f /var/lib/neutron/ns-metadata-proxy/d19dabb2-f8c8-4608-8387-1f356a9f0f14.conf
[root@<neutron server> neutron]# cat /var/lib/neutron/dhcp/d19dabb2-f8c8-4608-8387-1f356a9f0f14/opts
tag:tag0,option:classless-static-route,169.254.169.254/32,10.10.10.1,0.0.0.0/0,10.10.10.254
tag:tag0,249,169.254.169.254/32,10.10.10.1,0.0.0.0/0,10.10.10.254
tag:tag0,option:router,10.10.10.254
tag:tag0,option:dns-server,10.10.10.2,10.10.10.1
[root@<neutron server> neutron]#
** Affects: neutron
Importance: Undecided
Status: New
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1759694
Title:
DHCP agent doesn't respawn metadata when enable_isolated_metadata and
gateway removed
Status in neutron:
New
Bug description:
Hi,
We are running Neutron Pike with OVS and DVR.
When enable_isolated_metadata is True and we remove the gateway port
for a network from a router, a metadata process is not respawned to
start serving metadata.
How to replicate :
[root@5c1fced0888e /]# openstack network create test_nw
+---------------------------+--------------------------------------+
| Field | Value |
+---------------------------+--------------------------------------+
| admin_state_up | UP |
| availability_zone_hints | |
| availability_zones | |
| created_at | 2018-03-28T21:18:29Z |
| description | |
| dns_domain | |
| id | d19dabb2-f8c8-4608-8387-1f356a9f0f14 |
| ipv4_address_scope | None |
| ipv6_address_scope | None |
| is_default | False |
| is_vlan_transparent | None |
| mtu | 1500 |
| name | test_nw |
| port_security_enabled | True |
| project_id | c053ae2460e741008fa0ea908ae7da8c |
| provider:network_type | vxlan |
| provider:physical_network | None |
| provider:segmentation_id | 65035 |
| qos_policy_id | None |
| revision_number | 2 |
| router:external | Internal |
| segments | None |
| shared | False |
| status | ACTIVE |
| subnets | |
| tags | |
| updated_at | 2018-03-28T21:18:30Z |
+---------------------------+--------------------------------------+
[root@5c1fced0888e /]# openstack subnet create --network d19dabb2-f8c8-4608-8387-1f356a9f0f14 --subnet-range 10.10.10.0/24 --gateway 10.10.10.254 test_sn
+-------------------------+--------------------------------------+
| Field | Value |
+-------------------------+--------------------------------------+
| allocation_pools | 10.10.10.1-10.10.10.253 |
| cidr | 10.10.10.0/24 |
| created_at | 2018-03-28T21:20:03Z |
| description | |
| dns_nameservers | |
| enable_dhcp | True |
| gateway_ip | 10.10.10.254 |
| host_routes | |
| id | 1cd9d1f4-8c43-411b-85db-9514fe7b5e06 |
| ip_version | 4 |
| ipv6_address_mode | None |
| ipv6_ra_mode | None |
| name | test_sn |
| network_id | d19dabb2-f8c8-4608-8387-1f356a9f0f14 |
| project_id | c053ae2460e741008fa0ea908ae7da8c |
| revision_number | 0 |
| segment_id | None |
| service_types | |
| subnetpool_id | None |
| tags | |
| updated_at | 2018-03-28T21:20:03Z |
| use_default_subnet_pool | None |
+-------------------------+--------------------------------------+
[root@5c1fced0888e /]#
If I look on the neutron edge router, I can see the haproxy process
spawned and that the options in DHCP are going to get 169.254.169.254
routed to this server (10.10.10.1).
[root@<neutron server> neutron]# ps -ef | grep d19dabb2-f8c8-4608-8387-1f356a9f0f14 | grep haproxy
neutron 14768 1 0 21:20 ? 00:00:00 haproxy -f /var/lib/neutron/ns-metadata-proxy/d19dabb2-f8c8-4608-8387-1f356a9f0f14.conf
[root@<neutron server> neutron]# cat /var/lib/neutron/dhcp/d19dabb2-f8c8-4608-8387-1f356a9f0f14/opts
tag:tag0,option:classless-static-route,169.254.169.254/32,10.10.10.1,0.0.0.0/0,10.10.10.254
tag:tag0,249,169.254.169.254/32,10.10.10.1,0.0.0.0/0,10.10.10.254
tag:tag0,option:router,10.10.10.254
tag:tag0,option:dns-server,10.10.10.2,10.10.10.1
[root@<neutron server> neutron]#
I then create a router and plug this network in :
[root@5c1fced0888e /]# openstack router create test_rtr
+-------------------------+--------------------------------------+
| Field | Value |
+-------------------------+--------------------------------------+
| admin_state_up | UP |
| availability_zone_hints | |
| availability_zones | |
| created_at | 2018-03-28T21:23:34Z |
| description | |
| distributed | True |
| external_gateway_info | None |
| flavor_id | None |
| ha | True |
| id | 98d5603a-e476-4144-84ae-fca8da837d70 |
| name | test_rtr |
| project_id | c053ae2460e741008fa0ea908ae7da8c |
| revision_number | None |
| routes | |
| status | ACTIVE |
| tags | |
| updated_at | 2018-03-28T21:23:34Z |
+-------------------------+--------------------------------------+
[root@5c1fced0888e /]# openstack router add subnet 98d5603a-e476-4144-84ae-fca8da837d70 1cd9d1f4-8c43-411b-85db-9514fe7b5e06
[root@5c1fced0888e /]#
If I look on the neutron edge router, I can see the haproxy process is
still running and that the options in DHCP are going to get
169.254.169.254 routed to it's gateway (10.10.10.254). This works but
haproxy on the neutron server is unused.
[root@<neutron server> neutron]# ps -ef | grep d19dabb2-f8c8-4608-8387-1f356a9f0f14 | grep haproxy
neutron 14768 1 0 21:20 ? 00:00:00 haproxy -f /var/lib/neutron/ns-metadata-proxy/d19dabb2-f8c8-4608-8387-1f356a9f0f14.conf
[root@<neutron server> neutron]# cat /var/lib/neutron/dhcp/d19dabb2-f8c8-4608-8387-1f356a9f0f14/opts
tag:tag0,option:classless-static-route,169.254.169.254/32,10.10.10.254,0.0.0.0/0,10.10.10.254
tag:tag0,249,169.254.169.254/32,10.10.10.254,0.0.0.0/0,10.10.10.254
tag:tag0,option:router,10.10.10.254
tag:tag0,option:dns-server,10.10.10.2,10.10.10.1
[root@<neutron server> neutron]#
If I then restart the dhcp agent and wait a couple of minutes the
haproxy process is killed
[root@<neutron server> neutron]# systemctl restart neutron-dhcp-agent
[root@<neutron server> neutron]# ps -ef | grep d19dabb2-f8c8-4608-8387-1f356a9f0f14 | grep haproxy
[root@<neutron server> neutron]#
This all seems normal to me. If I now remove the subnet port from the
router though :
[root@5c1fced0888e /]# openstack router remove subnet 98d5603a-e476-4144-84ae-fca8da837d70 1cd9d1f4-8c43-411b-85db-9514fe7b5e06
[root@5c1fced0888e /]#
If I look on the neutron edge router, I can see the haproxy process is
NOT running and that the options in DHCP are going to get
169.254.169.254 routed to this server (10.10.10.1).
[root@<neutron server> neutron]# ps -ef | grep d19dabb2-f8c8-4608-8387-1f356a9f0f14 | grep haproxy
[root@<neutron server> neutron]# cat /var/lib/neutron/dhcp/d19dabb2-f8c8-4608-8387-1f356a9f0f14/opts
tag:tag0,option:classless-static-route,169.254.169.254/32,10.10.10.1,0.0.0.0/0,10.10.10.254
tag:tag0,249,169.254.169.254/32,10.10.10.1,0.0.0.0/0,10.10.10.254
tag:tag0,option:router,10.10.10.254
tag:tag0,option:dns-server,10.10.10.2,10.10.10.1
[root@<neutron server> neutron]#
At this point, metadata won't work until I restart the dhcp agent and
it does a full resync.
[root@<neutron server> neutron]# systemctl restart neutron-dhcp-agent
[root@<neutron server> neutron]# ps -ef | grep d19dabb2-f8c8-4608-8387-1f356a9f0f14 | grep haproxy
neutron 15376 1 0 21:36 ? 00:00:00 haproxy -f /var/lib/neutron/ns-metadata-proxy/d19dabb2-f8c8-4608-8387-1f356a9f0f14.conf
[root@<neutron server> neutron]# cat /var/lib/neutron/dhcp/d19dabb2-f8c8-4608-8387-1f356a9f0f14/opts
tag:tag0,option:classless-static-route,169.254.169.254/32,10.10.10.1,0.0.0.0/0,10.10.10.254
tag:tag0,249,169.254.169.254/32,10.10.10.1,0.0.0.0/0,10.10.10.254
tag:tag0,option:router,10.10.10.254
tag:tag0,option:dns-server,10.10.10.2,10.10.10.1
[root@<neutron server> neutron]#
To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1759694/+subscriptions
Follow ups
