yahoo-eng-team team mailing list archive
-
yahoo-eng-team team
-
Mailing list archive
-
Message #94675
[Bug 2028159] Re: Invalid IPv6 subnet in self-service network breaks DHCP agent
Given the lack of interest expressed for my proposal 8 months ago to
cover this with an advisory, it doesn't seem to rise to the level of
urgency where we'd issue an OSSA (some branches didn't get backports
before they reached end of normal maintenance either). As such, I'm
closing the Security Advisory task as Won't Fix, but if there are any
dissenting opinions I'm happy to reopen and revisit that decision.
** Changed in: ossa
Status: Incomplete => Won't Fix
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/2028159
Title:
Invalid IPv6 subnet in self-service network breaks DHCP agent
Status in neutron:
Fix Released
Status in OpenStack Security Advisory:
Won't Fix
Bug description:
High level description:
A user creates self-service network (vxlan) with IPv6 subnet with the address ::/24, gateway ::
After that new instances in other networks do not receive addresses via DHCP.
Pre-conditions:
Neutron 20.3.1 (Yoga) with OVS ML2 plugin
3 DHCP agents for each network running on each of 3 controllers
A user account with a user role in some project
Step-by-step reproduction steps:
1. Launch a new instance in any DHCP-enabled network.
2. Verify that the instance receives an address.
3. Create a new network with a subnet with the following options:
a) via Dashboard:
Network Address: ::/24
IP Version: IPv6
Gateway IP: ::
Enable DHCP: true
IPv6 Address Configuration Mode: No options specified
b) or via CLI:
openstack network create bad
openstack subnet create --network bad --dhcp --ip-version 6 --subnet-range "::/24" --gateway "::" badsub
4. Launch another instance in the same network as #1.
5. Verify that the instance does not receive an address.
6. Delete the network from step 3.
7. Reboot the last instance.
8. Verify that it receives an address.
Expected output:
Either Neutron does not allow to create such subnet, or
New instances do receive addresses (DHCP agent stays uninterrupted)
Actual output:
Neutron did not perform verification of the subnet options.
DHCP agent enters a broken state, new instances do not receive addresses.
Version:
# rpm -qa | grep neutron | sort
openstack-neutron-20.3.1-1.el8.noarch
openstack-neutron-common-20.3.1-1.el8.noarch
openstack-neutron-ml2-20.3.1-1.el8.noarch
openstack-neutron-openvswitch-20.3.1-1.el8.noarch
python3-neutron-20.3.1-1.el8.noarch
python3-neutronclient-7.8.0-1.el8.noarch
python3-neutron-lib-2.20.2-1.el8.noarch
# cat /etc/redhat-release
CentOS Stream release 8
# uname -srvmpio
Linux 4.18.0-383.el8.x86_64 #1 SMP Wed Apr 20 15:38:08 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Environment:
# openstack compute service list --sort-column Host
+--------------------------------------+----------------+------+----------+---------+-------+----------------------------+
| ID | Binary | Host | Zone | Status | State | Updated At |
+--------------------------------------+----------------+------+----------+---------+-------+----------------------------+
| c45e81ed-e173-4e36-b209-01c80b99036d | nova-conductor | s5 | internal | enabled | up | 2023-07-19T12:05:47.000000 |
| c0310488-c0c5-4c37-9847-44259c86f776 | nova-scheduler | s5 | internal | enabled | up | 2023-07-19T12:05:48.000000 |
| b30d037e-90c2-4624-b8a0-91822ecf85a8 | nova-conductor | s6 | internal | enabled | up | 2023-07-19T12:05:55.000000 |
| da00e178-c2a5-487c-affa-10ed60cc3a2f | nova-scheduler | s6 | internal | enabled | up | 2023-07-19T12:05:49.000000 |
| 49e63486-c55f-428b-a1a1-defac0f47bb7 | nova-conductor | s7 | internal | enabled | up | 2023-07-19T12:05:53.000000 |
| ae929e33-a114-4446-8c7a-a1f9a8aa5c21 | nova-scheduler | s7 | internal | enabled | up | 2023-07-19T12:05:55.000000 |
| 0e10eb67-8150-4a3d-a268-ec9e1a3cc0ec | nova-compute | s8 | nova | enabled | up | 2023-07-19T12:05:46.000000 |
| d271bf37-4d47-4150-8cd2-7119fcebc1a6 | nova-compute | s9 | nova | enabled | up | 2023-07-19T12:05:54.000000 |
+--------------------------------------+----------------+------+----------+---------+-------+----------------------------+
# openstack network agent list --sort-column Binary --sort-column Host
+--------------------------------------+--------------------+------+-------------------+-------+-------+---------------------------+
| ID | Agent Type | Host | Availability Zone | Alive | State | Binary |
+--------------------------------------+--------------------+------+-------------------+-------+-------+---------------------------+
| d749fb1b-2bda-42bf-b5a4-dd6a6c0f56c2 | DHCP agent | s5 | nova | :-) | UP | neutron-dhcp-agent |
| cceea512-154c-44ea-a571-9e5a542ccde9 | DHCP agent | s6 | nova | :-) | UP | neutron-dhcp-agent |
| 5c5ad312-c1ab-4d33-9e54-b62e7112b218 | DHCP agent | s7 | nova | :-) | UP | neutron-dhcp-agent |
| 7dc0b55f-6a3c-45bc-866a-28540128147d | L3 agent | s5 | nova | :-) | UP | neutron-l3-agent |
| 6171f6e5-66b6-475a-ba6b-6cc113dd2729 | L3 agent | s6 | nova | :-) | UP | neutron-l3-agent |
| df9b3796-181b-46ab-8adb-52083cbc5d1a | L3 agent | s7 | nova | :-) | UP | neutron-l3-agent |
| 03cffc3b-3e27-48bf-a633-b5ffed011fa6 | L3 agent | s8 | nova | :-) | UP | neutron-l3-agent |
| 1430f493-57e4-436d-8fcb-d8344fdbb2b0 | L3 agent | s9 | nova | :-) | UP | neutron-l3-agent |
| 52bd49c0-96d3-410f-88bb-ea99550851bc | Metadata agent | s5 | None | :-) | UP | neutron-metadata-agent |
| 699aca37-efc3-4c42-ad2c-eb6d5897a203 | Metadata agent | s6 | None | :-) | UP | neutron-metadata-agent |
| 89588d09-93ca-4c92-b544-0fd16274f4c9 | Metadata agent | s7 | None | :-) | UP | neutron-metadata-agent |
| e9af410b-7237-4e25-adcc-c13483917bf4 | Metadata agent | s8 | None | :-) | UP | neutron-metadata-agent |
| b4e9bef5-36fe-4953-a2f9-8d437fe7b30f | Metadata agent | s9 | None | :-) | UP | neutron-metadata-agent |
| 7173b0ed-4ec5-4177-ba29-3782e3e5f8be | Open vSwitch agent | s5 | None | :-) | UP | neutron-openvswitch-agent |
| d58ca721-f56d-4b3a-85d7-5e6c0d04f9db | Open vSwitch agent | s6 | None | :-) | UP | neutron-openvswitch-agent |
| 2924fb03-7e16-42c5-8af8-c1a3b25b0905 | Open vSwitch agent | s7 | None | :-) | UP | neutron-openvswitch-agent |
| b2118af9-a418-469f-9fea-379a92aa8548 | Open vSwitch agent | s8 | None | :-) | UP | neutron-openvswitch-agent |
| ee1c3f12-be03-4891-8895-b8f72f417585 | Open vSwitch agent | s9 | None | :-) | UP | neutron-openvswitch-agent |
+--------------------------------------+--------------------+------+-------------------+-------+-------+---------------------------+
Perceived severity:
High
dhcp-agent.log contains the following:
2023-07-14 16:26:03.589 93091 ERROR neutron.agent.dhcp.agent [-] Unable to enable dhcp for eb2e3a84-87fa-4d03-87fa-8986a70f5d57.: pr2modules.netlink.exceptions.NetlinkError: (99, 'Cannot assign requested address')
2023-07-14 16:26:03.589 93091 ERROR neutron.agent.dhcp.agent Traceback (most recent call last):
2023-07-14 16:26:03.589 93091 ERROR neutron.agent.dhcp.agent File "/usr/lib/python3.6/site-packages/neutron/agent/dhcp/agent.py", line 218, in call_driver
2023-07-14 16:26:03.589 93091 ERROR neutron.agent.dhcp.agent rv = getattr(driver, action)(**action_kwargs)
2023-07-14 16:26:03.589 93091 ERROR neutron.agent.dhcp.agent File "/usr/lib/python3.6/site-packages/neutron/agent/linux/dhcp.py", line 275, in enable
2023-07-14 16:26:03.589 93091 ERROR neutron.agent.dhcp.agent common_utils.wait_until_true(self._enable, timeout=300)
2023-07-14 16:26:03.589 93091 ERROR neutron.agent.dhcp.agent File "/usr/lib/python3.6/site-packages/neutron/common/utils.py", line 717, in wait_until_true
2023-07-14 16:26:03.589 93091 ERROR neutron.agent.dhcp.agent while not predicate():
2023-07-14 16:26:03.589 93091 ERROR neutron.agent.dhcp.agent File "/usr/lib/python3.6/site-packages/neutron/agent/linux/dhcp.py", line 287, in _enable
2023-07-14 16:26:03.589 93091 ERROR neutron.agent.dhcp.agent interface_name = self.device_manager.setup(self.network)
2023-07-14 16:26:03.589 93091 ERROR neutron.agent.dhcp.agent File "/usr/lib/python3.6/site-packages/neutron/agent/linux/dhcp.py", line 1780, in setup
2023-07-14 16:26:03.589 93091 ERROR neutron.agent.dhcp.agent namespace=network.namespace)
2023-07-14 16:26:03.589 93091 ERROR neutron.agent.dhcp.agent File "/usr/lib/python3.6/site-packages/neutron/agent/linux/interface.py", line 152, in init_l3
2023-07-14 16:26:03.589 93091 ERROR neutron.agent.dhcp.agent device.addr.add(ip_cidr)
2023-07-14 16:26:03.589 93091 ERROR neutron.agent.dhcp.agent File "/usr/lib/python3.6/site-packages/neutron/agent/linux/ip_lib.py", line 541, in add
2023-07-14 16:26:03.589 93091 ERROR neutron.agent.dhcp.agent add_broadcast)
2023-07-14 16:26:03.589 93091 ERROR neutron.agent.dhcp.agent File "/usr/lib/python3.6/site-packages/neutron/agent/linux/ip_lib.py", line 830, in add_ip_address
2023-07-14 16:26:03.589 93091 ERROR neutron.agent.dhcp.agent device, namespace, scope, broadcast)
2023-07-14 16:26:03.589 93091 ERROR neutron.agent.dhcp.agent File "/usr/lib/python3.6/site-packages/oslo_privsep/priv_context.py", line 272, in _wrap
2023-07-14 16:26:03.589 93091 ERROR neutron.agent.dhcp.agent r_call_timeout)
2023-07-14 16:26:03.589 93091 ERROR neutron.agent.dhcp.agent File "/usr/lib/python3.6/site-packages/oslo_privsep/daemon.py", line 215, in remote_call
2023-07-14 16:26:03.589 93091 ERROR neutron.agent.dhcp.agent raise exc_type(*result[2])
2023-07-14 16:26:03.589 93091 ERROR neutron.agent.dhcp.agent pr2modules.netlink.exceptions.NetlinkError: (99, 'Cannot assign requested address')
or
2023-07-19 13:58:39.777 98250 DEBUG neutron.agent.linux.dhcp [req-82f865b9-f787-4983-acb2-145c7db53877 - - - - -] Setting IPv6 gateway for dhcp netns on net 94355373-4bb8-4117-bec3-c6f492f26a93 to :: _set_default_route_ip_version /usr/lib/python3.6/site-packages/neutron/agent/linux/dhcp.py:1464
2023-07-19 13:58:39.832 98645 DEBUG oslo.privsep.daemon [-] privsep: Exception during request[14221983-9b1e-49c3-8248-59325d3e4069]: (22, 'Invalid argument') _process_cmd /usr/lib/python3.6/site-packages/oslo_privsep/daemon.py:481
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/oslo_privsep/daemon.py", line 476, in _process_cmd
ret = func(*f_args, **f_kwargs)
File "/usr/lib/python3.6/site-packages/oslo_privsep/priv_context.py", line 274, in _wrap
return func(*args, **kwargs)
File "/usr/lib/python3.6/site-packages/neutron/privileged/agent/linux/ip_lib.py", line 752, in add_ip_route
ip.route('replace', **kwargs)
File "/usr/lib/python3.6/site-packages/pr2modules/iproute/linux.py", line 2042, in route
callback=callback)
File "/usr/lib/python3.6/site-packages/pr2modules/netlink/nlsocket.py", line 397, in nlm_request
return tuple(self._genlm_request(*argv, **kwarg))
File "/usr/lib/python3.6/site-packages/pr2modules/netlink/nlsocket.py", line 891, in nlm_request
callback=callback):
File "/usr/lib/python3.6/site-packages/pr2modules/netlink/nlsocket.py", line 400, in get
return tuple(self._genlm_get(*argv, **kwarg))
File "/usr/lib/python3.6/site-packages/pr2modules/netlink/nlsocket.py", line 725, in get
raise msg['header']['error']
pr2modules.netlink.exceptions.NetlinkError: (22, 'Invalid argument')
2023-07-19 13:58:39.834 98645 DEBUG oslo.privsep.daemon [-] privsep: reply[14221983-9b1e-49c3-8248-59325d3e4069]: (5, 'pr2modules.netlink.exceptions.NetlinkError', (22, 'Invalid argument')) _call_back /usr/lib/python3.6/site-packages/oslo_privsep/daemon.py:502
2023-07-19 13:58:39.837 98250 ERROR neutron.agent.dhcp.agent [req-82f865b9-f787-4983-acb2-145c7db53877 - - - - -] Unable to enable dhcp for 94355373-4bb8-4117-bec3-c6f492f26a93.: pr2modules.netlink.exceptions.NetlinkError: (22, 'Invalid argument')
2023-07-19 13:58:39.837 98250 ERROR neutron.agent.dhcp.agent [req-82f865b9-f787-4983-acb2-145c7db53877 - - - - -] Unable to enable dhcp for 94355373-4bb8-4117-bec3-c6f492f26a93.: pr2modules.netlink.exceptions.NetlinkError: (22, 'Invalid argument')
2023-07-19 13:58:39.837 98250 ERROR neutron.agent.dhcp.agent Traceback (most recent call last):
2023-07-19 13:58:39.837 98250 ERROR neutron.agent.dhcp.agent File "/usr/lib/python3.6/site-packages/neutron/agent/dhcp/agent.py", line 218, in call_driver
2023-07-19 13:58:39.837 98250 ERROR neutron.agent.dhcp.agent rv = getattr(driver, action)(**action_kwargs)
2023-07-19 13:58:39.837 98250 ERROR neutron.agent.dhcp.agent File "/usr/lib/python3.6/site-packages/neutron/agent/linux/dhcp.py", line 275, in enable
2023-07-19 13:58:39.837 98250 ERROR neutron.agent.dhcp.agent common_utils.wait_until_true(self._enable, timeout=300)
2023-07-19 13:58:39.837 98250 ERROR neutron.agent.dhcp.agent File "/usr/lib/python3.6/site-packages/neutron/common/utils.py", line 717, in wait_until_true
2023-07-19 13:58:39.837 98250 ERROR neutron.agent.dhcp.agent while not predicate():
2023-07-19 13:58:39.837 98250 ERROR neutron.agent.dhcp.agent File "/usr/lib/python3.6/site-packages/neutron/agent/linux/dhcp.py", line 287, in _enable
2023-07-19 13:58:39.837 98250 ERROR neutron.agent.dhcp.agent interface_name = self.device_manager.setup(self.network)
2023-07-19 13:58:39.837 98250 ERROR neutron.agent.dhcp.agent File "/usr/lib/python3.6/site-packages/neutron/agent/linux/dhcp.py", line 1782, in setup
2023-07-19 13:58:39.837 98250 ERROR neutron.agent.dhcp.agent self._set_default_route(network, interface_name)
2023-07-19 13:58:39.837 98250 ERROR neutron.agent.dhcp.agent File "/usr/lib/python3.6/site-packages/neutron/agent/linux/dhcp.py", line 1505, in _set_default_route
2023-07-19 13:58:39.837 98250 ERROR neutron.agent.dhcp.agent ip_version)
2023-07-19 13:58:39.837 98250 ERROR neutron.agent.dhcp.agent File "/usr/lib/python3.6/site-packages/neutron/agent/linux/dhcp.py", line 1483, in _set_default_route_ip_version
2023-07-19 13:58:39.837 98250 ERROR neutron.agent.dhcp.agent device.route.add_gateway(subnet.gateway_ip)
2023-07-19 13:58:39.837 98250 ERROR neutron.agent.dhcp.agent File "/usr/lib/python3.6/site-packages/neutron/agent/linux/ip_lib.py", line 620, in add_gateway
2023-07-19 13:58:39.837 98250 ERROR neutron.agent.dhcp.agent scope=scope)
2023-07-19 13:58:39.837 98250 ERROR neutron.agent.dhcp.agent File "/usr/lib/python3.6/site-packages/neutron/agent/linux/ip_lib.py", line 658, in add_route
2023-07-19 13:58:39.837 98250 ERROR neutron.agent.dhcp.agent table=table, metric=metric, scope=scope, **kwargs)
2023-07-19 13:58:39.837 98250 ERROR neutron.agent.dhcp.agent File "/usr/lib/python3.6/site-packages/neutron/agent/linux/ip_lib.py", line 1532, in add_ip_route
2023-07-19 13:58:39.837 98250 ERROR neutron.agent.dhcp.agent metric=metric, scope=scope, proto=proto, **kwargs)
2023-07-19 13:58:39.837 98250 ERROR neutron.agent.dhcp.agent File "/usr/lib/python3.6/site-packages/oslo_privsep/priv_context.py", line 272, in _wrap
2023-07-19 13:58:39.837 98250 ERROR neutron.agent.dhcp.agent r_call_timeout)
2023-07-19 13:58:39.837 98250 ERROR neutron.agent.dhcp.agent File "/usr/lib/python3.6/site-packages/oslo_privsep/daemon.py", line 215, in remote_call
2023-07-19 13:58:39.837 98250 ERROR neutron.agent.dhcp.agent raise exc_type(*result[2])
2023-07-19 13:58:39.837 98250 ERROR neutron.agent.dhcp.agent pr2modules.netlink.exceptions.NetlinkError: (22, 'Invalid argument')
To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/2028159/+subscriptions
