yahoo-eng-team team mailing list archive
-
yahoo-eng-team team
-
Mailing list archive
-
Message #94388
[Bug 2076181] [NEW] Accidentally no-allocation-pool for subnet leads to poor diagnosis of build-in dhcp service issues
Public bug reported:
Our customer reported an issue where he does not understand what is
wrong with the DHCP service.
As it turned out, during the management of subnet allocation pools, he
accidentally deleted the allocation pool (thinking that the entire range
would be used).
Then he reset (off/on) the DHCP service, but he didn't understand why
the DHCP service stopped working.
Here is a demo that I reproduced on my DevStack installation from the
master branch:
[root@devstack ~]# openstack network create net0
[root@devstack ~]# openstack subnet create sub0 --network net0 --subnet-range 192.168.1.0/24
[root@devstack ~]# openstack subnet set sub0 --no-allocation-pool
[root@devstack ~]# openstack subnet set sub0 --no-dhcp
[root@devstack ~]# openstack subnet set sub0 --dhcp
Neutron DHCP agent prints the following traceback:
ERROR neutron.agent.dhcp.agent Traceback (most recent call last):
ERROR neutron.agent.dhcp.agent File "/opt/stack/neutron/neutron/agent/dhcp/agent.py", line 265, in _call_driver
ERROR neutron.agent.dhcp.agent rv = getattr(driver, action)(**action_kwargs)
ERROR neutron.agent.dhcp.agent File "/opt/stack/neutron/neutron/agent/linux/dhcp.py", line 331, in enable
ERROR neutron.agent.dhcp.agent try:
ERROR neutron.agent.dhcp.agent File "/opt/stack/neutron/neutron/common/utils.py", line 739, in wait_until_true
ERROR neutron.agent.dhcp.agent while not predicate():
ERROR neutron.agent.dhcp.agent File "/opt/stack/neutron/neutron/agent/linux/dhcp.py", line 343, in _enable
ERROR neutron.agent.dhcp.agent return True
ERROR neutron.agent.dhcp.agent File "/opt/stack/neutron/neutron/agent/linux/dhcp.py", line 1826, in setup
ERROR neutron.agent.dhcp.agent # them on the device. This must be done before any interfaces
ERROR neutron.agent.dhcp.agent File "/usr/local/lib/python3.9/site-packages/oslo_utils/excutils.py", line 227, in __exit__
ERROR neutron.agent.dhcp.agent self.force_reraise()
ERROR neutron.agent.dhcp.agent File "/usr/local/lib/python3.9/site-packages/oslo_utils/excutils.py", line 200, in force_reraise
ERROR neutron.agent.dhcp.agent raise self.value
ERROR neutron.agent.dhcp.agent File "/opt/stack/neutron/neutron/agent/linux/dhcp.py", line 1821, in setup
ERROR neutron.agent.dhcp.agent self._update_dhcp_port(network, port)
ERROR neutron.agent.dhcp.agent File "/opt/stack/neutron/neutron/agent/linux/dhcp.py", line 1757, in setup_dhcp_port
ERROR neutron.agent.dhcp.agent self._check_dhcp_port_subnet(dhcp_port, dhcp_subnets, network)
ERROR neutron.agent.dhcp.agent File "/opt/stack/neutron/neutron/agent/linux/dhcp.py", line 1705, in _setup_new_dhcp_port
ERROR neutron.agent.dhcp.agent can be removed once bug/1627480 is fixed
ERROR neutron.agent.dhcp.agent File "/opt/stack/neutron/neutron/agent/dhcp/agent.py", line 898, in create_dhcp_port
ERROR neutron.agent.dhcp.agent port = cctxt.call(self.context, 'create_dhcp_port',
ERROR neutron.agent.dhcp.agent File "/usr/local/lib/python3.9/site-packages/neutron_lib/rpc.py", line 149, in call
ERROR neutron.agent.dhcp.agent return self._original_context.call(ctxt, method, **kwargs)
ERROR neutron.agent.dhcp.agent File "/usr/local/lib/python3.9/site-packages/oslo_messaging/rpc/client.py", line 190, in call
ERROR neutron.agent.dhcp.agent result = self.transport._send(
ERROR neutron.agent.dhcp.agent File "/usr/local/lib/python3.9/site-packages/oslo_messaging/transport.py", line 123, in _send
ERROR neutron.agent.dhcp.agent return self._driver.send(target, ctxt, message,
ERROR neutron.agent.dhcp.agent File "/usr/local/lib/python3.9/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
ERROR neutron.agent.dhcp.agent return self._send(target, ctxt, message, wait_for_reply, timeout,
ERROR neutron.agent.dhcp.agent File "/usr/local/lib/python3.9/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
ERROR neutron.agent.dhcp.agent raise result
ERROR neutron.agent.dhcp.agent oslo_messaging.rpc.client.RemoteError: Remote error: IpAddressGenerationFailure No more IP addresses available on network 6d48c986-266c-48b4-835c-f0e215aa5713.
The traceback indicates that no IP addresses are available. However, upon checking, I found many IP addresses were free:
[root@devstack ~]# openstack ip availability show 6d48c986-266c-48b4-835c-f0e215aa5713
+------------------------+--------------------------------------------------------------------------------------------------------------------------------------------+
| Field | Value |
+------------------------+--------------------------------------------------------------------------------------------------------------------------------------------+
| network_id | 6d48c986-266c-48b4-835c-f0e215aa5713 |
| network_name | net0 |
| project_id | af61bf69ee0a4a7db97d2dd640d967c2 |
| subnet_ip_availability | cidr='192.168.1.0/24', ip_version='4', subnet_id='072c445f-b401-4fe4-acb6-cf772ce65738', subnet_name='sub0', total_ips='256', used_ips='0' |
| total_ips | 256 |
| used_ips | 0 |
+------------------------+--------------------------------------------------------------------------------------------------------------------------------------------+
[root@devstack ~]#
Maybe it's worth adding a validation that DHCP can't be enabled if there is no allocation pool, because this is a request that cannot be executed. Conversely, don't allow removing the allocation pool if DHCP is enabled.
Maybe I missed this behavior in the documentation.
** Affects: neutron
Importance: Undecided
Status: New
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/2076181
Title:
Accidentally no-allocation-pool for subnet leads to poor diagnosis of
build-in dhcp service issues
Status in neutron:
New
Bug description:
Our customer reported an issue where he does not understand what is
wrong with the DHCP service.
As it turned out, during the management of subnet allocation pools, he
accidentally deleted the allocation pool (thinking that the entire
range would be used).
Then he reset (off/on) the DHCP service, but he didn't understand why
the DHCP service stopped working.
Here is a demo that I reproduced on my DevStack installation from the
master branch:
[root@devstack ~]# openstack network create net0
[root@devstack ~]# openstack subnet create sub0 --network net0 --subnet-range 192.168.1.0/24
[root@devstack ~]# openstack subnet set sub0 --no-allocation-pool
[root@devstack ~]# openstack subnet set sub0 --no-dhcp
[root@devstack ~]# openstack subnet set sub0 --dhcp
Neutron DHCP agent prints the following traceback:
ERROR neutron.agent.dhcp.agent Traceback (most recent call last):
ERROR neutron.agent.dhcp.agent File "/opt/stack/neutron/neutron/agent/dhcp/agent.py", line 265, in _call_driver
ERROR neutron.agent.dhcp.agent rv = getattr(driver, action)(**action_kwargs)
ERROR neutron.agent.dhcp.agent File "/opt/stack/neutron/neutron/agent/linux/dhcp.py", line 331, in enable
ERROR neutron.agent.dhcp.agent try:
ERROR neutron.agent.dhcp.agent File "/opt/stack/neutron/neutron/common/utils.py", line 739, in wait_until_true
ERROR neutron.agent.dhcp.agent while not predicate():
ERROR neutron.agent.dhcp.agent File "/opt/stack/neutron/neutron/agent/linux/dhcp.py", line 343, in _enable
ERROR neutron.agent.dhcp.agent return True
ERROR neutron.agent.dhcp.agent File "/opt/stack/neutron/neutron/agent/linux/dhcp.py", line 1826, in setup
ERROR neutron.agent.dhcp.agent # them on the device. This must be done before any interfaces
ERROR neutron.agent.dhcp.agent File "/usr/local/lib/python3.9/site-packages/oslo_utils/excutils.py", line 227, in __exit__
ERROR neutron.agent.dhcp.agent self.force_reraise()
ERROR neutron.agent.dhcp.agent File "/usr/local/lib/python3.9/site-packages/oslo_utils/excutils.py", line 200, in force_reraise
ERROR neutron.agent.dhcp.agent raise self.value
ERROR neutron.agent.dhcp.agent File "/opt/stack/neutron/neutron/agent/linux/dhcp.py", line 1821, in setup
ERROR neutron.agent.dhcp.agent self._update_dhcp_port(network, port)
ERROR neutron.agent.dhcp.agent File "/opt/stack/neutron/neutron/agent/linux/dhcp.py", line 1757, in setup_dhcp_port
ERROR neutron.agent.dhcp.agent self._check_dhcp_port_subnet(dhcp_port, dhcp_subnets, network)
ERROR neutron.agent.dhcp.agent File "/opt/stack/neutron/neutron/agent/linux/dhcp.py", line 1705, in _setup_new_dhcp_port
ERROR neutron.agent.dhcp.agent can be removed once bug/1627480 is fixed
ERROR neutron.agent.dhcp.agent File "/opt/stack/neutron/neutron/agent/dhcp/agent.py", line 898, in create_dhcp_port
ERROR neutron.agent.dhcp.agent port = cctxt.call(self.context, 'create_dhcp_port',
ERROR neutron.agent.dhcp.agent File "/usr/local/lib/python3.9/site-packages/neutron_lib/rpc.py", line 149, in call
ERROR neutron.agent.dhcp.agent return self._original_context.call(ctxt, method, **kwargs)
ERROR neutron.agent.dhcp.agent File "/usr/local/lib/python3.9/site-packages/oslo_messaging/rpc/client.py", line 190, in call
ERROR neutron.agent.dhcp.agent result = self.transport._send(
ERROR neutron.agent.dhcp.agent File "/usr/local/lib/python3.9/site-packages/oslo_messaging/transport.py", line 123, in _send
ERROR neutron.agent.dhcp.agent return self._driver.send(target, ctxt, message,
ERROR neutron.agent.dhcp.agent File "/usr/local/lib/python3.9/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
ERROR neutron.agent.dhcp.agent return self._send(target, ctxt, message, wait_for_reply, timeout,
ERROR neutron.agent.dhcp.agent File "/usr/local/lib/python3.9/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
ERROR neutron.agent.dhcp.agent raise result
ERROR neutron.agent.dhcp.agent oslo_messaging.rpc.client.RemoteError: Remote error: IpAddressGenerationFailure No more IP addresses available on network 6d48c986-266c-48b4-835c-f0e215aa5713.
The traceback indicates that no IP addresses are available. However, upon checking, I found many IP addresses were free:
[root@devstack ~]# openstack ip availability show 6d48c986-266c-48b4-835c-f0e215aa5713
+------------------------+--------------------------------------------------------------------------------------------------------------------------------------------+
| Field | Value |
+------------------------+--------------------------------------------------------------------------------------------------------------------------------------------+
| network_id | 6d48c986-266c-48b4-835c-f0e215aa5713 |
| network_name | net0 |
| project_id | af61bf69ee0a4a7db97d2dd640d967c2 |
| subnet_ip_availability | cidr='192.168.1.0/24', ip_version='4', subnet_id='072c445f-b401-4fe4-acb6-cf772ce65738', subnet_name='sub0', total_ips='256', used_ips='0' |
| total_ips | 256 |
| used_ips | 0 |
+------------------------+--------------------------------------------------------------------------------------------------------------------------------------------+
[root@devstack ~]#
Maybe it's worth adding a validation that DHCP can't be enabled if there is no allocation pool, because this is a request that cannot be executed. Conversely, don't allow removing the allocation pool if DHCP is enabled.
Maybe I missed this behavior in the documentation.
To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/2076181/+subscriptions
