yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1427015] Re: too many subnet-create cause q-dhcp failure

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Armando Migliaccio <1427015@xxxxxxxxxxxxxxxxxx>
Date: Fri, 02 Oct 2015 21:59:29 -0000
Reply-to: Bug 1427015 <1427015@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
*** This bug is a duplicate of bug 1179713 ***
    https://bugs.launchpad.net/bugs/1179713

I don't think tempest has anything to do with this.

** This bug has been marked a duplicate of bug 1179713
   too many subnet-create cause q-dhcp failure

** This bug is no longer a duplicate of bug 1179713
   too many subnet-create cause q-dhcp failure

** Changed in: tempest
       Status: In Progress => Invalid

** This bug has been marked a duplicate of bug 1179713
   too many subnet-create cause q-dhcp failure

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1427015

Title:
  too many subnet-create cause q-dhcp failure

Status in neutron:
  In Progress
Status in tempest:
  Invalid

Bug description:
  ============================[NEUTRON]============================

  DHCP max port is only validated when port is creating.
  But when create_port_repuest has been sent in subnet create or update, validate will not be functionally.
  This results the number of total DHCP ports excess max_fixed_ips_per_port.
  If so, the DHCP agent will export error, and cannot restart itself.
  Also, user is not announced about that Fixed IP not been created after the subnet creation, even the "enable_dhcp" of subnet shows "True".

  [reproduce]
  1. neutron net create testnet
  2. neutron dhcp-agent-network-add <dhcp_agent_id> testnet
  3. neutron subnet-create testnet CIDR1 --name testsub1
  4. neutron subnet-create testnet CIDR2 --name testsub2
  5. neutron subnet-create testnet CIDR3 --name testsub3
  6. neutron subnet-create testnet CIDR4 --name testsub4
  7. neutron subnet-create testnet CIDR5 --name testsub5
  >>>since default value of max_fixed_ips_per_port is 5, it is ok till here.
  8-1. neutron subnet-create testnet CIDR6 --name testsub6
  >>>error repetly occured in q-dhcp.log.

  Also, confirmed the following case cause the same error
  9-1. neutron subnet-create testnet CIDR6 --name testsub6 --enable_dhcp False
  9-2. neutron subnet-update testsub6 --enable_dhcp True

  [trace log]
  2015-02-28 00:31:45.548 3011 TRACE neutron.agent.dhcp.agent Traceback (most recent call last):
  2015-02-28 00:31:45.548 3011 TRACE neutron.agent.dhcp.agent   File "/opt/stack/neutron/neutron/agent/dhcp/agent.py", line 112, in call_driver
  2015-02-28 00:31:45.548 3011 TRACE neutron.agent.dhcp.agent     getattr(driver, action)(**action_kwargs)
  2015-02-28 00:31:45.548 3011 TRACE neutron.agent.dhcp.agent   File "/opt/stack/neutron/neutron/agent/linux/dhcp.py", line 132, in restart
  2015-02-28 00:31:45.548 3011 TRACE neutron.agent.dhcp.agent     self.enable()
  2015-02-28 00:31:45.548 3011 TRACE neutron.agent.dhcp.agent   File "/opt/stack/neutron/neutron/agent/linux/dhcp.py", line 205, in enable
  2015-02-28 00:31:45.548 3011 TRACE neutron.agent.dhcp.agent     interface_name = self.device_manager.setup(self.network)
  2015-02-28 00:31:45.548 3011 TRACE neutron.agent.dhcp.agent   File "/opt/stack/neutron/neutron/agent/linux/dhcp.py", line 919, in setup
  2015-02-28 00:31:45.548 3011 TRACE neutron.agent.dhcp.agent     port = self.setup_dhcp_port(network)
  2015-02-28 00:31:45.548 3011 TRACE neutron.agent.dhcp.agent   File "/opt/stack/neutron/neutron/agent/linux/dhcp.py", line 863, in setup_dhcp_port
  2015-02-28 00:31:45.548 3011 TRACE neutron.agent.dhcp.agent     'fixed_ips': port_fixed_ips}})
  2015-02-28 00:31:45.548 3011 TRACE neutron.agent.dhcp.agent   File "/opt/stack/neutron/neutron/agent/dhcp/agent.py", line 441, in update_dhcp_port
  2015-02-28 00:31:45.548 3011 TRACE neutron.agent.dhcp.agent     port_id=port_id, port=port, host=self.host)
  2015-02-28 00:31:45.548 3011 TRACE neutron.agent.dhcp.agent   File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/client.py", line 156, in call
  2015-02-28 00:31:45.548 3011 TRACE neutron.agent.dhcp.agent     retry=self.retry)
  2015-02-28 00:31:45.548 3011 TRACE neutron.agent.dhcp.agent   File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/transport.py", line 90, in _send
  2015-02-28 00:31:45.548 3011 TRACE neutron.agent.dhcp.agent     timeout=timeout, retry=retry)
  2015-02-28 00:31:45.548 3011 TRACE neutron.agent.dhcp.agent   File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 349, in send
  2015-02-28 00:31:45.548 3011 TRACE neutron.agent.dhcp.agent     retry=retry)
  2015-02-28 00:31:45.548 3011 TRACE neutron.agent.dhcp.agent   File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 340, in _send
  2015-02-28 00:31:45.548 3011 TRACE neutron.agent.dhcp.agent     raise result
  2015-02-28 00:31:45.548 3011 TRACE neutron.agent.dhcp.agent RemoteError: Remote error: InvalidInput Invalid input for operation: Exceeded maximim amount of fixed ips per port.
  2015-02-28 00:31:45.548 3011 TRACE neutron.agent.dhcp.agent [u'Traceback (most recent call last):\n', u'  File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 142, in _dispatch_and_reply\n    executor_callback))\n', u'  File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 186, in _dispatch\n    executor_callback)\n', u'  File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 130, in _do_dispatch\n    result = func(ctxt, **new_args)\n', u'  File "/opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py", line 312, in update_dhcp_port\n    return self._port_action(plugin, context, port, \'update_port\')\n', u'  File "/opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py", line 75, in _port_action\n    return plugin.update_port(context, port[\'id\'], port)\n', u'  File "/opt/stack/neutron/neutron/plugins/ml2/plugin.py", line 1014, in update_port\n    port)\n', u'  File "/opt/stack/neutron/neutron/db/db_base_plugin_v2.py", line 1389, in update_port\n    original[\'mac_address\'], port[\'device_owner\'])\n', u'  File "/opt/stack/neutron/neutron/db/db_base_plugin_v2.py", line 466, in _update_ips_for_port\n    raise n_exc.InvalidInput(error_message=msg)\n', u'InvalidInput: Invalid input for operation: Exceeded maximim amount of fixed ips per port.\n'].
  2015-02-28 00:31:45.548 3011 TRACE neutron.agent.dhcp.agent
  2015-02-28 00:31:45.553 DEBUG oslo_concurrency.lockutils [req-41e2c225-2f9f-4e82-a18e-c79faf13cc49 admin admin] Lock "dhcp-agent" released by "subnet_update_end" :: held 0.358s inner /usr/local/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py:442
  2015-02-28 00:31:45.732 3011 DEBUG neutron.agent.dhcp.agent [-] resync (b682f8e6-5250-4c8c-bb83-93427cfd6185): [RemoteError(u'Remote error: InvalidInput Invalid input for operation: Exceeded maximim amount of fixed ips per port.\n[u\'Traceback (most recent call last):\\n\', u\'  File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 142, in _dispatch_and_reply\\n    executor_callback))\\n\', u\'  File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 186, in _dispatch\\n    executor_callback)\\n\', u\'  File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 130, in _do_dispatch\\n    result = func(ctxt, **new_args)\\n\', u\'  File "/opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py", line 312, in update_dhcp_port\\n    return self._port_action(plugin, context, port, \\\'update_port\\\')\\n\', u\'  File "/opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py", line 75, in _port_action\\n    return plugin.update_port(context, port[\\\'id\\\'], port)\\n\', u\'  File "/opt/stack/neutron/neutron/plugins/ml2/plugin.py", line 1014, in update_port\\n    port)\\n\', u\'  File "/opt/stack/neutron/neutron/db/db_base_plugin_v2.py", line 1389, in update_port\\n    original[\\\'mac_address\\\'], port[\\\'device_owner\\\'])\\n\', u\'  File "/opt/stack/neutron/neutron/db/db_base_plugin_v2.py", line 466, in _update_ips_for_port\\n    raise n_exc.InvalidInput(error_message=msg)\\n\', u\'InvalidInput: Invalid input for operation: Exceeded maximim amount of fixed ips per port.\\n\'].',)] _periodic_resync_helper /opt/stack/neutron/neutron/agent/dhcp/agent.py:185
  2015-02-28 00:31:45.733 3011 DEBUG oslo_concurrency.lockutils [-] Lock "dhcp-agent" acquired by "sync_state" :: waited 0.000s inner /usr/local/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py:430
  2015-02-28 00:31:45.733 3011 INFO neutron.agent.dhcp.agent [-] Synchronizing state
  2015-02-28 00:31:45.734 3011 DEBUG oslo_messaging._drivers.amqpdriver [-] MSG_ID is 81477b08ea9f4328bafe8f90ef2d3f33 _send /usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py:310
  2015-02-28 00:31:45.735 3011 DEBUG oslo_messaging._drivers.amqp [-] UNIQUE_ID is 0f306089fd55406bba1b5e7af7c489ce. _add_unique_id /usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqp.py:226
  2015-02-28 00:31:45.861 3011 DEBUG neutron.agent.dhcp.agent [-] Calling driver for network: b682f8e6-5250-4c8c-bb83-93427cfd6185 action: enable call_driver /opt/stack/neutron/neutron/agent/dhcp/agent.py:103
  2015-02-28 00:31:45.862 3011 DEBUG neutron.agent.linux.utils [-] Unable to access /opt/stack/data/neutron/dhcp/b682f8e6-5250-4c8c-bb83-93427cfd6185/pid get_value_from_file /opt/stack/neutron/neutron/agent/linux/utils.py:171
  2015-02-28 00:31:45.862 3011 DEBUG oslo_messaging._drivers.amqpdriver [-] MSG_ID is 2f0b3ab027e74f819d7969701ee4a414 _send /usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py:310
  2015-02-28 00:31:45.862 3011 DEBUG oslo_messaging._drivers.amqp [-] UNIQUE_ID is 8008ae78501a4269990536eb149cc6b7. _add_unique_id /usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqp.py:226
  2015-02-28 00:31:45.891 3011 ERROR neutron.agent.dhcp.agent [-] Unable to enable dhcp for b682f8e6-5250-4c8c-bb83-93427cfd6185.

  ==========================[TEMPEST]============================

  DHCP port 'technically' is not a tenant resource. So, the operations of DHCP port from both of admin and non-admin user shouldn't been restricted. Moreover, the best solution here is the non-admin user shouldn’t has permissions to operate with DHCP port.
  Currently, the operations of DHCP port from both of admin and non-admin user are restricted by "max_fixed_ips_per_port" parameter in neutron.conf file or it's fixed to 5 by default.
  https://github.com/openstack/neutron/blob/master/etc/neutron.conf#L151
  On Tempest, all test cases regarding to the operations of DHCP port are ignored or tested for the cases that:
  + Both of admin and non-admin user have permissions to perform the operations with DHCP port.
  + The number of ips per DHCP port is restricted.
  For example, please refer the below test case:
  https://github.com/openstack/tempest/blob/master/tempest/api/network/test_ports.py#L55
  >>> In this test case, we can see that Non-admin user has permissions to create/update/delete port but does NOT care about whether that’s DHCP port or not.

  So now, we need to change/add test cases relate to DHCP port operations as the following:
  + Admin user has permissions to get/create/update/delete DHCP port and no restricted.
  + Non-admin user has permissions to get/delete but cannot create/update DHCP port.

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1427015/+subscriptions
References

[Bug 1427015] [NEW] too many subnet-create cause q-dhcp failure
From: watanabe.isao, 2015-03-02