yahoo-eng-team team mailing list archive

[Rossella Sblendido <rsblendido@xxxxxxxx>]

You should modify the port quota and use something > 100 if you plan to launch 100 instances. Some ports are created automatically by Neutron and are included in the quota (like dhcp ports for example). 
Modify the port quota in horizon. Or you can use the command line,  specifying in the credential OS_USERNAME=admin_user, OS_TENANT_NAME= tenant_that_will_create_the_VMs 

neutron quota-update --port 120

** Changed in: neutron
       Status: New => Invalid

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1337275

Title:
  fail to launch more than 12 instances after updating qouta with
  'PortLimitExceeded: Maximum number of ports exceeded'

Status in OpenStack Neutron (virtual network service):
  Invalid

Bug description:
  I installed openstack with packstack as AIO + 3 computes. 
  Trying to run 100 instances, we fail to launch more than 12 with 'PortLimitExceeded: Maximum number of ports exceeded' ERROR. 

  to reproduce - launch 100 instances at once after changing admin
  tenant project default quota.

  attaching the answer file + logs but here is the ERROR from nova-
  compute.log

  2014-07-03 12:52:17.061 3045 TRACE nova.compute.manager [instance: bc4a3dcc-80a3-42ae-b2a1-8f4b852ce7f8] Traceback (most recent call last):
  2014-07-03 12:52:17.061 3045 TRACE nova.compute.manager [instance: bc4a3dcc-80a3-42ae-b2a1-8f4b852ce7f8]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1311, in _build_instance
  2014-07-03 12:52:17.061 3045 TRACE nova.compute.manager [instance: bc4a3dcc-80a3-42ae-b2a1-8f4b852ce7f8]     set_access_ip=set_access_ip)
  2014-07-03 12:52:17.061 3045 TRACE nova.compute.manager [instance: bc4a3dcc-80a3-42ae-b2a1-8f4b852ce7f8]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 399, in decorated_function
  2014-07-03 12:52:17.061 3045 TRACE nova.compute.manager [instance: bc4a3dcc-80a3-42ae-b2a1-8f4b852ce7f8]     return function(self, context, *args, **kwargs)
  2014-07-03 12:52:17.061 3045 TRACE nova.compute.manager [instance: bc4a3dcc-80a3-42ae-b2a1-8f4b852ce7f8]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1723, in _spawn
  2014-07-03 12:52:17.061 3045 TRACE nova.compute.manager [instance: bc4a3dcc-80a3-42ae-b2a1-8f4b852ce7f8]     LOG.exception(_('Instance failed to spawn'), instance=instance)
  2014-07-03 12:52:17.061 3045 TRACE nova.compute.manager [instance: bc4a3dcc-80a3-42ae-b2a1-8f4b852ce7f8]   File "/usr/lib/python2.7/site-packages/nova/openstack/common/excutils.py", line 68, in __exit__
  2014-07-03 12:52:17.061 3045 TRACE nova.compute.manager [instance: bc4a3dcc-80a3-42ae-b2a1-8f4b852ce7f8]     six.reraise(self.type_, self.value, self.tb)
  2014-07-03 12:52:17.061 3045 TRACE nova.compute.manager [instance: bc4a3dcc-80a3-42ae-b2a1-8f4b852ce7f8]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1720, in _spawn
  2014-07-03 12:52:17.061 3045 TRACE nova.compute.manager [instance: bc4a3dcc-80a3-42ae-b2a1-8f4b852ce7f8]     block_device_info)
  2014-07-03 12:52:17.061 3045 TRACE nova.compute.manager [instance: bc4a3dcc-80a3-42ae-b2a1-8f4b852ce7f8]   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 2253, in spawn
  2014-07-03 12:52:17.061 3045 TRACE nova.compute.manager [instance: bc4a3dcc-80a3-42ae-b2a1-8f4b852ce7f8]     admin_pass=admin_password)
  2014-07-03 12:52:17.061 3045 TRACE nova.compute.manager [instance: bc4a3dcc-80a3-42ae-b2a1-8f4b852ce7f8]   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 2704, in _create_image
  2014-07-03 12:52:17.061 3045 TRACE nova.compute.manager [instance: bc4a3dcc-80a3-42ae-b2a1-8f4b852ce7f8]     instance, network_info, admin_pass, files, suffix)
  2014-07-03 12:52:17.061 3045 TRACE nova.compute.manager [instance: bc4a3dcc-80a3-42ae-b2a1-8f4b852ce7f8]   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 2522, in _inject_data
  2014-07-03 12:52:17.061 3045 TRACE nova.compute.manager [instance: bc4a3dcc-80a3-42ae-b2a1-8f4b852ce7f8]     net = netutils.get_injected_network_template(network_info)
  2014-07-03 12:52:17.061 3045 TRACE nova.compute.manager [instance: bc4a3dcc-80a3-42ae-b2a1-8f4b852ce7f8]   File "/usr/lib/python2.7/site-packages/nova/virt/netutils.py", line 71, in get_injected_network_template
  2014-07-03 12:52:17.061 3045 TRACE nova.compute.manager [instance: bc4a3dcc-80a3-42ae-b2a1-8f4b852ce7f8]     if not (network_info and template):
  2014-07-03 12:52:17.061 3045 TRACE nova.compute.manager [instance: bc4a3dcc-80a3-42ae-b2a1-8f4b852ce7f8]   File "/usr/lib/python2.7/site-packages/nova/network/model.py", line 420, in __len__
  2014-07-03 12:52:17.061 3045 TRACE nova.compute.manager [instance: bc4a3dcc-80a3-42ae-b2a1-8f4b852ce7f8]     return self._sync_wrapper(fn, *args, **kwargs)
  2014-07-03 12:52:17.061 3045 TRACE nova.compute.manager [instance: bc4a3dcc-80a3-42ae-b2a1-8f4b852ce7f8]   File "/usr/lib/python2.7/site-packages/nova/network/model.py", line 407, in _sync_wrapper
  2014-07-03 12:52:17.061 3045 TRACE nova.compute.manager [instance: bc4a3dcc-80a3-42ae-b2a1-8f4b852ce7f8]     self.wait()
  2014-07-03 12:52:17.061 3045 TRACE nova.compute.manager [instance: bc4a3dcc-80a3-42ae-b2a1-8f4b852ce7f8]   File "/usr/lib/python2.7/site-packages/nova/network/model.py", line 439, in wait
  2014-07-03 12:52:17.061 3045 TRACE nova.compute.manager [instance: bc4a3dcc-80a3-42ae-b2a1-8f4b852ce7f8]     self[:] = self._gt.wait()
  2014-07-03 12:52:17.061 3045 TRACE nova.compute.manager [instance: bc4a3dcc-80a3-42ae-b2a1-8f4b852ce7f8]   File "/usr/lib/python2.7/site-packages/eventlet/greenthread.py", line 168, in wait
  2014-07-03 12:52:17.061 3045 TRACE nova.compute.manager [instance: bc4a3dcc-80a3-42ae-b2a1-8f4b852ce7f8]     return self._exit_event.wait()
  2014-07-03 12:52:17.061 3045 TRACE nova.compute.manager [instance: bc4a3dcc-80a3-42ae-b2a1-8f4b852ce7f8]   File "/usr/lib/python2.7/site-packages/eventlet/event.py", line 120, in wait
  2014-07-03 12:52:17.061 3045 TRACE nova.compute.manager [instance: bc4a3dcc-80a3-42ae-b2a1-8f4b852ce7f8]     current.throw(*self._exc)
  2014-07-03 12:52:17.061 3045 TRACE nova.compute.manager [instance: bc4a3dcc-80a3-42ae-b2a1-8f4b852ce7f8]   File "/usr/lib/python2.7/site-packages/eventlet/greenthread.py", line 194, in main
  2014-07-03 12:52:17.061 3045 TRACE nova.compute.manager [instance: bc4a3dcc-80a3-42ae-b2a1-8f4b852ce7f8]     result = function(*args, **kwargs)
  2014-07-03 12:52:17.061 3045 TRACE nova.compute.manager [instance: bc4a3dcc-80a3-42ae-b2a1-8f4b852ce7f8]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1510, in _allocate_network_async
  2014-07-03 12:52:17.061 3045 TRACE nova.compute.manager [instance: bc4a3dcc-80a3-42ae-b2a1-8f4b852ce7f8]     dhcp_options=dhcp_options)
  2014-07-03 12:52:17.061 3045 TRACE nova.compute.manager [instance: bc4a3dcc-80a3-42ae-b2a1-8f4b852ce7f8]   File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 361, in allocate_for_instance
  2014-07-03 12:52:17.061 3045 TRACE nova.compute.manager [instance: bc4a3dcc-80a3-42ae-b2a1-8f4b852ce7f8]     LOG.exception(msg, port_id)
  2014-07-03 12:52:17.061 3045 TRACE nova.compute.manager [instance: bc4a3dcc-80a3-42ae-b2a1-8f4b852ce7f8]   File "/usr/lib/python2.7/site-packages/nova/openstack/common/excutils.py", line 68, in __exit__
  2014-07-03 12:52:17.061 3045 TRACE nova.compute.manager [instance: bc4a3dcc-80a3-42ae-b2a1-8f4b852ce7f8]     six.reraise(self.type_, self.value, self.tb)
  2014-07-03 12:52:17.061 3045 TRACE nova.compute.manager [instance: bc4a3dcc-80a3-42ae-b2a1-8f4b852ce7f8]   File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 336, in allocate_for_instance
  2014-07-03 12:52:17.061 3045 TRACE nova.compute.manager [instance: bc4a3dcc-80a3-42ae-b2a1-8f4b852ce7f8]     security_group_ids, available_macs, dhcp_opts)
  2014-07-03 12:52:17.061 3045 TRACE nova.compute.manager [instance: bc4a3dcc-80a3-42ae-b2a1-8f4b852ce7f8]   File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 192, in _create_port
  2014-07-03 12:52:17.061 3045 TRACE nova.compute.manager [instance: bc4a3dcc-80a3-42ae-b2a1-8f4b852ce7f8]     raise exception.PortLimitExceeded()
  2014-07-03 12:52:17.061 3045 TRACE nova.compute.manager [instance: bc4a3dcc-80a3-42ae-b2a1-8f4b852ce7f8] PortLimitExceeded: Maximum number of ports exceeded
  2014-07-03 12:52:17.061 3045 TRACE nova.compute.manager [instance: bc4a3dcc-80a3-42ae-b2a1-8f4b852ce7f8] 
  2014-07-03 12:52:17.062 3045 DEBUG nova.compute.utils [req-fe69d609-e0a5-4f80-ad3c-d3958f274ff5 68e04b924e084a489984fdc63eb4157d 73df6022d9eb4158ac4d81f5d2fa6942] [instance: bc4a3dcc-80a3-42ae-b2a1-8f4b852ce7f8] Build of instance bc4a3dcc-80a3-42ae-b2a1-8f4b852ce7f8 was re-scheduled: Maximum number of ports exceeded notify_about_instance_usage /usr/lib/python2.7/site-packages/nova/compute/utils.py:336
  2014-07-03 12:52:17.062 3045 TRACE nova.compute.utils [instance: bc4a3dcc-80a3-42ae-b2a1-8f4b852ce7f8] Traceback (most recent call last):
  2014-07-03 12:52:17.062 3045 TRACE nova.compute.utils [instance: bc4a3dcc-80a3-42ae-b2a1-8f4b852ce7f8]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1191, in _run_instance
  2014-07-03 12:52:17.062 3045 TRACE nova.compute.utils [instance: bc4a3dcc-80a3-42ae-b2a1-8f4b852ce7f8]     instance, image_meta, legacy_bdm_in_spec)
  2014-07-03 12:52:17.062 3045 TRACE nova.compute.utils [instance: bc4a3dcc-80a3-42ae-b2a1-8f4b852ce7f8]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1361, in _build_instance
  2014-07-03 12:52:17.062 3045 TRACE nova.compute.utils [instance: bc4a3dcc-80a3-42ae-b2a1-8f4b852ce7f8]     reason=unicode(exc_info[1]))
  2014-07-03 12:52:17.062 3045 TRACE nova.compute.utils [instance: bc4a3dcc-80a3-42ae-b2a1-8f4b852ce7f8] RescheduledException: Build of instance bc4a3dcc-80a3-42ae-b2a1-8f4b852ce7f8 was re-scheduled: Maximum number of ports exceeded
  2014-07-03 12:52:17.062 3045 TRACE nova.compute.utils [instance: bc4a3dcc-80a3-42ae-b2a1-8f4b852ce7f8] 
  2014-07-03 12:52:17.065 3045 DEBUG nova.openstack.common.lockutils [req-fe69d609-e0a5-4f80-ad3c-d3958f274ff5 68e04b924e084a489984fdc63eb4157d 73df6022d9eb4158ac4d81f5d2fa6942] Semaphore / lock released "do_run_instance" inner /usr/lib/p:

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1337275/+subscriptions