← Back to team overview

yahoo-eng-team team mailing list archive

  1. yahoo-eng-team team
  2. Mailing list archive
  3. Message #53227

[Bug 1192287] Re: Creating server did not fail when exceeded Quantum quota limit

  Thread Previous • Date Previous • • Thread Next 
This is an automated cleanup. This bug report has been closed because it
is older than 18 months and there is no open code change to fix this.
After this time it is unlikely that the circumstances which lead to
the observed issue can be reproduced.

If you can reproduce the bug, please:
* reopen the bug report (set to status "New")
* AND add the detailed steps to reproduce the issue (if applicable)
* AND leave a comment "CONFIRMED FOR: <RELEASE_NAME>"
  Only still supported release names are valid (LIBERTY, MITAKA, OCATA, NEWTON).
  Valid example: CONFIRMED FOR: LIBERTY


** Changed in: nova
       Status: Confirmed => Expired

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1192287

Title:
  Creating server did not fail when exceeded Quantum quota limit

Status in OpenStack Compute (nova):
  Expired

Bug description:
  I using the released Grizzly bits 2013.1.1 with one controller node,
  one quantum node, and 4 compute nodes.

  I encountered the issue where the Quantum port quota was exceeded, but
  the nova create instance did not return error condition for status. It
  appear to be bouncing the request to the 4 compute node where 3 of the
  nodes responded with quantum quota exceeded. Shouldn't the create
  request fail immediately on the first failure? Or from the log file,
  shouldn't it fail after 3 attempt?

  Portion of my test log showing create instances:

  Date                 Time                                                       Test server name                                 task                status
  ==========  =======                                                ==========================  =========   =====
  [2013-06-17][13:07:09][A:0,E:0] Testing server testserver-20130617-130707-25 - networking, BUILD
  [2013-06-17][13:07:15][A:0,E:0] Testing server testserver-20130617-130707-25 - None, ACTIVE
  [2013-06-17][13:07:16][A:0,E:0] Waiting for server testserver-20130617-130715-26 with ID f6fb6968-82e7-4b36-87cc-c18eba189f71 to get spawned
  [2013-06-17][13:07:18][A:0,E:0] Testing server testserver-20130617-130715-26 - networking, BUILD
  [2013-06-17][13:07:24][A:0,E:0] Testing server testserver-20130617-130715-26 - None, ACTIVE
  [2013-06-17][13:07:25][A:0,E:0] Waiting for server testserver-20130617-130724-27 with ID 4ed9a635-6fc4-4d4b-9a64-8dfd4574fddd to get spawned
  [2013-06-17][13:07:26][A:0,E:0] Testing server testserver-20130617-130724-27 - networking, BUILD
  [2013-06-17][13:07:32][A:0,E:0] Testing server testserver-20130617-130724-27 - scheduling, BUILD
  [2013-06-17][13:07:37][A:0,E:0] Testing server testserver-20130617-130724-27 - scheduling, BUILD
  [2013-06-17][13:07:43][A:0,E:0] Testing server testserver-20130617-130724-27 - scheduling, BUILD
  [2013-06-17][13:07:49][A:0,E:0] Testing server testserver-20130617-130724-27 - scheduling, BUILD
  [2013-06-17][13:07:55][A:0,E:0] Testing server testserver-20130617-130724-27 - scheduling, BUILD
  [2013-06-17][13:08:01][A:0,E:0] Testing server testserver-20130617-130724-27 - scheduling, BUILD
  [2013-06-17][13:08:07][A:0,E:0] Testing server testserver-20130617-130724-27 - scheduling, BUILD
  [2013-06-17][13:08:13][A:0,E:0] Testing server testserver-20130617-130724-27 - scheduling, BUILD
  [2013-06-17][13:08:19][A:0,E:0] Testing server testserver-20130617-130724-27 - scheduling, BUILD
  [2013-06-17][13:08:25][A:0,E:0] Testing server testserver-20130617-130724-27 - scheduling, BUILD
  [2013-06-17][13:08:31][A:0,E:0] Testing server testserver-20130617-130724-27 - scheduling, BUILD
  [2013-06-17][13:08:36][A:0,E:0] Testing server testserver-20130617-130724-27 - scheduling, BUILD
  [2013-06-17][13:08:42][A:0,E:0] Testing server testserver-20130617-130724-27 - scheduling, BUILD
  [2013-06-17][13:08:48][A:0,E:0] Testing server testserver-20130617-130724-27 - scheduling, BUILD
  [2013-06-17][13:08:54][A:0,E:0] Testing server testserver-20130617-130724-27 - scheduling, BUILD
  [2013-06-17][13:09:00][A:0,E:0] Testing server testserver-20130617-130724-27 - scheduling, BUILD

  
  nova-scheduler.log
  2013-06-17 13:07:26.287 ERROR nova.scheduler.filter_scheduler [req-816e9730-fe41-493a-93c5-ca79237705eb 6891ad89906646e4934ae6014eb07521 29ca96af08984e9288ec3f98a743197a] [instance: 4ed9a635-6fc4-4d4b-9a64-8dfd4574fddd] Error from last host: cld5b3 (node cld5b3.casl.adapps.hp.com): [u'Traceback (most recent call last):\n', u'  File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 831, in _run_instance\n    requested_networks, macs, security_groups)\n', u'  File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 1075, in _allocate_network\n    instance=instance)\n', u'  File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__\n    self.gen.next()\n', u'  File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 1071, in _allocate_network\n    security_groups=security_groups)\n', u'  File "/usr/lib/python2.7/dist-packages/nova/network/api.py", line 46, in wrapper\n    res = f(self, context, *args, **kwargs)\n', u'  File "/usr/lib/python2.7/dist-packages/nova/network/quantumv2/api.py", line 281, in allocate_for_instance\n    \'exception\': ex})\n', u'  File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__\n    self.gen.next()\n', u'  File "/usr/lib/python2.7/dist-packages/nova/network/quantumv2/api.py", line 264, in allocate_for_instance\n    quantum.create_port(port_req_body)[\'port\'][\'id\'])\n', u'  File "/usr/lib/python2.7/dist-packages/quantumclient/v2_0/client.py", line 107, in with_params\n    ret = self.function(instance, *args, **kwargs)\n', u'  File "/usr/lib/python2.7/dist-packages/quantumclient/v2_0/client.py", line 269, in create_port\n    return self.post(self.ports_path, body=body)\n', u'  File "/usr/lib/python2.7/dist-packages/quantumclient/v2_0/client.py", line 987, in post\n    headers=headers, params=params)\n', u'  File "/usr/lib/python2.7/dist-packages/quantumclient/v2_0/client.py", line 912, in do_request\n    self._handle_fault_response(status_code, replybody)\n', u'  File "/usr/lib/python2.7/dist-packages/quantumclient/v2_0/client.py", line 893, in _handle_fault_response\n    exception_handler_v20(status_code, des_error_body)\n', u'  File "/usr/lib/python2.7/dist-packages/quantumclient/v2_0/client.py", line 80, in exception_handler_v20\n    message=error_dict)\n', u"QuantumClientException: Quota exceeded for resources: ['port']\n"]
  2013-06-17 13:07:28.584 ERROR nova.scheduler.filter_scheduler [req-816e9730-fe41-493a-93c5-ca79237705eb 6891ad89906646e4934ae6014eb07521 29ca96af08984e9288ec3f98a743197a] [instance: 4ed9a635-6fc4-4d4b-9a64-8dfd4574fddd] Error from last host: cld5b5 (node cld5b5.casl.adapps.hp.com): [u'Traceback (most recent call last):\n', u'  File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 831, in _run_instance\n    requested_networks, macs, security_groups)\n', u'  File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 1075, in _allocate_network\n    instance=instance)\n', u'  File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__\n    self.gen.next()\n', u'  File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 1071, in _allocate_network\n    security_groups=security_groups)\n', u'  File "/usr/lib/python2.7/dist-packages/nova/network/api.py", line 46, in wrapper\n    res = f(self, context, *args, **kwargs)\n', u'  File "/usr/lib/python2.7/dist-packages/nova/network/quantumv2/api.py", line 281, in allocate_for_instance\n    \'exception\': ex})\n', u'  File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__\n    self.gen.next()\n', u'  File "/usr/lib/python2.7/dist-packages/nova/network/quantumv2/api.py", line 264, in allocate_for_instance\n    quantum.create_port(port_req_body)[\'port\'][\'id\'])\n', u'  File "/usr/lib/python2.7/dist-packages/quantumclient/v2_0/client.py", line 107, in with_params\n    ret = self.function(instance, *args, **kwargs)\n', u'  File "/usr/lib/python2.7/dist-packages/quantumclient/v2_0/client.py", line 269, in create_port\n    return self.post(self.ports_path, body=body)\n', u'  File "/usr/lib/python2.7/dist-packages/quantumclient/v2_0/client.py", line 987, in post\n    headers=headers, params=params)\n', u'  File "/usr/lib/python2.7/dist-packages/quantumclient/v2_0/client.py", line 912, in do_request\n    self._handle_fault_response(status_code, replybody)\n', u'  File "/usr/lib/python2.7/dist-packages/quantumclient/v2_0/client.py", line 893, in _handle_fault_response\n    exception_handler_v20(status_code, des_error_body)\n', u'  File "/usr/lib/python2.7/dist-packages/quantumclient/v2_0/client.py", line 80, in exception_handler_v20\n    message=error_dict)\n', u"QuantumClientException: Quota exceeded for resources: ['port']\n"]
  2013-06-17 13:07:30.398 ERROR nova.scheduler.filter_scheduler [req-816e9730-fe41-493a-93c5-ca79237705eb 6891ad89906646e4934ae6014eb07521 29ca96af08984e9288ec3f98a743197a] [instance: 4ed9a635-6fc4-4d4b-9a64-8dfd4574fddd] Error from last host: cld5b4 (node cld5b4.casl.adapps.hp.com): [u'Traceback (most recent call last):\n', u'  File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 831, in _run_instance\n    requested_networks, macs, security_groups)\n', u'  File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 1075, in _allocate_network\n    instance=instance)\n', u'  File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__\n    self.gen.next()\n', u'  File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 1071, in _allocate_network\n    security_groups=security_groups)\n', u'  File "/usr/lib/python2.7/dist-packages/nova/network/api.py", line 46, in wrapper\n    res = f(self, context, *args, **kwargs)\n', u'  File "/usr/lib/python2.7/dist-packages/nova/network/quantumv2/api.py", line 281, in allocate_for_instance\n    \'exception\': ex})\n', u'  File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__\n    self.gen.next()\n', u'  File "/usr/lib/python2.7/dist-packages/nova/network/quantumv2/api.py", line 264, in allocate_for_instance\n    quantum.create_port(port_req_body)[\'port\'][\'id\'])\n', u'  File "/usr/lib/python2.7/dist-packages/quantumclient/v2_0/client.py", line 107, in with_params\n    ret = self.function(instance, *args, **kwargs)\n', u'  File "/usr/lib/python2.7/dist-packages/quantumclient/v2_0/client.py", line 269, in create_port\n    return self.post(self.ports_path, body=body)\n', u'  File "/usr/lib/python2.7/dist-packages/quantumclient/v2_0/client.py", line 987, in post\n    headers=headers, params=params)\n', u'  File "/usr/lib/python2.7/dist-packages/quantumclient/v2_0/client.py", line 912, in do_request\n    self._handle_fault_response(status_code, replybody)\n', u'  File "/usr/lib/python2.7/dist-packages/quantumclient/v2_0/client.py", line 893, in _handle_fault_response\n    exception_handler_v20(status_code, des_error_body)\n', u'  File "/usr/lib/python2.7/dist-packages/quantumclient/v2_0/client.py", line 80, in exception_handler_v20\n    message=error_dict)\n', u"QuantumClientException: Quota exceeded for resources: ['port']\n"]
  2013-06-17 13:07:30.399 WARNING nova.scheduler.manager [req-816e9730-fe41-493a-93c5-ca79237705eb 6891ad89906646e4934ae6014eb07521 29ca96af08984e9288ec3f98a743197a] Failed to schedule_run_instance: No valid host was found. Exceeded max scheduling attempts 3 for instance 4ed9a635-6fc4-4d4b-9a64-8dfd4574fddd

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1192287/+subscriptions
  Thread Previous • Date Previous • • Thread Next