yahoo-eng-team team mailing list archive

[Michael Semenov <msemenov@xxxxxxxxxxxx>]

** Changed in: nova
       Status: New => Invalid

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1597848

Title:
  Resource has ERROR status: Exceeded maximum number of retries

Status in OpenStack Compute (nova):
  Invalid

Bug description:
  Description
  ===========
  During 9.0RC2 Certification Testing Sutie run, the following error log appeared when
  executing "NovaSecGroup.boot_and_delete_server_with_secgroups" task:

  2016-06-28 01:58:17.619 30929 DEBUG novaclient.v2.client [-] GET call to compute for http://172.16.184.3:8774/v2.1/servers/e3a9f1fd-d7a9-43af-a911-903297545f4e used request id req-33956e34-3b21-4f97-a0ae-fc975f9cc6b5 _log_request_id /opt/stack/.venv/lib/python2.7/site-packages/novaclient/client.py:85
  2016-06-28 01:58:17.623 30929 ERROR rally.task.runner [-] Resource <Server: s_rally_02cfda0f_nOe66YVK> has ERROR status.
  Fault: {u'message': u'Exceeded maximum number of retries. Exceeded max scheduling attempts 3 for instance e3a9f1fd-d7a9-43af-a911-903297545f4e. Last exception: Binding failed for port c3c6e915-3395-4d6f-90bf-02a565da4146, please check neutron logs for more information.', u'code': 500, u'created': u'2016-06-28T01:58:16Z'}
  2016-06-28 01:58:17.623 30929 ERROR rally.task.runner Traceback (most recent call last):
  2016-06-28 01:58:17.623 30929 ERROR rally.task.runner   File "/opt/stack/.venv/lib/python2.7/site-packages/rally/task/runner.py", line 66, in _run_scenario_once
  2016-06-28 01:58:17.623 30929 ERROR rally.task.runner     deprecated_output = getattr(scenario_inst, method_name)(**kwargs)
  2016-06-28 01:58:17.623 30929 ERROR rally.task.runner   File "/opt/stack/.venv/lib/python2.7/site-packages/rally/plugins/openstack/scenarios/nova/security_group.py", line 137, in boot_and_delete_server_with_secgroups
  2016-06-28 01:58:17.623 30929 ERROR rally.task.runner     **kwargs)
  2016-06-28 01:58:17.623 30929 ERROR rally.task.runner   File "/opt/stack/.venv/lib/python2.7/site-packages/rally/task/atomic.py", line 84, in func_atomic_actions
  2016-06-28 01:58:17.623 30929 ERROR rally.task.runner     f = func(self, *args, **kwargs)
  2016-06-28 01:58:17.623 30929 ERROR rally.task.runner   File "/opt/stack/.venv/lib/python2.7/site-packages/rally/plugins/openstack/scenarios/nova/utils.py", line 147, in _boot_server
  2016-06-28 01:58:17.623 30929 ERROR rally.task.runner     check_interval=CONF.benchmark.nova_server_boot_poll_interval
  2016-06-28 01:58:17.623 30929 ERROR rally.task.runner   File "/opt/stack/.venv/lib/python2.7/site-packages/rally/common/logging.py", line 236, in wrapper
  2016-06-28 01:58:17.623 30929 ERROR rally.task.runner     return f(*args, **kwargs)
  2016-06-28 01:58:17.623 30929 ERROR rally.task.runner   File "/opt/stack/.venv/lib/python2.7/site-packages/rally/task/utils.py", line 147, in wait_for
  2016-06-28 01:58:17.623 30929 ERROR rally.task.runner     check_interval=check_interval)
  2016-06-28 01:58:17.623 30929 ERROR rally.task.runner   File "/opt/stack/.venv/lib/python2.7/site-packages/rally/task/utils.py", line 211, in wait_for_status
  2016-06-28 01:58:17.623 30929 ERROR rally.task.runner     resource = update_resource(resource)
  2016-06-28 01:58:17.623 30929 ERROR rally.task.runner   File "/opt/stack/.venv/lib/python2.7/site-packages/rally/task/utils.py", line 90, in _get_from_manager
  2016-06-28 01:58:17.623 30929 ERROR rally.task.runner     fault=getattr(res, "fault", "n/a"))
  2016-06-28 01:58:17.623 30929 ERROR rally.task.runner GetResourceErrorStatus: Resource <Server: s_rally_02cfda0f_nOe66YVK> has ERROR status.
  2016-06-28 01:58:17.623 30929 ERROR rally.task.runner Fault: {u'message': u'Exceeded maximum number of retries. Exceeded max scheduling attempts 3 for instance e3a9f1fd-d7a9-43af-a911-903297545f4e. Last exception: Binding failed for port c3c6e915-3395-4d6f-90bf-02a565da4146, please check neutron logs for more information.', u'code': 500, u'created': u'2016-06-28T01:58:16Z'}

  Steps to reproduce
  ==================
  Install Rally on masternode and run task "NovaSecGroup.boot_and_delete_server_with_secgroups":

  1. cd /opt/stack
  2. source .venv/bin/activate
  3. rally task start /opt/stack/rally-scenarios/nova/boot_and_delete_server_with_secgroups.yaml

  Expected result
  ===============
  Task run without errors

  Actual result
  =============
  Errors described above in logs

  Environment
  ===========
  Scale 200-node lab (ENV-10):
    3 controllers
    199 compute nodes
    20 ceph nodes
    DVR enabled
    XVLAN
  Image RC2 build:
    fuel-9.0-mos-495-2016-06-16_18-18-00.iso

  Diagnostic snapshot:
    http://mos-scale-share.mirantis.com/fuel-snapshot-2016-06-28_13-13-50.tar.gz

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1597848/+subscriptions