yahoo-eng-team team mailing list archive

[Ihar Hrachyshka <1741954@xxxxxxxxxxxxxxxxxx>]

Public bug reported:

This happened once in Pike.

http://logs.openstack.org/16/529616/1/check/legacy-rally-dsvm-neutron-
neutron/72b511a/rally-
plot/results.html.gz#/NeutronTrunks.create_and_list_trunk_subports/failures

Traceback (most recent call last):
  File "/opt/stack/new/rally/rally/task/runner.py", line 71, in _run_scenario_once
    getattr(scenario_inst, method_name)(**scenario_kwargs)
  File "/home/zuul/.rally/plugins/plugins/trunk_scenario.py", line 41, in run
    trunk = self._create_trunk(trunk_payload)
  File "/opt/stack/new/rally/rally/task/atomic.py", line 91, in func_atomic_actions
    f = func(self, *args, **kwargs)
  File "/home/zuul/.rally/plugins/plugins/trunk_scenario.py", line 53, in _create_trunk
    return self.clients("neutron").create_trunk({'trunk': trunk_payload})
  File "/usr/local/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 2067, in create_trunk
    return self.post(self.trunks_path, body=body)
  File "/usr/local/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 357, in post
    headers=headers, params=params)
  File "/usr/local/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 280, in do_request
    resp, replybody = self.httpclient.do_request(action, method, body=body)
  File "/usr/local/lib/python2.7/dist-packages/neutronclient/client.py", line 342, in do_request
    return self.request(url, method, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/neutronclient/client.py", line 330, in request
    resp = super(SessionClient, self).request(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/keystoneauth1/adapter.py", line 192, in request
    return self.session.request(url, method, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/positional/__init__.py", line 108, in inner
    return wrapped(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/keystoneauth1/session.py", line 703, in request
    resp = send(**kwargs)
  File "/usr/local/lib/python2.7/dist-packages/keystoneauth1/session.py", line 768, in _send_request
    raise exceptions.ConnectTimeout(msg)
ConnectTimeout: Request to http://104.239.168.166:9696/v2.0/trunks timed out


All four attempts failed the same way. Other scenarios worked just fine.

When you look at q-svc log, you see that all four POST requests for
trunks took a very long time, for example, req-b98d273b-
8c38-4d54-98f5-1d74008795d8 took 3:30 minutes to get from quotas check
to precommit_create notification:

http://logs.openstack.org/16/529616/1/check/legacy-rally-dsvm-neutron-
neutron/72b511a/logs/screen-q-svc.txt.gz?#_Jan_05_19_46_52_971085

The only major code between quota check and this notification is
validation of the request. Requests are, granted, heavy loaded, with 125
subports per trunk, so it may have something to do with it.

** Affects: neutron
     Importance: High
         Status: Confirmed


** Tags: gate-failure

** Tags added: gate-failure

** Changed in: neutron
   Importance: Undecided => High

** Changed in: neutron
       Status: New => Confirmed

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1741954

Title:
  create_and_list_trunk_subports rally scenario failed with timeouts

Status in neutron:
  Confirmed

Bug description:
  This happened once in Pike.

  http://logs.openstack.org/16/529616/1/check/legacy-rally-dsvm-neutron-
  neutron/72b511a/rally-
  plot/results.html.gz#/NeutronTrunks.create_and_list_trunk_subports/failures

  Traceback (most recent call last):
    File "/opt/stack/new/rally/rally/task/runner.py", line 71, in _run_scenario_once
      getattr(scenario_inst, method_name)(**scenario_kwargs)
    File "/home/zuul/.rally/plugins/plugins/trunk_scenario.py", line 41, in run
      trunk = self._create_trunk(trunk_payload)
    File "/opt/stack/new/rally/rally/task/atomic.py", line 91, in func_atomic_actions
      f = func(self, *args, **kwargs)
    File "/home/zuul/.rally/plugins/plugins/trunk_scenario.py", line 53, in _create_trunk
      return self.clients("neutron").create_trunk({'trunk': trunk_payload})
    File "/usr/local/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 2067, in create_trunk
      return self.post(self.trunks_path, body=body)
    File "/usr/local/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 357, in post
      headers=headers, params=params)
    File "/usr/local/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 280, in do_request
      resp, replybody = self.httpclient.do_request(action, method, body=body)
    File "/usr/local/lib/python2.7/dist-packages/neutronclient/client.py", line 342, in do_request
      return self.request(url, method, **kwargs)
    File "/usr/local/lib/python2.7/dist-packages/neutronclient/client.py", line 330, in request
      resp = super(SessionClient, self).request(*args, **kwargs)
    File "/usr/local/lib/python2.7/dist-packages/keystoneauth1/adapter.py", line 192, in request
      return self.session.request(url, method, **kwargs)
    File "/usr/local/lib/python2.7/dist-packages/positional/__init__.py", line 108, in inner
      return wrapped(*args, **kwargs)
    File "/usr/local/lib/python2.7/dist-packages/keystoneauth1/session.py", line 703, in request
      resp = send(**kwargs)
    File "/usr/local/lib/python2.7/dist-packages/keystoneauth1/session.py", line 768, in _send_request
      raise exceptions.ConnectTimeout(msg)
  ConnectTimeout: Request to http://104.239.168.166:9696/v2.0/trunks timed out

  
  All four attempts failed the same way. Other scenarios worked just fine.

  When you look at q-svc log, you see that all four POST requests for
  trunks took a very long time, for example, req-b98d273b-
  8c38-4d54-98f5-1d74008795d8 took 3:30 minutes to get from quotas check
  to precommit_create notification:

  http://logs.openstack.org/16/529616/1/check/legacy-rally-dsvm-neutron-
  neutron/72b511a/logs/screen-q-svc.txt.gz?#_Jan_05_19_46_52_971085

  The only major code between quota check and this notification is
  validation of the request. Requests are, granted, heavy loaded, with
  125 subports per trunk, so it may have something to do with it.

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1741954/+subscriptions