← Back to team overview

yahoo-eng-team team mailing list archive

  1. yahoo-eng-team team
  2. Mailing list archive
  3. Message #81956

[Bug 1867936] [NEW] test_update_delete_extra_route failing due to timeout when creating subnets

  Thread Previous • Date Previous • • Thread Next 
Public bug reported:

tempest.api.network.test_routers.RoutersTest.test_update_delete_extra_route
is failing due to a http timeout when creating multiple subnets. AFAICT
a request to the dhcp-agent appears to get stuck causing the overall
timeout.

https://efb4199e6af2b2032769-ac03866ed76044a5727f648521330347.ssl.cf5.rackcdn.com/696834/9/check
/tempest-integrated-compute/1796646/testr_results.html

Traceback (most recent call last):
  File "/opt/stack/tempest/tempest/common/utils/__init__.py", line 108, in wrapper
    return func(*func_args, **func_kwargs)
  File "/opt/stack/tempest/tempest/api/network/test_routers.py", line 202, in test_update_delete_extra_route
    subnet = self.create_subnet(network, cidr=next_cidr)
  File "/opt/stack/tempest/tempest/api/network/base.py", line 142, in create_subnet
    **kwargs)
  File "/opt/stack/tempest/tempest/lib/services/network/subnets_client.py", line 27, in create_subnet
    return self.create_resource(uri, post_data)
  File "/opt/stack/tempest/tempest/lib/services/network/base.py", line 61, in create_resource
    resp, body = self.post(req_uri, req_post_data)
  File "/opt/stack/tempest/tempest/lib/common/rest_client.py", line 283, in post
    return self.request('POST', url, extra_headers, headers, body, chunked)
  File "/opt/stack/tempest/tempest/lib/common/rest_client.py", line 672, in request
    body=body, chunked=chunked)
  File "/opt/stack/tempest/tempest/lib/common/rest_client.py", line 552, in _request
    chunked=chunked
  File "/opt/stack/tempest/tempest/lib/common/rest_client.py", line 592, in raw_request
    body=body, chunked=chunked)
  File "/opt/stack/tempest/tempest/lib/common/http.py", line 112, in request
    *args, **new_kwargs)
  File "/opt/stack/tempest/.tox/tempest/lib/python3.6/site-packages/urllib3/request.py", line 80, in request
    method, url, fields=fields, headers=headers, **urlopen_kw
  File "/opt/stack/tempest/.tox/tempest/lib/python3.6/site-packages/urllib3/request.py", line 171, in request_encode_body
    return self.urlopen(method, url, **extra_kw)
  File "/opt/stack/tempest/.tox/tempest/lib/python3.6/site-packages/urllib3/poolmanager.py", line 330, in urlopen
    response = conn.urlopen(method, u.request_uri, **kw)
  File "/opt/stack/tempest/.tox/tempest/lib/python3.6/site-packages/urllib3/connectionpool.py", line 720, in urlopen
    method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
  File "/opt/stack/tempest/.tox/tempest/lib/python3.6/site-packages/urllib3/util/retry.py", line 400, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/opt/stack/tempest/.tox/tempest/lib/python3.6/site-packages/urllib3/packages/six.py", line 735, in reraise
    raise value
  File "/opt/stack/tempest/.tox/tempest/lib/python3.6/site-packages/urllib3/connectionpool.py", line 672, in urlopen
    chunked=chunked,
  File "/opt/stack/tempest/.tox/tempest/lib/python3.6/site-packages/urllib3/connectionpool.py", line 423, in _make_request
    self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
  File "/opt/stack/tempest/.tox/tempest/lib/python3.6/site-packages/urllib3/connectionpool.py", line 331, in _raise_timeout
    self, url, "Read timed out. (read timeout=%s)" % timeout_value
urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='198.72.124.39', port=9696): Read timed out. (read timeout=60)

https://zuul.opendev.org/t/openstack/build/1796646647854d128ba128b45d5c7a14/log/controller/logs/screen-q-svc.txt

80868 Mar 17 11:45:25.433123 ubuntu-bionic-inap-mtl01-0015265350 neutron-server[6406]: DEBUG neutron.api.v2.base [None req-223326d8-28e2-42ab-894f-377fd49092b5 tempest-RoutersTest-1688904247 tempest-RoutersTest-1688904247] Request body: {'subnet': {'network_id': '1ef7ad5b-f937-43f9-a435-7721df24ae61', 'cidr': '10.1.16      .0/28', 'ip_version': 4, 'gateway_ip': '10.1.16.1'}} {{(pid=6783) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:719}}
[..]
80922 Mar 17 11:45:43.117787 ubuntu-bionic-inap-mtl01-0015265350 neutron-server[6406]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-223326d8-28e2-42ab-894f-377fd49092b5 tempest-RoutersTest-1688904247 tempest-RoutersTest-1688904247] Network 1ef7ad5b-f937-43f9-a435-7721df24ae61 is scheduled to be hosted by DHC      P agent 0510cea8-b218-496b-8272-000af85039a2 with binding_index 1 {{(pid=6783) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:246}}
[..]
80945 Mar 17 11:46:32.421880 ubuntu-bionic-inap-mtl01-0015265350 neutron-server[6406]: INFO neutron.wsgi [None req-223326d8-28e2-42ab-894f-377fd49092b5 tempest-RoutersTest-1688904247 tempest-RoutersTest-1688904247] 198.72.124.39,198.72.124.39 "POST /v2.0/subnets HTTP/1.1" status: 201  len: 802 time: 66.9933891

** Affects: neutron
     Importance: Undecided
         Status: New

** Summary changed:

- test_update_delete_extra_route failing due to timeout when creating subnet
+ test_update_delete_extra_route failing due to timeout when creating subnets

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1867936

Title:
  test_update_delete_extra_route failing due to timeout when creating
  subnets

Status in neutron:
  New

Bug description:
  tempest.api.network.test_routers.RoutersTest.test_update_delete_extra_route
  is failing due to a http timeout when creating multiple subnets.
  AFAICT a request to the dhcp-agent appears to get stuck causing the
  overall timeout.

  https://efb4199e6af2b2032769-ac03866ed76044a5727f648521330347.ssl.cf5.rackcdn.com/696834/9/check
  /tempest-integrated-compute/1796646/testr_results.html

  Traceback (most recent call last):
    File "/opt/stack/tempest/tempest/common/utils/__init__.py", line 108, in wrapper
      return func(*func_args, **func_kwargs)
    File "/opt/stack/tempest/tempest/api/network/test_routers.py", line 202, in test_update_delete_extra_route
      subnet = self.create_subnet(network, cidr=next_cidr)
    File "/opt/stack/tempest/tempest/api/network/base.py", line 142, in create_subnet
      **kwargs)
    File "/opt/stack/tempest/tempest/lib/services/network/subnets_client.py", line 27, in create_subnet
      return self.create_resource(uri, post_data)
    File "/opt/stack/tempest/tempest/lib/services/network/base.py", line 61, in create_resource
      resp, body = self.post(req_uri, req_post_data)
    File "/opt/stack/tempest/tempest/lib/common/rest_client.py", line 283, in post
      return self.request('POST', url, extra_headers, headers, body, chunked)
    File "/opt/stack/tempest/tempest/lib/common/rest_client.py", line 672, in request
      body=body, chunked=chunked)
    File "/opt/stack/tempest/tempest/lib/common/rest_client.py", line 552, in _request
      chunked=chunked
    File "/opt/stack/tempest/tempest/lib/common/rest_client.py", line 592, in raw_request
      body=body, chunked=chunked)
    File "/opt/stack/tempest/tempest/lib/common/http.py", line 112, in request
      *args, **new_kwargs)
    File "/opt/stack/tempest/.tox/tempest/lib/python3.6/site-packages/urllib3/request.py", line 80, in request
      method, url, fields=fields, headers=headers, **urlopen_kw
    File "/opt/stack/tempest/.tox/tempest/lib/python3.6/site-packages/urllib3/request.py", line 171, in request_encode_body
      return self.urlopen(method, url, **extra_kw)
    File "/opt/stack/tempest/.tox/tempest/lib/python3.6/site-packages/urllib3/poolmanager.py", line 330, in urlopen
      response = conn.urlopen(method, u.request_uri, **kw)
    File "/opt/stack/tempest/.tox/tempest/lib/python3.6/site-packages/urllib3/connectionpool.py", line 720, in urlopen
      method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
    File "/opt/stack/tempest/.tox/tempest/lib/python3.6/site-packages/urllib3/util/retry.py", line 400, in increment
      raise six.reraise(type(error), error, _stacktrace)
    File "/opt/stack/tempest/.tox/tempest/lib/python3.6/site-packages/urllib3/packages/six.py", line 735, in reraise
      raise value
    File "/opt/stack/tempest/.tox/tempest/lib/python3.6/site-packages/urllib3/connectionpool.py", line 672, in urlopen
      chunked=chunked,
    File "/opt/stack/tempest/.tox/tempest/lib/python3.6/site-packages/urllib3/connectionpool.py", line 423, in _make_request
      self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
    File "/opt/stack/tempest/.tox/tempest/lib/python3.6/site-packages/urllib3/connectionpool.py", line 331, in _raise_timeout
      self, url, "Read timed out. (read timeout=%s)" % timeout_value
  urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='198.72.124.39', port=9696): Read timed out. (read timeout=60)

  https://zuul.opendev.org/t/openstack/build/1796646647854d128ba128b45d5c7a14/log/controller/logs/screen-q-svc.txt

  80868 Mar 17 11:45:25.433123 ubuntu-bionic-inap-mtl01-0015265350 neutron-server[6406]: DEBUG neutron.api.v2.base [None req-223326d8-28e2-42ab-894f-377fd49092b5 tempest-RoutersTest-1688904247 tempest-RoutersTest-1688904247] Request body: {'subnet': {'network_id': '1ef7ad5b-f937-43f9-a435-7721df24ae61', 'cidr': '10.1.16      .0/28', 'ip_version': 4, 'gateway_ip': '10.1.16.1'}} {{(pid=6783) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:719}}
  [..]
  80922 Mar 17 11:45:43.117787 ubuntu-bionic-inap-mtl01-0015265350 neutron-server[6406]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-223326d8-28e2-42ab-894f-377fd49092b5 tempest-RoutersTest-1688904247 tempest-RoutersTest-1688904247] Network 1ef7ad5b-f937-43f9-a435-7721df24ae61 is scheduled to be hosted by DHC      P agent 0510cea8-b218-496b-8272-000af85039a2 with binding_index 1 {{(pid=6783) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:246}}
  [..]
  80945 Mar 17 11:46:32.421880 ubuntu-bionic-inap-mtl01-0015265350 neutron-server[6406]: INFO neutron.wsgi [None req-223326d8-28e2-42ab-894f-377fd49092b5 tempest-RoutersTest-1688904247 tempest-RoutersTest-1688904247] 198.72.124.39,198.72.124.39 "POST /v2.0/subnets HTTP/1.1" status: 201  len: 802 time: 66.9933891

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1867936/+subscriptions

Follow ups

  Thread Previous • Date Previous • • Thread Next