yahoo-eng-team team mailing list archive
-
yahoo-eng-team team
-
Mailing list archive
-
Message #71269
[Bug 1750337] Re: Fullstack tests fail due to "block_until_boot" timeout
Reviewed: https://review.openstack.org/546069
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=465ad6f3197b8591a401a9f0db2fabf6c70fdfce
Submitter: Zuul
Branch: master
commit 465ad6f3197b8591a401a9f0db2fabf6c70fdfce
Author: Sławek Kapłoński <slawek@xxxxxxxxxxxx>
Date: Tue Feb 20 09:24:38 2018 +0100
[Fullstack] Limit number of Neutron's api workers
Default number of api workers in Neutron is set to be equal to
number of CPU cores on host. That is fine on production environment
but on fullstack tests, where each test spawns own neutron-server
process it might cause host overload.
This patch limits number of api_workers to 2 which should be enough
for single test case and should make significantly lower load on host.
Change-Id: I1e970e35883d5240f0bd30eaea50313d93900580
Closes-Bug: #1750337
** Changed in: neutron
Status: In Progress => Fix Released
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1750337
Title:
Fullstack tests fail due to "block_until_boot" timeout
Status in neutron:
Fix Released
Bug description:
Sometimes in tests like "neutron.tests.fullstack.test_connectivity.TestOvsConnectivitySameNetworkOnOvsBridgeControllerStop.test_controller_timeout_does_not_break_connectivity_sigterm(VLANs,openflow-native)" there is timeout error during waiting for all vms to be boot.
Example of such error can be checked e.g. on http://logs.openstack.org/81/545681/1/check/neutron-fullstack/8285bf3/logs/testr_results.html.gz
This example is done on patch with some additional logging added to debug tests. What is strange there is fact that test environment makes GET /v2.0/ports/{port_id} call properly: http://logs.openstack.org/81/545681/1/check/neutron-fullstack/8285bf3/logs/dsvm-fullstack-logs/TestOvsConnectivitySameNetworkOnOvsBridgeControllerStop.test_controller_timeout_does_not_break_connectivity_sigterm_VLANs,openflow-native_.txt.gz#_2018-02-18_20_34_47_950
but there is no this call logged in neutron-server logs. First GET call for this port in neutron-server logs is about 1m 30seconds later: http://logs.openstack.org/81/545681/1/check/neutron-fullstack/8285bf3/logs/dsvm-fullstack-logs/TestOvsConnectivitySameNetworkOnOvsBridgeControllerStop.test_controller_timeout_does_not_break_connectivity_sigterm_VLANs,openflow-native_/neutron-server--2018-02-18--20-31-43-830810.txt.gz#_2018-02-18_20_36_18_516 and this is already too late as test reached timeout and it is failed.
Above failed test run is just an example. I saw similar errors more
times than only this one.
To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1750337/+subscriptions
References