yahoo-eng-team team mailing list archive
-
yahoo-eng-team team
-
Mailing list archive
-
Message #82328
[Bug 1872693] Re: Timeout exception in scenario tests with advanced image
Reviewed: https://review.opendev.org/719927
Committed: https://git.openstack.org/cgit/openstack/neutron-tempest-plugin/commit/?id=eb354239a9f036a9e22e6f07daa03a7656948e4e
Submitter: Zuul
Branch: master
commit eb354239a9f036a9e22e6f07daa03a7656948e4e
Author: Slawek Kaplonski <skaplons@xxxxxxxxxx>
Date: Tue Apr 14 14:04:28 2020 +0200
Set higher timeout for tests in scenario jobs
It seems that in case of scenario tests which uses advanced image, when
tests are run on slow node, it may happen that around 20 minutes test
timeout will be not enough to perform all cleanup after test and it will
fail due to that.
So lets give more time for such tests to make sure all will be cleaned
properly.
Change-Id: Ic2177dc90a6fc8db4b50f05649ad794f6c3d5a23
Closes-Bug: #1872693
** Changed in: neutron
Status: In Progress => Fix Released
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1872693
Title:
Timeout exception in scenario tests with advanced image
Status in neutron:
Fix Released
Bug description:
It seems that sometimes on slow nodes, tests which requires advanced image, like e.g. neutron_tempest_plugin.scenario.test_multicast.MulticastTestIPv4.test_multicast_between_vms_on_same_network or neutron_tempest_plugin.scenario.test_trunk.TrunkTest.test_subport_connectivity may fail due to global timeout reach.
So we should give some more time for such tests, just to make sure that cleanup after test can perform fine and test will not fail in that last phase.
Examples of failure:
Body: None
Response - Headers: {'date': 'Tue, 24 Mar 2020 19:34:39 GMT', 'server': 'Apache/2.4.29 (Ubuntu)', 'content-type': 'application/json', 'openstack-api-version': 'compute 2.1', 'x-openstack-nova-api-version': '2.1', 'vary': 'OpenStack-API-Version,X-OpenStack-Nova-API-Version', 'x-openstack-request-id': 'req-f6803ed4-cd44-4aaf-852b-9642166f66fd', 'x-compute-request-id': 'req-f6803ed4-cd44-4aaf-852b-9642166f66fd', 'connection': 'close', 'status': '204', 'content-location': 'https://158.69.70.109/compute/v2.1/servers/6d8abb96-e8de-4833-86f0-2307493e42f9'}
Body: b''
2020-03-24 19:34:41,164 19073 INFO [tempest.lib.common.rest_client] Request (MulticastTestIPv4:_run_cleanups): 200 GET https://158.69.70.109/compute/v2.1/servers/6d8abb96-e8de-4833-86f0-2307493e42f9 0.664s
2020-03-24 19:34:41,164 19073 DEBUG [tempest.lib.common.rest_client] Request - Headers: {'Content-Type': 'application/json', 'Accept': 'application/json', 'X-Auth-Token': '<omitted>'}
Body: None
Response - Headers: {'date': 'Tue, 24 Mar 2020 19:34:40 GMT', 'server': 'Apache/2.4.29 (Ubuntu)', 'content-length': '1636', 'content-type': 'application/json', 'openstack-api-version': 'compute 2.1', 'x-openstack-nova-api-version': '2.1', 'vary': 'OpenStack-API-Version,X-OpenStack-Nova-API-Version', 'x-openstack-request-id': 'req-9e05c04f-d73f-410e-a593-9732cd0d368b', 'x-compute-request-id': 'req-9e05c04f-d73f-410e-a593-9732cd0d368b', 'connection': 'close', 'status': '200', 'content-location': 'https://158.69.70.109/compute/v2.1/servers/6d8abb96-e8de-4833-86f0-2307493e42f9'}
Body: b'{"server": {"id": "6d8abb96-e8de-4833-86f0-2307493e42f9", "name": "tempest-multicast-server-968193459", "status": "ACTIVE", "tenant_id": "1247c419c0bd41bb948eacb87e8bf8e3", "user_id": "3b79839ed47d4c95b8f363497a767ff2", "metadata": {}, "hostId": "9ee32c342fa222fa161f31fa44b4ad381d7a977bf32d6c6db0070294", "image": {"id": "3a21d3c5-44f2-4543-97cd-aee290e1a6a8", "links": [{"rel": "bookmark", "href": "https://158.69.70.109/compute/images/3a21d3c5-44f2-4543-97cd-aee290e1a6a8"}]}, "flavor": {"id": "d1", "links": [{"rel": "bookmark", "href": "https://158.69.70.109/compute/flavors/d1"}]}, "created": "2020-03-24T19:24:33Z", "updated": "2020-03-24T19:34:40Z", "addresses": {"tempest-test-network--2075552321": [{"version": 4, "addr": "10.1.0.12", "OS-EXT-IPS:type": "fixed", "OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:e2:f4:69"}, {"version": 4, "addr": "172.24.5.147", "OS-EXT-IPS:type": "floating", "OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:e2:f4:69"}]}, "accessIPv4": "", "accessIPv6": "", "links": [{"rel": "self", "href": "https://158.69.70.109/compute/v2.1/servers/6d8abb96-e8de-4833-86f0-2307493e42f9"}, {"rel": "bookmark", "href": "https://158.69.70.109/compute/servers/6d8abb96-e8de-4833-86f0-2307493e42f9"}], "OS-DCF:diskConfig": "MANUAL", "progress": 0, "OS-EXT-AZ:availability_zone": "nova", "config_drive": "", "key_name": "tempest-keypair-test-913711661", "OS-SRV-USG:launched_at": "2020-03-24T19:24:40.000000", "OS-SRV-USG:terminated_at": null, "security_groups": [{"name": "secgroup_mtu"}], "OS-EXT-STS:task_state": "deleting", "OS-EXT-STS:vm_state": "active", "OS-EXT-STS:power_state": 1, "os-extended-volumes:volumes_attached": []}}'
2020-03-24 19:34:41,762 19073 INFO [tempest.lib.common.rest_client] Request (MulticastTestIPv4:_run_cleanups): 204 DELETE https://158.69.70.109/compute/v2.1/servers/fde7c8ae-7fea-4aa9-9f3b-e2d8ac864d4f 0.322s
2020-03-24 19:34:41,762 19073 DEBUG [tempest.lib.common.rest_client] Request - Headers: {'Content-Type': 'application/json', 'Accept': 'application/json', 'X-Auth-Token': '<omitted>'}
Body: None
Response - Headers: {'date': 'Tue, 24 Mar 2020 19:34:41 GMT', 'server': 'Apache/2.4.29 (Ubuntu)', 'content-type': 'application/json', 'openstack-api-version': 'compute 2.1', 'x-openstack-nova-api-version': '2.1', 'vary': 'OpenStack-API-Version,X-OpenStack-Nova-API-Version', 'x-openstack-request-id': 'req-06f636f8-ebd2-4974-979b-cf7a81e1f29f', 'x-compute-request-id': 'req-06f636f8-ebd2-4974-979b-cf7a81e1f29f', 'connection': 'close', 'status': '204', 'content-location': 'https://158.69.70.109/compute/v2.1/servers/fde7c8ae-7fea-4aa9-9f3b-e2d8ac864d4f'}
Body: b''
2020-03-24 19:34:42,583 19073 INFO [tempest.lib.common.rest_client] Request (MulticastTestIPv4:_run_cleanups): 200 GET https://158.69.70.109/compute/v2.1/servers/fde7c8ae-7fea-4aa9-9f3b-e2d8ac864d4f 0.820s
2020-03-24 19:34:42,583 19073 DEBUG [tempest.lib.common.rest_client] Request - Headers: {'Content-Type': 'application/json', 'Accept': 'application/json', 'X-Auth-Token': '<omitted>'}
Body: None
Response - Headers: {'date': 'Tue, 24 Mar 2020 19:34:41 GMT', 'server': 'Apache/2.4.29 (Ubuntu)', 'content-length': '1636', 'content-type': 'application/json', 'openstack-api-version': 'compute 2.1', 'x-openstack-nova-api-version': '2.1', 'vary': 'OpenStack-API-Version,X-OpenStack-Nova-API-Version', 'x-openstack-request-id': 'req-6b4b40e9-7181-497d-b8dd-f5e32189b897', 'x-compute-request-id': 'req-6b4b40e9-7181-497d-b8dd-f5e32189b897', 'connection': 'close', 'status': '200', 'content-location': 'https://158.69.70.109/compute/v2.1/servers/fde7c8ae-7fea-4aa9-9f3b-e2d8ac864d4f'}
Body: b'{"server": {"id": "fde7c8ae-7fea-4aa9-9f3b-e2d8ac864d4f", "name": "tempest-multicast-server-1504820408", "status": "ACTIVE", "tenant_id": "1247c419c0bd41bb948eacb87e8bf8e3", "user_id": "3b79839ed47d4c95b8f363497a767ff2", "metadata": {}, "hostId": "9ee32c342fa222fa161f31fa44b4ad381d7a977bf32d6c6db0070294", "image": {"id": "3a21d3c5-44f2-4543-97cd-aee290e1a6a8", "links": [{"rel": "bookmark", "href": "https://158.69.70.109/compute/images/3a21d3c5-44f2-4543-97cd-aee290e1a6a8"}]}, "flavor": {"id": "d1", "links": [{"rel": "bookmark", "href": "https://158.69.70.109/compute/flavors/d1"}]}, "created": "2020-03-24T19:19:26Z", "updated": "2020-03-24T19:34:42Z", "addresses": {"tempest-test-network--2075552321": [{"version": 4, "addr": "10.1.0.8", "OS-EXT-IPS:type": "fixed", "OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:30:7e:e5"}, {"version": 4, "addr": "172.24.5.125", "OS-EXT-IPS:type": "floating", "OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:30:7e:e5"}]}, "accessIPv4": "", "accessIPv6": "", "links": [{"rel": "self", "href": "https://158.69.70.109/compute/v2.1/servers/fde7c8ae-7fea-4aa9-9f3b-e2d8ac864d4f"}, {"rel": "bookmark", "href": "https://158.69.70.109/compute/servers/fde7c8ae-7fea-4aa9-9f3b-e2d8ac864d4f"}], "OS-DCF:diskConfig": "MANUAL", "progress": 0, "OS-EXT-AZ:availability_zone": "nova", "config_drive": "", "key_name": "tempest-keypair-test-913711661", "OS-SRV-USG:launched_at": "2020-03-24T19:19:36.000000", "OS-SRV-USG:terminated_at": null, "security_groups": [{"name": "secgroup_mtu"}], "OS-EXT-STS:task_state": "deleting", "OS-EXT-STS:vm_state": "active", "OS-EXT-STS:power_state": 1, "os-extended-volumes:volumes_attached": []}}'
2020-03-24 19:34:44,202 19073 INFO [tempest.lib.common.rest_client] Request (MulticastTestIPv4:_run_cleanups): 200 GET https://158.69.70.109/compute/v2.1/servers/fde7c8ae-7fea-4aa9-9f3b-e2d8ac864d4f 0.612s
2020-03-24 19:34:44,203 19073 DEBUG [tempest.lib.common.rest_client] Request - Headers: {'Content-Type': 'application/json', 'Accept': 'application/json', 'X-Auth-Token': '<omitted>'}
Body: None
Response - Headers: {'date': 'Tue, 24 Mar 2020 19:34:43 GMT', 'server': 'Apache/2.4.29 (Ubuntu)', 'content-length': '1636', 'content-type': 'application/json', 'openstack-api-version': 'compute 2.1', 'x-openstack-nova-api-version': '2.1', 'vary': 'OpenStack-API-Version,X-OpenStack-Nova-API-Version', 'x-openstack-request-id': 'req-54679f2b-ea1a-43c2-920c-757819b2845f', 'x-compute-request-id': 'req-54679f2b-ea1a-43c2-920c-757819b2845f', 'connection': 'close', 'status': '200', 'content-location': 'https://158.69.70.109/compute/v2.1/servers/fde7c8ae-7fea-4aa9-9f3b-e2d8ac864d4f'}
Body: b'{"server": {"id": "fde7c8ae-7fea-4aa9-9f3b-e2d8ac864d4f", "name": "tempest-multicast-server-1504820408", "status": "ACTIVE", "tenant_id": "1247c419c0bd41bb948eacb87e8bf8e3", "user_id": "3b79839ed47d4c95b8f363497a767ff2", "metadata": {}, "hostId": "9ee32c342fa222fa161f31fa44b4ad381d7a977bf32d6c6db0070294", "image": {"id": "3a21d3c5-44f2-4543-97cd-aee290e1a6a8", "links": [{"rel": "bookmark", "href": "https://158.69.70.109/compute/images/3a21d3c5-44f2-4543-97cd-aee290e1a6a8"}]}, "flavor": {"id": "d1", "links": [{"rel": "bookmark", "href": "https://158.69.70.109/compute/flavors/d1"}]}, "created": "2020-03-24T19:19:26Z", "updated": "2020-03-24T19:34:43Z", "addresses": {"tempest-test-network--2075552321": [{"version": 4, "addr": "10.1.0.8", "OS-EXT-IPS:type": "fixed", "OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:30:7e:e5"}, {"version": 4, "addr": "172.24.5.125", "OS-EXT-IPS:type": "floating", "OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:30:7e:e5"}]}, "accessIPv4": "", "accessIPv6": "", "links": [{"rel": "self", "href": "https://158.69.70.109/compute/v2.1/servers/fde7c8ae-7fea-4aa9-9f3b-e2d8ac864d4f"}, {"rel": "bookmark", "href": "https://158.69.70.109/compute/servers/fde7c8ae-7fea-4aa9-9f3b-e2d8ac864d4f"}], "OS-DCF:diskConfig": "MANUAL", "progress": 0, "OS-EXT-AZ:availability_zone": "nova", "config_drive": "", "key_name": "tempest-keypair-test-913711661", "OS-SRV-USG:launched_at": "2020-03-24T19:19:36.000000", "OS-SRV-USG:terminated_at": null, "security_groups": [{"name": "secgroup_mtu"}], "OS-EXT-STS:task_state": "deleting", "OS-EXT-STS:vm_state": "active", "OS-EXT-STS:power_state": 1, "os-extended-volumes:volumes_attached": []}}'
2020-03-24 19:34:45,520 19073 INFO [tempest.lib.common.rest_client] Request (MulticastTestIPv4:_run_cleanups): 200 GET https://158.69.70.109/compute/v2.1/servers/fde7c8ae-7fea-4aa9-9f3b-e2d8ac864d4f 0.307s
2020-03-24 19:34:45,521 19073 DEBUG [tempest.lib.common.rest_client] Request - Headers: {'Content-Type': 'application/json', 'Accept': 'application/json', 'X-Auth-Token': '<omitted>'}
Body: None
Response - Headers: {'date': 'Tue, 24 Mar 2020 19:34:45 GMT', 'server': 'Apache/2.4.29 (Ubuntu)', 'content-length': '1636', 'content-type': 'application/json', 'openstack-api-version': 'compute 2.1', 'x-openstack-nova-api-version': '2.1', 'vary': 'OpenStack-API-Version,X-OpenStack-Nova-API-Version', 'x-openstack-request-id': 'req-7affb2b1-108d-4de5-b6f4-a42602c7c6e7', 'x-compute-request-id': 'req-7affb2b1-108d-4de5-b6f4-a42602c7c6e7', 'connection': 'close', 'status': '200', 'content-location': 'https://158.69.70.109/compute/v2.1/servers/fde7c8ae-7fea-4aa9-9f3b-e2d8ac864d4f'}
Body: b'{"server": {"id": "fde7c8ae-7fea-4aa9-9f3b-e2d8ac864d4f", "name": "tempest-multicast-server-1504820408", "status": "ACTIVE", "tenant_id": "1247c419c0bd41bb948eacb87e8bf8e3", "user_id": "3b79839ed47d4c95b8f363497a767ff2", "metadata": {}, "hostId": "9ee32c342fa222fa161f31fa44b4ad381d7a977bf32d6c6db0070294", "image": {"id": "3a21d3c5-44f2-4543-97cd-aee290e1a6a8", "links": [{"rel": "bookmark", "href": "https://158.69.70.109/compute/images/3a21d3c5-44f2-4543-97cd-aee290e1a6a8"}]}, "flavor": {"id": "d1", "links": [{"rel": "bookmark", "href": "https://158.69.70.109/compute/flavors/d1"}]}, "created": "2020-03-24T19:19:26Z", "updated": "2020-03-24T19:34:43Z", "addresses": {"tempest-test-network--2075552321": [{"version": 4, "addr": "10.1.0.8", "OS-EXT-IPS:type": "fixed", "OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:30:7e:e5"}, {"version": 4, "addr": "172.24.5.125", "OS-EXT-IPS:type": "floating", "OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:30:7e:e5"}]}, "accessIPv4": "", "accessIPv6": "", "links": [{"rel": "self", "href": "https://158.69.70.109/compute/v2.1/servers/fde7c8ae-7fea-4aa9-9f3b-e2d8ac864d4f"}, {"rel": "bookmark", "href": "https://158.69.70.109/compute/servers/fde7c8ae-7fea-4aa9-9f3b-e2d8ac864d4f"}], "OS-DCF:diskConfig": "MANUAL", "progress": 0, "OS-EXT-AZ:availability_zone": "nova", "config_drive": "", "key_name": "tempest-keypair-test-913711661", "OS-SRV-USG:launched_at": "2020-03-24T19:19:36.000000", "OS-SRV-USG:terminated_at": null, "security_groups": [{"name": "secgroup_mtu"}], "OS-EXT-STS:task_state": "deleting", "OS-EXT-STS:vm_state": "active", "OS-EXT-STS:power_state": 1, "os-extended-volumes:volumes_attached": []}}'
2020-03-24 19:34:46,735 19073 INFO [tempest.lib.common.rest_client] Request (MulticastTestIPv4:_run_cleanups): 200 GET https://158.69.70.109/compute/v2.1/servers/fde7c8ae-7fea-4aa9-9f3b-e2d8ac864d4f 0.206s
2020-03-24 19:34:46,735 19073 DEBUG [tempest.lib.common.rest_client] Request - Headers: {'Content-Type': 'application/json', 'Accept': 'application/json', 'X-Auth-Token': '<omitted>'}
Body: None
Response - Headers: {'date': 'Tue, 24 Mar 2020 19:34:46 GMT', 'server': 'Apache/2.4.29 (Ubuntu)', 'content-length': '1322', 'content-type': 'application/json', 'openstack-api-version': 'compute 2.1', 'x-openstack-nova-api-version': '2.1', 'vary': 'OpenStack-API-Version,X-OpenStack-Nova-API-Version', 'x-openstack-request-id': 'req-d0381bc4-6667-48c3-8c04-0faa3bb95f00', 'x-compute-request-id': 'req-d0381bc4-6667-48c3-8c04-0faa3bb95f00', 'connection': 'close', 'status': '200', 'content-location': 'https://158.69.70.109/compute/v2.1/servers/fde7c8ae-7fea-4aa9-9f3b-e2d8ac864d4f'}
Body: b'{"server": {"id": "fde7c8ae-7fea-4aa9-9f3b-e2d8ac864d4f", "name": "tempest-multicast-server-1504820408", "status": "ACTIVE", "tenant_id": "1247c419c0bd41bb948eacb87e8bf8e3", "user_id": "3b79839ed47d4c95b8f363497a767ff2", "metadata": {}, "hostId": "9ee32c342fa222fa161f31fa44b4ad381d7a977bf32d6c6db0070294", "image": {"id": "3a21d3c5-44f2-4543-97cd-aee290e1a6a8", "links": [{"rel": "bookmark", "href": "https://158.69.70.109/compute/images/3a21d3c5-44f2-4543-97cd-aee290e1a6a8"}]}, "flavor": {"id": "d1", "links": [{"rel": "bookmark", "href": "https://158.69.70.109/compute/flavors/d1"}]}, "created": "2020-03-24T19:19:26Z", "updated": "2020-03-24T19:34:43Z", "addresses": {}, "accessIPv4": "", "accessIPv6": "", "links": [{"rel": "self", "href": "https://158.69.70.109/compute/v2.1/servers/fde7c8ae-7fea-4aa9-9f3b-e2d8ac864d4f"}, {"rel": "bookmark", "href": "https://158.69.70.109/compute/servers/fde7c8ae-7fea-4aa9-9f3b-e2d8ac864d4f"}], "OS-DCF:diskConfig": "MANUAL", "progress": 0, "OS-EXT-AZ:availability_zone": "nova", "config_drive": "", "key_name": "tempest-keypair-test-913711661", "OS-SRV-USG:launched_at": "2020-03-24T19:19:36.000000", "OS-SRV-USG:terminated_at": null, "OS-EXT-STS:task_state": "deleting", "OS-EXT-STS:vm_state": "active", "OS-EXT-STS:power_state": 1, "os-extended-volumes:volumes_attached": []}}'
2020-03-24 19:34:47,847 19073 INFO [tempest.lib.common.rest_client] Request (MulticastTestIPv4:_run_cleanups): 404 GET https://158.69.70.109/compute/v2.1/servers/fde7c8ae-7fea-4aa9-9f3b-e2d8ac864d4f 0.100s
2020-03-24 19:34:47,848 19073 DEBUG [tempest.lib.common.rest_client] Request - Headers: {'Content-Type': 'application/json', 'Accept': 'application/json', 'X-Auth-Token': '<omitted>'}
Body: None
Response - Headers: {'date': 'Tue, 24 Mar 2020 19:34:47 GMT', 'server': 'Apache/2.4.29 (Ubuntu)', 'openstack-api-version': 'compute 2.1', 'x-openstack-nova-api-version': '2.1', 'vary': 'OpenStack-API-Version,X-OpenStack-Nova-API-Version', 'content-type': 'application/json; charset=UTF-8', 'content-length': '111', 'x-openstack-request-id': 'req-3af792f4-16b1-4320-8e66-d6a192ad7a5f', 'x-compute-request-id': 'req-3af792f4-16b1-4320-8e66-d6a192ad7a5f', 'connection': 'close', 'status': '404', 'content-location': 'https://158.69.70.109/compute/v2.1/servers/fde7c8ae-7fea-4aa9-9f3b-e2d8ac864d4f'}
Body: b'{"itemNotFound": {"code": 404, "message": "Instance fde7c8ae-7fea-4aa9-9f3b-e2d8ac864d4f could not be found."}}'
2020-03-24 19:34:48,060 19073 INFO [tempest.lib.common.rest_client] Request (MulticastTestIPv4:_run_cleanups): 204 DELETE https://158.69.70.109/compute/v2.1/servers/dcc1b3f3-d7a5-4b63-b72b-0e250e2386e6 0.211s
2020-03-24 19:34:48,060 19073 DEBUG [tempest.lib.common.rest_client] Request - Headers: {'Content-Type': 'application/json', 'Accept': 'application/json', 'X-Auth-Token': '<omitted>'}
Body: None
Response - Headers: {'date': 'Tue, 24 Mar 2020 19:34:47 GMT', 'server': 'Apache/2.4.29 (Ubuntu)', 'content-type': 'application/json', 'openstack-api-version': 'compute 2.1', 'x-openstack-nova-api-version': '2.1', 'vary': 'OpenStack-API-Version,X-OpenStack-Nova-API-Version', 'x-openstack-request-id': 'req-60a7c83f-4b76-4c94-9cf6-e57fbed2b982', 'x-compute-request-id': 'req-60a7c83f-4b76-4c94-9cf6-e57fbed2b982', 'connection': 'close', 'status': '204', 'content-location': 'https://158.69.70.109/compute/v2.1/servers/dcc1b3f3-d7a5-4b63-b72b-0e250e2386e6'}
Body: b''
2020-03-24 19:34:48,438 19073 INFO [tempest.lib.common.rest_client] Request (MulticastTestIPv4:_run_cleanups): 200 GET https://158.69.70.109/compute/v2.1/servers/dcc1b3f3-d7a5-4b63-b72b-0e250e2386e6 0.377s
2020-03-24 19:34:48,438 19073 DEBUG [tempest.lib.common.rest_client] Request - Headers: {'Content-Type': 'application/json', 'Accept': 'application/json', 'X-Auth-Token': '<omitted>'}
Body: None
Response - Headers: {'date': 'Tue, 24 Mar 2020 19:34:48 GMT', 'server': 'Apache/2.4.29 (Ubuntu)', 'content-length': '1637', 'content-type': 'application/json', 'openstack-api-version': 'compute 2.1', 'x-openstack-nova-api-version': '2.1', 'vary': 'OpenStack-API-Version,X-OpenStack-Nova-API-Version', 'x-openstack-request-id': 'req-980b088a-799e-432c-ad02-bf6204a115f6', 'x-compute-request-id': 'req-980b088a-799e-432c-ad02-bf6204a115f6', 'connection': 'close', 'status': '200', 'content-location': 'https://158.69.70.109/compute/v2.1/servers/dcc1b3f3-d7a5-4b63-b72b-0e250e2386e6'}
Body: b'{"server": {"id": "dcc1b3f3-d7a5-4b63-b72b-0e250e2386e6", "name": "tempest-multicast-server-1305662029", "status": "ACTIVE", "tenant_id": "1247c419c0bd41bb948eacb87e8bf8e3", "user_id": "3b79839ed47d4c95b8f363497a767ff2", "metadata": {}, "hostId": "9ee32c342fa222fa161f31fa44b4ad381d7a977bf32d6c6db0070294", "image": {"id": "3a21d3c5-44f2-4543-97cd-aee290e1a6a8", "links": [{"rel": "bookmark", "href": "https://158.69.70.109/compute/images/3a21d3c5-44f2-4543-97cd-aee290e1a6a8"}]}, "flavor": {"id": "d1", "links": [{"rel": "bookmark", "href": "https://158.69.70.109/compute/flavors/d1"}]}, "created": "2020-03-24T19:14:43Z", "updated": "2020-03-24T19:34:48Z", "addresses": {"tempest-test-network--2075552321": [{"version": 4, "addr": "10.1.0.11", "OS-EXT-IPS:type": "fixed", "OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:aa:46:f7"}, {"version": 4, "addr": "172.24.5.249", "OS-EXT-IPS:type": "floating", "OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:aa:46:f7"}]}, "accessIPv4": "", "accessIPv6": "", "links": [{"rel": "self", "href": "https://158.69.70.109/compute/v2.1/servers/dcc1b3f3-d7a5-4b63-b72b-0e250e2386e6"}, {"rel": "bookmark", "href": "https://158.69.70.109/compute/servers/dcc1b3f3-d7a5-4b63-b72b-0e250e2386e6"}], "OS-DCF:diskConfig": "MANUAL", "progress": 0, "OS-EXT-AZ:availability_zone": "nova", "config_drive": "", "key_name": "tempest-keypair-test-913711661", "OS-SRV-USG:launched_at": "2020-03-24T19:15:13.000000", "OS-SRV-USG:terminated_at": null, "security_groups": [{"name": "secgroup_mtu"}], "OS-EXT-STS:task_state": "deleting", "OS-EXT-STS:vm_state": "active", "OS-EXT-STS:power_state": 1, "os-extended-volumes:volumes_attached": []}}'
2020-03-24 19:34:50,655 19073 INFO [tempest.lib.common.rest_client] Request (MulticastTestIPv4:_run_cleanups): 200 GET https://158.69.70.109/compute/v2.1/servers/dcc1b3f3-d7a5-4b63-b72b-0e250e2386e6 1.210s
2020-03-24 19:34:50,656 19073 DEBUG [tempest.lib.common.rest_client] Request - Headers: {'Content-Type': 'application/json', 'Accept': 'application/json', 'X-Auth-Token': '<omitted>'}
Body: None
Response - Headers: {'date': 'Tue, 24 Mar 2020 19:34:49 GMT', 'server': 'Apache/2.4.29 (Ubuntu)', 'content-length': '1637', 'content-type': 'application/json', 'openstack-api-version': 'compute 2.1', 'x-openstack-nova-api-version': '2.1', 'vary': 'OpenStack-API-Version,X-OpenStack-Nova-API-Version', 'x-openstack-request-id': 'req-846415cc-4906-4239-b483-2725e4cb4800', 'x-compute-request-id': 'req-846415cc-4906-4239-b483-2725e4cb4800', 'connection': 'close', 'status': '200', 'content-location': 'https://158.69.70.109/compute/v2.1/servers/dcc1b3f3-d7a5-4b63-b72b-0e250e2386e6'}
Body: b'{"server": {"id": "dcc1b3f3-d7a5-4b63-b72b-0e250e2386e6", "name": "tempest-multicast-server-1305662029", "status": "ACTIVE", "tenant_id": "1247c419c0bd41bb948eacb87e8bf8e3", "user_id": "3b79839ed47d4c95b8f363497a767ff2", "metadata": {}, "hostId": "9ee32c342fa222fa161f31fa44b4ad381d7a977bf32d6c6db0070294", "image": {"id": "3a21d3c5-44f2-4543-97cd-aee290e1a6a8", "links": [{"rel": "bookmark", "href": "https://158.69.70.109/compute/images/3a21d3c5-44f2-4543-97cd-aee290e1a6a8"}]}, "flavor": {"id": "d1", "links": [{"rel": "bookmark", "href": "https://158.69.70.109/compute/flavors/d1"}]}, "created": "2020-03-24T19:14:43Z", "updated": "2020-03-24T19:34:49Z", "addresses": {"tempest-test-network--2075552321": [{"version": 4, "addr": "10.1.0.11", "OS-EXT-IPS:type": "fixed", "OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:aa:46:f7"}, {"version": 4, "addr": "172.24.5.249", "OS-EXT-IPS:type": "floating", "OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:aa:46:f7"}]}, "accessIPv4": "", "accessIPv6": "", "links": [{"rel": "self", "href": "https://158.69.70.109/compute/v2.1/servers/dcc1b3f3-d7a5-4b63-b72b-0e250e2386e6"}, {"rel": "bookmark", "href": "https://158.69.70.109/compute/servers/dcc1b3f3-d7a5-4b63-b72b-0e250e2386e6"}], "OS-DCF:diskConfig": "MANUAL", "progress": 0, "OS-EXT-AZ:availability_zone": "nova", "config_drive": "", "key_name": "tempest-keypair-test-913711661", "OS-SRV-USG:launched_at": "2020-03-24T19:15:13.000000", "OS-SRV-USG:terminated_at": null, "security_groups": [{"name": "secgroup_mtu"}], "OS-EXT-STS:task_state": "deleting", "OS-EXT-STS:vm_state": "active", "OS-EXT-STS:power_state": 1, "os-extended-volumes:volumes_attached": []}}'
2020-03-24 19:34:51,764 19073 INFO [tempest.lib.common.rest_client] Request (MulticastTestIPv4:_run_cleanups): 404 GET https://158.69.70.109/compute/v2.1/servers/dcc1b3f3-d7a5-4b63-b72b-0e250e2386e6 0.098s
2020-03-24 19:34:51,765 19073 DEBUG [tempest.lib.common.rest_client] Request - Headers: {'Content-Type': 'application/json', 'Accept': 'application/json', 'X-Auth-Token': '<omitted>'}
Body: None
Response - Headers: {'date': 'Tue, 24 Mar 2020 19:34:51 GMT', 'server': 'Apache/2.4.29 (Ubuntu)', 'openstack-api-version': 'compute 2.1', 'x-openstack-nova-api-version': '2.1', 'vary': 'OpenStack-API-Version,X-OpenStack-Nova-API-Version', 'content-type': 'application/json; charset=UTF-8', 'content-length': '111', 'x-openstack-request-id': 'req-12f4f0a5-7fbc-4f3c-8000-1d3e5c8e3fa1', 'x-compute-request-id': 'req-12f4f0a5-7fbc-4f3c-8000-1d3e5c8e3fa1', 'connection': 'close', 'status': '404', 'content-location': 'https://158.69.70.109/compute/v2.1/servers/dcc1b3f3-d7a5-4b63-b72b-0e250e2386e6'}
Body: b'{"itemNotFound": {"code": 404, "message": "Instance dcc1b3f3-d7a5-4b63-b72b-0e250e2386e6 could not be found."}}'
}}}
Traceback (most recent call last):
File "/opt/stack/tempest/tempest/lib/common/utils/test_utils.py", line 84, in call_and_ignore_notfound_exc
return func(*args, **kwargs)
File "/opt/stack/tempest/tempest/common/waiters.py", line 111, in wait_for_server_termination
time.sleep(client.build_interval)
File "/usr/local/lib/python3.6/dist-packages/fixtures/_fixtures/timeout.py", line 52, in signal_handler
raise TimeoutException()
fixtures._fixtures.timeout.TimeoutException
links to examples of failures:
https://103900b4a03cdd60217d-625a0eb0440aa527fbdb216e8991f5a6.ssl.cf5.rackcdn.com/714726/1/check/neutron-ovn-tempest-ovs-release/2b52c20/testr_results.html
https://5481f30c7f7864f4d07b-85bb929ac58f1a419407783b8f581cf5.ssl.cf5.rackcdn.com/710755/11/check/neutron-tempest-plugin-scenario-openvswitch/2d4ae79/testr_results.html
logstash query:
http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22line%20111%2C%20in%20wait_for_server_termination%5C%22
To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1872693/+subscriptions
References
