yahoo-eng-team team mailing list archive
-
yahoo-eng-team team
-
Mailing list archive
-
Message #63625
[Bug 1661360] Re: InstanceNotFound due to missing osapi_compute service version when running nova-api under wsgi
Reviewed: https://review.openstack.org/457283
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=d3c084f23448d1890bfda4a06de246f2be3c1279
Submitter: Jenkins
Branch: master
commit d3c084f23448d1890bfda4a06de246f2be3c1279
Author: Chris Dent <cdent@xxxxxxxxxxxxx>
Date: Mon Apr 17 16:38:49 2017 +0000
Register osapi_compute when nova-api is wsgi
When the nova-api services starts from its own standalone binary it
registers itself in the services table. The original wsgi script in
nova/wsgi/nova-api.py did not, leading to the bug referenced below.
The new wsgi script at nova.api.openstack.compute.wsgi, modelled on
a similar thing used for the placement API, provides the necessary
service registration.
If a ServiceTooOld exception happens while trying to register the
service then a very simple (currently very stubby) application is
loaded instead of the compute api. This application returns a 500
and a message.
Some caveats/todos:
* wsgi apps managed under mod-wsgi (and presumably other containers)
are not imported/compiled/run until the first request is made. In
this case that means the service handling does not happen until
that first request, somewhat defeating the purpose if the api is a
bit idle.
Change-Id: I7c4acfaa6c50ac0e4d6de69eb62ec5bbad72ff85
Closes-Bug: #1661360
** Changed in: nova
Status: In Progress => Fix Released
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1661360
Title:
InstanceNotFound due to missing osapi_compute service version when
running nova-api under wsgi
Status in OpenStack Compute (nova):
Fix Released
Status in tripleo:
Fix Released
Bug description:
Running OpenStack services from master, when we try to run tempest
test
tempest.scenario.test_server_basic_ops.TestServerBasicOps.test_server_basic_ops
(among others). It always fails with message "u'message': u'Instance
bf33af04-6b55-4835-bb17-02484c196f13 could not be found.'" (full log
in http://logs.openstack.org/15/424915/8/check/gate-puppet-openstack-
integration-4-scenario001-tempest-centos-7/b29f35b/console.html)
According to the sequence in the log, this is what happens:
1. tempest creates an instance:
http://logs.openstack.org/15/424915/8/check/gate-puppet-openstack-
integration-4-scenario001-tempest-
centos-7/b29f35b/console.html#_2017-02-02_13_04_48_291997
2. nova server returns instance bf33af04-6b55-4835-bb17-02484c196f13
so it seems it has been properly created:
http://logs.openstack.org/15/424915/8/check/gate-puppet-openstack-
integration-4-scenario001-tempest-
centos-7/b29f35b/console.html#_2017-02-02_13_04_48_292483
3. tempest try to get status of the instance right after creating it
and nova server returns 404, instance not found:
http://logs.openstack.org/15/424915/8/check/gate-puppet-openstack-
integration-4-scenario001-tempest-
centos-7/b29f35b/console.html#_2017-02-02_13_04_48_292565
http://logs.openstack.org/15/424915/8/check/gate-puppet-openstack-
integration-4-scenario001-tempest-
centos-7/b29f35b/console.html#_2017-02-02_13_04_48_292845
At that time following messages are found in nova log:
2017-02-02 12:58:10.823 7439 DEBUG nova.compute.api [req-eec92d3e-9f78-4915-b3b9-ca6858f8dd6a - - - - -] [instance: bf33af04-6b55-4835-bb17-02484c196f13] Fetching instance by UUID get /usr/lib/python2.7/site-packages/nova/compute/api.py:2312
2017-02-02 12:58:10.879 7439 INFO nova.api.openstack.wsgi [req-eec92d3e-9f78-4915-b3b9-ca6858f8dd6a - - - - -] HTTP exception thrown: Instance bf33af04-6b55-4835-bb17-02484c196f13 could not be found.
2017-02-02 12:58:10.880 7439 DEBUG nova.api.openstack.wsgi [req-eec92d3e-9f78-4915-b3b9-ca6858f8dd6a - - - - -] Returning 404 to user: Instance bf33af04-6b55-4835-bb17-02484c196f13 could not be found. __call__ /usr/lib/python2.7/site-packages/nova/api/openstack/wsgi.py:1039
http://logs.openstack.org/15/424915/8/check/gate-puppet-openstack-
integration-4-scenario001-tempest-centos-7/b29f35b/logs/nova/nova-
api.txt.gz#_2017-02-02_12_58_10_879
4. Then tempest start cleaning up environment, deleting security
group, etc...
We are hitting this with nova from commit
f40467b0eb2b58a369d24a0e832df1ace6c400c3
Tempest starts cleaning up securitygroup
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1661360/+subscriptions
References