yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1696848] [NEW] POST /openstack/2013-10-17/password returns 500 if instance is not found

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Matt Riedemann <mriedem.os@xxxxxxxxx>
Date: Thu, 08 Jun 2017 20:04:55 -0000
Reply-to: Bug 1696848 <1696848@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
Public bug reported:

Seen here:

http://logs.openstack.org/94/436094/33/check/gate-tempest-dsvm-neutron-
full-ubuntu-
xenial/bed0e6a/logs/screen-n-api.txt.gz#_Jun_08_18_36_06_589590

Jun 08 18:36:06.589590 ubuntu-xenial-osic-cloud1-s3700-9216660 nova-api[18643]: INFO nova.metadata.wsgi.server [None req-97c06652-908d-4603-a84d-40f818726b5d None None] Traceback (most recent call last):
Jun 08 18:36:06.589719 ubuntu-xenial-osic-cloud1-s3700-9216660 nova-api[18643]:   File "/usr/local/lib/python2.7/dist-packages/eventlet/wsgi.py", line 490, in handle_one_response
Jun 08 18:36:06.589804 ubuntu-xenial-osic-cloud1-s3700-9216660 nova-api[18643]:     result = self.application(self.environ, start_response)
Jun 08 18:36:06.589911 ubuntu-xenial-osic-cloud1-s3700-9216660 nova-api[18643]:   File "/usr/local/lib/python2.7/dist-packages/paste/urlmap.py", line 216, in __call__
Jun 08 18:36:06.590006 ubuntu-xenial-osic-cloud1-s3700-9216660 nova-api[18643]:     return app(environ, start_response)
Jun 08 18:36:06.590090 ubuntu-xenial-osic-cloud1-s3700-9216660 nova-api[18643]:   File "/usr/local/lib/python2.7/dist-packages/webob/dec.py", line 131, in __call__
Jun 08 18:36:06.590179 ubuntu-xenial-osic-cloud1-s3700-9216660 nova-api[18643]:     resp = self.call_func(req, *args, **self.kwargs)
Jun 08 18:36:06.590270 ubuntu-xenial-osic-cloud1-s3700-9216660 nova-api[18643]:   File "/usr/local/lib/python2.7/dist-packages/webob/dec.py", line 196, in call_func
Jun 08 18:36:06.590351 ubuntu-xenial-osic-cloud1-s3700-9216660 nova-api[18643]:     return self.func(req, *args, **kwargs)
Jun 08 18:36:06.590428 ubuntu-xenial-osic-cloud1-s3700-9216660 nova-api[18643]:   File "/usr/local/lib/python2.7/dist-packages/oslo_middleware/base.py", line 125, in __call__
Jun 08 18:36:06.590508 ubuntu-xenial-osic-cloud1-s3700-9216660 nova-api[18643]:     response = req.get_response(self.application)
Jun 08 18:36:06.590585 ubuntu-xenial-osic-cloud1-s3700-9216660 nova-api[18643]:   File "/usr/local/lib/python2.7/dist-packages/webob/request.py", line 1316, in send
Jun 08 18:36:06.590661 ubuntu-xenial-osic-cloud1-s3700-9216660 nova-api[18643]:     application, catch_exc_info=False)
Jun 08 18:36:06.590742 ubuntu-xenial-osic-cloud1-s3700-9216660 nova-api[18643]:   File "/usr/local/lib/python2.7/dist-packages/webob/request.py", line 1280, in call_application
Jun 08 18:36:06.590824 ubuntu-xenial-osic-cloud1-s3700-9216660 nova-api[18643]:     app_iter = application(self.environ, start_response)
Jun 08 18:36:06.590906 ubuntu-xenial-osic-cloud1-s3700-9216660 nova-api[18643]:   File "/usr/local/lib/python2.7/dist-packages/webob/dec.py", line 131, in __call__
Jun 08 18:36:06.590983 ubuntu-xenial-osic-cloud1-s3700-9216660 nova-api[18643]:     resp = self.call_func(req, *args, **self.kwargs)
Jun 08 18:36:06.591060 ubuntu-xenial-osic-cloud1-s3700-9216660 nova-api[18643]:   File "/usr/local/lib/python2.7/dist-packages/webob/dec.py", line 196, in call_func
Jun 08 18:36:06.591136 ubuntu-xenial-osic-cloud1-s3700-9216660 nova-api[18643]:     return self.func(req, *args, **kwargs)
Jun 08 18:36:06.591212 ubuntu-xenial-osic-cloud1-s3700-9216660 nova-api[18643]:   File "/opt/stack/new/nova/nova/api/metadata/handler.py", line 121, in __call__
Jun 08 18:36:06.591287 ubuntu-xenial-osic-cloud1-s3700-9216660 nova-api[18643]:     return data(req, meta_data)
Jun 08 18:36:06.591370 ubuntu-xenial-osic-cloud1-s3700-9216660 nova-api[18643]:   File "/opt/stack/new/nova/nova/api/metadata/password.py", line 70, in handle_password
Jun 08 18:36:06.591446 ubuntu-xenial-osic-cloud1-s3700-9216660 nova-api[18643]:     instance = objects.Instance.get_by_uuid(ctxt, meta_data.uuid)
Jun 08 18:36:06.591531 ubuntu-xenial-osic-cloud1-s3700-9216660 nova-api[18643]:   File "/usr/local/lib/python2.7/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper
Jun 08 18:36:06.591610 ubuntu-xenial-osic-cloud1-s3700-9216660 nova-api[18643]:     result = fn(cls, context, *args, **kwargs)
Jun 08 18:36:06.591687 ubuntu-xenial-osic-cloud1-s3700-9216660 nova-api[18643]:   File "/opt/stack/new/nova/nova/objects/instance.py", line 463, in get_by_uuid
Jun 08 18:36:06.591763 ubuntu-xenial-osic-cloud1-s3700-9216660 nova-api[18643]:     use_slave=use_slave)
Jun 08 18:36:06.591839 ubuntu-xenial-osic-cloud1-s3700-9216660 nova-api[18643]:   File "/opt/stack/new/nova/nova/db/sqlalchemy/api.py", line 236, in wrapper
Jun 08 18:36:06.591914 ubuntu-xenial-osic-cloud1-s3700-9216660 nova-api[18643]:     return f(*args, **kwargs)
Jun 08 18:36:06.592000 ubuntu-xenial-osic-cloud1-s3700-9216660 nova-api[18643]:   File "/opt/stack/new/nova/nova/objects/instance.py", line 455, in _db_instance_get_by_uuid
Jun 08 18:36:06.592069 ubuntu-xenial-osic-cloud1-s3700-9216660 nova-api[18643]:     columns_to_join=columns_to_join)
Jun 08 18:36:06.592159 ubuntu-xenial-osic-cloud1-s3700-9216660 nova-api[18643]:   File "/opt/stack/new/nova/nova/db/api.py", line 745, in instance_get_by_uuid
Jun 08 18:36:06.592253 ubuntu-xenial-osic-cloud1-s3700-9216660 nova-api[18643]:     return IMPL.instance_get_by_uuid(context, uuid, columns_to_join)
Jun 08 18:36:06.592339 ubuntu-xenial-osic-cloud1-s3700-9216660 nova-api[18643]:   File "/opt/stack/new/nova/nova/db/sqlalchemy/api.py", line 180, in wrapper
Jun 08 18:36:06.592420 ubuntu-xenial-osic-cloud1-s3700-9216660 nova-api[18643]:     return f(*args, **kwargs)
Jun 08 18:36:06.592501 ubuntu-xenial-osic-cloud1-s3700-9216660 nova-api[18643]:   File "/opt/stack/new/nova/nova/db/sqlalchemy/api.py", line 281, in wrapped
Jun 08 18:36:06.592582 ubuntu-xenial-osic-cloud1-s3700-9216660 nova-api[18643]:     return f(context, *args, **kwargs)
Jun 08 18:36:06.592664 ubuntu-xenial-osic-cloud1-s3700-9216660 nova-api[18643]:   File "/opt/stack/new/nova/nova/db/sqlalchemy/api.py", line 1921, in instance_get_by_uuid
Jun 08 18:36:06.592745 ubuntu-xenial-osic-cloud1-s3700-9216660 nova-api[18643]:     columns_to_join=columns_to_join)
Jun 08 18:36:06.593201 ubuntu-xenial-osic-cloud1-s3700-9216660 nova-api[18643]:   File "/opt/stack/new/nova/nova/db/sqlalchemy/api.py", line 1930, in _instance_get_by_uuid
Jun 08 18:36:06.593299 ubuntu-xenial-osic-cloud1-s3700-9216660 nova-api[18643]:     raise exception.InstanceNotFound(instance_id=uuid)
Jun 08 18:36:06.593384 ubuntu-xenial-osic-cloud1-s3700-9216660 nova-api[18643]: InstanceNotFound: Instance 6cd26f78-9c8a-4b37-8c62-38ad003a095f could not be found.
Jun 08 18:36:06.593487 ubuntu-xenial-osic-cloud1-s3700-9216660 nova-api[18643]: : InstanceNotFound: Instance 6cd26f78-9c8a-4b37-8c62-38ad003a095f could not be found.
Jun 08 18:36:06.594103 ubuntu-xenial-osic-cloud1-s3700-9216660 nova-api[18643]: INFO nova.metadata.wsgi.server [None req-97c06652-908d-4603-a84d-40f818726b5d None None] 10.1.0.11,10.40.184.140 "POST /openstack/2013-10-17/password HTTP/1.1" status: 500 len: 139 time: 0.0445061: InstanceNotFound: Instance 6cd26f78-9c8a-4b37-8c62-38ad003a095f could not be found.

The handle_password metadata request handler doesn't handle an
InstanceNotFound and that results in a 500 error.

** Affects: nova
     Importance: Medium
         Status: Triaged

** Affects: nova/newton
     Importance: Medium
         Status: Confirmed

** Affects: nova/ocata
     Importance: Medium
         Status: Confirmed


** Tags: api metadata

** Tags added: api metadata

** Also affects: nova/newton
   Importance: Undecided
       Status: New

** Also affects: nova/ocata
   Importance: Undecided
       Status: New

** Changed in: nova
       Status: New => Triaged

** Changed in: nova/newton
       Status: New => Confirmed

** Changed in: nova/newton
   Importance: Undecided => Medium

** Changed in: nova
   Importance: Undecided => Medium

** Changed in: nova/ocata
   Importance: Undecided => Medium

** Changed in: nova/ocata
       Status: New => Confirmed

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1696848

Title:
  POST /openstack/2013-10-17/password returns 500 if instance is not
  found

Status in OpenStack Compute (nova):
  Triaged
Status in OpenStack Compute (nova) newton series:
  Confirmed
Status in OpenStack Compute (nova) ocata series:
  Confirmed

Bug description:
  Seen here:

  http://logs.openstack.org/94/436094/33/check/gate-tempest-dsvm-
  neutron-full-ubuntu-
  xenial/bed0e6a/logs/screen-n-api.txt.gz#_Jun_08_18_36_06_589590

  Jun 08 18:36:06.589590 ubuntu-xenial-osic-cloud1-s3700-9216660 nova-api[18643]: INFO nova.metadata.wsgi.server [None req-97c06652-908d-4603-a84d-40f818726b5d None None] Traceback (most recent call last):
  Jun 08 18:36:06.589719 ubuntu-xenial-osic-cloud1-s3700-9216660 nova-api[18643]:   File "/usr/local/lib/python2.7/dist-packages/eventlet/wsgi.py", line 490, in handle_one_response
  Jun 08 18:36:06.589804 ubuntu-xenial-osic-cloud1-s3700-9216660 nova-api[18643]:     result = self.application(self.environ, start_response)
  Jun 08 18:36:06.589911 ubuntu-xenial-osic-cloud1-s3700-9216660 nova-api[18643]:   File "/usr/local/lib/python2.7/dist-packages/paste/urlmap.py", line 216, in __call__
  Jun 08 18:36:06.590006 ubuntu-xenial-osic-cloud1-s3700-9216660 nova-api[18643]:     return app(environ, start_response)
  Jun 08 18:36:06.590090 ubuntu-xenial-osic-cloud1-s3700-9216660 nova-api[18643]:   File "/usr/local/lib/python2.7/dist-packages/webob/dec.py", line 131, in __call__
  Jun 08 18:36:06.590179 ubuntu-xenial-osic-cloud1-s3700-9216660 nova-api[18643]:     resp = self.call_func(req, *args, **self.kwargs)
  Jun 08 18:36:06.590270 ubuntu-xenial-osic-cloud1-s3700-9216660 nova-api[18643]:   File "/usr/local/lib/python2.7/dist-packages/webob/dec.py", line 196, in call_func
  Jun 08 18:36:06.590351 ubuntu-xenial-osic-cloud1-s3700-9216660 nova-api[18643]:     return self.func(req, *args, **kwargs)
  Jun 08 18:36:06.590428 ubuntu-xenial-osic-cloud1-s3700-9216660 nova-api[18643]:   File "/usr/local/lib/python2.7/dist-packages/oslo_middleware/base.py", line 125, in __call__
  Jun 08 18:36:06.590508 ubuntu-xenial-osic-cloud1-s3700-9216660 nova-api[18643]:     response = req.get_response(self.application)
  Jun 08 18:36:06.590585 ubuntu-xenial-osic-cloud1-s3700-9216660 nova-api[18643]:   File "/usr/local/lib/python2.7/dist-packages/webob/request.py", line 1316, in send
  Jun 08 18:36:06.590661 ubuntu-xenial-osic-cloud1-s3700-9216660 nova-api[18643]:     application, catch_exc_info=False)
  Jun 08 18:36:06.590742 ubuntu-xenial-osic-cloud1-s3700-9216660 nova-api[18643]:   File "/usr/local/lib/python2.7/dist-packages/webob/request.py", line 1280, in call_application
  Jun 08 18:36:06.590824 ubuntu-xenial-osic-cloud1-s3700-9216660 nova-api[18643]:     app_iter = application(self.environ, start_response)
  Jun 08 18:36:06.590906 ubuntu-xenial-osic-cloud1-s3700-9216660 nova-api[18643]:   File "/usr/local/lib/python2.7/dist-packages/webob/dec.py", line 131, in __call__
  Jun 08 18:36:06.590983 ubuntu-xenial-osic-cloud1-s3700-9216660 nova-api[18643]:     resp = self.call_func(req, *args, **self.kwargs)
  Jun 08 18:36:06.591060 ubuntu-xenial-osic-cloud1-s3700-9216660 nova-api[18643]:   File "/usr/local/lib/python2.7/dist-packages/webob/dec.py", line 196, in call_func
  Jun 08 18:36:06.591136 ubuntu-xenial-osic-cloud1-s3700-9216660 nova-api[18643]:     return self.func(req, *args, **kwargs)
  Jun 08 18:36:06.591212 ubuntu-xenial-osic-cloud1-s3700-9216660 nova-api[18643]:   File "/opt/stack/new/nova/nova/api/metadata/handler.py", line 121, in __call__
  Jun 08 18:36:06.591287 ubuntu-xenial-osic-cloud1-s3700-9216660 nova-api[18643]:     return data(req, meta_data)
  Jun 08 18:36:06.591370 ubuntu-xenial-osic-cloud1-s3700-9216660 nova-api[18643]:   File "/opt/stack/new/nova/nova/api/metadata/password.py", line 70, in handle_password
  Jun 08 18:36:06.591446 ubuntu-xenial-osic-cloud1-s3700-9216660 nova-api[18643]:     instance = objects.Instance.get_by_uuid(ctxt, meta_data.uuid)
  Jun 08 18:36:06.591531 ubuntu-xenial-osic-cloud1-s3700-9216660 nova-api[18643]:   File "/usr/local/lib/python2.7/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper
  Jun 08 18:36:06.591610 ubuntu-xenial-osic-cloud1-s3700-9216660 nova-api[18643]:     result = fn(cls, context, *args, **kwargs)
  Jun 08 18:36:06.591687 ubuntu-xenial-osic-cloud1-s3700-9216660 nova-api[18643]:   File "/opt/stack/new/nova/nova/objects/instance.py", line 463, in get_by_uuid
  Jun 08 18:36:06.591763 ubuntu-xenial-osic-cloud1-s3700-9216660 nova-api[18643]:     use_slave=use_slave)
  Jun 08 18:36:06.591839 ubuntu-xenial-osic-cloud1-s3700-9216660 nova-api[18643]:   File "/opt/stack/new/nova/nova/db/sqlalchemy/api.py", line 236, in wrapper
  Jun 08 18:36:06.591914 ubuntu-xenial-osic-cloud1-s3700-9216660 nova-api[18643]:     return f(*args, **kwargs)
  Jun 08 18:36:06.592000 ubuntu-xenial-osic-cloud1-s3700-9216660 nova-api[18643]:   File "/opt/stack/new/nova/nova/objects/instance.py", line 455, in _db_instance_get_by_uuid
  Jun 08 18:36:06.592069 ubuntu-xenial-osic-cloud1-s3700-9216660 nova-api[18643]:     columns_to_join=columns_to_join)
  Jun 08 18:36:06.592159 ubuntu-xenial-osic-cloud1-s3700-9216660 nova-api[18643]:   File "/opt/stack/new/nova/nova/db/api.py", line 745, in instance_get_by_uuid
  Jun 08 18:36:06.592253 ubuntu-xenial-osic-cloud1-s3700-9216660 nova-api[18643]:     return IMPL.instance_get_by_uuid(context, uuid, columns_to_join)
  Jun 08 18:36:06.592339 ubuntu-xenial-osic-cloud1-s3700-9216660 nova-api[18643]:   File "/opt/stack/new/nova/nova/db/sqlalchemy/api.py", line 180, in wrapper
  Jun 08 18:36:06.592420 ubuntu-xenial-osic-cloud1-s3700-9216660 nova-api[18643]:     return f(*args, **kwargs)
  Jun 08 18:36:06.592501 ubuntu-xenial-osic-cloud1-s3700-9216660 nova-api[18643]:   File "/opt/stack/new/nova/nova/db/sqlalchemy/api.py", line 281, in wrapped
  Jun 08 18:36:06.592582 ubuntu-xenial-osic-cloud1-s3700-9216660 nova-api[18643]:     return f(context, *args, **kwargs)
  Jun 08 18:36:06.592664 ubuntu-xenial-osic-cloud1-s3700-9216660 nova-api[18643]:   File "/opt/stack/new/nova/nova/db/sqlalchemy/api.py", line 1921, in instance_get_by_uuid
  Jun 08 18:36:06.592745 ubuntu-xenial-osic-cloud1-s3700-9216660 nova-api[18643]:     columns_to_join=columns_to_join)
  Jun 08 18:36:06.593201 ubuntu-xenial-osic-cloud1-s3700-9216660 nova-api[18643]:   File "/opt/stack/new/nova/nova/db/sqlalchemy/api.py", line 1930, in _instance_get_by_uuid
  Jun 08 18:36:06.593299 ubuntu-xenial-osic-cloud1-s3700-9216660 nova-api[18643]:     raise exception.InstanceNotFound(instance_id=uuid)
  Jun 08 18:36:06.593384 ubuntu-xenial-osic-cloud1-s3700-9216660 nova-api[18643]: InstanceNotFound: Instance 6cd26f78-9c8a-4b37-8c62-38ad003a095f could not be found.
  Jun 08 18:36:06.593487 ubuntu-xenial-osic-cloud1-s3700-9216660 nova-api[18643]: : InstanceNotFound: Instance 6cd26f78-9c8a-4b37-8c62-38ad003a095f could not be found.
  Jun 08 18:36:06.594103 ubuntu-xenial-osic-cloud1-s3700-9216660 nova-api[18643]: INFO nova.metadata.wsgi.server [None req-97c06652-908d-4603-a84d-40f818726b5d None None] 10.1.0.11,10.40.184.140 "POST /openstack/2013-10-17/password HTTP/1.1" status: 500 len: 139 time: 0.0445061: InstanceNotFound: Instance 6cd26f78-9c8a-4b37-8c62-38ad003a095f could not be found.

  The handle_password metadata request handler doesn't handle an
  InstanceNotFound and that results in a 500 error.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1696848/+subscriptions
Follow ups

[Bug 1696848] Re: POST /openstack/2013-10-17/password returns 500 if instance is not found
From: Matt Riedemann, 2017-11-14