yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1372218] [NEW] servers.list, filtering on metadata doesn't work. unicode error

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Sam Morrison <1372218@xxxxxxxxxxxxxxxxxx>
Date: Mon, 22 Sep 2014 01:38:09 -0000
Reply-to: Bug 1372218 <1372218@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
Public bug reported:

I'm trying to list servers by filtering on system_metadata or metadata.

I should be able to do something like (looking into the code)

nclient.servers.list(search_opts={'system_metadata': {"some_value":
"some_key"}, 'all_tenants': 1})

But this dictionary gets turned into a unicode string. I get a 500 back
from nova with the below stack trace in nova-api.

The offending code is in exact_filter in the db api. It is expecting a
list of dicts or a single dict when using system_metadata or metadata
key when searching. It looks like this used to work but now somewhere
higher up is ensuring this is a string.


2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack Traceback (most recent call last):
2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack   File "/opt/nova/nova/api/openstack/__init__.py", line 125, in __call__
2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack     return req.get_response(self.application)
2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack   File "/usr/lib/python2.7/dist-packages/webob/request.py", line 1320, in send
2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack     application, catch_exc_info=False)
2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack   File "/usr/lib/python2.7/dist-packages/webob/request.py", line 1284, in call_application
2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack     app_iter = application(self.environ, start_response)
2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack   File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 144, in __call__
2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack     return resp(environ, start_response)
2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack   File "/usr/lib/python2.7/dist-packages/keystoneclient/middleware/auth_token.py", line 582, in __call__
2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack     return self.app(env, start_response)
2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack   File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 144, in __call__
2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack     return resp(environ, start_response)
2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack   File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 144, in __call__
2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack     return resp(environ, start_response)
2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack   File "/usr/local/lib/python2.7/dist-packages/routes/middleware.py", line 131, in __call__
2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack     response = self.app(environ, start_response)
2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack   File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 144, in __call__
2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack     return resp(environ, start_response)
2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack   File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 130, in __call__
2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack     resp = self.call_func(req, *args, **self.kwargs)
2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack   File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 195, in call_func
2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack     return self.func(req, *args, **kwargs)
2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack   File "/opt/nova/nova/api/openstack/wsgi.py", line 917, in __call__
2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack     content_type, body, accept)
2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack   File "/opt/nova/nova/api/openstack/wsgi.py", line 983, in _process_stack
2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack     action_result = self.dispatch(meth, request, action_args)
2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack   File "/opt/nova/nova/api/openstack/wsgi.py", line 1070, in dispatch
2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack     return method(req=request, **action_args)
2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack   File "/opt/nova/nova/api/openstack/compute/servers.py", line 520, in detail
2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack     servers = self._get_servers(req, is_detail=True)
2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack   File "/opt/nova/nova/api/openstack/compute/servers.py", line 603, in _get_servers
2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack     want_objects=True)
2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack   File "/opt/nova/nova/compute/api.py", line 1887, in get_all
2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack     expected_attrs=expected_attrs)
2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack   File "/opt/nova/nova/compute/cells_api.py", line 224, in _get_instances_by_filters
2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack     limit=limit, marker=marker, expected_attrs=fields)
2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack   File "/opt/nova/nova/objects/base.py", line 112, in wrapper
2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack     result = fn(cls, context, *args, **kwargs)
2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack   File "/opt/nova/nova/objects/instance.py", line 629, in get_by_filters
2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack     use_slave=use_slave)
2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack   File "/opt/nova/nova/db/api.py", line 665, in instance_get_all_by_filters
2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack     use_slave=use_slave)
2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack   File "/opt/nova/nova/db/sqlalchemy/api.py", line 164, in wrapper
2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack     return f(*args, **kwargs)
2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack   File "/opt/nova/nova/db/sqlalchemy/api.py", line 2008, in instance_get_all_by_filters
2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack     filters, exact_match_filter_names)
2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack   File "/opt/nova/nova/db/sqlalchemy/api.py", line 388, in exact_filter
2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack     for k, v in value.iteritems():
2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack AttributeError: 'unicode' object has no attribute 'iteritems'

** Affects: nova
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1372218

Title:
  servers.list, filtering on metadata doesn't work. unicode error

Status in OpenStack Compute (Nova):
  New

Bug description:
  I'm trying to list servers by filtering on system_metadata or
  metadata.

  I should be able to do something like (looking into the code)

  nclient.servers.list(search_opts={'system_metadata': {"some_value":
  "some_key"}, 'all_tenants': 1})

  But this dictionary gets turned into a unicode string. I get a 500
  back from nova with the below stack trace in nova-api.

  The offending code is in exact_filter in the db api. It is expecting a
  list of dicts or a single dict when using system_metadata or metadata
  key when searching. It looks like this used to work but now somewhere
  higher up is ensuring this is a string.

  
  2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack Traceback (most recent call last):
  2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack   File "/opt/nova/nova/api/openstack/__init__.py", line 125, in __call__
  2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack     return req.get_response(self.application)
  2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack   File "/usr/lib/python2.7/dist-packages/webob/request.py", line 1320, in send
  2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack     application, catch_exc_info=False)
  2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack   File "/usr/lib/python2.7/dist-packages/webob/request.py", line 1284, in call_application
  2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack     app_iter = application(self.environ, start_response)
  2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack   File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 144, in __call__
  2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack     return resp(environ, start_response)
  2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack   File "/usr/lib/python2.7/dist-packages/keystoneclient/middleware/auth_token.py", line 582, in __call__
  2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack     return self.app(env, start_response)
  2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack   File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 144, in __call__
  2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack     return resp(environ, start_response)
  2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack   File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 144, in __call__
  2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack     return resp(environ, start_response)
  2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack   File "/usr/local/lib/python2.7/dist-packages/routes/middleware.py", line 131, in __call__
  2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack     response = self.app(environ, start_response)
  2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack   File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 144, in __call__
  2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack     return resp(environ, start_response)
  2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack   File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 130, in __call__
  2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack     resp = self.call_func(req, *args, **self.kwargs)
  2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack   File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 195, in call_func
  2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack     return self.func(req, *args, **kwargs)
  2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack   File "/opt/nova/nova/api/openstack/wsgi.py", line 917, in __call__
  2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack     content_type, body, accept)
  2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack   File "/opt/nova/nova/api/openstack/wsgi.py", line 983, in _process_stack
  2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack     action_result = self.dispatch(meth, request, action_args)
  2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack   File "/opt/nova/nova/api/openstack/wsgi.py", line 1070, in dispatch
  2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack     return method(req=request, **action_args)
  2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack   File "/opt/nova/nova/api/openstack/compute/servers.py", line 520, in detail
  2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack     servers = self._get_servers(req, is_detail=True)
  2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack   File "/opt/nova/nova/api/openstack/compute/servers.py", line 603, in _get_servers
  2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack     want_objects=True)
  2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack   File "/opt/nova/nova/compute/api.py", line 1887, in get_all
  2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack     expected_attrs=expected_attrs)
  2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack   File "/opt/nova/nova/compute/cells_api.py", line 224, in _get_instances_by_filters
  2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack     limit=limit, marker=marker, expected_attrs=fields)
  2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack   File "/opt/nova/nova/objects/base.py", line 112, in wrapper
  2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack     result = fn(cls, context, *args, **kwargs)
  2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack   File "/opt/nova/nova/objects/instance.py", line 629, in get_by_filters
  2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack     use_slave=use_slave)
  2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack   File "/opt/nova/nova/db/api.py", line 665, in instance_get_all_by_filters
  2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack     use_slave=use_slave)
  2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack   File "/opt/nova/nova/db/sqlalchemy/api.py", line 164, in wrapper
  2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack     return f(*args, **kwargs)
  2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack   File "/opt/nova/nova/db/sqlalchemy/api.py", line 2008, in instance_get_all_by_filters
  2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack     filters, exact_match_filter_names)
  2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack   File "/opt/nova/nova/db/sqlalchemy/api.py", line 388, in exact_filter
  2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack     for k, v in value.iteritems():
  2014-09-22 11:31:28.916 20196 TRACE nova.api.openstack AttributeError: 'unicode' object has no attribute 'iteritems'

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1372218/+subscriptions
Follow ups

[Bug 1372218] Re: servers.list, filtering on metadata doesn't work. unicode error
From: Thierry Carrez, 2014-10-01
[Bug 1372218] [NEW] servers.list, filtering on metadata doesn't work. unicode error
From: Sam Morrison, 2014-09-22
References

[Bug 1372218] [NEW] servers.list, filtering on metadata doesn't work. unicode error
From: Sam Morrison, 2014-09-22