yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1682693] Re: tags and not-tags cannot work properly

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: OpenStack Infra <1682693@xxxxxxxxxxxxxxxxxx>
Date: Tue, 18 Apr 2017 03:49:52 -0000
Reply-to: Bug 1682693 <1682693@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
Reviewed:  https://review.openstack.org/456872
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=c4820305d2f9ee8d62bcc708baf3fa6dfe7ca960
Submitter: Jenkins
Branch:    master

commit c4820305d2f9ee8d62bcc708baf3fa6dfe7ca960
Author: Kevin_Zheng <zhengzhenyu@xxxxxxxxxx>
Date:   Fri Apr 14 11:57:59 2017 +0800

    Use deepcopy when process filters in db api
    
    In db API when we process filters, we didn't
    use deepcopy. In cases of "tags" and "not-tags"
    we used pop to get the first tag, filtered out
    results, and then joined with other tags for
    later filtering. When we did pop(), the original
    value was deleted, the key "tags"/"not-tags" remains.
    
    In the cell scenario, both single cell(we will
    query cell0 and the other cell) and multicell,
    as we have to query all the cells in a loop and
    the tags list in the filter will keep popping,
    this will lead to either a HTTP 500 error(popping
    from an empty list) or incorrect result(when
    number of tags in the list is larger than cell
    number, no HTTP 500 will show, but the filter
    results for each cell will be different as
    each loop will pop one tag).
    
    closes-bug: #1682693
    
    Change-Id: Ia2738dd0c7d1842b68c83d0a9e75e26b2f8d492a


** Changed in: nova
       Status: In Progress => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1682693

Title:
  tags and not-tags cannot work properly

Status in OpenStack Compute (nova):
  Fix Released
Status in OpenStack Compute (nova) newton series:
  In Progress
Status in OpenStack Compute (nova) ocata series:
  In Progress

Bug description:
  When there is no tags in the system and user uses "nova list --not-tags xxx",
  HTTP 500 will return:

  root@SZX1000291918:/var/log/nova# nova list --not-tags 123
  /usr/local/lib/python2.7/dist-packages/novaclient/client.py:278: UserWarning: The 'tenant_id' argument is deprecated in Ocata and its use may result in errors in future releases. As 'project_id' is provided, the 'tenant_id' argument will be ignored.
    warnings.warn(msg)
  ERROR (ClientException): Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
  <type 'exceptions.IndexError'> (HTTP 500) (Request-ID: req-225940e7-1976-4ec9-8970-a8c2b4b103ea)

  
  nova-api Logs:
  2017-04-14 09:34:29.952 111211 DEBUG oslo_db.sqlalchemy.engines [req-225940e7-1976-4ec9-8970-a8c2b4b103ea f9e4d94bbfbd4cf1b774108650faa855 d4fd94accf2a4718b2ec8100e4e2e5f1 - default default] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION _check_effective_sql_mode /usr/local/lib/python2.7/dist-packages/oslo_db/sqlalchemy/engines.py:260
  2017-04-14 09:34:29.955 111211 ERROR nova.api.openstack.extensions [req-225940e7-1976-4ec9-8970-a8c2b4b103ea f9e4d94bbfbd4cf1b774108650faa855 d4fd94accf2a4718b2ec8100e4e2e5f1 - default default] Unexpected exception in API method
  2017-04-14 09:34:29.955 111211 ERROR nova.api.openstack.extensions Traceback (most recent call last):
  2017-04-14 09:34:29.955 111211 ERROR nova.api.openstack.extensions   File "/opt/stack/nova/nova/api/openstack/extensions.py", line 336, in wrapped
  2017-04-14 09:34:29.955 111211 ERROR nova.api.openstack.extensions     return f(*args, **kwargs)
  2017-04-14 09:34:29.955 111211 ERROR nova.api.openstack.extensions   File "/opt/stack/nova/nova/api/validation/__init__.py", line 181, in wrapper
  2017-04-14 09:34:29.955 111211 ERROR nova.api.openstack.extensions     return func(*args, **kwargs)
  2017-04-14 09:34:29.955 111211 ERROR nova.api.openstack.extensions   File "/opt/stack/nova/nova/api/validation/__init__.py", line 181, in wrapper
  2017-04-14 09:34:29.955 111211 ERROR nova.api.openstack.extensions     return func(*args, **kwargs)
  2017-04-14 09:34:29.955 111211 ERROR nova.api.openstack.extensions   File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 167, in detail
  2017-04-14 09:34:29.955 111211 ERROR nova.api.openstack.extensions     servers = self._get_servers(req, is_detail=True)
  2017-04-14 09:34:29.955 111211 ERROR nova.api.openstack.extensions   File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 310, in _get_servers
  2017-04-14 09:34:29.955 111211 ERROR nova.api.openstack.extensions     sort_keys=sort_keys, sort_dirs=sort_dirs)
  2017-04-14 09:34:29.955 111211 ERROR nova.api.openstack.extensions   File "/opt/stack/nova/nova/compute/api.py", line 2572, in get_all
  2017-04-14 09:34:29.955 111211 ERROR nova.api.openstack.extensions     sort_dirs=sort_dirs)
  2017-04-14 09:34:29.955 111211 ERROR nova.api.openstack.extensions   File "/opt/stack/nova/nova/compute/api.py", line 2657, in _get_instances_by_filters_all_cells
  2017-04-14 09:34:29.955 111211 ERROR nova.api.openstack.extensions     **kwargs)
  2017-04-14 09:34:29.955 111211 ERROR nova.api.openstack.extensions   File "/opt/stack/nova/nova/compute/api.py", line 2688, in _get_instances_by_filters
  2017-04-14 09:34:29.955 111211 ERROR nova.api.openstack.extensions     expected_attrs=fields, sort_keys=sort_keys, sort_dirs=sort_dirs)
  2017-04-14 09:34:29.955 111211 ERROR nova.api.openstack.extensions   File "/usr/local/lib/python2.7/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper
  2017-04-14 09:34:29.955 111211 ERROR nova.api.openstack.extensions     result = fn(cls, context, *args, **kwargs)
  2017-04-14 09:34:29.955 111211 ERROR nova.api.openstack.extensions   File "/opt/stack/nova/nova/objects/instance.py", line 1220, in get_by_filters
  2017-04-14 09:34:29.955 111211 ERROR nova.api.openstack.extensions     use_slave=use_slave, sort_keys=sort_keys, sort_dirs=sort_dirs)
  2017-04-14 09:34:29.955 111211 ERROR nova.api.openstack.extensions   File "/opt/stack/nova/nova/db/sqlalchemy/api.py", line 236, in wrapper
  2017-04-14 09:34:29.955 111211 ERROR nova.api.openstack.extensions     return f(*args, **kwargs)
  2017-04-14 09:34:29.955 111211 ERROR nova.api.openstack.extensions   File "/opt/stack/nova/nova/objects/instance.py", line 1204, in _get_by_filters_impl
  2017-04-14 09:34:29.955 111211 ERROR nova.api.openstack.extensions     sort_keys=sort_keys, sort_dirs=sort_dirs)
  2017-04-14 09:34:29.955 111211 ERROR nova.api.openstack.extensions   File "/opt/stack/nova/nova/db/api.py", line 763, in instance_get_all_by_filters_sort
  2017-04-14 09:34:29.955 111211 ERROR nova.api.openstack.extensions     sort_dirs=sort_dirs)
  2017-04-14 09:34:29.955 111211 ERROR nova.api.openstack.extensions   File "/opt/stack/nova/nova/db/sqlalchemy/api.py", line 180, in wrapper
  2017-04-14 09:34:29.955 111211 ERROR nova.api.openstack.extensions     return f(*args, **kwargs)
  2017-04-14 09:34:29.955 111211 ERROR nova.api.openstack.extensions   File "/opt/stack/nova/nova/db/sqlalchemy/api.py", line 281, in wrapped
  2017-04-14 09:34:29.955 111211 ERROR nova.api.openstack.extensions     return f(context, *args, **kwargs)
  2017-04-14 09:34:29.955 111211 ERROR nova.api.openstack.extensions   File "/opt/stack/nova/nova/db/sqlalchemy/api.py", line 2211, in instance_get_all_by_filters_sort
  2017-04-14 09:34:29.955 111211 ERROR nova.api.openstack.extensions     first_tag = tags.pop(0)
  2017-04-14 09:34:29.955 111211 ERROR nova.api.openstack.extensions IndexError: pop from empty list
  2017-04-14 09:34:29.955 111211 ERROR nova.api.openstack.extensions
  2017-04-14 09:34:29.958 111211 INFO nova.api.openstack.wsgi [req-225940e7-1976-4ec9-8970-a8c2b4b103ea f9e4d94bbfbd4cf1b774108650faa855 d4fd94accf2a4718b2ec8100e4e2e5f1 - default default] HTTP exception thrown: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
  <type 'exceptions.IndexError'>
  2017-04-14 09:34:29.959 111211 DEBUG nova.api.openstack.wsgi [req-225940e7-1976-4ec9-8970-a8c2b4b103ea f9e4d94bbfbd4cf1b774108650faa855 d4fd94accf2a4718b2ec8100e4e2e5f1 - default default] Returning 500 to user: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
  <type 'exceptions.IndexError'> __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:1041
  2017-04-14 09:34:29.960 111211 INFO nova.osapi_compute.wsgi.server [req-225940e7-1976-4ec9-8970-a8c2b4b103ea f9e4d94bbfbd4cf1b774108650faa855 d4fd94accf2a4718b2ec8100e4e2e5f1 - default default] 10.229.45.145 "GET /v2.1/servers/detail?not-tags=123 HTTP/1.1" status: 500 len: 563 time: 0.1029530

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1682693/+subscriptions
References

[Bug 1682693] [NEW] not-tags cannot work properly
From: Zhenyu Zheng, 2017-04-14