yahoo-eng-team team mailing list archive

[OpenStack Infra <1746202@xxxxxxxxxxxxxxxxxx>]

Reviewed:  https://review.openstack.org/539387
Committed: https://git.openstack.org/cgit/openstack/cinder/commit/?id=0e2193982ab1e4941ed82c3785e306eb8e4e46ec
Submitter: Zuul
Branch:    master

commit 0e2193982ab1e4941ed82c3785e306eb8e4e46ec
Author: TommyLike <tommylikehu@xxxxxxxxx>
Date:   Wed Jan 31 09:41:13 2018 +0800

    Fix UnicodeDecodeError when decode API input
    
    Convert UnicodeDecodeError to HTTPBadRequest in
    FaultWrapper.
    
    **NOTE**: Cinder will raise 500 error when failed to decode invalid
    unicode character at anyplace where try to collect url
    parematers as below:
    
    ```
    params = req.params
    ```
    
    This patch converts this kind of exception into BadRequest, plus
    an explicit error message.
    
    Change-Id: I816f05084b0a0ef670ef293d381868409b96ed7d
    Closes-Bug: #1746202


** Changed in: cinder
       Status: In Progress => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1746202

Title:
  Invalid query parameter could lead to HTTP 500

Status in Cinder:
  Fix Released
Status in Manila:
  Fix Released
Status in OpenStack Compute (nova):
  In Progress

Bug description:
  Invalid query parameter could lead to HTTP 500, although Nova used JSON Schema verification
  to check input query params, but query like:
  GET /servers?limit=%88 will still lead to HTTP 500, as it failed to parse at webob which is
  pre JSON Schema check.

  GET http://10.76.150.18/compute/v2.1/servers/detail?limit=%88

  Response:

  {
      "computeFault": {
          "message": "Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.\n<type 'exceptions.UnicodeDecodeError'>",
          "code": 500
      }
  }

  Traceback:

  DEBUG nova.api.openstack.wsgi [None req-ee355759-13c3-4f63-a41f-920d7385878d admin admin] Calling method '<bound method ServersController.detail of <nova.api.openstack.compute.servers.ServersController object at 0x
  Jan 30 17:46:56 kevin-dev devstack@n-api.service[4374]: ERROR nova.api.openstack.wsgi [None req-ee355759-13c3-4f63-a41f-920d7385878d admin admin] Unexpected exception in API method: UnicodeDecodeError: 'utf8' codec can't decode byte 0x89 in position 0: invalid start byt
  Jan 30 17:46:56 kevin-dev devstack@n-api.service[4374]: ERROR nova.api.openstack.wsgi Traceback (most recent call last):
  Jan 30 17:46:56 kevin-dev devstack@n-api.service[4374]: ERROR nova.api.openstack.wsgi   File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 803, in wrapped
  Jan 30 17:46:56 kevin-dev devstack@n-api.service[4374]: ERROR nova.api.openstack.wsgi     return f(*args, **kwargs)
  Jan 30 17:46:56 kevin-dev devstack@n-api.service[4374]: ERROR nova.api.openstack.wsgi   File "/opt/stack/nova/nova/api/validation/__init__.py", line 171, in wrapper
  Jan 30 17:46:56 kevin-dev devstack@n-api.service[4374]: ERROR nova.api.openstack.wsgi     req.GET.dict_of_lists(),
  Jan 30 17:46:56 kevin-dev devstack@n-api.service[4374]: ERROR nova.api.openstack.wsgi   File "/usr/local/lib/python2.7/dist-packages/webob/request.py", line 833, in GET
  Jan 30 17:46:56 kevin-dev devstack@n-api.service[4374]: ERROR nova.api.openstack.wsgi     vars = GetDict(data, env)
  Jan 30 17:46:56 kevin-dev devstack@n-api.service[4374]: ERROR nova.api.openstack.wsgi   File "/usr/local/lib/python2.7/dist-packages/webob/multidict.py", line 287, in __init__
  Jan 30 17:46:56 kevin-dev devstack@n-api.service[4374]: ERROR nova.api.openstack.wsgi     MultiDict.__init__(self, data)
  Jan 30 17:46:56 kevin-dev devstack@n-api.service[4374]: ERROR nova.api.openstack.wsgi   File "/usr/local/lib/python2.7/dist-packages/webob/multidict.py", line 38, in __init__
  Jan 30 17:46:56 kevin-dev devstack@n-api.service[4374]: ERROR nova.api.openstack.wsgi     items = list(args[0])
  Jan 30 17:46:56 kevin-dev devstack@n-api.service[4374]: ERROR nova.api.openstack.wsgi   File "/usr/local/lib/python2.7/dist-packages/webob/compat.py", line 126, in parse_qsl_text
  Jan 30 17:46:56 kevin-dev devstack@n-api.service[4374]: ERROR nova.api.openstack.wsgi     yield (x.decode(encoding), y.decode(encoding))
  Jan 30 17:46:56 kevin-dev devstack@n-api.service[4374]: ERROR nova.api.openstack.wsgi   File "/usr/lib/python2.7/encodings/utf_8.py", line 16, in decode
  Jan 30 17:46:56 kevin-dev devstack@n-api.service[4374]: ERROR nova.api.openstack.wsgi     return codecs.utf_8_decode(input, errors, True)
  Jan 30 17:46:56 kevin-dev devstack@n-api.service[4374]: ERROR nova.api.openstack.wsgi UnicodeDecodeError: 'utf8' codec can't decode byte 0x89 in position 0: invalid start byte
  Jan 30 17:46:56 kevin-dev devstack@n-api.service[4374]: ERROR nova.api.openstack.wsgi 
  Jan 30 17:46:56 kevin-dev devstack@n-api.service[4374]: INFO nova.api.openstack.wsgi [None req-ee355759-13c3-4f63-a41f-920d7385878d admin admin] HTTP exception thrown: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API lo
  Jan 30 17:46:56 kevin-dev devstack@n-api.service[4374]: <type 'exceptions.UnicodeDecodeError'>
  Jan 30 17:46:56 kevin-dev devstack@n-api.service[4374]: DEBUG nova.api.openstack.wsgi [None req-ee355759-13c3-4f63-a41f-920d7385878d admin admin] Returning 500 to user: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API l
  Jan 30 17:46:56 kevin-dev devstack@n-api.service[4374]: <type 'exceptions.UnicodeDecodeError'> {{(pid=4377) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:1079}}
  Jan 30 17:46:56 kevin-dev devstack@n-api.service[4374]: INFO nova.api.openstack.requestlog [None req-ee355759-13c3-4f63-a41f-920d7385878d admin admin] 10.8.4.18 "GET /compute/v2.1/servers/detail?limit=%89" status: 500 len: 202 microversion: 2.49 time: 0.531050

To manage notifications about this bug go to:
https://bugs.launchpad.net/cinder/+bug/1746202/+subscriptions