yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1688189] [NEW] Member create raises 500 error for unicode charater values

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Abhishek Kekane <abhishek.kekane@xxxxxxxxxxx>
Date: Thu, 04 May 2017 07:00:39 -0000
Reply-to: Bug 1688189 <1688189@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
Public bug reported:

If user passes member as Unicode character while creating member for
image then he will get HTTP 500 error.

Steps to reproduce:

1. Create image
2. Assign member to image using
   $ glance member-create e64f4347-51d6-4f97-8b6e-02e40c7ecb30 𠜎

   or using curl
   $ curl -g -i -X POST http://10.232.48.198:9292/v2/images/e64f4347-51d6-4f97-8b6e-02e40c7ecb30/members -H "User-Agent: python-glanceclient" -H "Content-Type: application/json" -H "X-Auth-Token: gAAAAABZCs01HPjCjKDkYnWQECtu9dYOxySXXrMH-lH4xO9xZBtl4MXNIPbTwkuCWSQ4EOh0tKvOPz55DmMdyOM0RYziM-qNE2Jikncq2oExZvf6k8OZYj_Vad5Q04p_uCU0Rg-9b94mVFfv_HaImCnT9ofO6RQZyNLOf1zc-AOzQPOMnjv9e4g" -d '{"member": "𠜎"}'

Output:
<html>
 <head>
  <title>500 Internal Server Error</title>
 </head>
 <body>
  <h1>500 Internal Server Error</h1>
  The server has either erred or is incapable of performing the requested operation.<br /><br />

 </body>
</html>

API Logs:
500 Internal Server Error: The server has either erred or is incapable of performing the requested operation. (HTTP 500)

2017-05-04 12:18:14.460 TRACE glance.common.wsgi     self._flush(objects)
2017-05-04 12:18:14.460 TRACE glance.common.wsgi   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 2259, in _flush
2017-05-04 12:18:14.460 TRACE glance.common.wsgi     transaction.rollback(_capture_exception=True)
2017-05-04 12:18:14.460 TRACE glance.common.wsgi   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/util/langhelpers.py", line 66, in __exit__
2017-05-04 12:18:14.460 TRACE glance.common.wsgi     compat.reraise(exc_type, exc_value, exc_tb)
2017-05-04 12:18:14.460 TRACE glance.common.wsgi   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 2223, in _flush
2017-05-04 12:18:14.460 TRACE glance.common.wsgi     flush_context.execute()
2017-05-04 12:18:14.460 TRACE glance.common.wsgi   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/unitofwork.py", line 389, in execute
2017-05-04 12:18:14.460 TRACE glance.common.wsgi     rec.execute(self)
2017-05-04 12:18:14.460 TRACE glance.common.wsgi   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/unitofwork.py", line 548, in execute
2017-05-04 12:18:14.460 TRACE glance.common.wsgi     uow
2017-05-04 12:18:14.460 TRACE glance.common.wsgi   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/persistence.py", line 181, in save_obj
2017-05-04 12:18:14.460 TRACE glance.common.wsgi     mapper, table, insert)
2017-05-04 12:18:14.460 TRACE glance.common.wsgi   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/persistence.py", line 835, in _emit_insert_statements
2017-05-04 12:18:14.460 TRACE glance.common.wsgi     execute(statement, params)
2017-05-04 12:18:14.460 TRACE glance.common.wsgi   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 945, in execute
2017-05-04 12:18:14.460 TRACE glance.common.wsgi     return meth(self, multiparams, params)
2017-05-04 12:18:14.460 TRACE glance.common.wsgi   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/sql/elements.py", line 263, in _execute_on_connection
2017-05-04 12:18:14.460 TRACE glance.common.wsgi     return connection._execute_clauseelement(self, multiparams, params)
2017-05-04 12:18:14.460 TRACE glance.common.wsgi   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1053, in _execute_clauseelement
2017-05-04 12:18:14.460 TRACE glance.common.wsgi     compiled_sql, distilled_params
2017-05-04 12:18:14.460 TRACE glance.common.wsgi   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1189, in _execute_context
2017-05-04 12:18:14.460 TRACE glance.common.wsgi     context)
2017-05-04 12:18:14.460 TRACE glance.common.wsgi   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1398, in _handle_dbapi_exception
2017-05-04 12:18:14.460 TRACE glance.common.wsgi     util.raise_from_cause(newraise, exc_info)
2017-05-04 12:18:14.460 TRACE glance.common.wsgi   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/util/compat.py", line 203, in raise_from_cause
2017-05-04 12:18:14.460 TRACE glance.common.wsgi     reraise(type(exception), exception, tb=exc_tb, cause=cause)
2017-05-04 12:18:14.460 TRACE glance.common.wsgi   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1182, in _execute_context
2017-05-04 12:18:14.460 TRACE glance.common.wsgi     context)
2017-05-04 12:18:14.460 TRACE glance.common.wsgi   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", line 470, in do_execute
2017-05-04 12:18:14.460 TRACE glance.common.wsgi     cursor.execute(statement, parameters)
2017-05-04 12:18:14.460 TRACE glance.common.wsgi   File "/usr/local/lib/python2.7/dist-packages/pymysql/cursors.py", line 166, in execute
2017-05-04 12:18:14.460 TRACE glance.common.wsgi     result = self._query(query)
2017-05-04 12:18:14.460 TRACE glance.common.wsgi   File "/usr/local/lib/python2.7/dist-packages/pymysql/cursors.py", line 322, in _query
2017-05-04 12:18:14.460 TRACE glance.common.wsgi     conn.query(q)
2017-05-04 12:18:14.460 TRACE glance.common.wsgi   File "/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 856, in query
2017-05-04 12:18:14.460 TRACE glance.common.wsgi     self._affected_rows = self._read_query_result(unbuffered=unbuffered)
2017-05-04 12:18:14.460 TRACE glance.common.wsgi   File "/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 1057, in _read_query_result
2017-05-04 12:18:14.460 TRACE glance.common.wsgi     result.read()
2017-05-04 12:18:14.460 TRACE glance.common.wsgi   File "/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 1340, in read
2017-05-04 12:18:14.460 TRACE glance.common.wsgi     first_packet = self.connection._read_packet()
2017-05-04 12:18:14.460 TRACE glance.common.wsgi   File "/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 1014, in _read_packet
2017-05-04 12:18:14.460 TRACE glance.common.wsgi     packet.check_error()
2017-05-04 12:18:14.460 TRACE glance.common.wsgi   File "/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 393, in check_error
2017-05-04 12:18:14.460 TRACE glance.common.wsgi     err.raise_mysql_exception(self._data)
2017-05-04 12:18:14.460 TRACE glance.common.wsgi   File "/usr/local/lib/python2.7/dist-packages/pymysql/err.py", line 107, in raise_mysql_exception
2017-05-04 12:18:14.460 TRACE glance.common.wsgi     raise errorclass(errno, errval)
2017-05-04 12:18:14.460 TRACE glance.common.wsgi DBDataError: (pymysql.err.InternalError) (1366, u"Incorrect string value: '\\xF0\\xA0\\x9C\\x8E' for column 'member' at row 1") [SQL: u'INSERT INTO image_members (created_at, updated_at, deleted_at, deleted, image_id, member, can_share, status) VALUES (%(created_at)s, %(updated_at)s, %(deleted_at)s, %(deleted)s, %(image_id)s, %(member)s, %(can_share)s, %(status)s)'] [parameters: {'status': 'pending', 'deleted': 0, 'created_at': datetime.datetime(2017, 5, 4, 6, 48, 14, 458976), 'updated_at': datetime.datetime(2017, 5, 4, 6, 48, 14, 458985), 'member': u'\U0002070e', 'image_id': u'e64f4347-51d6-4f97-8b6e-02e40c7ecb30', 'can_share': 0, 'deleted_at': None}]

** Affects: glance
     Importance: Undecided
     Assignee: Neha Pandey (nehapandey)
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to Glance.
https://bugs.launchpad.net/bugs/1688189

Title:
  Member create raises 500 error for unicode charater values

Status in Glance:
  New

Bug description:
  If user passes member as Unicode character while creating member for
  image then he will get HTTP 500 error.

  Steps to reproduce:

  1. Create image
  2. Assign member to image using
     $ glance member-create e64f4347-51d6-4f97-8b6e-02e40c7ecb30 𠜎

     or using curl
     $ curl -g -i -X POST http://10.232.48.198:9292/v2/images/e64f4347-51d6-4f97-8b6e-02e40c7ecb30/members -H "User-Agent: python-glanceclient" -H "Content-Type: application/json" -H "X-Auth-Token: gAAAAABZCs01HPjCjKDkYnWQECtu9dYOxySXXrMH-lH4xO9xZBtl4MXNIPbTwkuCWSQ4EOh0tKvOPz55DmMdyOM0RYziM-qNE2Jikncq2oExZvf6k8OZYj_Vad5Q04p_uCU0Rg-9b94mVFfv_HaImCnT9ofO6RQZyNLOf1zc-AOzQPOMnjv9e4g" -d '{"member": "𠜎"}'

  Output:
  <html>
   <head>
    <title>500 Internal Server Error</title>
   </head>
   <body>
    <h1>500 Internal Server Error</h1>
    The server has either erred or is incapable of performing the requested operation.<br /><br />

   </body>
  </html>

  API Logs:
  500 Internal Server Error: The server has either erred or is incapable of performing the requested operation. (HTTP 500)

  2017-05-04 12:18:14.460 TRACE glance.common.wsgi     self._flush(objects)
  2017-05-04 12:18:14.460 TRACE glance.common.wsgi   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 2259, in _flush
  2017-05-04 12:18:14.460 TRACE glance.common.wsgi     transaction.rollback(_capture_exception=True)
  2017-05-04 12:18:14.460 TRACE glance.common.wsgi   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/util/langhelpers.py", line 66, in __exit__
  2017-05-04 12:18:14.460 TRACE glance.common.wsgi     compat.reraise(exc_type, exc_value, exc_tb)
  2017-05-04 12:18:14.460 TRACE glance.common.wsgi   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 2223, in _flush
  2017-05-04 12:18:14.460 TRACE glance.common.wsgi     flush_context.execute()
  2017-05-04 12:18:14.460 TRACE glance.common.wsgi   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/unitofwork.py", line 389, in execute
  2017-05-04 12:18:14.460 TRACE glance.common.wsgi     rec.execute(self)
  2017-05-04 12:18:14.460 TRACE glance.common.wsgi   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/unitofwork.py", line 548, in execute
  2017-05-04 12:18:14.460 TRACE glance.common.wsgi     uow
  2017-05-04 12:18:14.460 TRACE glance.common.wsgi   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/persistence.py", line 181, in save_obj
  2017-05-04 12:18:14.460 TRACE glance.common.wsgi     mapper, table, insert)
  2017-05-04 12:18:14.460 TRACE glance.common.wsgi   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/persistence.py", line 835, in _emit_insert_statements
  2017-05-04 12:18:14.460 TRACE glance.common.wsgi     execute(statement, params)
  2017-05-04 12:18:14.460 TRACE glance.common.wsgi   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 945, in execute
  2017-05-04 12:18:14.460 TRACE glance.common.wsgi     return meth(self, multiparams, params)
  2017-05-04 12:18:14.460 TRACE glance.common.wsgi   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/sql/elements.py", line 263, in _execute_on_connection
  2017-05-04 12:18:14.460 TRACE glance.common.wsgi     return connection._execute_clauseelement(self, multiparams, params)
  2017-05-04 12:18:14.460 TRACE glance.common.wsgi   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1053, in _execute_clauseelement
  2017-05-04 12:18:14.460 TRACE glance.common.wsgi     compiled_sql, distilled_params
  2017-05-04 12:18:14.460 TRACE glance.common.wsgi   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1189, in _execute_context
  2017-05-04 12:18:14.460 TRACE glance.common.wsgi     context)
  2017-05-04 12:18:14.460 TRACE glance.common.wsgi   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1398, in _handle_dbapi_exception
  2017-05-04 12:18:14.460 TRACE glance.common.wsgi     util.raise_from_cause(newraise, exc_info)
  2017-05-04 12:18:14.460 TRACE glance.common.wsgi   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/util/compat.py", line 203, in raise_from_cause
  2017-05-04 12:18:14.460 TRACE glance.common.wsgi     reraise(type(exception), exception, tb=exc_tb, cause=cause)
  2017-05-04 12:18:14.460 TRACE glance.common.wsgi   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1182, in _execute_context
  2017-05-04 12:18:14.460 TRACE glance.common.wsgi     context)
  2017-05-04 12:18:14.460 TRACE glance.common.wsgi   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", line 470, in do_execute
  2017-05-04 12:18:14.460 TRACE glance.common.wsgi     cursor.execute(statement, parameters)
  2017-05-04 12:18:14.460 TRACE glance.common.wsgi   File "/usr/local/lib/python2.7/dist-packages/pymysql/cursors.py", line 166, in execute
  2017-05-04 12:18:14.460 TRACE glance.common.wsgi     result = self._query(query)
  2017-05-04 12:18:14.460 TRACE glance.common.wsgi   File "/usr/local/lib/python2.7/dist-packages/pymysql/cursors.py", line 322, in _query
  2017-05-04 12:18:14.460 TRACE glance.common.wsgi     conn.query(q)
  2017-05-04 12:18:14.460 TRACE glance.common.wsgi   File "/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 856, in query
  2017-05-04 12:18:14.460 TRACE glance.common.wsgi     self._affected_rows = self._read_query_result(unbuffered=unbuffered)
  2017-05-04 12:18:14.460 TRACE glance.common.wsgi   File "/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 1057, in _read_query_result
  2017-05-04 12:18:14.460 TRACE glance.common.wsgi     result.read()
  2017-05-04 12:18:14.460 TRACE glance.common.wsgi   File "/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 1340, in read
  2017-05-04 12:18:14.460 TRACE glance.common.wsgi     first_packet = self.connection._read_packet()
  2017-05-04 12:18:14.460 TRACE glance.common.wsgi   File "/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 1014, in _read_packet
  2017-05-04 12:18:14.460 TRACE glance.common.wsgi     packet.check_error()
  2017-05-04 12:18:14.460 TRACE glance.common.wsgi   File "/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 393, in check_error
  2017-05-04 12:18:14.460 TRACE glance.common.wsgi     err.raise_mysql_exception(self._data)
  2017-05-04 12:18:14.460 TRACE glance.common.wsgi   File "/usr/local/lib/python2.7/dist-packages/pymysql/err.py", line 107, in raise_mysql_exception
  2017-05-04 12:18:14.460 TRACE glance.common.wsgi     raise errorclass(errno, errval)
  2017-05-04 12:18:14.460 TRACE glance.common.wsgi DBDataError: (pymysql.err.InternalError) (1366, u"Incorrect string value: '\\xF0\\xA0\\x9C\\x8E' for column 'member' at row 1") [SQL: u'INSERT INTO image_members (created_at, updated_at, deleted_at, deleted, image_id, member, can_share, status) VALUES (%(created_at)s, %(updated_at)s, %(deleted_at)s, %(deleted)s, %(image_id)s, %(member)s, %(can_share)s, %(status)s)'] [parameters: {'status': 'pending', 'deleted': 0, 'created_at': datetime.datetime(2017, 5, 4, 6, 48, 14, 458976), 'updated_at': datetime.datetime(2017, 5, 4, 6, 48, 14, 458985), 'member': u'\U0002070e', 'image_id': u'e64f4347-51d6-4f97-8b6e-02e40c7ecb30', 'can_share': 0, 'deleted_at': None}]

To manage notifications about this bug go to:
https://bugs.launchpad.net/glance/+bug/1688189/+subscriptions
Follow ups

[Bug 1688189] Re: Member create raises 500 error for unicode charater values
From: OpenStack Infra, 2018-01-02