yahoo-eng-team team mailing list archive
-
yahoo-eng-team team
-
Mailing list archive
-
Message #21598
[Bug 1370954] [NEW] glance 500's when passed image name with a 4-byte utf-8 character
Public bug reported:
Glance currently 500's when passed an image name with a 4-byte utf-8
character in the name. This is because the mysql utf-8 type only handles
up to 3-byte utf-8 characters:
See http://stackoverflow.com/questions/10957238/incorrect-string-value-
when-trying-to-insert-utf-8-into-mysql-via-jdbc
You can replicate this by using the tempest test:
test_create_image_specify_multibyte_character_image_name (the positive not negative one)
You'll need to change utf8_name to
utf8_name = data_utils.rand_name('\xF0\x9F\x92\xA9')
(note removal of unicode prefix)
Backtrace from nova request to glance that triggers this:
2014-09-18 07:35:47.343 843 DEBUG routes.middleware [797abe72-9a70-488e-9254-c71888536278 a2bb050fa6e647398991ebc635741cb1 33760203c2944644
a9ee2a0433f45d0b - - -] Route path: '/images', defaults: {'action': u'create', 'controller': <glance.common.wsgi.Resource object at 0x7f90c
47a6d10>} __call__ /usr/local/lib/python2.7/dist-packages/routes/middleware.py:102
2014-09-18 07:35:47.343 843 DEBUG routes.middleware [797abe72-9a70-488e-9254-c71888536278 a2bb050fa6e647398991ebc635741cb1 33760203c2944644
a9ee2a0433f45d0b - - -] Match dict: {'action': u'create', 'controller': <glance.common.wsgi.Resource object at 0x7f90c47a6d10>} __call__ /u
sr/local/lib/python2.7/dist-packages/routes/middleware.py:103
2014-09-18 07:35:47.348 843 ERROR glance.registry.api.v1.images [797abe72-9a70-488e-9254-c71888536278 a2bb050fa6e647398991ebc635741cb1 3376
0203c2944644a9ee2a0433f45d0b - - -] Unable to create image None
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images Traceback (most recent call last):
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images File "/opt/stack/glance/glance/registry/api/v1/images.py", line 424, in c
reate
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images image_data = self.db_api.image_create(req.context, image_data)
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images File "/opt/stack/glance/glance/db/sqlalchemy/api.py", line 124, in image_
create
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images return _image_update(context, values, None, purge_props=False)
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images File "/usr/local/lib/python2.7/dist-packages/retrying.py", line 92, in wr
apped_f
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images return Retrying(*dargs, **dkw).call(f, *args, **kw)
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images File "/usr/local/lib/python2.7/dist-packages/retrying.py", line 239, in c
all
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images return attempt.get(self._wrap_exception)
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images reraise(self.value[0], self.value[1], self.value[2])
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images File "/usr/local/lib/python2.7/dist-packages/retrying.py", line 233, in call
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images File "/opt/stack/glance/glance/db/sqlalchemy/api.py", line 759, in _image_update
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images image_ref.save(session=session)
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images File "/opt/stack/glance/glance/db/sqlalchemy/models.py", line 77, in save
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images super(GlanceBase, self).save(session or db_api.get_session())
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images File "/usr/local/lib/python2.7/dist-packages/oslo/db/sqlalchemy/models.py", line 48, in save
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images session.flush()
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 1818, in flush
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images self._flush(objects)
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 1936, in _flush
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images transaction.rollback(_capture_exception=True)
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images File "/usr/lib/python2.7/dist-packages/sqlalchemy/util/langhelpers.py", line 58, in __exit__
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images compat.reraise(exc_type, exc_value, exc_tb)
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 1900, in _flush
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images flush_context.execute()
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/unitofwork.py", line 372, in execute
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images rec.execute(self)
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/unitofwork.py", lin
e 525, in execute
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images uow
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/persistence.py", line 64, in save_obj
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images table, insert)
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/persistence.py", line 569, in _emit_insert_statements
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images execute(statement, params)
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 662, in execute
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images params)
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 761, in _execute_clauseelement
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images compiled_sql, distilled_params
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 874, in _execute_context
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images context)
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images File "/usr/local/lib/python2.7/dist-packages/oslo/db/sqlalchemy/compat/handle_error.py", line 125, in _handle_dbapi_exception
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images six.reraise(type(newraise), newraise, sys.exc_info()[2])
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images File "/usr/local/lib/python2.7/dist-packages/oslo/db/sqlalchemy/compat/handle_error.py", line 102, in _handle_dbapi_exception
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images per_fn = fn(ctx)
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images File "/usr/local/lib/python2.7/dist-packages/oslo/db/sqlalchemy/exc_filters.py", line 323, in handler
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images context.is_disconnect)
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images File "/usr/local/lib/python2.7/dist-packages/oslo/db/sqlalchemy/exc_filters.py", line 254, in _raise_operational_errors_directly_filter
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images raise operational_error
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images OperationalError: (OperationalError) (1366, "Incorrect string value: '\\xF0\\x9F\\x92\\xA9-1...' for column 'name' at row 1") 'INSERT INTO images (created_at, updated_at, deleted_at, deleted, id, name, disk_format, container_format, size, virtual_size, status, is_public, checksum, min_disk, min_ram, owner, protected) VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s)' (datetime.datetime(2014, 9, 18, 7, 35, 47, 347058), datetime.datetime(2014, 9, 18, 7, 35, 47, 347072), None, 0, '5ff754de-4b8c-4cca-818c-d235dda03b83', '\xf0\x9f\x92\xa9-1786046285', 'ami', 'ami', 0, None, 'queued', 0, None, 0, 0, '33760203c2944644a9ee2a0433f45d0b', 0)
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images
2014-09-18 07:35:47.356 843 INFO glance.wsgi.server [797abe72-9a70-488e-9254-c71888536278 a2bb050fa6e647398991ebc635741cb1 33760203c2944644a9ee2a0433f45d0b - - -] Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/eventlet/wsgi.py", line 433, in handle_one_response
result = self.application(self.environ, start_response)
** Affects: glance
Importance: Undecided
Status: New
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to Glance.
https://bugs.launchpad.net/bugs/1370954
Title:
glance 500's when passed image name with a 4-byte utf-8 character
Status in OpenStack Image Registry and Delivery Service (Glance):
New
Bug description:
Glance currently 500's when passed an image name with a 4-byte utf-8
character in the name. This is because the mysql utf-8 type only
handles up to 3-byte utf-8 characters:
See http://stackoverflow.com/questions/10957238/incorrect-string-
value-when-trying-to-insert-utf-8-into-mysql-via-jdbc
You can replicate this by using the tempest test:
test_create_image_specify_multibyte_character_image_name (the positive not negative one)
You'll need to change utf8_name to
utf8_name = data_utils.rand_name('\xF0\x9F\x92\xA9')
(note removal of unicode prefix)
Backtrace from nova request to glance that triggers this:
2014-09-18 07:35:47.343 843 DEBUG routes.middleware [797abe72-9a70-488e-9254-c71888536278 a2bb050fa6e647398991ebc635741cb1 33760203c2944644
a9ee2a0433f45d0b - - -] Route path: '/images', defaults: {'action': u'create', 'controller': <glance.common.wsgi.Resource object at 0x7f90c
47a6d10>} __call__ /usr/local/lib/python2.7/dist-packages/routes/middleware.py:102
2014-09-18 07:35:47.343 843 DEBUG routes.middleware [797abe72-9a70-488e-9254-c71888536278 a2bb050fa6e647398991ebc635741cb1 33760203c2944644
a9ee2a0433f45d0b - - -] Match dict: {'action': u'create', 'controller': <glance.common.wsgi.Resource object at 0x7f90c47a6d10>} __call__ /u
sr/local/lib/python2.7/dist-packages/routes/middleware.py:103
2014-09-18 07:35:47.348 843 ERROR glance.registry.api.v1.images [797abe72-9a70-488e-9254-c71888536278 a2bb050fa6e647398991ebc635741cb1 3376
0203c2944644a9ee2a0433f45d0b - - -] Unable to create image None
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images Traceback (most recent call last):
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images File "/opt/stack/glance/glance/registry/api/v1/images.py", line 424, in c
reate
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images image_data = self.db_api.image_create(req.context, image_data)
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images File "/opt/stack/glance/glance/db/sqlalchemy/api.py", line 124, in image_
create
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images return _image_update(context, values, None, purge_props=False)
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images File "/usr/local/lib/python2.7/dist-packages/retrying.py", line 92, in wr
apped_f
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images return Retrying(*dargs, **dkw).call(f, *args, **kw)
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images File "/usr/local/lib/python2.7/dist-packages/retrying.py", line 239, in c
all
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images return attempt.get(self._wrap_exception)
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images reraise(self.value[0], self.value[1], self.value[2])
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images File "/usr/local/lib/python2.7/dist-packages/retrying.py", line 233, in call
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images File "/opt/stack/glance/glance/db/sqlalchemy/api.py", line 759, in _image_update
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images image_ref.save(session=session)
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images File "/opt/stack/glance/glance/db/sqlalchemy/models.py", line 77, in save
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images super(GlanceBase, self).save(session or db_api.get_session())
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images File "/usr/local/lib/python2.7/dist-packages/oslo/db/sqlalchemy/models.py", line 48, in save
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images session.flush()
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 1818, in flush
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images self._flush(objects)
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 1936, in _flush
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images transaction.rollback(_capture_exception=True)
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images File "/usr/lib/python2.7/dist-packages/sqlalchemy/util/langhelpers.py", line 58, in __exit__
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images compat.reraise(exc_type, exc_value, exc_tb)
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 1900, in _flush
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images flush_context.execute()
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/unitofwork.py", line 372, in execute
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images rec.execute(self)
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/unitofwork.py", lin
e 525, in execute
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images uow
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/persistence.py", line 64, in save_obj
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images table, insert)
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/persistence.py", line 569, in _emit_insert_statements
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images execute(statement, params)
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 662, in execute
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images params)
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 761, in _execute_clauseelement
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images compiled_sql, distilled_params
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 874, in _execute_context
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images context)
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images File "/usr/local/lib/python2.7/dist-packages/oslo/db/sqlalchemy/compat/handle_error.py", line 125, in _handle_dbapi_exception
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images six.reraise(type(newraise), newraise, sys.exc_info()[2])
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images File "/usr/local/lib/python2.7/dist-packages/oslo/db/sqlalchemy/compat/handle_error.py", line 102, in _handle_dbapi_exception
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images per_fn = fn(ctx)
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images File "/usr/local/lib/python2.7/dist-packages/oslo/db/sqlalchemy/exc_filters.py", line 323, in handler
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images context.is_disconnect)
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images File "/usr/local/lib/python2.7/dist-packages/oslo/db/sqlalchemy/exc_filters.py", line 254, in _raise_operational_errors_directly_filter
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images raise operational_error
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images OperationalError: (OperationalError) (1366, "Incorrect string value: '\\xF0\\x9F\\x92\\xA9-1...' for column 'name' at row 1") 'INSERT INTO images (created_at, updated_at, deleted_at, deleted, id, name, disk_format, container_format, size, virtual_size, status, is_public, checksum, min_disk, min_ram, owner, protected) VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s)' (datetime.datetime(2014, 9, 18, 7, 35, 47, 347058), datetime.datetime(2014, 9, 18, 7, 35, 47, 347072), None, 0, '5ff754de-4b8c-4cca-818c-d235dda03b83', '\xf0\x9f\x92\xa9-1786046285', 'ami', 'ami', 0, None, 'queued', 0, None, 0, 0, '33760203c2944644a9ee2a0433f45d0b', 0)
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images
2014-09-18 07:35:47.356 843 INFO glance.wsgi.server [797abe72-9a70-488e-9254-c71888536278 a2bb050fa6e647398991ebc635741cb1 33760203c2944644a9ee2a0433f45d0b - - -] Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/eventlet/wsgi.py", line 433, in handle_one_response
result = self.application(self.environ, start_response)
To manage notifications about this bug go to:
https://bugs.launchpad.net/glance/+bug/1370954/+subscriptions
Follow ups
References
