yahoo-eng-team team mailing list archive

[OpenStack Infra <1580728@xxxxxxxxxxxxxxxxxx>]

Reviewed:  https://review.openstack.org/461912
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=564958dba6c280eb8e11ae9bbc819c7da6e204bd
Submitter: Jenkins
Branch:    master

commit 564958dba6c280eb8e11ae9bbc819c7da6e204bd
Author: melanie witt <melwittt@xxxxxxxxx>
Date:   Tue May 2 21:47:12 2017 +0000

    Use six.text_type() when logging Instance object
    
    We're seeing a trace in gate jobs, for example:
    
      UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position
      402: ordinal not in range(128)
    
    when attempting to log an Instance object with a unicode display name.
    
    This resurfaced relatively recently because of the change in devstack
    to use the new OSJournalHandler with use_journal=True which is
    suspected of causing some deadlock issues [1] unrelated to this bug.
    
    The problem occurs in code that logs an entire Instance object when
    the object has a field with unicode characters in it (display_name).
    When the object is sent to logging, the UnicodeDecodeError is raised
    while formatting the log record here [2]. This implies an implicit
    conversion attempt to unicode at this point.
    
    I found that with the Instance object, the conversion to unicode fails
    with the UnicodeDecodeError unless the encoding 'utf-8' is explicitly
    specified to six.text_type(). And when specifying an encoding to
    six.text_type(), the argument to convert must be a string, not an
    Instance object, so this does the conversion in two steps as a utility
    function:
    
      1. Get the string representation of the Instance with repr()
      2. Call six.text_type(instance_repr, 'utf-8') passing the encoding
         if not six.PY3
    
    Closes-Bug: #1580728
    
    [1] https://review.openstack.org/#/c/462163
    [2] https://github.com/python/cpython/blob/2e576f5/Lib/logging/__init__.py#L338
    
    Change-Id: I0fc3ae02cb2e401b3240faf0d8b6aa5dc52b91fc


** Changed in: nova
       Status: In Progress => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1580728

Title:
  UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position
  386: ordinal not in range(128) in nova.virt.libvirt.vif:unplug with
  unicode instance.display_name

Status in devstack:
  Confirmed
Status in OpenStack Compute (nova):
  Fix Released
Status in oslo.log:
  New
Status in oslo.versionedobjects:
  New

Bug description:
  I saw this in the n-cpu logs for a xenproject CI run:

  http://logs.openstack.xenproject.org/00/315100/1/check/dsvm-tempest-
  xen/9649dc5/logs/screen-n-cpu.txt.gz

  2016-05-11 16:19:09.457 27252 INFO nova.virt.libvirt.driver [-] [instance: 76c4ad96-87dd-4300-acdc-cbe65d3aa0a6] Instance destroyed successfully.
  Traceback (most recent call last):
    File "/usr/lib/python2.7/logging/__init__.py", line 851, in emit
      msg = self.format(record)
    File "/usr/local/lib/python2.7/dist-packages/oslo_log/handlers.py", line 73, in format
      return logging.StreamHandler.format(self, record)
    File "/usr/lib/python2.7/logging/__init__.py", line 724, in format
      return fmt.format(record)
    File "/usr/local/lib/python2.7/dist-packages/oslo_log/formatters.py", line 265, in format
      return logging.Formatter.format(self, record)
    File "/usr/lib/python2.7/logging/__init__.py", line 464, in format
      record.message = record.getMessage()
    File "/usr/lib/python2.7/logging/__init__.py", line 328, in getMessage
      msg = msg % self.args
  UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 386: ordinal not in range(128)
  Logged from file vif.py, line 966

  That would be logging the vif object in unplug:

  https://github.com/openstack/nova/blob/15abb39ef20ae76d602d50e67e43c3500a00cd3e/nova/virt/libvirt/vif.py#L966

To manage notifications about this bug go to:
https://bugs.launchpad.net/devstack/+bug/1580728/+subscriptions