yahoo-eng-team team mailing list archive

[OpenStack Infra <1795425@xxxxxxxxxxxxxxxxxx>]

Reviewed:  https://review.openstack.org/606967
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=68a689b0f3e5bcdd8939fdadef21de38d06f4dd2
Submitter: Zuul
Branch:    master

commit 68a689b0f3e5bcdd8939fdadef21de38d06f4dd2
Author: Chris Dent <cdent@xxxxxxxxxxxxx>
Date:   Mon Oct 1 15:56:05 2018 +0100

    Clean up header encoding handling in compute API
    
    PEP 3333[1] says request and response headers (within the application)
    should be treated as native `str` (whatever the Python version). It's
    the job of the WSGI server to translate from `str` to reasonable output
    on the outgoing socket connection.
    
    This was already mostly correct but two issues were discovered while
    trying to create integration tests that use the value of the location
    response header when POSTing to create a server. In python2 it was
    working. In Python3 the header had a value of
    
    location: http://192.168.1.76/compute/v2.1/b'http:/192.168.1.76/compute/v2.1/servers/fad04042-850b-443a-9e48-773111cbc981'
    
    (note the b'...' bounding the full url on the end).
    
    This was happening for two reasons:
    
    * nova/api/openstack/compute/servers.py independently encodes the
      location header to utf-8, instead of using the centralized handling
      in nova/api/openstack/wsgi.py
    
      This meant that the value of the location header would arrive, in
      Python 3, at the centralized handling as a bytestring.
    
    * The centralized handling in nova/api/openstack/wsgi.py was incorrectly
      using the six.text_type() method. That is simply an alias to unicode
      in python 2 and str in python3. In python3 when given a bytestring
      as the only argument object.__str()__ is called on the argument.
      Which yields b'whatever'.
    
    At that stage, the handling in the web server which processes a location
    header to check for the presence of a host and prefix already at the
    start of the provided location will find b'...' and do a concatenation
    without any replace.
    
    So, because of all that, this patch includes three changes:
    
    * The server creation location header code does no encoding and relies
      on the centralized handling.
    
    * The centralized handling removes the use of text_type() as a function
      because that is redundant with the safe_encode and safe_decode changes
      in the same loop.
    
    * Doc strings and comments in the ResponseObject are clarified with
      regard to this encoding topic.
    
    Also, comments in Resource._process_stack are updated to correspond with
    the changes above. The code is not changed, as they are already doing
    the right thing: The comment was misrepresenting what was going on.
    There is some duplication of code between these two areas, but the code
    is too inscrutable for me to be willing to change a part that isn't
    presenting an explicit bug.
    
    Tests for the internal production of the location header are adjusted
    to reflect these changes.
    
    [1] https://www.python.org/dev/peps/pep-3333/#a-note-on-string-types
    
    Change-Id: I163c417375678b428ac77aac87ccafaf1f6186ab
    Closes-Bug: #1795425


** Changed in: nova
       Status: In Progress => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1795425

Title:
  create server api sends location header as bytestring in py3

Status in OpenStack Compute (nova):
  Fix Released

Bug description:
  PEP 3333 points out that request and response headers, inside a WSGI
  application, should be native strings. That is: whatever `str` is in
  the version of Python being used:
  https://www.python.org/dev/peps/pep-3333/#a-note-on-string-types

  The create server api returns a location header which is encoded to
  UTF-8 in python, making it a bytestring in python3. This violates the
  spec but also leads to issues when testing nova under wsgi-intercept
  (which removes whatever normalisation most WSGI servers helpfully do
  for "bad" applications). The issues show up when concatenating the
  response header with other values, such as base URLs.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1795425/+subscriptions