yahoo-eng-team team mailing list archive

[OpenStack Infra <2120723@xxxxxxxxxxxxxxxxxx>]

Reviewed:  https://review.opendev.org/c/openstack/neutron/+/959687
Committed: https://opendev.org/openstack/neutron/commit/c031b59ec116db7c9d4c5e102e5cb7d274ae1975
Submitter: "Zuul (22348)"
Branch:    master

commit c031b59ec116db7c9d4c5e102e5cb7d274ae1975
Author: Rodolfo Alonso Hernandez <ralonsoh@xxxxxxxxxx>
Date:   Thu Sep 4 16:52:41 2025 +0000

    Decode the metadata response before sending it to the VM
    
    The metadata server response can be encoded. The metadata proxy decodes
    it before crafting the HTTP message that will be delivered to the
    virtual machine.
    
    It requires that the HTTP message sent by the metadata server contains
    the proper "Context-Encoding" header, defining the encoding type used.
    This header is not provided by default (the content is not encoded)
    or is empty.
    
    NOTE: unless we provide a method to encode the Nova metadata server
    content, it would not be possible to properly test this patch.
    
    Closes-Bug: #2120723
    Signed-off-by: Rodolfo Alonso Hernandez <ralonsoh@xxxxxxxxxx>
    Change-Id: I747872f031cc5a1a87ced69bb0af645c088143f3


** Changed in: neutron
       Status: In Progress => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/2120723

Title:
  [OVN] Metadata agent errors when returning gzip-encoded data

Status in neutron:
  Fix Released

Bug description:
  In 2025.1 (deployed with OpenStack-Ansible 31.0.0), when requesting
  the user_data for an instance that is gzip-encoded, the client gets a
  502 Bad Gateway response. This worked previously in our 2024.2
  deployment (and rolling back just the neutron-ovn-metadata-agent to
  2024.2 fixes the issue).

  This is the error found in the neutron-ovn-metadata-agent logs:

  2025-08-15 14:20:07.225 657820 ERROR neutron.agent.ovn.metadata.server_socket [-] Error while receiving data.: UnicodeDecodeError: 'utf-8' codec can't decode byte 0x8b in position 1: invalid start byte
  2025-08-15 14:20:07.225 657820 ERROR neutron.agent.ovn.metadata.server_socket Traceback (most recent call last):
  2025-08-15 14:20:07.225 657820 ERROR neutron.agent.ovn.metadata.server_socket   File "/openstack/venvs/neutron-31.0.0/lib/python3.11/site-packages/neutron/agent/ovn/metadata/server_socket.py", line 144, in handle
  2025-08-15 14:20:07.225 657820 ERROR neutron.agent.ovn.metadata.server_socket     res = self._proxy_request(instance_id, project_id, req)
  2025-08-15 14:20:07.225 657820 ERROR neutron.agent.ovn.metadata.server_socket           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  2025-08-15 14:20:07.225 657820 ERROR neutron.agent.ovn.metadata.server_socket   File "/openstack/venvs/neutron-31.0.0/lib/python3.11/site-packages/neutron/agent/ovn/metadata/server_socket.py", line 103, in _proxy_request
  2025-08-15 14:20:07.225 657820 ERROR neutron.agent.ovn.metadata.server_socket     return self._http_response(resp, req)
  2025-08-15 14:20:07.225 657820 ERROR neutron.agent.ovn.metadata.server_socket            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  2025-08-15 14:20:07.225 657820 ERROR neutron.agent.ovn.metadata.server_socket   File "/openstack/venvs/neutron-31.0.0/lib/python3.11/site-packages/neutron/agent/ovn/metadata/server_socket.py", line 49, in _http_response
  2025-08-15 14:20:07.225 657820 ERROR neutron.agent.ovn.metadata.server_socket     out = request.http_version + ' ' + str(_res)
  2025-08-15 14:20:07.225 657820 ERROR neutron.agent.ovn.metadata.server_socket                                        ^^^^^^^^^
  2025-08-15 14:20:07.225 657820 ERROR neutron.agent.ovn.metadata.server_socket   File "/openstack/venvs/neutron-31.0.0/lib/python3.11/site-packages/webob/response.py", line 403, in __str__
  2025-08-15 14:20:07.225 657820 ERROR neutron.agent.ovn.metadata.server_socket     parts += ['', self.body if PY2 else self.text]
  2025-08-15 14:20:07.225 657820 ERROR neutron.agent.ovn.metadata.server_socket                                         ^^^^^^^^^
  2025-08-15 14:20:07.225 657820 ERROR neutron.agent.ovn.metadata.server_socket   File "/openstack/venvs/neutron-31.0.0/lib/python3.11/site-packages/webob/response.py", line 622, in _text__get
  2025-08-15 14:20:07.225 657820 ERROR neutron.agent.ovn.metadata.server_socket     return body.decode(decoding, self.unicode_errors)
  2025-08-15 14:20:07.225 657820 ERROR neutron.agent.ovn.metadata.server_socket            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  2025-08-15 14:20:07.225 657820 ERROR neutron.agent.ovn.metadata.server_socket UnicodeDecodeError: 'utf-8' codec can't decode byte 0x8b in position 1: invalid start byte
  2025-08-15 14:20:07.225 657820 ERROR neutron.agent.ovn.metadata.server_socket
  ----------------------------------------
  Exception occurred during processing of request from
  Traceback (most recent call last):
    File "/usr/lib/python3.11/socketserver.py", line 691, in process_request_thread
      self.finish_request(request, client_address)
    File "/usr/lib/python3.11/socketserver.py", line 361, in finish_request
      self.RequestHandlerClass(request, client_address, self)
    File "/openstack/venvs/neutron-31.0.0/lib/python3.11/site-packages/neutron/agent/ovn/metadata/server_socket.py", line 133, in __init__
      super().__init__(self._conf, has_cache=False, request=request,
    File "/openstack/venvs/neutron-31.0.0/lib/python3.11/site-packages/neutron/agent/metadata/proxy_base.py", line 48, in __init__
      super().__init__(**kwargs)
    File "/usr/lib/python3.11/socketserver.py", line 755, in __init__
      self.handle()
    File "/openstack/venvs/neutron-31.0.0/lib/python3.11/site-packages/neutron/agent/ovn/metadata/server_socket.py", line 165, in handle
      raise exc
    File "/openstack/venvs/neutron-31.0.0/lib/python3.11/site-packages/neutron/agent/ovn/metadata/server_socket.py", line 144, in handle
      res = self._proxy_request(instance_id, project_id, req)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/openstack/venvs/neutron-31.0.0/lib/python3.11/site-packages/neutron/agent/ovn/metadata/server_socket.py", line 103, in _proxy_request
      return self._http_response(resp, req)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/openstack/venvs/neutron-31.0.0/lib/python3.11/site-packages/neutron/agent/ovn/metadata/server_socket.py", line 49, in _http_response
      out = request.http_version + ' ' + str(_res)
                                         ^^^^^^^^^
    File "/openstack/venvs/neutron-31.0.0/lib/python3.11/site-packages/webob/response.py", line 403, in __str__
      parts += ['', self.body if PY2 else self.text]
                                          ^^^^^^^^^
    File "/openstack/venvs/neutron-31.0.0/lib/python3.11/site-packages/webob/response.py", line 622, in _text__get
      return body.decode(decoding, self.unicode_errors)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  UnicodeDecodeError: 'utf-8' codec can't decode byte 0x8b in position 1: invalid start byte
  ----------------------------------------

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/2120723/+subscriptions