yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1769286] [NEW] NoVNCConsoleTestJSON.test_novnc intermittently fails with: SecurityProxyNegotiationFailed: Failed to negotiate security type with server: No compute auth available: No matching auth scheme: allowed types: 'AuthType.NONE', desired types: '19'

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Matt Riedemann <mriedem.os@xxxxxxxxx>
Date: Sat, 05 May 2018 01:15:56 -0000
Reply-to: Bug 1769286 <1769286@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
Public bug reported:

http://logs.openstack.org/67/565367/10/gate/nova-
next/3e991f7/logs/screen-n-novnc-
cell1.txt.gz?level=TRACE#_May_04_21_50_42_066687

May 04 21:50:42.066687 ubuntu-xenial-rax-iad-0003876942 nova-novncproxy[30706]: ERROR nova.console.websocketproxy Traceback (most recent call last):
May 04 21:50:42.067054 ubuntu-xenial-rax-iad-0003876942 nova-novncproxy[30706]: ERROR nova.console.websocketproxy   File "/opt/stack/new/nova/nova/console/websocketproxy.py", line 293, in new_websocket_client
May 04 21:50:42.067426 ubuntu-xenial-rax-iad-0003876942 nova-novncproxy[30706]: ERROR nova.console.websocketproxy     tsock = self.server.security_proxy.connect(tenant_sock, tsock)
May 04 21:50:42.067805 ubuntu-xenial-rax-iad-0003876942 nova-novncproxy[30706]: ERROR nova.console.websocketproxy   File "/opt/stack/new/nova/nova/console/securityproxy/rfb.py", line 173, in connect
May 04 21:50:42.068089 ubuntu-xenial-rax-iad-0003876942 nova-novncproxy[30706]: ERROR nova.console.websocketproxy     reason=_("No compute auth available: %s") % six.text_type(e))
May 04 21:50:42.068314 ubuntu-xenial-rax-iad-0003876942 nova-novncproxy[30706]: ERROR nova.console.websocketproxy SecurityProxyNegotiationFailed: Failed to negotiate security type with server: No compute auth available: No matching auth scheme: allowed types: 'AuthType.NONE', desired types: '19'
May 04 21:50:42.068528 ubuntu-xenial-rax-iad-0003876942 nova-novncproxy[30706]: ERROR nova.console.websocketproxy 
May 04 21:50:42.090681 ubuntu-xenial-rax-iad-0003876942 nova-novncproxy[30706]: <INFO nova.console.websocketproxy [None req-8e52a9a2-3573-4c6b-85da-e7a75dfb8e89 None None] handler exception: [Errno 32] Broken pipe
May 04 21:50:42.112940 ubuntu-xenial-rax-iad-0003876942 nova-novncproxy[30706]: ERROR nova.console.websocketproxy Traceback (most recent call last):
May 04 21:50:42.113187 ubuntu-xenial-rax-iad-0003876942 nova-novncproxy[30706]: ERROR nova.console.websocketproxy   File "/usr/lib/python2.7/dist-packages/websockify/websocket.py", line 933, in top_new_client
May 04 21:50:42.113427 ubuntu-xenial-rax-iad-0003876942 nova-novncproxy[30706]: ERROR nova.console.websocketproxy     client = self.do_handshake(startsock, address)
May 04 21:50:42.113657 ubuntu-xenial-rax-iad-0003876942 nova-novncproxy[30706]: ERROR nova.console.websocketproxy   File "/usr/lib/python2.7/dist-packages/websockify/websocket.py", line 863, in do_handshake
May 04 21:50:42.113873 ubuntu-xenial-rax-iad-0003876942 nova-novncproxy[30706]: ERROR nova.console.websocketproxy     self.RequestHandlerClass(retsock, address, self)
May 04 21:50:42.114086 ubuntu-xenial-rax-iad-0003876942 nova-novncproxy[30706]: ERROR nova.console.websocketproxy   File "/opt/stack/new/nova/nova/console/websocketproxy.py", line 324, in __init__
May 04 21:50:42.114306 ubuntu-xenial-rax-iad-0003876942 nova-novncproxy[30706]: ERROR nova.console.websocketproxy     websockify.ProxyRequestHandler.__init__(self, *args, **kwargs)
May 04 21:50:42.114517 ubuntu-xenial-rax-iad-0003876942 nova-novncproxy[30706]: ERROR nova.console.websocketproxy   File "/usr/lib/python2.7/dist-packages/websockify/websocket.py", line 114, in __init__
May 04 21:50:42.114727 ubuntu-xenial-rax-iad-0003876942 nova-novncproxy[30706]: ERROR nova.console.websocketproxy     SimpleHTTPRequestHandler.__init__(self, req, addr, server)
May 04 21:50:42.114939 ubuntu-xenial-rax-iad-0003876942 nova-novncproxy[30706]: ERROR nova.console.websocketproxy   File "/usr/lib/python2.7/SocketServer.py", line 652, in __init__
May 04 21:50:42.115150 ubuntu-xenial-rax-iad-0003876942 nova-novncproxy[30706]: ERROR nova.console.websocketproxy     self.handle()
May 04 21:50:42.115359 ubuntu-xenial-rax-iad-0003876942 nova-novncproxy[30706]: ERROR nova.console.websocketproxy   File "/usr/lib/python2.7/dist-packages/websockify/websocket.py", line 581, in handle
May 04 21:50:42.115585 ubuntu-xenial-rax-iad-0003876942 nova-novncproxy[30706]: ERROR nova.console.websocketproxy     SimpleHTTPRequestHandler.handle(self)
May 04 21:50:42.115874 ubuntu-xenial-rax-iad-0003876942 nova-novncproxy[30706]: ERROR nova.console.websocketproxy   File "/usr/lib/python2.7/BaseHTTPServer.py", line 340, in handle
May 04 21:50:42.116125 ubuntu-xenial-rax-iad-0003876942 nova-novncproxy[30706]: ERROR nova.console.websocketproxy     self.handle_one_request()
May 04 21:50:42.116351 ubuntu-xenial-rax-iad-0003876942 nova-novncproxy[30706]: ERROR nova.console.websocketproxy   File "/usr/lib/python2.7/BaseHTTPServer.py", line 328, in handle_one_request
May 04 21:50:42.116611 ubuntu-xenial-rax-iad-0003876942 nova-novncproxy[30706]: ERROR nova.console.websocketproxy     method()
May 04 21:50:42.116838 ubuntu-xenial-rax-iad-0003876942 nova-novncproxy[30706]: ERROR nova.console.websocketproxy   File "/usr/lib/python2.7/dist-packages/websockify/websocket.py", line 543, in do_GET
May 04 21:50:42.117186 ubuntu-xenial-rax-iad-0003876942 nova-novncproxy[30706]: ERROR nova.console.websocketproxy     if not self.handle_websocket():
May 04 21:50:42.117428 ubuntu-xenial-rax-iad-0003876942 nova-novncproxy[30706]: ERROR nova.console.websocketproxy   File "/usr/lib/python2.7/dist-packages/websockify/websocket.py", line 531, in handle_websocket
May 04 21:50:42.117688 ubuntu-xenial-rax-iad-0003876942 nova-novncproxy[30706]: ERROR nova.console.websocketproxy     self.new_websocket_client()
May 04 21:50:42.118061 ubuntu-xenial-rax-iad-0003876942 nova-novncproxy[30706]: ERROR nova.console.websocketproxy   File "/opt/stack/new/nova/nova/console/websocketproxy.py", line 297, in new_websocket_client
May 04 21:50:42.118432 ubuntu-xenial-rax-iad-0003876942 nova-novncproxy[30706]: ERROR nova.console.websocketproxy     tenant_sock.close()
May 04 21:50:42.118709 ubuntu-xenial-rax-iad-0003876942 nova-novncproxy[30706]: ERROR nova.console.websocketproxy   File "/opt/stack/new/nova/nova/console/websocketproxy.py", line 89, in close
May 04 21:50:42.118984 ubuntu-xenial-rax-iad-0003876942 nova-novncproxy[30706]: ERROR nova.console.websocketproxy     self.reqhandler.send_close()
May 04 21:50:42.119261 ubuntu-xenial-rax-iad-0003876942 nova-novncproxy[30706]: ERROR nova.console.websocketproxy   File "/usr/lib/python2.7/dist-packages/websockify/websocket.py", line 416, in send_close
May 04 21:50:42.119727 ubuntu-xenial-rax-iad-0003876942 nova-novncproxy[30706]: ERROR nova.console.websocketproxy     self.request.send(buf)
May 04 21:50:42.120023 ubuntu-xenial-rax-iad-0003876942 nova-novncproxy[30706]: ERROR nova.console.websocketproxy   File "/usr/local/lib/python2.7/dist-packages/eventlet/greenio/base.py", line 384, in send
May 04 21:50:42.120255 ubuntu-xenial-rax-iad-0003876942 nova-novncproxy[30706]: ERROR nova.console.websocketproxy     return self._send_loop(self.fd.send, data, flags)
May 04 21:50:42.120468 ubuntu-xenial-rax-iad-0003876942 nova-novncproxy[30706]: ERROR nova.console.websocketproxy   File "/usr/local/lib/python2.7/dist-packages/eventlet/greenio/base.py", line 371, in _send_loop
May 04 21:50:42.120859 ubuntu-xenial-rax-iad-0003876942 nova-novncproxy[30706]: ERROR nova.console.websocketproxy     return send_method(data, *args)
May 04 21:50:42.121162 ubuntu-xenial-rax-iad-0003876942 nova-novncproxy[30706]: ERROR nova.console.websocketproxy error: [Errno 32] Broken pipe

http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22SecurityProxyNegotiationFailed%3A%20Failed%20to%20negotiate%20security%20type%20with%20server%3A%20No%20compute%20auth%20available%3A%20No%20matching%20auth%20scheme%3A%20allowed%20types%3A%20'AuthType.NONE'%2C%20desired%20types%3A%20'19'%5C%22%20AND%20tags%3A%5C%22screen-n
-novnc-cell1.txt%5C%22&from=7d

This might be a fluke since it's only happened on one change, but it was
in the gate and just a docs patch, so it should be unrelated. Could be
after these changes merged:

https://review.openstack.org/#/c/333990/

https://review.openstack.org/#/c/527812/

** Affects: nova
     Importance: High
         Status: Confirmed


** Tags: console vnc

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1769286

Title:
  NoVNCConsoleTestJSON.test_novnc intermittently fails with:
  SecurityProxyNegotiationFailed: Failed to negotiate security type with
  server: No compute auth available: No matching auth scheme: allowed
  types: 'AuthType.NONE', desired types: '19'

Status in OpenStack Compute (nova):
  Confirmed

Bug description:
  http://logs.openstack.org/67/565367/10/gate/nova-
  next/3e991f7/logs/screen-n-novnc-
  cell1.txt.gz?level=TRACE#_May_04_21_50_42_066687

  May 04 21:50:42.066687 ubuntu-xenial-rax-iad-0003876942 nova-novncproxy[30706]: ERROR nova.console.websocketproxy Traceback (most recent call last):
  May 04 21:50:42.067054 ubuntu-xenial-rax-iad-0003876942 nova-novncproxy[30706]: ERROR nova.console.websocketproxy   File "/opt/stack/new/nova/nova/console/websocketproxy.py", line 293, in new_websocket_client
  May 04 21:50:42.067426 ubuntu-xenial-rax-iad-0003876942 nova-novncproxy[30706]: ERROR nova.console.websocketproxy     tsock = self.server.security_proxy.connect(tenant_sock, tsock)
  May 04 21:50:42.067805 ubuntu-xenial-rax-iad-0003876942 nova-novncproxy[30706]: ERROR nova.console.websocketproxy   File "/opt/stack/new/nova/nova/console/securityproxy/rfb.py", line 173, in connect
  May 04 21:50:42.068089 ubuntu-xenial-rax-iad-0003876942 nova-novncproxy[30706]: ERROR nova.console.websocketproxy     reason=_("No compute auth available: %s") % six.text_type(e))
  May 04 21:50:42.068314 ubuntu-xenial-rax-iad-0003876942 nova-novncproxy[30706]: ERROR nova.console.websocketproxy SecurityProxyNegotiationFailed: Failed to negotiate security type with server: No compute auth available: No matching auth scheme: allowed types: 'AuthType.NONE', desired types: '19'
  May 04 21:50:42.068528 ubuntu-xenial-rax-iad-0003876942 nova-novncproxy[30706]: ERROR nova.console.websocketproxy 
  May 04 21:50:42.090681 ubuntu-xenial-rax-iad-0003876942 nova-novncproxy[30706]: <INFO nova.console.websocketproxy [None req-8e52a9a2-3573-4c6b-85da-e7a75dfb8e89 None None] handler exception: [Errno 32] Broken pipe
  May 04 21:50:42.112940 ubuntu-xenial-rax-iad-0003876942 nova-novncproxy[30706]: ERROR nova.console.websocketproxy Traceback (most recent call last):
  May 04 21:50:42.113187 ubuntu-xenial-rax-iad-0003876942 nova-novncproxy[30706]: ERROR nova.console.websocketproxy   File "/usr/lib/python2.7/dist-packages/websockify/websocket.py", line 933, in top_new_client
  May 04 21:50:42.113427 ubuntu-xenial-rax-iad-0003876942 nova-novncproxy[30706]: ERROR nova.console.websocketproxy     client = self.do_handshake(startsock, address)
  May 04 21:50:42.113657 ubuntu-xenial-rax-iad-0003876942 nova-novncproxy[30706]: ERROR nova.console.websocketproxy   File "/usr/lib/python2.7/dist-packages/websockify/websocket.py", line 863, in do_handshake
  May 04 21:50:42.113873 ubuntu-xenial-rax-iad-0003876942 nova-novncproxy[30706]: ERROR nova.console.websocketproxy     self.RequestHandlerClass(retsock, address, self)
  May 04 21:50:42.114086 ubuntu-xenial-rax-iad-0003876942 nova-novncproxy[30706]: ERROR nova.console.websocketproxy   File "/opt/stack/new/nova/nova/console/websocketproxy.py", line 324, in __init__
  May 04 21:50:42.114306 ubuntu-xenial-rax-iad-0003876942 nova-novncproxy[30706]: ERROR nova.console.websocketproxy     websockify.ProxyRequestHandler.__init__(self, *args, **kwargs)
  May 04 21:50:42.114517 ubuntu-xenial-rax-iad-0003876942 nova-novncproxy[30706]: ERROR nova.console.websocketproxy   File "/usr/lib/python2.7/dist-packages/websockify/websocket.py", line 114, in __init__
  May 04 21:50:42.114727 ubuntu-xenial-rax-iad-0003876942 nova-novncproxy[30706]: ERROR nova.console.websocketproxy     SimpleHTTPRequestHandler.__init__(self, req, addr, server)
  May 04 21:50:42.114939 ubuntu-xenial-rax-iad-0003876942 nova-novncproxy[30706]: ERROR nova.console.websocketproxy   File "/usr/lib/python2.7/SocketServer.py", line 652, in __init__
  May 04 21:50:42.115150 ubuntu-xenial-rax-iad-0003876942 nova-novncproxy[30706]: ERROR nova.console.websocketproxy     self.handle()
  May 04 21:50:42.115359 ubuntu-xenial-rax-iad-0003876942 nova-novncproxy[30706]: ERROR nova.console.websocketproxy   File "/usr/lib/python2.7/dist-packages/websockify/websocket.py", line 581, in handle
  May 04 21:50:42.115585 ubuntu-xenial-rax-iad-0003876942 nova-novncproxy[30706]: ERROR nova.console.websocketproxy     SimpleHTTPRequestHandler.handle(self)
  May 04 21:50:42.115874 ubuntu-xenial-rax-iad-0003876942 nova-novncproxy[30706]: ERROR nova.console.websocketproxy   File "/usr/lib/python2.7/BaseHTTPServer.py", line 340, in handle
  May 04 21:50:42.116125 ubuntu-xenial-rax-iad-0003876942 nova-novncproxy[30706]: ERROR nova.console.websocketproxy     self.handle_one_request()
  May 04 21:50:42.116351 ubuntu-xenial-rax-iad-0003876942 nova-novncproxy[30706]: ERROR nova.console.websocketproxy   File "/usr/lib/python2.7/BaseHTTPServer.py", line 328, in handle_one_request
  May 04 21:50:42.116611 ubuntu-xenial-rax-iad-0003876942 nova-novncproxy[30706]: ERROR nova.console.websocketproxy     method()
  May 04 21:50:42.116838 ubuntu-xenial-rax-iad-0003876942 nova-novncproxy[30706]: ERROR nova.console.websocketproxy   File "/usr/lib/python2.7/dist-packages/websockify/websocket.py", line 543, in do_GET
  May 04 21:50:42.117186 ubuntu-xenial-rax-iad-0003876942 nova-novncproxy[30706]: ERROR nova.console.websocketproxy     if not self.handle_websocket():
  May 04 21:50:42.117428 ubuntu-xenial-rax-iad-0003876942 nova-novncproxy[30706]: ERROR nova.console.websocketproxy   File "/usr/lib/python2.7/dist-packages/websockify/websocket.py", line 531, in handle_websocket
  May 04 21:50:42.117688 ubuntu-xenial-rax-iad-0003876942 nova-novncproxy[30706]: ERROR nova.console.websocketproxy     self.new_websocket_client()
  May 04 21:50:42.118061 ubuntu-xenial-rax-iad-0003876942 nova-novncproxy[30706]: ERROR nova.console.websocketproxy   File "/opt/stack/new/nova/nova/console/websocketproxy.py", line 297, in new_websocket_client
  May 04 21:50:42.118432 ubuntu-xenial-rax-iad-0003876942 nova-novncproxy[30706]: ERROR nova.console.websocketproxy     tenant_sock.close()
  May 04 21:50:42.118709 ubuntu-xenial-rax-iad-0003876942 nova-novncproxy[30706]: ERROR nova.console.websocketproxy   File "/opt/stack/new/nova/nova/console/websocketproxy.py", line 89, in close
  May 04 21:50:42.118984 ubuntu-xenial-rax-iad-0003876942 nova-novncproxy[30706]: ERROR nova.console.websocketproxy     self.reqhandler.send_close()
  May 04 21:50:42.119261 ubuntu-xenial-rax-iad-0003876942 nova-novncproxy[30706]: ERROR nova.console.websocketproxy   File "/usr/lib/python2.7/dist-packages/websockify/websocket.py", line 416, in send_close
  May 04 21:50:42.119727 ubuntu-xenial-rax-iad-0003876942 nova-novncproxy[30706]: ERROR nova.console.websocketproxy     self.request.send(buf)
  May 04 21:50:42.120023 ubuntu-xenial-rax-iad-0003876942 nova-novncproxy[30706]: ERROR nova.console.websocketproxy   File "/usr/local/lib/python2.7/dist-packages/eventlet/greenio/base.py", line 384, in send
  May 04 21:50:42.120255 ubuntu-xenial-rax-iad-0003876942 nova-novncproxy[30706]: ERROR nova.console.websocketproxy     return self._send_loop(self.fd.send, data, flags)
  May 04 21:50:42.120468 ubuntu-xenial-rax-iad-0003876942 nova-novncproxy[30706]: ERROR nova.console.websocketproxy   File "/usr/local/lib/python2.7/dist-packages/eventlet/greenio/base.py", line 371, in _send_loop
  May 04 21:50:42.120859 ubuntu-xenial-rax-iad-0003876942 nova-novncproxy[30706]: ERROR nova.console.websocketproxy     return send_method(data, *args)
  May 04 21:50:42.121162 ubuntu-xenial-rax-iad-0003876942 nova-novncproxy[30706]: ERROR nova.console.websocketproxy error: [Errno 32] Broken pipe

  http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22SecurityProxyNegotiationFailed%3A%20Failed%20to%20negotiate%20security%20type%20with%20server%3A%20No%20compute%20auth%20available%3A%20No%20matching%20auth%20scheme%3A%20allowed%20types%3A%20'AuthType.NONE'%2C%20desired%20types%3A%20'19'%5C%22%20AND%20tags%3A%5C%22screen-n
  -novnc-cell1.txt%5C%22&from=7d

  This might be a fluke since it's only happened on one change, but it
  was in the gate and just a docs patch, so it should be unrelated.
  Could be after these changes merged:

  https://review.openstack.org/#/c/333990/

  https://review.openstack.org/#/c/527812/

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1769286/+subscriptions
Follow ups

[Bug 1769286] Re: NoVNCConsoleTestJSON.test_novnc intermittently fails with: SecurityProxyNegotiationFailed: Failed to negotiate security type with server: No compute auth available: No matching auth scheme: allowed types: 'AuthType.NONE', desired types: '19'
From: Matt Riedemann, 2018-05-08