yahoo-eng-team team mailing list archive

[Matt Riedemann <mriedem.os@xxxxxxxxx>]

** Changed in: nova
   Importance: Undecided => High

** Also affects: nova/rocky
   Importance: Undecided
       Status: New

** Changed in: nova/rocky
       Status: New => Triaged

** Changed in: nova/rocky
   Importance: Undecided => High

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1809418

Title:
  ironic serial console port validate always failed

Status in OpenStack Compute (nova):
  In Progress
Status in OpenStack Compute (nova) rocky series:
  Triaged

Bug description:
  I deployed latest ironic and used IPMI_socat console interface alongs
  with nova-serialproxy service, but always failed to open console in
  dashboard, and the nova-serialproxy error as follows:

  2018-12-20 19:11:16.642 15391 ERROR nova.console.websocketproxy Traceback (most recent call last):
  2018-12-20 19:11:16.642 15391 ERROR nova.console.websocketproxy   File "/usr/lib/python2.7/site-packages/websockify/websocket.py", line 930, in top_new_client
  2018-12-20 19:11:16.642 15391 ERROR nova.console.websocketproxy     client = self.do_handshake(startsock, address)
  2018-12-20 19:11:16.642 15391 ERROR nova.console.websocketproxy   File "/usr/lib/python2.7/site-packages/websockify/websocket.py", line 860, in do_handshake
  2018-12-20 19:11:16.642 15391 ERROR nova.console.websocketproxy     self.RequestHandlerClass(retsock, address, self)
  2018-12-20 19:11:16.642 15391 ERROR nova.console.websocketproxy   File "/usr/lib/python2.7/site-packages/nova/console/websocketproxy.py", line 324, in __init__
  2018-12-20 19:11:16.642 15391 ERROR nova.console.websocketproxy     websockify.ProxyRequestHandler.__init__(self, *args, **kwargs)
  2018-12-20 19:11:16.642 15391 ERROR nova.console.websocketproxy   File "/usr/lib/python2.7/site-packages/websockify/websocket.py", line 114, in __init__
  2018-12-20 19:11:16.642 15391 ERROR nova.console.websocketproxy     SimpleHTTPRequestHandler.__init__(self, req, addr, server)
  2018-12-20 19:11:16.642 15391 ERROR nova.console.websocketproxy   File "/usr/lib64/python2.7/SocketServer.py", line 649, in __init__
  2018-12-20 19:11:16.642 15391 ERROR nova.console.websocketproxy     self.handle()
  2018-12-20 19:11:16.642 15391 ERROR nova.console.websocketproxy   File "/usr/lib/python2.7/site-packages/websockify/websocket.py", line 581, in handle
  2018-12-20 19:11:16.642 15391 ERROR nova.console.websocketproxy     SimpleHTTPRequestHandler.handle(self)
  2018-12-20 19:11:16.642 15391 ERROR nova.console.websocketproxy   File "/usr/lib64/python2.7/BaseHTTPServer.py", line 340, in handle
  2018-12-20 19:11:16.642 15391 ERROR nova.console.websocketproxy     self.handle_one_request()
  2018-12-20 19:11:16.642 15391 ERROR nova.console.websocketproxy   File "/usr/lib64/python2.7/BaseHTTPServer.py", line 328, in handle_one_request
  2018-12-20 19:11:16.642 15391 ERROR nova.console.websocketproxy     method()
  2018-12-20 19:11:16.642 15391 ERROR nova.console.websocketproxy   File "/usr/lib/python2.7/site-packages/websockify/websocket.py", line 543, in do_GET
  2018-12-20 19:11:16.642 15391 ERROR nova.console.websocketproxy     if not self.handle_websocket():
  2018-12-20 19:11:16.642 15391 ERROR nova.console.websocketproxy   File "/usr/lib/python2.7/site-packages/websockify/websocket.py", line 531, in handle_websocket
  2018-12-20 19:11:16.642 15391 ERROR nova.console.websocketproxy     self.new_websocket_client()
  2018-12-20 19:11:16.642 15391 ERROR nova.console.websocketproxy   File "/usr/lib/python2.7/site-packages/nova/console/websocketproxy.py", line 236, in new_websocket_client
  2018-12-20 19:11:16.642 15391 ERROR nova.console.websocketproxy     connect_info = self._get_connect_info(ctxt, token)
  2018-12-20 19:11:16.642 15391 ERROR nova.console.websocketproxy   File "/usr/lib/python2.7/site-packages/nova/console/websocketproxy.py", line 192, in _get_connect_info
  2018-12-20 19:11:16.642 15391 ERROR nova.console.websocketproxy     connect_info = self._get_connect_info_database(ctxt, token)
  2018-12-20 19:11:16.642 15391 ERROR nova.console.websocketproxy   File "/usr/lib/python2.7/site-packages/nova/console/websocketproxy.py", line 166, in _get_connect_info_database
  2018-12-20 19:11:16.642 15391 ERROR nova.console.websocketproxy     raise exception.InvalidToken(token='***')
  2018-12-20 19:11:16.642 15391 ERROR nova.console.websocketproxy InvalidToken: The token '***' is invalid or has expired
  2018-12-20 19:11:16.642 15391 ERROR nova.console.websocketproxy
  2018-12-20 19:11:40.867 14996 DEBUG nova.console.websocketproxy [-] 197.3.133.254: new handler Process vmsg /usr/lib/python2.7/site-packages/websockify/websocket.py:875

  So I created a breakpoint in this code and found the port type must be
  str in nova-serialproxy side as we converted it to str type forcefully
  before rpc validate_console to compute service[1]. But on the compute
  side, the drive may be return int type like ironic serial console[2].
  So the validate_console_port may always return False as type
  inconsistence[3].

  
  [1] https://github.com/openstack/nova/blob/master/nova/console/websocketproxy.py#L123-L125
  [2] https://github.com/openstack/nova/blob/master/nova/virt/ironic/driver.py#L1819
  [3] https://github.com/openstack/nova/blob/master/nova/compute/manager.py#L5369

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1809418/+subscriptions