yahoo-eng-team team mailing list archive

[Sean Dague <sean@xxxxxxxxx>]

Public bug reported:

The Nova Meta Data service is full of stack traces. Largely all about no
route to host. These can be seen like this:

2014-04-27 19:10:41.463 30617 INFO nova.metadata.wsgi.server [-] 10.1.0.71 "GET /2009-04-04/meta-data/placement/availability-zone HTTP/1.1" status: 200 len: 119 time: 0.0008161
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/eventlet/greenpool.py", line 80, in _spawn_n_impl
    func(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/eventlet/wsgi.py", line 594, in process_request
    proto.__init__(sock, address, self)
  File "/usr/lib/python2.7/SocketServer.py", line 638, in __init__
    self.handle()
  File "/usr/lib/python2.7/BaseHTTPServer.py", line 342, in handle
    self.handle_one_request()
  File "/usr/local/lib/python2.7/dist-packages/eventlet/wsgi.py", line 231, in handle_one_request
    self.raw_requestline = self.rfile.readline(self.server.url_length_limit)
  File "/usr/lib/python2.7/socket.py", line 476, in readline
    data = self._sock.recv(self._rbufsize)
  File "/usr/local/lib/python2.7/dist-packages/eventlet/greenio.py", line 253, in recv
    return fd.recv(buflen, flags)
error: [Errno 113] No route to host

Example log: http://logs.openstack.org/43/90043/1/check/check-tempest-
dsvm-postgres-full-icehouse/7c4098a/logs/screen-n-api-meta.txt.gz

While these are mostly non fatal, it appears sometimes these trigger a
crash, causing test runs to fail.

This is happening *a lot*: message:"error: [Errno 113] No route to host"
AND (tags:screen-n-api-meta.txt OR tags:screen-n-api.txt)

http://logstash.openstack.org/#eyJzZWFyY2giOiJtZXNzYWdlOlwiZXJyb3I6IFtFcnJubyAxMTNdIE5vIHJvdXRlIHRvIGhvc3RcIiBBTkQgKHRhZ3M6c2NyZWVuLW4tYXBpLW1ldGEudHh0IE9SIHRhZ3M6c2NyZWVuLW4tYXBpLnR4dCkiLCJmaWVsZHMiOltdLCJvZmZzZXQiOjAsInRpbWVmcmFtZSI6IjYwNDgwMCIsImdyYXBobW9kZSI6ImNvdW50IiwidGltZSI6eyJ1c2VyX2ludGVydmFsIjowfSwic3RhbXAiOjEzOTg2ODM5MzQyNTh9

** Affects: nova
     Importance: High
         Status: New

** Changed in: nova
   Importance: Undecided => High

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1313661

Title:
  Nova Meta Data service stack traces with error: [Errno 113] No route
  to host all the time

Status in OpenStack Compute (Nova):
  New

Bug description:
  The Nova Meta Data service is full of stack traces. Largely all about
  no route to host. These can be seen like this:

  2014-04-27 19:10:41.463 30617 INFO nova.metadata.wsgi.server [-] 10.1.0.71 "GET /2009-04-04/meta-data/placement/availability-zone HTTP/1.1" status: 200 len: 119 time: 0.0008161
  Traceback (most recent call last):
    File "/usr/local/lib/python2.7/dist-packages/eventlet/greenpool.py", line 80, in _spawn_n_impl
      func(*args, **kwargs)
    File "/usr/local/lib/python2.7/dist-packages/eventlet/wsgi.py", line 594, in process_request
      proto.__init__(sock, address, self)
    File "/usr/lib/python2.7/SocketServer.py", line 638, in __init__
      self.handle()
    File "/usr/lib/python2.7/BaseHTTPServer.py", line 342, in handle
      self.handle_one_request()
    File "/usr/local/lib/python2.7/dist-packages/eventlet/wsgi.py", line 231, in handle_one_request
      self.raw_requestline = self.rfile.readline(self.server.url_length_limit)
    File "/usr/lib/python2.7/socket.py", line 476, in readline
      data = self._sock.recv(self._rbufsize)
    File "/usr/local/lib/python2.7/dist-packages/eventlet/greenio.py", line 253, in recv
      return fd.recv(buflen, flags)
  error: [Errno 113] No route to host

  Example log: http://logs.openstack.org/43/90043/1/check/check-tempest-
  dsvm-postgres-full-icehouse/7c4098a/logs/screen-n-api-meta.txt.gz

  While these are mostly non fatal, it appears sometimes these trigger a
  crash, causing test runs to fail.

  This is happening *a lot*: message:"error: [Errno 113] No route to
  host" AND (tags:screen-n-api-meta.txt OR tags:screen-n-api.txt)

  http://logstash.openstack.org/#eyJzZWFyY2giOiJtZXNzYWdlOlwiZXJyb3I6IFtFcnJubyAxMTNdIE5vIHJvdXRlIHRvIGhvc3RcIiBBTkQgKHRhZ3M6c2NyZWVuLW4tYXBpLW1ldGEudHh0IE9SIHRhZ3M6c2NyZWVuLW4tYXBpLnR4dCkiLCJmaWVsZHMiOltdLCJvZmZzZXQiOjAsInRpbWVmcmFtZSI6IjYwNDgwMCIsImdyYXBobW9kZSI6ImNvdW50IiwidGltZSI6eyJ1c2VyX2ludGVydmFsIjowfSwic3RhbXAiOjEzOTg2ODM5MzQyNTh9

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1313661/+subscriptions