← Back to team overview

yahoo-eng-team team mailing list archive

[Bug 1179709] Re: keystone shuts down it's own listening socket with 'too many files open'

 

Seems we hit this today on a Havana build.  Nothing really going on with
the system... went to dinner, user came back and couldn't log in.  I
logged on to controller and noticed traces in keystone, restarted
services and back in business, but not sure what's actually causing
this.

** Changed in: keystone
       Status: Expired => New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to Keystone.
https://bugs.launchpad.net/bugs/1179709

Title:
  keystone shuts down it's own listening socket with 'too many files
  open'

Status in OpenStack Identity (Keystone):
  New

Bug description:
  We're running with mysql - I can grab a rendered config file for you
  if needed.

  Here is what I see in the log:
  Traceback (most recent call last):
    File "/opt/stack/venvs/keystone/local/lib/python2.7/site-packages/eventlet/hubs/poll.py", line 97, in wait
      readers.get(fileno, noop).cb(fileno)
    File "/opt/stack/venvs/keystone/local/lib/python2.7/site-packages/eventlet/greenthread.py", line 194, in main
      result = function(*args, **kwargs)
    File "/opt/stack/venvs/keystone/local/lib/python2.7/site-packages/keystone/common/wsgi.py", line 135, in _run
      log=WritableLogger(log))
    File "/opt/stack/venvs/keystone/local/lib/python2.7/site-packages/eventlet/wsgi.py", line 663, in server
      client_socket = sock.accept()
    File "/opt/stack/venvs/keystone/local/lib/python2.7/site-packages/eventlet/greenio.py", line 166, in accept
      res = socket_accept(fd)
    File "/opt/stack/venvs/keystone/local/lib/python2.7/site-packages/eventlet/greenio.py", line 56, in socket_accept
      return descriptor.accept()
    File "/usr/lib/python2.7/socket.py", line 202, in accept
      sock, addr = self._sock.accept()
  error: [Errno 24] Too many open files
  Removing descriptor: 5

  keystone-all is still running, netstat -anp shows:
  tcp        0      0 0.0.0.0:35357           0.0.0.0:*               LISTEN      25530/python    
  unix  3      [ ]         STREAM     CONNECTED     44834812 25530/python        

  restarting it gets me:
  root@ubuntu:~# netstat -anp | grep 25267
  tcp        0      0 0.0.0.0:35357           0.0.0.0:*               LISTEN      25267/python    
  tcp        0      0 0.0.0.0:5000            0.0.0.0:*               LISTEN      25267/python    
  unix  3      [ ]         STREAM     CONNECTED     44848565 25267/python        

  Which is rather more useful.

To manage notifications about this bug go to:
https://bugs.launchpad.net/keystone/+bug/1179709/+subscriptions