yahoo-eng-team team mailing list archive

[Doug Fish <drfish@xxxxxxxxxx>]

I spent some time trying to reproduce this. I don’t believe the problem
is still occurring.

Here’s what I did:

I used devstack to set up an environment on an ubuntu image.
I edited /etc/apache2/sites-available/keystone.conf and changed processes=5 to processes=1 for both virtualhosts (to reduce the number of processes I needed to watch)
and restarted the apache service.

Use ps aux | grep keystone and noted the PIDs for processes named
(wsgi:keystone-pu -k start  
and
(wsgi:keystone-ad -k start
after my keystone.conf http config edit there is only one each of these processes.

I opened two windows and monitored each process with a  loop like:
while true; do lsof -p <processid> | wc -l; sleep 2; done

Then I opened Horizon. I launched 10 instances, terminated them and
launched 10 again. The output from my loops did not change at all during
this time.

** Changed in: django-openstack-auth
       Status: Confirmed => Invalid

** Changed in: horizon
       Status: Confirmed => Invalid

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Dashboard (Horizon).
https://bugs.launchpad.net/bugs/1282089

Title:
  keystone client is leaving hanging connections to the server

Status in django-openstack-auth:
  Invalid
Status in OpenStack Dashboard (Horizon):
  Invalid
Status in python-keystoneclient:
  Fix Released

Bug description:
  This is remarkable noticeable from Horizon which use keystoneclient to
  connect to the keystone server and at each request this later is left
  hanged there which consume the keystone server and at one point this
  will result to having keystone server process exceeding the limit of
  connection that is allowed to handle (ulimit of open filed).

  ## How to check:

  If you have horizon installed so just keep using it normally (creating
  instances ....) while keeping an eye on the server number of opened
  files "lsof -p <keystone-pid>" you can see that the number increment
  pretty quickly.

  To reproduce this bug very fast try launching 40 instances at the same time
  for example using "Instance Count" field.

  ## Why:

  This because keystone client doesn't reuse the http connection pool,
  so in a long running service (e.g. horizon) the effect will be a new
  connections created for each request no connection reuse.

  Patch coming soon with more details.

To manage notifications about this bug go to:
https://bugs.launchpad.net/django-openstack-auth/+bug/1282089/+subscriptions