yahoo-eng-team team mailing list archive

[Ken'ichi Ohmichi <ken1ohmichi@xxxxxxxxx>]

ah, my point is duplicated to
https://bugs.launchpad.net/nova/+bug/1630664

** Changed in: keystone
       Status: Confirmed => Opinion

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Identity (keystone).
https://bugs.launchpad.net/bugs/1535640

Title:
  Sometimes getting unable to establish connection error in Keystone
  client

Status in OpenStack Identity (keystone):
  Opinion

Bug description:
  In Openstack HA environment, sometimes  Kesystone client is throwing
  the error "unable to establish connection error". This is happening
  once in fifteen times. The error and stack trace of the error log is
  given below.

  Error: 1) keystoneauth1.exceptions.connection.ConnectFailure: Unable to establish connection to http://Controller-VIP:8004/v1/ca1d10786fdc41698e276a8f22d98741/stacks
  2) ConnectionRefused: Unable to establish connection to http://Controller-VIP:9696/v2.0/ports.json?tenant_id=ca1d10786fdc41698e276a8f22d98741&device_id=183bea96-b0da-42a5-a731-713882598624

  Stack trace of the error:

  NFO (connectionpool) Starting new HTTP connection (1): Controller-VIP
  Traceback (most recent call last):
    File "/usr/bin/heat", line 10, in <module>
      sys.exit(main())
    File "/usr/lib/python2.7/dist-packages/heatclient/shell.py", line 706, in main
      HeatShell().main(args)
    File "/usr/lib/python2.7/dist-packages/heatclient/shell.py", line 656, in main
      args.func(client, args)
    File "/usr/lib/python2.7/dist-packages/heatclient/v1/shell.py", line 137, in do_stack_create
      hc.stacks.create(**fields)
    File "/usr/lib/python2.7/dist-packages/heatclient/v1/stacks.py", line 134, in create
      data=kwargs, headers=headers)
    File "/usr/lib/python2.7/dist-packages/heatclient/common/http.py", line 266, in json_request
      resp = self._http_request(url, method, **kwargs)
    File "/usr/lib/python2.7/dist-packages/heatclient/common/http.py", line 358, in _http_request
      raise_exc=False, **kwargs)
    File "/usr/local/lib/python2.7/dist-packages/keystoneclient/utils.py", line 337, in inner
      return func(*args, **kwargs)
    File "/usr/local/lib/python2.7/dist-packages/keystoneclient/session.py", line 390, in request
      resp = send(**kwargs)
    File "/usr/local/lib/python2.7/dist-packages/keystoneclient/session.py", line 434, in _send_request
      raise exceptions.ConnectionRefused(msg)
  keystoneauth1.exceptions.connection.ConnectFailure: Unable to establish connection to http://Controller-VIP:8004/v1/ca1d10786fdc41698e276a8f22d98741/stacks

  I think we should retry the API connection from keystone client
  whenever there is a failure of connection to API. When I added code to
  retry it is working fine. In the existing code Keystoneclient is
  trying only one time and returning an error if it is not able to
  connect to any other service API.

To manage notifications about this bug go to:
https://bugs.launchpad.net/keystone/+bug/1535640/+subscriptions