yahoo-eng-team team mailing list archive

[Adam Gandelman <1241275@xxxxxxxxxxxxxxxxxx>]

** Also affects: nova/havana
   Importance: Undecided
       Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1241275

Title:
  Nova / Neutron Client failing upon re-authentication after token
  expiration

Status in OpenStack Compute (Nova):
  Fix Released
Status in OpenStack Compute (nova) havana series:
  New
Status in Python client library for Neutron:
  Fix Committed

Bug description:
  By default, the token length for clients is 24 hours.  When that token
  expires (or is invalidated for any reason), nova should obtain a new
  token.

  Currently, when the token expires, it leads to the following fault:
      File "/usr/lib/python2.6/site-packages/nova/network/neutronv2/api.py", line 136, in _get_available_networks
        nets = neutron.list_networks(**search_opts).get('networks', [])
      File "/usr/lib/python2.6/site-packages/neutronclient/v2_0/client.py", line 108, in with_params
        ret = self.function(instance, *args, **kwargs)
      File "/usr/lib/python2.6/site-packages/neutronclient/v2_0/client.py", line 325, in list_networks
        **_params)
      File "/usr/lib/python2.6/site-packages/neutronclient/v2_0/client.py", line 1197, in list
        for r in self._pagination(collection, path, **params):
      File "/usr/lib/python2.6/site-packages/neutronclient/v2_0/client.py", line 1210, in _pagination
        res = self.get(path, params=params)
      File "/usr/lib/python2.6/site-packages/neutronclient/v2_0/client.py", line 1183, in get
        headers=headers, params=params)
      File "/usr/lib/python2.6/site-packages/neutronclient/v2_0/client.py", line 1168, in retry_request
        headers=headers, params=params)
      File "/usr/lib/python2.6/site-packages/neutronclient/v2_0/client.py", line 1103, in do_request
        resp, replybody = self.httpclient.do_request(action, method, body=body)
      File "/usr/lib/python2.6/site-packages/neutronclient/client.py", line 188, in do_request
        self.authenticate()
      File "/usr/lib/python2.6/site-packages/neutronclient/client.py", line 224, in authenticate
        token_url = self.auth_url + "/tokens"
      TRACE nova.openstack.common.rpc.amqp TypeError: unsupported operand type(s) for +: 'NoneType' and 'str'

  This error is occurring because nova/network/neutronv2/__init__.py
  obtains a token for communication with neutron.  Nova is then
  authenticating the token (nova/network/neutronv2/__init__.py -
  _get_auth_token).  Upon authentication, it passes in the token into
  the neutron client (via the _get_client method).  It should be noted
  that the token is the main element passed into the neutron client
  (auth_url, username, password, etc... are not passed in as part of the
  request)

  Since nova is passing the token directly into the neutron client, nova
  does not validate whether or not the token is authenticated.

  After the 24 hour period of time, the token naturally expires.
  Therefore, when the neutron client goes to make a request, it catches
  an exceptions.Unauthorized block.  Upon catching this exception, the
  neutron client attempts to re-authenticate and then make the request
  again.

  The issue arises in the re-authentication of the token.  The neutron client's authenticate method requires that the following parameters are sent in from its users:
   - username
   - password
   - tenant_id or tenant_name
   - auth_url
   - auth_strategy

  Since the nova client is not passing these parameters in, the neutron
  client is failing with the exception above.

  Not all methods from the nova client are exposed to this.  Invocations
  to nova/network/neutronv2/__init__.py - get_client with an 'admin'
  value set to True will always get a new token.  However, the clients
  that invoke the get_client method without specifying the admin flag,
  or by explicitly setting it to False will be affected by this.  Note
  that the admin flag IS NOT determined based off the context's admin
  attribute.

  Methods from nova/network/neutronv2/api.py that are currently affected appear to be:
   - _get_available_networks
   - allocate_for_instance
   - deallocate_for_instance
   - deallocate_port_for_instance
   - list_ports
   - show_port
   - add_fixed_ip_to_instance
   - remove_fixed_ip_from_instance
   - validate_networks
   - _get_instance_uuids_by_ip
   - associate_floating_ip
   - get_all
   - get
   - get_floating_ip
   - get_floating_ip_pools
   - get_floating_ip_by_address
   - get_floating_ips_by_project
   - get_instance_id_by_floating_address
   - allocate_floating_ip
   - release_floating_ip
   - disassociate_floating_ip
   - _get_subnets_from_port

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1241275/+subscriptions