yahoo-eng-team team mailing list archive

[Adam Gandelman <1255577@xxxxxxxxxxxxxxxxxx>]

** Also affects: nova/havana
   Importance: Undecided
       Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1255577

Title:
  Requests to Metadata API fail with 500 if Neutron network plugin  is
  used

Status in OpenStack Compute (Nova):
  Fix Released
Status in OpenStack Compute (nova) havana series:
  New

Bug description:
  In TripleO devtest story we are using Nova + Baremetal Driver +
  Neutron. The provisioned baremetal instance obtains its configuration
  from Metadata API. Currently all requests to Metadata API fail with
  error 500.

  In nova-api log I can see the following traceback:

  2013-11-27 11:44:01,423.423 5895 ERROR nova.api.metadata.handler [req-0d22f3c7-663e-452e-bfa9-747b728fc13b None None] Failed to get metadata for ip: 192.0.2.2
  2013-11-27 11:44:01,423.423 5895 TRACE nova.api.metadata.handler Traceback (most recent call last):
  2013-11-27 11:44:01,423.423 5895 TRACE nova.api.metadata.handler   File "/opt/stack/venvs/nova/local/lib/python2.7/site-packages/nova/api/metadata/handler.py", line 136, in _handle_remote_ip_request
  2013-11-27 11:44:01,423.423 5895 TRACE nova.api.metadata.handler     meta_data = self.get_metadata_by_remote_address(remote_address)
  2013-11-27 11:44:01,423.423 5895 TRACE nova.api.metadata.handler   File "/opt/stack/venvs/nova/local/lib/python2.7/site-packages/nova/api/metadata/handler.py", line 78, in get_metadata_by_remote_address
  2013-11-27 11:44:01,423.423 5895 TRACE nova.api.metadata.handler     data = base.get_metadata_by_address(self.conductor_api, address)
  2013-11-27 11:44:01,423.423 5895 TRACE nova.api.metadata.handler   File "/opt/stack/venvs/nova/local/lib/python2.7/site-packages/nova/api/metadata/base.py", line 466, in get_metadata_by_address
  2013-11-27 11:44:01,423.423 5895 TRACE nova.api.metadata.handler     fixed_ip = network.API().get_fixed_ip_by_address(ctxt, address)
  2013-11-27 11:44:01,423.423 5895 TRACE nova.api.metadata.handler   File "/opt/stack/venvs/nova/local/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 680, in get_fixed_ip_by_address
  2013-11-27 11:44:01,423.423 5895 TRACE nova.api.metadata.handler     uuid_maps = self._get_instance_uuids_by_ip(context, address)
  2013-11-27 11:44:01,423.423 5895 TRACE nova.api.metadata.handler   File "/opt/stack/venvs/nova/local/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 582, in _get_instance_uuids_by_ip
  2013-11-27 11:44:01,423.423 5895 TRACE nova.api.metadata.handler     data = neutronv2.get_client(context).list_ports(**search_opts)
  2013-11-27 11:44:01,423.423 5895 TRACE nova.api.metadata.handler   File "/opt/stack/venvs/nova/local/lib/python2.7/site-packages/nova/network/neutronv2/__init__.py", line 69, in get_client
  2013-11-27 11:44:01,423.423 5895 TRACE nova.api.metadata.handler     raise exceptions.Unauthorized()
  2013-11-27 11:44:01,423.423 5895 TRACE nova.api.metadata.handler Unauthorized: Unauthorized: bad credentials

  Analyzing this issue we found that Metadata API stopped working since
  change https://review.openstack.org/#/c/56174/4 was merged (it seems
  that change of line 57 in
  https://review.openstack.org/#/c/56174/4/nova/network/neutronv2/__init__.py
  is the reason).

  The commit message looks pretty sane and that fix seems to be the
  right thing to do, because we don't want to do neutron requests on
  behalf of neutron service user we have in nova config, but rather on
  behalf of the admin user instead who made the original request to nova
  api. So it seems that context.is_admin should be extended to make it
  possible to distinguish between those two cases of admin users: the
  real admin users, and the cases when nova api needs to talk to
  neutron.

  The problem is that all metadata queries are handled using default
  admin context (user and other vars are set to None while
  is_admin=True), so with https://review.openstack.org/#/c/56174/4
  applied, get_client() always raises an exception when Metadata API
  requests are handled.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1255577/+subscriptions