yahoo-eng-team team mailing list archive

[Matt Riedemann <mriedem@xxxxxxxxxx>]

Public bug reported:

The nova API metadata handler has some flows like this where it's
getting instance metadata by the fixed IP:

https://github.com/openstack/nova/blob/7fc982e19f94a4624d54c3ac113057bed7750ec4/nova/api/metadata/handler.py#L88

That leads to queries to neutron (if using neutron) to list ports by
that fixed IP and then get the instance uuid (via device_id on the port)
for the fixed_ip:

https://github.com/openstack/nova/blob/7fc982e19f94a4624d54c3ac113057bed7750ec4/nova/api/metadata/base.py#L533

And then we get the instance object via that uuid:

https://github.com/openstack/nova/blob/7fc982e19f94a4624d54c3ac113057bed7750ec4/nova/api/metadata/base.py#L544

Note we're only joining on these fields:

expected_attrs=['ec2_ids', 'flavor', 'info_cache']

But when constructing the InstanceMetadata object, we're loading up
security groups separately:

https://github.com/openstack/nova/blob/7fc982e19f94a4624d54c3ac113057bed7750ec4/nova/api/metadata/base.py#L130

Lazy loading 'metadata':

https://github.com/openstack/nova/blob/7fc982e19f94a4624d54c3ac113057bed7750ec4/nova/api/metadata/base.py#L143

And lazy loading system_metadata:

https://github.com/openstack/nova/blob/7fc982e19f94a4624d54c3ac113057bed7750ec4/nova/api/metadata/base.py#L145

https://github.com/openstack/nova/blob/7fc982e19f94a4624d54c3ac113057bed7750ec4/nova/api/metadata/password.py#L32

We can load the metadata/system_metadata/security_groups when we get the
instance object from the database the first time, which would avoid
these extra queries to the database, which requires more round trips
through to conductor.

** Affects: nova
     Importance: Medium
     Assignee: Matt Riedemann (mriedem)
         Status: Confirmed


** Tags: api metadata performance

** Changed in: nova
       Status: New => Confirmed

** Changed in: nova
     Assignee: (unassigned) => Matt Riedemann (mriedem)

** Changed in: nova
   Importance: Undecided => Medium

** Tags added: api metadata performance

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1521675

Title:
  n-api-meta handler could be more efficient with db

Status in OpenStack Compute (nova):
  Confirmed

Bug description:
  The nova API metadata handler has some flows like this where it's
  getting instance metadata by the fixed IP:

  https://github.com/openstack/nova/blob/7fc982e19f94a4624d54c3ac113057bed7750ec4/nova/api/metadata/handler.py#L88

  That leads to queries to neutron (if using neutron) to list ports by
  that fixed IP and then get the instance uuid (via device_id on the
  port) for the fixed_ip:

  https://github.com/openstack/nova/blob/7fc982e19f94a4624d54c3ac113057bed7750ec4/nova/api/metadata/base.py#L533

  And then we get the instance object via that uuid:

  https://github.com/openstack/nova/blob/7fc982e19f94a4624d54c3ac113057bed7750ec4/nova/api/metadata/base.py#L544

  Note we're only joining on these fields:

  expected_attrs=['ec2_ids', 'flavor', 'info_cache']

  But when constructing the InstanceMetadata object, we're loading up
  security groups separately:

  https://github.com/openstack/nova/blob/7fc982e19f94a4624d54c3ac113057bed7750ec4/nova/api/metadata/base.py#L130

  Lazy loading 'metadata':

  https://github.com/openstack/nova/blob/7fc982e19f94a4624d54c3ac113057bed7750ec4/nova/api/metadata/base.py#L143

  And lazy loading system_metadata:

  https://github.com/openstack/nova/blob/7fc982e19f94a4624d54c3ac113057bed7750ec4/nova/api/metadata/base.py#L145

  https://github.com/openstack/nova/blob/7fc982e19f94a4624d54c3ac113057bed7750ec4/nova/api/metadata/password.py#L32

  We can load the metadata/system_metadata/security_groups when we get
  the instance object from the database the first time, which would
  avoid these extra queries to the database, which requires more round
  trips through to conductor.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1521675/+subscriptions