yahoo-eng-team team mailing list archive

[OpenStack Infra <1599256@xxxxxxxxxxxxxxxxxx>]

Reviewed:  https://review.openstack.org/342299
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=dc7c21f3cfbd5f68e9bc6b88f75e160826fadb82
Submitter: Jenkins
Branch:    master

commit dc7c21f3cfbd5f68e9bc6b88f75e160826fadb82
Author: Pushkar Umaranikar <pushkar.umaranikar@xxxxxxxxx>
Date:   Thu Jul 14 17:32:02 2016 +0000

    Removing duplicates from columns_to_join list
    
    columns_to_join list contains duplicate values as we keep
    extending expected_attrs list in api level.
    This commit involves removing duplicates from columns_to_join
    attribute of instance model to avoid unnecessary joins in
    instance_get_all_by_filters method.
    Sorting resultant set based on list index is required to maintain
    the sequence.
    
    Change-Id: I2c93cef0babad23fa456bf10ba2ccaec7844b4cd
    Closes-Bug: #1599256


** Changed in: nova
       Status: In Progress => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1599256

Title:
  instance_get_all_by_filters can perform unnecessary joins

Status in OpenStack Compute (nova):
  Fix Released

Bug description:
  When listing server details, instance_get_all_by_filters() can be
  invoked with duplicates in the columns_to_join list.  This can result
  in both a join and a separate query against the potential
  "manual_join" tables.

  This appears to have been introduced with the behavior in this commit:

  https://github.com/openstack/nova/commit/2e68b2298e94a15d1282c0fb46804b9efa6c8b3a

  Where the list of expected_attr here:

  https://github.com/openstack/nova/blob/master/nova/api/openstack/compute/views/servers.py#L55
  https://github.com/openstack/nova/blob/master/nova/api/openstack/compute/views/servers.py#L110

  is further extended with the expected_attr list in the compute_api
  here:

  https://github.com/openstack/nova/blob/master/nova/compute/api.py#L2092

  Resulting in a columns_to_join list resembling:

  ['flavor', 'info_cache', 'metadata', 'metadata', 'system_metadata',
  'info_cache', 'security_groups']

  In nova.db.sqlalchemy.api:_manual_join_columns(), only the first
  'metadata' entry gets removed resulting in both a sqlalchemy
  joinedload() hint (joining against instance_metadata) and separately
  querying instance_metadata via _instances_fill_metadata().  The
  'metadata' join in particular can be rather inefficient.  In some
  cases  this results in about 10x the data being pulled from the
  database compared to just the "manual join" - this problem is
  particularly amplified for projects with a large number of associated
  instances.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1599256/+subscriptions