yahoo-eng-team team mailing list archive

[OpenStack Infra <2067770@xxxxxxxxxxxxxxxxxx>]

Reviewed:  https://review.opendev.org/c/openstack/neutron/+/920933
Committed: https://opendev.org/openstack/neutron/commit/05fcfef6ced908d7ccf9b2bc13b76b13f055d096
Submitter: "Zuul (22348)"
Branch:    master

commit 05fcfef6ced908d7ccf9b2bc13b76b13f055d096
Author: Brian Haley <haleyb.dev@xxxxxxxxx>
Date:   Fri May 31 16:14:32 2024 -0400

    Change to use selectin for DB load strategy
    
    During a mailing list discussion on some OOM issues neutron
    has been seeing [0], Mike Bayer recommended we should change
    from using subquery to selectin DB load strategy.
    
    A full description of this strategy can be found here [1],
    but in short:
    
    - “subquery” loading incurs additional performance / complexity
      issues when used on a many-levels-deep eager load, as
      subqueries will be nested repeatedly.
    
    - "The subqueryload() eager loader is mostly legacy at this
      point, superseded by selectinload()
    
    - "The only scenario in which selectin eager loading is not
      feasible is when the model is using composite primary keys,
      and the backend database does not support tuples with IN,
      which currently includes SQL Server." So that does not
      apply to us.
    
    The plan agreed to at the neutron drivers meeting [2] was to
    make this change early in the cycle so we would be able to
    see if there were any issues through the D cycle.
    
    Added hacking checks so new code using subquery loads is
    not added back.
    
    [0] https://lists.openstack.org/archives/list/openstack-discuss@xxxxxxxxxxxxxxxxxxx/thread/EHLQQXNG3NLLZYPDGG2ES3DINIJ7YT3N/
    [1] https://docs.sqlalchemy.org/en/20/orm/queryguide/relationships.html#selectin-eager-loading
    [2] https://meetings.opendev.org/meetings/neutron_drivers/2024/neutron_drivers.2024-05-31-14.00.log.html#l-67
    
    Closes-bug: #2067770
    Depends-on: https://review.opendev.org/c/openstack/neutron-lib/+/920936
    Change-Id: I6e40a15284da392a3d48d45205a7a5770c14c297


** Changed in: neutron
       Status: In Progress => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/2067770

Title:
  Neutron should use selectin for DB load strategy

Status in neutron:
  Fix Released

Bug description:
  During a mailing list discussion on some OOM issues neutron has been
  seeing [0], Mike Bayer recommended we should change from using
  subquery to selectin DB load strategy.

  A full description of this strategy can be found here [1], but in
  short:

  - “subquery” loading also incurs additional performance / complexity
  issues when used on a many-levels-deep eager load, as subqueries will
  be nested repeatedly.

  - "The subqueryload() eager loader is mostly legacy at this point,
  superseded by the selectinload()

  - "The only scenario in which selectin eager loading is not feasible
  is when the model is using composite primary keys, and the backend
  database does not support tuples with IN, which currently includes SQL
  Server." So that does not apply to us.

  The plan agreed to at the neutron drivers meeting [2] was to make this
  change early in the cycle so we would be able to see if there were any
  issues through the D cycle.

  [0] https://lists.openstack.org/archives/list/openstack-discuss@xxxxxxxxxxxxxxxxxxx/thread/EHLQQXNG3NLLZYPDGG2ES3DINIJ7YT3N/
  [1] https://docs.sqlalchemy.org/en/20/orm/queryguide/relationships.html#selectin-eager-loading
  [2] https://meetings.opendev.org/meetings/neutron_drivers/2024/neutron_drivers.2024-05-31-14.00.log.html#l-67

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/2067770/+subscriptions