yahoo-eng-team team mailing list archive

[Pierre Crégut <1649317@xxxxxxxxxxxxxxxxxx>]

Public bug reported:

A regular tenant can create objects that will require a lot of time to
enumerate because of the strategy used by the ORM to build back the
object from the different tables in the database.

The script attached can be used to reproduce the problem. It creates a
network with several subnetworks (with several outes and DNS servers),
several tags, several RBAC policies but it does not exceed any typical
quota. Because the network is retrieved at each stage it is modified, it
is hard to run this script until its end in a typical settings.

Using the strategy lazy='joined' means that a single request is
performed to retrieve an object and all its parts that may be expressed
in several tables. For example when one asks for the network list, a
complex query will be issued that also retrieves subnets, subnetpools,
dns agents, etc. The exact query is visible at
http://paste.openstack.org/show/592120/

Unfortunately using the strategy lazy=joined has another impact when the
relation between the parent object and the sub-object has a ?-n arity.
Rather than giving back exactly the row needed, the single query builds
a kind of cross-product of the answers sharing the join keys. For
example if we have a network with 4 tags and 4 subnetworks, we will have
at least 16 rows for each combination of tags and subnetworks. Other
fields like rbac rules, special routes, dns servers can amplify the
problem.

It is not clear if the heavy usage of the database server and neutron
server could lead to a real deny of service for other users.

** Affects: neutron
     Importance: Undecided
         Status: New

** Attachment added: "Script reproducing the problem"
   https://bugs.launchpad.net/bugs/1649317/+attachment/4790746/+files/combinatorial_blowup.sh

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1649317

Title:
  Combinatorial blow-up with the Alchemy strategy lazy='joined'

Status in neutron:
  New

Bug description:
  A regular tenant can create objects that will require a lot of time to
  enumerate because of the strategy used by the ORM to build back the
  object from the different tables in the database.

  The script attached can be used to reproduce the problem. It creates a
  network with several subnetworks (with several outes and DNS servers),
  several tags, several RBAC policies but it does not exceed any typical
  quota. Because the network is retrieved at each stage it is modified,
  it is hard to run this script until its end in a typical settings.

  Using the strategy lazy='joined' means that a single request is
  performed to retrieve an object and all its parts that may be
  expressed in several tables. For example when one asks for the network
  list, a complex query will be issued that also retrieves subnets,
  subnetpools, dns agents, etc. The exact query is visible at
  http://paste.openstack.org/show/592120/

  Unfortunately using the strategy lazy=joined has another impact when
  the relation between the parent object and the sub-object has a ?-n
  arity. Rather than giving back exactly the row needed, the single
  query builds a kind of cross-product of the answers sharing the join
  keys. For example if we have a network with 4 tags and 4 subnetworks,
  we will have at least 16 rows for each combination of tags and
  subnetworks. Other fields like rbac rules, special routes, dns servers
  can amplify the problem.

  It is not clear if the heavy usage of the database server and neutron
  server could lead to a real deny of service for other users.

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1649317/+subscriptions