yahoo-eng-team team mailing list archive

[OpenStack Infra <1827363@xxxxxxxxxxxxxxxxxx>]

Reviewed:  https://review.opendev.org/658777
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=94bc403078b1e5d2cddc697fc59b7f1d2d8a6045
Submitter: Zuul
Branch:    master

commit 94bc403078b1e5d2cddc697fc59b7f1d2d8a6045
Author: Gabriele Cerami <gcerami@xxxxxxxxxx>
Date:   Mon May 13 10:38:50 2019 +0100

    ML2 plugin: extract and postpone limit in port query
    
    as with https://review.opendev.org/#/c/656066/ if limit is applied in
    any place other than at the end of the filters, sql alchemy will return
    an error, and possibily we could return less result than intended.
    
    Change-Id: I9a54ae99d2d5dfda63cb0061bcf3d727ed7cc992
    Closes-Bug: #1827363


** Changed in: neutron
       Status: In Progress => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1827363

Title:
  Additional  port list / get_ports() failures when filtering and
  limiting at the same time

Status in neutron:
  Fix Released

Bug description:
  When doing a openstack port list that filters for a fixed-ip/subnet
  and at the same time limits the amount of results neutron returns a
  500 internal server error. This was already addressed in
  https://bugs.launchpad.net/neutron/+bug/1826186 but this bug is also
  present in other places.

  While running tempest against a Neutron Queens installation I came
  across another _get_ports_query() in neutron/plugins/ml2/plugin.py
  where filter is again called onto the result of an already limited
  query.

  See
  https://github.com/openstack/neutron/blob/6f4962dcf89aebf2552ee8ec0993c6389a953024/neutron/plugins/ml2/plugin.py#L2206

  InvalidRequestError: Query.filter() being called on a Query which already has LIMIT or OFFSET applied. To modify the row-limited results of a Query, call from_self() first. Otherwise, call filter() before limit() or offset() are applied.
    File "pecan/core.py", line 683, in __call__
      self.invoke_controller(controller, args, kwargs, state)
  [...]
    File "neutron/db/db_base_plugin_v2.py", line 1417, in get_ports
      page_reverse=page_reverse)
    File "neutron/plugins/ml2/plugin.py", line 1941, in _get_ports_query
      query = query.filter(substr_filter)
    File "<string>", line 2, in filter
    File "sqlalchemy/orm/base.py", line 200, in generate
      assertion(self, fn.__name__)
    File "sqlalchemy/orm/query.py", line 435, in _no_limit_offset
      % (meth, meth)

  I applied a patch similar to the one Gabriele Cerami proposed in
  https://review.opendev.org/#/c/656066/ on our production setup and
  this seems to have fixed the bug there as well.

  When doing a grep for _get_ports_query() in the neutron codebase I
  find a function with this name being called in
  neutron/db/dvr_mac_db.py in get_ports_on_host_by_subnet(), I do not
  have a stacktrace or test for that though.

  See
  https://github.com/openstack/neutron/blob/6f4962dcf89aebf2552ee8ec0993c6389a953024/neutron/db/dvr_mac_db.py#L162

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1827363/+subscriptions