yahoo-eng-team team mailing list archive

[Sebastian Lohff <1827363@xxxxxxxxxxxxxxxxxx>]

Public bug reported:

When doing a openstack port list that filters for a fixed-ip/subnet and
at the same time limits the amount of results neutron returns a 500
internal server error. This was already addressed in
https://bugs.launchpad.net/neutron/+bug/1826186 but this bug is also
present in other places.

While running tempest against a Neutron Queens installation I came
across another _get_ports_query() in neutron/plugins/ml2/plugin.py where
filter is again called onto the result of an already limited query.

See
https://github.com/openstack/neutron/blob/6f4962dcf89aebf2552ee8ec0993c6389a953024/neutron/plugins/ml2/plugin.py#L2206

InvalidRequestError: Query.filter() being called on a Query which already has LIMIT or OFFSET applied. To modify the row-limited results of a Query, call from_self() first. Otherwise, call filter() before limit() or offset() are applied.
  File "pecan/core.py", line 683, in __call__
    self.invoke_controller(controller, args, kwargs, state)
[...]
  File "neutron/db/db_base_plugin_v2.py", line 1417, in get_ports
    page_reverse=page_reverse)
  File "neutron/plugins/ml2/plugin.py", line 1941, in _get_ports_query
    query = query.filter(substr_filter)
  File "<string>", line 2, in filter
  File "sqlalchemy/orm/base.py", line 200, in generate
    assertion(self, fn.__name__)
  File "sqlalchemy/orm/query.py", line 435, in _no_limit_offset
    % (meth, meth)

I applied a patch similar to the one Gabriele Cerami proposed in
https://review.opendev.org/#/c/656066/ on our production setup and this
seems to have fixed the bug there as well.

When doing a grep for _get_ports_query() in the neutron codebase I find
a function with this name being called in neutron/db/dvr_mac_db.py in
get_ports_on_host_by_subnet(), I do not have a stacktrace or test for
that though.

See
https://github.com/openstack/neutron/blob/6f4962dcf89aebf2552ee8ec0993c6389a953024/neutron/db/dvr_mac_db.py#L162

** Affects: neutron
     Importance: Undecided
         Status: New

** Description changed:

  When doing a openstack port list that filters for a fixed-ip/subnet and
  at the same time limits the amount of results neutron returns a 500
  internal server error. This was already addressed in
  https://bugs.launchpad.net/neutron/+bug/1826186 but this bug is also
  present in other places.
  
  While running tempest against a Neutron Queens installation I came
  across another _get_ports_query() in neutron/plugins/ml2/plugin.py where
  filter is again called onto the result of an already limited query.
  
  See
  https://github.com/openstack/neutron/blob/6f4962dcf89aebf2552ee8ec0993c6389a953024/neutron/plugins/ml2/plugin.py#L2206
  
  InvalidRequestError: Query.filter() being called on a Query which already has LIMIT or OFFSET applied. To modify the row-limited results of a Query, call from_self() first. Otherwise, call filter() before limit() or offset() are applied.
-   File "pecan/core.py", line 683, in __call__
-     self.invoke_controller(controller, args, kwargs, state)
+   File "pecan/core.py", line 683, in __call__
+     self.invoke_controller(controller, args, kwargs, state)
  [...]
-   File "neutron/db/db_base_plugin_v2.py", line 1417, in get_ports
-     page_reverse=page_reverse)
-   File "neutron/plugins/ml2/plugin.py", line 1941, in _get_ports_query
-     query = query.filter(substr_filter)
-   File "<string>", line 2, in filter
-   File "sqlalchemy/orm/base.py", line 200, in generate
-     assertion(self, fn.__name__)
-   File "sqlalchemy/orm/query.py", line 435, in _no_limit_offset
-     % (meth, meth)
+   File "neutron/db/db_base_plugin_v2.py", line 1417, in get_ports
+     page_reverse=page_reverse)
+   File "neutron/plugins/ml2/plugin.py", line 1941, in _get_ports_query
+     query = query.filter(substr_filter)
+   File "<string>", line 2, in filter
+   File "sqlalchemy/orm/base.py", line 200, in generate
+     assertion(self, fn.__name__)
+   File "sqlalchemy/orm/query.py", line 435, in _no_limit_offset
+     % (meth, meth)
  
  I applied a patch similar to the one Gabriele Cerami proposed in
- https://review.opendev.org/#/c/656066/
+ https://review.opendev.org/#/c/656066/ on our production setup and this
+ seems to have fixed the bug there as well.
  
  When doing a grep for _get_ports_query() in the neutron codebase I find
  a function with this name being called in neutron/db/dvr_mac_db.py in
  get_ports_on_host_by_subnet(), I do not have a stacktrace or test for
  that though.
  
  See
  https://github.com/openstack/neutron/blob/6f4962dcf89aebf2552ee8ec0993c6389a953024/neutron/db/dvr_mac_db.py#L162

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1827363

Title:
  Additional  port list / get_ports() failures when filtering and
  limiting at the same time

Status in neutron:
  New

Bug description:
  When doing a openstack port list that filters for a fixed-ip/subnet
  and at the same time limits the amount of results neutron returns a
  500 internal server error. This was already addressed in
  https://bugs.launchpad.net/neutron/+bug/1826186 but this bug is also
  present in other places.

  While running tempest against a Neutron Queens installation I came
  across another _get_ports_query() in neutron/plugins/ml2/plugin.py
  where filter is again called onto the result of an already limited
  query.

  See
  https://github.com/openstack/neutron/blob/6f4962dcf89aebf2552ee8ec0993c6389a953024/neutron/plugins/ml2/plugin.py#L2206

  InvalidRequestError: Query.filter() being called on a Query which already has LIMIT or OFFSET applied. To modify the row-limited results of a Query, call from_self() first. Otherwise, call filter() before limit() or offset() are applied.
    File "pecan/core.py", line 683, in __call__
      self.invoke_controller(controller, args, kwargs, state)
  [...]
    File "neutron/db/db_base_plugin_v2.py", line 1417, in get_ports
      page_reverse=page_reverse)
    File "neutron/plugins/ml2/plugin.py", line 1941, in _get_ports_query
      query = query.filter(substr_filter)
    File "<string>", line 2, in filter
    File "sqlalchemy/orm/base.py", line 200, in generate
      assertion(self, fn.__name__)
    File "sqlalchemy/orm/query.py", line 435, in _no_limit_offset
      % (meth, meth)

  I applied a patch similar to the one Gabriele Cerami proposed in
  https://review.opendev.org/#/c/656066/ on our production setup and
  this seems to have fixed the bug there as well.

  When doing a grep for _get_ports_query() in the neutron codebase I
  find a function with this name being called in
  neutron/db/dvr_mac_db.py in get_ports_on_host_by_subnet(), I do not
  have a stacktrace or test for that though.

  See
  https://github.com/openstack/neutron/blob/6f4962dcf89aebf2552ee8ec0993c6389a953024/neutron/db/dvr_mac_db.py#L162

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1827363/+subscriptions