yahoo-eng-team team mailing list archive

[Sebastian Lohff <1826186@xxxxxxxxxxxxxxxxxx>]

Public bug reported:

When doing a openstack port list that filters for a fixed-ip/subnet and
at the same time limits the amount of results neutron returns a 500
internal server error.

Example command: openstack port list --fixed-ip ip-address=192.0.2.23
Limits should be applied automatically with a recent version of the openstacksdk with pagination turned on by default. Additionally, I attached a testcase that triggers this bug. This bug was found on neutron-queens, but the test-case also breaks current master (tested on commit id 1214e59cc2d818f6fde9c3e24c7f26c50d2a8a74).

It looks like _get_ports_query() gets a query with pre-applied limits by
calling model_query.get_collection_query() and then tries to filter the
results, which triggers a sqlalchemy assertion that disallows filtering
after a limit has been applied.

The corresponding exception neutron exception would be the following:
InvalidRequestError: Query.filter() being called on a Query which already has LIMIT or OFFSET applied. To modify the row-limited results of a  Query, call from_self() first.  Otherwise, call filter() before limit() or offset() are applied.
  File "pecan/core.py", line 683, in __call__
    self.invoke_controller(controller, args, kwargs, state)
  File "pecan/core.py", line 574, in invoke_controller
    result = controller(*args, **kwargs)
  File "neutron/db/api.py", line 91, in wrapped
    setattr(e, '_RETRY_EXCEEDED', True)
  File "oslo_utils/excutils.py", line 220, in __exit__
    self.force_reraise()
  File "oslo_utils/excutils.py", line 196, in force_reraise
    six.reraise(self.type_, self.value, self.tb)
  File "neutron/db/api.py", line 87, in wrapped
    return f(*args, **kwargs)
  File "oslo_db/api.py", line 147, in wrapper
    ectxt.value = e.inner_exc
  File "oslo_utils/excutils.py", line 220, in __exit__
    self.force_reraise()
  File "oslo_utils/excutils.py", line 196, in force_reraise
    six.reraise(self.type_, self.value, self.tb)
  File "oslo_db/api.py", line 135, in wrapper
    return f(*args, **kwargs)
  File "neutron/db/api.py", line 126, in wrapped
    LOG.debug("Retry wrapper got retriable exception: %s", e)
  File "oslo_utils/excutils.py", line 220, in __exit__
    self.force_reraise()
  File "oslo_utils/excutils.py", line 196, in force_reraise
    six.reraise(self.type_, self.value, self.tb)
  File "neutron/db/api.py", line 122, in wrapped
    return f(*dup_args, **dup_kwargs)
  File "neutron/pecan_wsgi/controllers/utils.py", line 76, in wrapped
    return f(*args, **kwargs)
  File "neutron/pecan_wsgi/controllers/resource.py", line 131, in index
    return self.get(*args, **kwargs)
  File "neutron/pecan_wsgi/controllers/resource.py", line 141, in get
    **query_params)}
  File "neutron/db/api.py", line 161, in wrapped
    return method(*args, **kwargs)
  File "neutron/db/api.py", line 91, in wrapped
    setattr(e, '_RETRY_EXCEEDED', True)
  File "oslo_utils/excutils.py", line 220, in __exit__
    self.force_reraise()
  File "oslo_utils/excutils.py", line 196, in force_reraise
    six.reraise(self.type_, self.value, self.tb)
  File "neutron/db/api.py", line 87, in wrapped
    return f(*args, **kwargs)
  File "oslo_db/api.py", line 147, in wrapper
    ectxt.value = e.inner_exc
  File "oslo_utils/excutils.py", line 220, in __exit__
    self.force_reraise()
  File "oslo_utils/excutils.py", line 196, in force_reraise
    six.reraise(self.type_, self.value, self.tb)
  File "oslo_db/api.py", line 135, in wrapper
    return f(*args, **kwargs)
  File "neutron/db/api.py", line 126, in wrapped
    LOG.debug("Retry wrapper got retriable exception: %s", e)
  File "oslo_utils/excutils.py", line 220, in __exit__
    self.force_reraise()
  File "oslo_utils/excutils.py", line 196, in force_reraise
    six.reraise(self.type_, self.value, self.tb)
  File "neutron/db/api.py", line 122, in wrapped
    return f(*dup_args, **dup_kwargs)
  File "neutron/db/db_base_plugin_v2.py", line 1417, in get_ports
    page_reverse=page_reverse)
  File "neutron/plugins/ml2/plugin.py", line 1941, in _get_ports_query
    query = query.filter(substr_filter)
  File "<string>", line 2, in filter
  File "sqlalchemy/orm/base.py", line 200, in generate
    assertion(self, fn.__name__)
  File "sqlalchemy/orm/query.py", line 435, in _no_limit_offset
    % (meth, meth)

** Affects: neutron
     Importance: Undecided
         Status: New

** Patch added: "neutron testcase for filtering and limiting on a get ports at the same time"
   https://bugs.launchpad.net/bugs/1826186/+attachment/5258554/+files/test_list_ports_filtered_by_fixed_ip_with_limit.patch

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1826186

Title:
  port list / get_ports() fails when filtering and limiting at the same
  time

Status in neutron:
  New

Bug description:
  When doing a openstack port list that filters for a fixed-ip/subnet
  and at the same time limits the amount of results neutron returns a
  500 internal server error.

  Example command: openstack port list --fixed-ip ip-address=192.0.2.23
  Limits should be applied automatically with a recent version of the openstacksdk with pagination turned on by default. Additionally, I attached a testcase that triggers this bug. This bug was found on neutron-queens, but the test-case also breaks current master (tested on commit id 1214e59cc2d818f6fde9c3e24c7f26c50d2a8a74).

  It looks like _get_ports_query() gets a query with pre-applied limits
  by calling model_query.get_collection_query() and then tries to filter
  the results, which triggers a sqlalchemy assertion that disallows
  filtering after a limit has been applied.

  The corresponding exception neutron exception would be the following:
  InvalidRequestError: Query.filter() being called on a Query which already has LIMIT or OFFSET applied. To modify the row-limited results of a  Query, call from_self() first.  Otherwise, call filter() before limit() or offset() are applied.
    File "pecan/core.py", line 683, in __call__
      self.invoke_controller(controller, args, kwargs, state)
    File "pecan/core.py", line 574, in invoke_controller
      result = controller(*args, **kwargs)
    File "neutron/db/api.py", line 91, in wrapped
      setattr(e, '_RETRY_EXCEEDED', True)
    File "oslo_utils/excutils.py", line 220, in __exit__
      self.force_reraise()
    File "oslo_utils/excutils.py", line 196, in force_reraise
      six.reraise(self.type_, self.value, self.tb)
    File "neutron/db/api.py", line 87, in wrapped
      return f(*args, **kwargs)
    File "oslo_db/api.py", line 147, in wrapper
      ectxt.value = e.inner_exc
    File "oslo_utils/excutils.py", line 220, in __exit__
      self.force_reraise()
    File "oslo_utils/excutils.py", line 196, in force_reraise
      six.reraise(self.type_, self.value, self.tb)
    File "oslo_db/api.py", line 135, in wrapper
      return f(*args, **kwargs)
    File "neutron/db/api.py", line 126, in wrapped
      LOG.debug("Retry wrapper got retriable exception: %s", e)
    File "oslo_utils/excutils.py", line 220, in __exit__
      self.force_reraise()
    File "oslo_utils/excutils.py", line 196, in force_reraise
      six.reraise(self.type_, self.value, self.tb)
    File "neutron/db/api.py", line 122, in wrapped
      return f(*dup_args, **dup_kwargs)
    File "neutron/pecan_wsgi/controllers/utils.py", line 76, in wrapped
      return f(*args, **kwargs)
    File "neutron/pecan_wsgi/controllers/resource.py", line 131, in index
      return self.get(*args, **kwargs)
    File "neutron/pecan_wsgi/controllers/resource.py", line 141, in get
      **query_params)}
    File "neutron/db/api.py", line 161, in wrapped
      return method(*args, **kwargs)
    File "neutron/db/api.py", line 91, in wrapped
      setattr(e, '_RETRY_EXCEEDED', True)
    File "oslo_utils/excutils.py", line 220, in __exit__
      self.force_reraise()
    File "oslo_utils/excutils.py", line 196, in force_reraise
      six.reraise(self.type_, self.value, self.tb)
    File "neutron/db/api.py", line 87, in wrapped
      return f(*args, **kwargs)
    File "oslo_db/api.py", line 147, in wrapper
      ectxt.value = e.inner_exc
    File "oslo_utils/excutils.py", line 220, in __exit__
      self.force_reraise()
    File "oslo_utils/excutils.py", line 196, in force_reraise
      six.reraise(self.type_, self.value, self.tb)
    File "oslo_db/api.py", line 135, in wrapper
      return f(*args, **kwargs)
    File "neutron/db/api.py", line 126, in wrapped
      LOG.debug("Retry wrapper got retriable exception: %s", e)
    File "oslo_utils/excutils.py", line 220, in __exit__
      self.force_reraise()
    File "oslo_utils/excutils.py", line 196, in force_reraise
      six.reraise(self.type_, self.value, self.tb)
    File "neutron/db/api.py", line 122, in wrapped
      return f(*dup_args, **dup_kwargs)
    File "neutron/db/db_base_plugin_v2.py", line 1417, in get_ports
      page_reverse=page_reverse)
    File "neutron/plugins/ml2/plugin.py", line 1941, in _get_ports_query
      query = query.filter(substr_filter)
    File "<string>", line 2, in filter
    File "sqlalchemy/orm/base.py", line 200, in generate
      assertion(self, fn.__name__)
    File "sqlalchemy/orm/query.py", line 435, in _no_limit_offset
      % (meth, meth)

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1826186/+subscriptions