yahoo-eng-team team mailing list archive

Thread
Date
[Bug 2028285] [NEW] [unit test][xena+] test_port_deletion_prevention fails when runs in isolation

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: yatin <2028285@xxxxxxxxxxxxxxxxxx>
Date: Thu, 20 Jul 2023 15:44:29 -0000
Reply-to: Bug 2028285 <2028285@xxxxxxxxxxxxxxxxxx>
Sender: noreply@xxxxxxxxxxxxx
Public bug reported:

Can be reproduced by Just running:-
tox -epy3 -- test_port_deletion_prevention
or run any of the below tests individually:-
neutron.tests.unit.extensions.test_l3.L3NatDBSepTestCase.test_port_deletion_prevention_handles_missing_port
neutron.tests.unit.extensions.test_extraroute.ExtraRouteDBSepTestCase.test_port_deletion_prevention_handles_missing_port

Fails as below:-
neutron.tests.unit.extensions.test_extraroute.ExtraRouteDBSepTestCase.test_port_deletion_prevention_handles_missing_port
------------------------------------------------------------------------------------------------------------------------

Captured traceback:
~~~~~~~~~~~~~~~~~~~
    Traceback (most recent call last):

      File "/home/ykarel/work/openstack/neutron/neutron/tests/base.py", line 178, in func
    return f(self, *args, **kwargs)

      File "/home/ykarel/work/openstack/neutron/neutron/tests/base.py", line 178, in func
    return f(self, *args, **kwargs)

      File "/home/ykarel/work/openstack/neutron/neutron/tests/unit/extensions/test_l3.py", line 4491, in test_port_deletion_prevention_handles_missing_port
    pl.prevent_l3_port_deletion(context.get_admin_context(), 'fakeid')

      File "/home/ykarel/work/openstack/neutron/neutron/db/l3_db.py", line 1742, in prevent_l3_port_deletion
    port = port or self._core_plugin.get_port(context, port_id)

      File "/home/ykarel/work/openstack/neutron/.tox/py3/lib/python3.10/site-packages/neutron_lib/db/api.py", line 223, in wrapped
    return f_with_retry(*args, **kwargs,

      File "/home/ykarel/work/openstack/neutron/.tox/py3/lib/python3.10/site-packages/neutron_lib/db/api.py", line 137, in wrapped
    with excutils.save_and_reraise_exception():

      File "/home/ykarel/work/openstack/neutron/.tox/py3/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__
    self.force_reraise()

      File "/home/ykarel/work/openstack/neutron/.tox/py3/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise
    raise self.value

      File "/home/ykarel/work/openstack/neutron/.tox/py3/lib/python3.10/site-packages/neutron_lib/db/api.py", line 135, in wrapped
    return f(*args, **kwargs)

      File "/home/ykarel/work/openstack/neutron/.tox/py3/lib/python3.10/site-packages/oslo_db/api.py", line 144, in wrapper
    with excutils.save_and_reraise_exception() as ectxt:

      File "/home/ykarel/work/openstack/neutron/.tox/py3/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__
    self.force_reraise()

      File "/home/ykarel/work/openstack/neutron/.tox/py3/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise
    raise self.value

      File "/home/ykarel/work/openstack/neutron/.tox/py3/lib/python3.10/site-packages/oslo_db/api.py", line 142, in wrapper
    return f(*args, **kwargs)

      File "/home/ykarel/work/openstack/neutron/.tox/py3/lib/python3.10/site-packages/neutron_lib/db/api.py", line 183, in wrapped
    with excutils.save_and_reraise_exception():

      File "/home/ykarel/work/openstack/neutron/.tox/py3/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__
    self.force_reraise()

      File "/home/ykarel/work/openstack/neutron/.tox/py3/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise
    raise self.value

      File "/home/ykarel/work/openstack/neutron/.tox/py3/lib/python3.10/site-packages/neutron_lib/db/api.py", line 181, in wrapped
    return f(*dup_args, **dup_kwargs)

      File "/home/ykarel/work/openstack/neutron/.tox/py3/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 1022, in wrapper
    return fn(*args, **kwargs)

      File "/home/ykarel/work/openstack/neutron/neutron/db/db_base_plugin_v2.py", line 1628, in get_port
    lazy_fields = [models_v2.Port.port_forwardings,

    AttributeError: type object 'Port' has no attribute
'port_forwardings'

It's reproducible Since Xena+ since the inclusion of patch
https://review.opendev.org/c/openstack/neutron/+/790691

It do not reproduce if there are other test runs(from the test class)
before this test which involve other requests(like network get/create
etc) apart from the ones modified in above patch.

Considering above point if this test is modified to run other requests like below then it succeeds:-
self.plugin.get_ports_count(context.get_admin_context())
or
self.plugin.get_networks_count(context.get_admin_context())
or
with self.port/network():
    self.assertIsNone(      
        pl.prevent_l3_port_deletion(context.get_admin_context(), 'fakeid')
    )


The issue was originally noticed in a downstream job where unit tests were executed as part of package build. As the tests suite was executed with 100+ concurrency, by chance this test got executed in a worker first of any other test of the class.

Me not sure if there are other test cases which will fail like this when
run in isolation.

** Affects: neutron
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/2028285

Title:
  [unit test][xena+] test_port_deletion_prevention fails when runs in
  isolation

Status in neutron:
  New

Bug description:
  Can be reproduced by Just running:-
  tox -epy3 -- test_port_deletion_prevention
  or run any of the below tests individually:-
  neutron.tests.unit.extensions.test_l3.L3NatDBSepTestCase.test_port_deletion_prevention_handles_missing_port
  neutron.tests.unit.extensions.test_extraroute.ExtraRouteDBSepTestCase.test_port_deletion_prevention_handles_missing_port

  Fails as below:-
  neutron.tests.unit.extensions.test_extraroute.ExtraRouteDBSepTestCase.test_port_deletion_prevention_handles_missing_port
  ------------------------------------------------------------------------------------------------------------------------

  Captured traceback:
  ~~~~~~~~~~~~~~~~~~~
      Traceback (most recent call last):

        File "/home/ykarel/work/openstack/neutron/neutron/tests/base.py", line 178, in func
      return f(self, *args, **kwargs)

        File "/home/ykarel/work/openstack/neutron/neutron/tests/base.py", line 178, in func
      return f(self, *args, **kwargs)

        File "/home/ykarel/work/openstack/neutron/neutron/tests/unit/extensions/test_l3.py", line 4491, in test_port_deletion_prevention_handles_missing_port
      pl.prevent_l3_port_deletion(context.get_admin_context(), 'fakeid')

        File "/home/ykarel/work/openstack/neutron/neutron/db/l3_db.py", line 1742, in prevent_l3_port_deletion
      port = port or self._core_plugin.get_port(context, port_id)

        File "/home/ykarel/work/openstack/neutron/.tox/py3/lib/python3.10/site-packages/neutron_lib/db/api.py", line 223, in wrapped
      return f_with_retry(*args, **kwargs,

        File "/home/ykarel/work/openstack/neutron/.tox/py3/lib/python3.10/site-packages/neutron_lib/db/api.py", line 137, in wrapped
      with excutils.save_and_reraise_exception():

        File "/home/ykarel/work/openstack/neutron/.tox/py3/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__
      self.force_reraise()

        File "/home/ykarel/work/openstack/neutron/.tox/py3/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise
      raise self.value

        File "/home/ykarel/work/openstack/neutron/.tox/py3/lib/python3.10/site-packages/neutron_lib/db/api.py", line 135, in wrapped
      return f(*args, **kwargs)

        File "/home/ykarel/work/openstack/neutron/.tox/py3/lib/python3.10/site-packages/oslo_db/api.py", line 144, in wrapper
      with excutils.save_and_reraise_exception() as ectxt:

        File "/home/ykarel/work/openstack/neutron/.tox/py3/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__
      self.force_reraise()

        File "/home/ykarel/work/openstack/neutron/.tox/py3/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise
      raise self.value

        File "/home/ykarel/work/openstack/neutron/.tox/py3/lib/python3.10/site-packages/oslo_db/api.py", line 142, in wrapper
      return f(*args, **kwargs)

        File "/home/ykarel/work/openstack/neutron/.tox/py3/lib/python3.10/site-packages/neutron_lib/db/api.py", line 183, in wrapped
      with excutils.save_and_reraise_exception():

        File "/home/ykarel/work/openstack/neutron/.tox/py3/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__
      self.force_reraise()

        File "/home/ykarel/work/openstack/neutron/.tox/py3/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise
      raise self.value

        File "/home/ykarel/work/openstack/neutron/.tox/py3/lib/python3.10/site-packages/neutron_lib/db/api.py", line 181, in wrapped
      return f(*dup_args, **dup_kwargs)

        File "/home/ykarel/work/openstack/neutron/.tox/py3/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 1022, in wrapper
      return fn(*args, **kwargs)

        File "/home/ykarel/work/openstack/neutron/neutron/db/db_base_plugin_v2.py", line 1628, in get_port
      lazy_fields = [models_v2.Port.port_forwardings,

      AttributeError: type object 'Port' has no attribute
  'port_forwardings'

  It's reproducible Since Xena+ since the inclusion of patch
  https://review.opendev.org/c/openstack/neutron/+/790691

  It do not reproduce if there are other test runs(from the test class)
  before this test which involve other requests(like network get/create
  etc) apart from the ones modified in above patch.

  Considering above point if this test is modified to run other requests like below then it succeeds:-
  self.plugin.get_ports_count(context.get_admin_context())
  or
  self.plugin.get_networks_count(context.get_admin_context())
  or
  with self.port/network():
      self.assertIsNone(      
          pl.prevent_l3_port_deletion(context.get_admin_context(), 'fakeid')
      )

  
  The issue was originally noticed in a downstream job where unit tests were executed as part of package build. As the tests suite was executed with 100+ concurrency, by chance this test got executed in a worker first of any other test of the class.

  Me not sure if there are other test cases which will fail like this
  when run in isolation.

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/2028285/+subscriptions
Follow ups

[Bug 2028285] Re: [unit test][xena+] test_port_deletion_prevention fails when runs in isolation
From: Brian Haley, 2024-02-19