yahoo-eng-team team mailing list archive

[Mike Bayer <mike_mp@xxxxxxxxxxxxxxxx>]

Public bug reported:

Currently only neutron/tests/functional/db/test_ipam.py seems to use
this fixture, however it is not interacting correctly with oslo.db such
that it actually uses the engine set up by oslo.

Add a test like this:

diff --git a/neutron/tests/functional/db/test_ipam.py b/neutron/tests/functional/db/test_ipam.py
index 0f28f74..d14bf6e 100644
--- a/neutron/tests/functional/db/test_ipam.py
+++ b/neutron/tests/functional/db/test_ipam.py
@@ -156,8 +156,8 @@ class IpamTestCase(base.BaseTestCase):
 
 
 class TestIpamMySql(common_base.MySQLTestCase, IpamTestCase):
-    pass
-
+    def test_we_are_on_mysql(self):
+        self.cxt.session.execute("SELECT CURDATE()")
 
 class TestIpamPsql(common_base.PostgreSQLTestCase, IpamTestCase):
     pass


then run:

[classic@photon2 neutron]$  tox -e functional neutron.tests.functional.db.test_ipam
functional develop-inst-nodeps: /home/classic/dev/redhat/openstack/neutron
functional installed:  ( ... output skipped ... )
functional runtests: PYTHONHASHSEED='545881821'
functional runtests: commands[0] | /home/classic/dev/redhat/openstack/neutron/tools/ostestr_compat_shim.sh neutron.tests.functional.db.test_ipam

( ... output skipped ... )

{3} neutron.tests.functional.db.test_ipam.IpamTestCase.test_allocate_fixed_ip [1.510751s] ... ok
{1} neutron.tests.functional.db.test_ipam.TestIpamMySql.test_allocate_fixed_ip [1.822431s] ... ok
{2} neutron.tests.functional.db.test_ipam.IpamTestCase.test_allocate_ip_exausted_pool [2.468420s] ... ok
{1} neutron.tests.functional.db.test_ipam.TestIpamPsql.test_allocate_ip_exausted_pool ... SKIPPED: backend 'postgresql' unavailable
{0} neutron.tests.functional.db.test_ipam.TestIpamMySql.test_allocate_ip_exausted_pool [2.873318s] ... ok
{2} neutron.tests.functional.db.test_ipam.TestIpamMySql.test_we_are_on_mysql [0.993651s] ... FAILED
{0} neutron.tests.functional.db.test_ipam.TestIpamPsql.test_allocate_fixed_ip ... SKIPPED: backend 'postgresql' unavailable
{1} neutron.tests.functional.db.test_ipam.TestPluggableIpamMySql.test_allocate_fixed_ip [1.133034s] ... ok
{0} neutron.tests.functional.db.test_ipam.TestPluggableIpamPsql.test_allocate_ip_exausted_pool ... SKIPPED: backend 'postgresql' unavailable
{2} neutron.tests.functional.db.test_ipam.TestPluggableIpamPsql.test_allocate_fixed_ip ... SKIPPED: backend 'postgresql' unavailable
{3} neutron.tests.functional.db.test_ipam.TestPluggableIpamMySql.test_allocate_ip_exausted_pool [2.740086s] ... ok

==============================
Failed 1 tests - output below:
==============================

neutron.tests.functional.db.test_ipam.TestIpamMySql.test_we_are_on_mysql
------------------------------------------------------------------------

Captured traceback:
~~~~~~~~~~~~~~~~~~~
    Traceback (most recent call last):
      File "neutron/tests/functional/db/test_ipam.py", line 160, in test_we_are_on_mysql
        self.cxt.session.execute("SELECT CURDATE()")
      File "/home/classic/dev/redhat/openstack/neutron/.tox/functional/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 1034, in execute
        bind, close_with_result=True).execute(clause, params or {})
      File "/home/classic/dev/redhat/openstack/neutron/.tox/functional/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 914, in execute
        return meth(self, multiparams, params)
      File "/home/classic/dev/redhat/openstack/neutron/.tox/functional/lib/python2.7/site-packages/sqlalchemy/sql/elements.py", line 323, in _execute_on_connection
        return connection._execute_clauseelement(self, multiparams, params)
      File "/home/classic/dev/redhat/openstack/neutron/.tox/functional/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1010, in _execute_clauseelement
        compiled_sql, distilled_params
      File "/home/classic/dev/redhat/openstack/neutron/.tox/functional/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1146, in _execute_context
        context)
      File "/home/classic/dev/redhat/openstack/neutron/.tox/functional/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1337, in _handle_dbapi_exception
        util.raise_from_cause(newraise, exc_info)
      File "/home/classic/dev/redhat/openstack/neutron/.tox/functional/lib/python2.7/site-packages/sqlalchemy/util/compat.py", line 202, in raise_from_cause
        reraise(type(exception), exception, tb=exc_tb, cause=cause)
      File "/home/classic/dev/redhat/openstack/neutron/.tox/functional/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1139, in _execute_context
        context)
      File "/home/classic/dev/redhat/openstack/neutron/.tox/functional/lib/python2.7/site-packages/sqlalchemy/engine/default.py", line 450, in do_execute
        cursor.execute(statement, parameters)
    sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) no such function: CURDATE [SQL: u'SELECT CURDATE()']
    

At the end there, that's a SQLite error.  You're not supposed to get
those in the MySQL test suite :).

The problem is that the SqlFixture is calling upon
neutron.db.api.get_engine() but this is in no way associated with the
context that oslo.db creates within the MySQLOpportunisticFixture
approach.   As neutron is using enginefacade now we need to swap in the
facade that's specific to oslo_db.sqlalchemy.test_base.DbFixture and
make sure everything is linked up.

Note that this problem does not impact the alembic migration tests, as
that test suite does its own set up of alembic fixtures.

I'm working on a reorg of the test fixtures here so this can work, as we
will need these fixtures to be effective for the upcoming CIDR stored
functions / triggers to be tested.

** Affects: neutron
     Importance: Undecided
     Assignee: Mike Bayer (zzzeek)
         Status: New

** Changed in: neutron
     Assignee: (unassigned) => Mike Bayer (zzzeek)

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1594898

Title:
  functional DB tests based on SqlFixture don't actually use non-sqlite
  DB

Status in neutron:
  New

Bug description:
  Currently only neutron/tests/functional/db/test_ipam.py seems to use
  this fixture, however it is not interacting correctly with oslo.db
  such that it actually uses the engine set up by oslo.

  Add a test like this:

  diff --git a/neutron/tests/functional/db/test_ipam.py b/neutron/tests/functional/db/test_ipam.py
  index 0f28f74..d14bf6e 100644
  --- a/neutron/tests/functional/db/test_ipam.py
  +++ b/neutron/tests/functional/db/test_ipam.py
  @@ -156,8 +156,8 @@ class IpamTestCase(base.BaseTestCase):
   
   
   class TestIpamMySql(common_base.MySQLTestCase, IpamTestCase):
  -    pass
  -
  +    def test_we_are_on_mysql(self):
  +        self.cxt.session.execute("SELECT CURDATE()")
   
   class TestIpamPsql(common_base.PostgreSQLTestCase, IpamTestCase):
       pass

  
  then run:

  [classic@photon2 neutron]$  tox -e functional neutron.tests.functional.db.test_ipam
  functional develop-inst-nodeps: /home/classic/dev/redhat/openstack/neutron
  functional installed:  ( ... output skipped ... )
  functional runtests: PYTHONHASHSEED='545881821'
  functional runtests: commands[0] | /home/classic/dev/redhat/openstack/neutron/tools/ostestr_compat_shim.sh neutron.tests.functional.db.test_ipam

  ( ... output skipped ... )

  {3} neutron.tests.functional.db.test_ipam.IpamTestCase.test_allocate_fixed_ip [1.510751s] ... ok
  {1} neutron.tests.functional.db.test_ipam.TestIpamMySql.test_allocate_fixed_ip [1.822431s] ... ok
  {2} neutron.tests.functional.db.test_ipam.IpamTestCase.test_allocate_ip_exausted_pool [2.468420s] ... ok
  {1} neutron.tests.functional.db.test_ipam.TestIpamPsql.test_allocate_ip_exausted_pool ... SKIPPED: backend 'postgresql' unavailable
  {0} neutron.tests.functional.db.test_ipam.TestIpamMySql.test_allocate_ip_exausted_pool [2.873318s] ... ok
  {2} neutron.tests.functional.db.test_ipam.TestIpamMySql.test_we_are_on_mysql [0.993651s] ... FAILED
  {0} neutron.tests.functional.db.test_ipam.TestIpamPsql.test_allocate_fixed_ip ... SKIPPED: backend 'postgresql' unavailable
  {1} neutron.tests.functional.db.test_ipam.TestPluggableIpamMySql.test_allocate_fixed_ip [1.133034s] ... ok
  {0} neutron.tests.functional.db.test_ipam.TestPluggableIpamPsql.test_allocate_ip_exausted_pool ... SKIPPED: backend 'postgresql' unavailable
  {2} neutron.tests.functional.db.test_ipam.TestPluggableIpamPsql.test_allocate_fixed_ip ... SKIPPED: backend 'postgresql' unavailable
  {3} neutron.tests.functional.db.test_ipam.TestPluggableIpamMySql.test_allocate_ip_exausted_pool [2.740086s] ... ok

  ==============================
  Failed 1 tests - output below:
  ==============================

  neutron.tests.functional.db.test_ipam.TestIpamMySql.test_we_are_on_mysql
  ------------------------------------------------------------------------

  Captured traceback:
  ~~~~~~~~~~~~~~~~~~~
      Traceback (most recent call last):
        File "neutron/tests/functional/db/test_ipam.py", line 160, in test_we_are_on_mysql
          self.cxt.session.execute("SELECT CURDATE()")
        File "/home/classic/dev/redhat/openstack/neutron/.tox/functional/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 1034, in execute
          bind, close_with_result=True).execute(clause, params or {})
        File "/home/classic/dev/redhat/openstack/neutron/.tox/functional/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 914, in execute
          return meth(self, multiparams, params)
        File "/home/classic/dev/redhat/openstack/neutron/.tox/functional/lib/python2.7/site-packages/sqlalchemy/sql/elements.py", line 323, in _execute_on_connection
          return connection._execute_clauseelement(self, multiparams, params)
        File "/home/classic/dev/redhat/openstack/neutron/.tox/functional/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1010, in _execute_clauseelement
          compiled_sql, distilled_params
        File "/home/classic/dev/redhat/openstack/neutron/.tox/functional/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1146, in _execute_context
          context)
        File "/home/classic/dev/redhat/openstack/neutron/.tox/functional/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1337, in _handle_dbapi_exception
          util.raise_from_cause(newraise, exc_info)
        File "/home/classic/dev/redhat/openstack/neutron/.tox/functional/lib/python2.7/site-packages/sqlalchemy/util/compat.py", line 202, in raise_from_cause
          reraise(type(exception), exception, tb=exc_tb, cause=cause)
        File "/home/classic/dev/redhat/openstack/neutron/.tox/functional/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1139, in _execute_context
          context)
        File "/home/classic/dev/redhat/openstack/neutron/.tox/functional/lib/python2.7/site-packages/sqlalchemy/engine/default.py", line 450, in do_execute
          cursor.execute(statement, parameters)
      sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) no such function: CURDATE [SQL: u'SELECT CURDATE()']
      

  At the end there, that's a SQLite error.  You're not supposed to get
  those in the MySQL test suite :).

  The problem is that the SqlFixture is calling upon
  neutron.db.api.get_engine() but this is in no way associated with the
  context that oslo.db creates within the MySQLOpportunisticFixture
  approach.   As neutron is using enginefacade now we need to swap in
  the facade that's specific to oslo_db.sqlalchemy.test_base.DbFixture
  and make sure everything is linked up.

  Note that this problem does not impact the alembic migration tests, as
  that test suite does its own set up of alembic fixtures.

  I'm working on a reorg of the test fixtures here so this can work, as
  we will need these fixtures to be effective for the upcoming CIDR
  stored functions / triggers to be tested.

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1594898/+subscriptions