yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1546762] [NEW] 8a6d8bdae39_migrate_neutron_resources_table is not postgres compliant

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Cedric Brandily <1546762@xxxxxxxxxxxxxxxxxx>
Date: Wed, 17 Feb 2016 21:18:07 -0000
Reply-to: Bug 1546762 <1546762@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
Public bug reported:

8a6d8bdae39_migrate_neutron_resources_table.py[1] is not postgres-
compliant[3] and perhaps not working with non empty tables because
generate_records_for_existing assumes that
session.execute(...).inserted_primary_key is a value BUT it's a list[2]!

[1] in package neutron.db.migration.alembic_migrations.versions.mitaka.contract
[2] http://docs.sqlalchemy.org/en/rel_1_0/core/connections.html?highlight=inserted_primary_key#sqlalchemy.engine.ResultProxy.inserted_primary_key
[3]
# Starting with a liberty neutron db
#$ neutron-db-manage upgrade head


No handlers could be found for logger "oslo_config.cfg"
INFO  [alembic.runtime.migration] Context impl PostgresqlImpl.
INFO  [alembic.runtime.migration] Will assume transactional DDL.
  Running upgrade for neutron ...
INFO  [alembic.runtime.migration] Context impl PostgresqlImpl.
INFO  [alembic.runtime.migration] Will assume transactional DDL.
INFO  [alembic.runtime.migration] Running upgrade 34af2b5c5a59 -> 59cb5b6cf4d, Add availability zone
INFO  [alembic.runtime.migration] Running upgrade 59cb5b6cf4d -> 13cfb89f881a, add is_default to subnetpool
INFO  [alembic.runtime.migration] Running upgrade 13cfb89f881a -> 32e5974ada25, Add standard attribute table
INFO  [alembic.runtime.migration] Running upgrade 4af11ca47297 -> 1b294093239c, Drop embrane plugin table
INFO  [alembic.runtime.migration] Running upgrade 1b294093239c, 32e5974ada25 -> 8a6d8bdae39, standardattributes migration
Traceback (most recent call last):
  File "/home/user/projects/os/neutron/.tox/py27/bin/neutron-db-manage", line 10, in <module>
    sys.exit(main())
  File "/home/user/projects/os/neutron/neutron/db/migration/cli.py", line 744, in main
    return_val |= bool(CONF.command.func(config, CONF.command.name)) 
  File "/home/user/projects/os/neutron/neutron/db/migration/cli.py", line 220, in do_upgrade
    desc=branch, sql=CONF.command.sql)
  File "/home/user/projects/os/neutron/neutron/db/migration/cli.py", line 127, in do_alembic_command
    getattr(alembic_command, cmd)(config, *args, **kwargs)
  File "/home/user/projects/os/neutron/.tox/py27/local/lib/python2.7/site-packages/alembic/command.py", line 174, in upgrade
    script.run_env()
  File "/home/user/projects/os/neutron/.tox/py27/local/lib/python2.7/site-packages/alembic/script/base.py", line 397, in run_env
    util.load_python_file(self.dir, 'env.py')
  File "/home/user/projects/os/neutron/.tox/py27/local/lib/python2.7/site-packages/alembic/util/pyfiles.py", line 81, in load_python_file
    module = load_module_py(module_id, path)
  File "/home/user/projects/os/neutron/.tox/py27/local/lib/python2.7/site-packages/alembic/util/compat.py", line 79, in load_module_py
    mod = imp.load_source(module_id, path, fp)
  File "/home/user/projects/os/neutron/neutron/db/migration/alembic_migrations/env.py", line 126, in <module>
    run_migrations_online()
  File "/home/user/projects/os/neutron/neutron/db/migration/alembic_migrations/env.py", line 120, in run_migrations_online
    context.run_migrations()
  File "<string>", line 8, in run_migrations
  File "/home/user/projects/os/neutron/.tox/py27/local/lib/python2.7/site-packages/alembic/runtime/environment.py", line 797, in run_migrations
    self.get_context().run_migrations(**kw)
  File "/home/user/projects/os/neutron/.tox/py27/local/lib/python2.7/site-packages/alembic/runtime/migration.py", line 312, in run_migrations
    step.migration_fn(**kw)
  File "/home/user/projects/os/neutron/neutron/db/migration/alembic_migrations/versions/mitaka/contract/8a6d8bdae39_migrate_neutron_resources_table.py", line 50, in upgrade
    generate_records_for_existing()
  File "/home/user/projects/os/neutron/neutron/db/migration/alembic_migrations/versions/mitaka/contract/8a6d8bdae39_migrate_neutron_resources_table.py", line 83, in generate_records_for_existing
    model.c.id == row[0]))
  File "/home/user/projects/os/neutron/.tox/py27/local/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 1034, in execute
    bind, close_with_result=True).execute(clause, params or {})
  File "/home/user/projects/os/neutron/.tox/py27/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 914, in execute
    return meth(self, multiparams, params)
  File "/home/user/projects/os/neutron/.tox/py27/local/lib/python2.7/site-packages/sqlalchemy/sql/elements.py", line 323, in _execute_on_connection
    return connection._execute_clauseelement(self, multiparams, params)
  File "/home/user/projects/os/neutron/.tox/py27/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1010, in _execute_clauseelement
    compiled_sql, distilled_params
  File "/home/user/projects/os/neutron/.tox/py27/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1146, in _execute_context
    context)
  File "/home/user/projects/os/neutron/.tox/py27/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1337, in _handle_dbapi_exception
    util.raise_from_cause(newraise, exc_info)
  File "/home/user/projects/os/neutron/.tox/py27/local/lib/python2.7/site-packages/sqlalchemy/util/compat.py", line 200, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb, cause=cause)
  File "/home/user/projects/os/neutron/.tox/py27/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1139, in _execute_context
    context)
  File "/home/user/projects/os/neutron/.tox/py27/local/lib/python2.7/site-packages/sqlalchemy/engine/default.py", line 450, in do_execute
    cursor.execute(statement, parameters)
oslo_db.exception.DBError: (psycopg2.ProgrammingError) column "standard_attr_id" is of type bigint but expression is of type integer[]
LINE 1: UPDATE ports SET standard_attr_id=ARRAY[1] WHERE ports.id = ...

** Affects: neutron
     Importance: Undecided
     Assignee: Cedric Brandily (cbrandily)
         Status: New


** Tags: db

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1546762

Title:
  8a6d8bdae39_migrate_neutron_resources_table is not postgres compliant

Status in neutron:
  New

Bug description:
  8a6d8bdae39_migrate_neutron_resources_table.py[1] is not postgres-
  compliant[3] and perhaps not working with non empty tables because
  generate_records_for_existing assumes that
  session.execute(...).inserted_primary_key is a value BUT it's a
  list[2]!

  [1] in package neutron.db.migration.alembic_migrations.versions.mitaka.contract
  [2] http://docs.sqlalchemy.org/en/rel_1_0/core/connections.html?highlight=inserted_primary_key#sqlalchemy.engine.ResultProxy.inserted_primary_key
  [3]
  # Starting with a liberty neutron db
  #$ neutron-db-manage upgrade head

  
  No handlers could be found for logger "oslo_config.cfg"
  INFO  [alembic.runtime.migration] Context impl PostgresqlImpl.
  INFO  [alembic.runtime.migration] Will assume transactional DDL.
    Running upgrade for neutron ...
  INFO  [alembic.runtime.migration] Context impl PostgresqlImpl.
  INFO  [alembic.runtime.migration] Will assume transactional DDL.
  INFO  [alembic.runtime.migration] Running upgrade 34af2b5c5a59 -> 59cb5b6cf4d, Add availability zone
  INFO  [alembic.runtime.migration] Running upgrade 59cb5b6cf4d -> 13cfb89f881a, add is_default to subnetpool
  INFO  [alembic.runtime.migration] Running upgrade 13cfb89f881a -> 32e5974ada25, Add standard attribute table
  INFO  [alembic.runtime.migration] Running upgrade 4af11ca47297 -> 1b294093239c, Drop embrane plugin table
  INFO  [alembic.runtime.migration] Running upgrade 1b294093239c, 32e5974ada25 -> 8a6d8bdae39, standardattributes migration
  Traceback (most recent call last):
    File "/home/user/projects/os/neutron/.tox/py27/bin/neutron-db-manage", line 10, in <module>
      sys.exit(main())
    File "/home/user/projects/os/neutron/neutron/db/migration/cli.py", line 744, in main
      return_val |= bool(CONF.command.func(config, CONF.command.name)) 
    File "/home/user/projects/os/neutron/neutron/db/migration/cli.py", line 220, in do_upgrade
      desc=branch, sql=CONF.command.sql)
    File "/home/user/projects/os/neutron/neutron/db/migration/cli.py", line 127, in do_alembic_command
      getattr(alembic_command, cmd)(config, *args, **kwargs)
    File "/home/user/projects/os/neutron/.tox/py27/local/lib/python2.7/site-packages/alembic/command.py", line 174, in upgrade
      script.run_env()
    File "/home/user/projects/os/neutron/.tox/py27/local/lib/python2.7/site-packages/alembic/script/base.py", line 397, in run_env
      util.load_python_file(self.dir, 'env.py')
    File "/home/user/projects/os/neutron/.tox/py27/local/lib/python2.7/site-packages/alembic/util/pyfiles.py", line 81, in load_python_file
      module = load_module_py(module_id, path)
    File "/home/user/projects/os/neutron/.tox/py27/local/lib/python2.7/site-packages/alembic/util/compat.py", line 79, in load_module_py
      mod = imp.load_source(module_id, path, fp)
    File "/home/user/projects/os/neutron/neutron/db/migration/alembic_migrations/env.py", line 126, in <module>
      run_migrations_online()
    File "/home/user/projects/os/neutron/neutron/db/migration/alembic_migrations/env.py", line 120, in run_migrations_online
      context.run_migrations()
    File "<string>", line 8, in run_migrations
    File "/home/user/projects/os/neutron/.tox/py27/local/lib/python2.7/site-packages/alembic/runtime/environment.py", line 797, in run_migrations
      self.get_context().run_migrations(**kw)
    File "/home/user/projects/os/neutron/.tox/py27/local/lib/python2.7/site-packages/alembic/runtime/migration.py", line 312, in run_migrations
      step.migration_fn(**kw)
    File "/home/user/projects/os/neutron/neutron/db/migration/alembic_migrations/versions/mitaka/contract/8a6d8bdae39_migrate_neutron_resources_table.py", line 50, in upgrade
      generate_records_for_existing()
    File "/home/user/projects/os/neutron/neutron/db/migration/alembic_migrations/versions/mitaka/contract/8a6d8bdae39_migrate_neutron_resources_table.py", line 83, in generate_records_for_existing
      model.c.id == row[0]))
    File "/home/user/projects/os/neutron/.tox/py27/local/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 1034, in execute
      bind, close_with_result=True).execute(clause, params or {})
    File "/home/user/projects/os/neutron/.tox/py27/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 914, in execute
      return meth(self, multiparams, params)
    File "/home/user/projects/os/neutron/.tox/py27/local/lib/python2.7/site-packages/sqlalchemy/sql/elements.py", line 323, in _execute_on_connection
      return connection._execute_clauseelement(self, multiparams, params)
    File "/home/user/projects/os/neutron/.tox/py27/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1010, in _execute_clauseelement
      compiled_sql, distilled_params
    File "/home/user/projects/os/neutron/.tox/py27/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1146, in _execute_context
      context)
    File "/home/user/projects/os/neutron/.tox/py27/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1337, in _handle_dbapi_exception
      util.raise_from_cause(newraise, exc_info)
    File "/home/user/projects/os/neutron/.tox/py27/local/lib/python2.7/site-packages/sqlalchemy/util/compat.py", line 200, in raise_from_cause
      reraise(type(exception), exception, tb=exc_tb, cause=cause)
    File "/home/user/projects/os/neutron/.tox/py27/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1139, in _execute_context
      context)
    File "/home/user/projects/os/neutron/.tox/py27/local/lib/python2.7/site-packages/sqlalchemy/engine/default.py", line 450, in do_execute
      cursor.execute(statement, parameters)
  oslo_db.exception.DBError: (psycopg2.ProgrammingError) column "standard_attr_id" is of type bigint but expression is of type integer[]
  LINE 1: UPDATE ports SET standard_attr_id=ARRAY[1] WHERE ports.id = ...

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1546762/+subscriptions
Follow ups

[Bug 1546762] Re: 8a6d8bdae39_migrate_neutron_resources_table is not postgres compliant
From: OpenStack Infra, 2016-02-24