yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1557757] [NEW] Fix add_is_default_to_subnetpool migration in PostgreSQL

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Cedric Brandily <1557757@xxxxxxxxxxxxxxxxxx>
Date: Tue, 15 Mar 2016 21:38:34 -0000
Reply-to: Bug 1557757 <1557757@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
Public bug reported:

add_is_default_to_subnetpool migration[1] is not working on PostgreSQL
with a non empty because it adds a new column non-nullable with no
default value and PostgreSQL cannot deduce it.

It could affect  also MySQL.


[1] neutron/db/migration/alembic_migrations/versions/mitaka/expand/13cfb89f881a_add_is_default_to_subnetpool.py
[2] stacktrace

user@host:/opt/stack/neutron$ neutron-db-manage --config-file ~/a upgrade head
No handlers could be found for logger "oslo_config.cfg"
INFO  [alembic.runtime.migration] Context impl PostgresqlImpl.
INFO  [alembic.runtime.migration] Will assume transactional DDL.
  Running upgrade for neutron ...
INFO  [alembic.runtime.migration] Context impl PostgresqlImpl.
INFO  [alembic.runtime.migration] Will assume transactional DDL.
INFO  [alembic.runtime.migration] Running upgrade 34af2b5c5a59 -> 59cb5b6cf4d, Add availability zone
INFO  [alembic.runtime.migration] Running upgrade 59cb5b6cf4d -> 13cfb89f881a, add is_default to subnetpool
Traceback (most recent call last):
  File "/opt/stack/neutron/.tox/py27/bin/neutron-db-manage", line 10, in <module>
    sys.exit(main())
  File "/opt/stack/neutron/neutron/db/migration/cli.py", line 749, in main
    return_val |= bool(CONF.command.func(config, CONF.command.name))
  File "/opt/stack/neutron/neutron/db/migration/cli.py", line 225, in do_upgrade
    desc=branch, sql=CONF.command.sql)
  File "/opt/stack/neutron/neutron/db/migration/cli.py", line 127, in do_alembic_command
    getattr(alembic_command, cmd)(config, *args, **kwargs)
  File "/opt/stack/neutron/.tox/py27/local/lib/python2.7/site-packages/alembic/command.py", line 174, in upgrade
    script.run_env()
  File "/opt/stack/neutron/.tox/py27/local/lib/python2.7/site-packages/alembic/script/base.py", line 397, in run_env
    util.load_python_file(self.dir, 'env.py')
  File "/opt/stack/neutron/.tox/py27/local/lib/python2.7/site-packages/alembic/util/pyfiles.py", line 81, in load_python_file
    module = load_module_py(module_id, path)
  File "/opt/stack/neutron/.tox/py27/local/lib/python2.7/site-packages/alembic/util/compat.py", line 79, in load_module_py
    mod = imp.load_source(module_id, path, fp)
  File "/opt/stack/neutron/neutron/db/migration/alembic_migrations/env.py", line 126, in <module>
    run_migrations_online()
  File "/opt/stack/neutron/neutron/db/migration/alembic_migrations/env.py", line 120, in run_migrations_online
    context.run_migrations()
  File "<string>", line 8, in run_migrations
  File "/opt/stack/neutron/.tox/py27/local/lib/python2.7/site-packages/alembic/runtime/environment.py", line 797, in run_migrations
    self.get_context().run_migrations(**kw)
  File "/opt/stack/neutron/.tox/py27/local/lib/python2.7/site-packages/alembic/runtime/migration.py", line 312, in run_migrations
    step.migration_fn(**kw)
  File "/opt/stack/neutron/neutron/db/migration/alembic_migrations/versions/mitaka/expand/13cfb89f881a_add_is_default_to_subnetpool.py", line 36, in upgrade
    nullable=False))
  File "<string>", line 8, in add_column
  File "<string>", line 3, in add_column
  File "/opt/stack/neutron/.tox/py27/local/lib/python2.7/site-packages/alembic/operations/ops.py", line 1535, in add_column
    return operations.invoke(op)
  File "/opt/stack/neutron/.tox/py27/local/lib/python2.7/site-packages/alembic/operations/base.py", line 318, in invoke
    return fn(self, operation)
  File "/opt/stack/neutron/.tox/py27/local/lib/python2.7/site-packages/alembic/operations/toimpl.py", line 123, in add_column
    schema=schema
  File "/opt/stack/neutron/.tox/py27/local/lib/python2.7/site-packages/alembic/ddl/impl.py", line 172, in add_column
    self._exec(base.AddColumn(table_name, column, schema=schema))
  File "/opt/stack/neutron/.tox/py27/local/lib/python2.7/site-packages/alembic/ddl/impl.py", line 118, in _exec
    return conn.execute(construct, *multiparams, **params)
  File "/opt/stack/neutron/.tox/py27/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 914, in execute
    return meth(self, multiparams, params)
  File "/opt/stack/neutron/.tox/py27/local/lib/python2.7/site-packages/sqlalchemy/sql/ddl.py", line 68, in _execute_on_connection
    return connection._execute_ddl(self, multiparams, params)
  File "/opt/stack/neutron/.tox/py27/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 968, in _execute_ddl
    compiled
  File "/opt/stack/neutron/.tox/py27/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1146, in _execute_context
    context)
  File "/opt/stack/neutron/.tox/py27/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1337, in _handle_dbapi_exception
    util.raise_from_cause(newraise, exc_info)
  File "/opt/stack/neutron/.tox/py27/local/lib/python2.7/site-packages/sqlalchemy/util/compat.py", line 200, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb, cause=cause)
  File "/opt/stack/neutron/.tox/py27/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1139, in _execute_context
    context)
  File "/opt/stack/neutron/.tox/py27/local/lib/python2.7/site-packages/sqlalchemy/engine/default.py", line 450, in do_execute
    cursor.execute(statement, parameters)
oslo_db.exception.DBError: (psycopg2.IntegrityError) column "is_default" contains null values
 [SQL: 'ALTER TABLE subnetpools ADD COLUMN is_default BOOLEAN NOT NULL']

** Affects: neutron
     Importance: Undecided
     Assignee: Cedric Brandily (cbrandily)
         Status: New


** Tags: db mitaka-rc-potential

** Tags added: mitaka-rc-potential

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1557757

Title:
  Fix add_is_default_to_subnetpool migration in PostgreSQL

Status in neutron:
  New

Bug description:
  add_is_default_to_subnetpool migration[1] is not working on PostgreSQL
  with a non empty because it adds a new column non-nullable with no
  default value and PostgreSQL cannot deduce it.

  It could affect  also MySQL.

  
  [1] neutron/db/migration/alembic_migrations/versions/mitaka/expand/13cfb89f881a_add_is_default_to_subnetpool.py
  [2] stacktrace

  user@host:/opt/stack/neutron$ neutron-db-manage --config-file ~/a upgrade head
  No handlers could be found for logger "oslo_config.cfg"
  INFO  [alembic.runtime.migration] Context impl PostgresqlImpl.
  INFO  [alembic.runtime.migration] Will assume transactional DDL.
    Running upgrade for neutron ...
  INFO  [alembic.runtime.migration] Context impl PostgresqlImpl.
  INFO  [alembic.runtime.migration] Will assume transactional DDL.
  INFO  [alembic.runtime.migration] Running upgrade 34af2b5c5a59 -> 59cb5b6cf4d, Add availability zone
  INFO  [alembic.runtime.migration] Running upgrade 59cb5b6cf4d -> 13cfb89f881a, add is_default to subnetpool
  Traceback (most recent call last):
    File "/opt/stack/neutron/.tox/py27/bin/neutron-db-manage", line 10, in <module>
      sys.exit(main())
    File "/opt/stack/neutron/neutron/db/migration/cli.py", line 749, in main
      return_val |= bool(CONF.command.func(config, CONF.command.name))
    File "/opt/stack/neutron/neutron/db/migration/cli.py", line 225, in do_upgrade
      desc=branch, sql=CONF.command.sql)
    File "/opt/stack/neutron/neutron/db/migration/cli.py", line 127, in do_alembic_command
      getattr(alembic_command, cmd)(config, *args, **kwargs)
    File "/opt/stack/neutron/.tox/py27/local/lib/python2.7/site-packages/alembic/command.py", line 174, in upgrade
      script.run_env()
    File "/opt/stack/neutron/.tox/py27/local/lib/python2.7/site-packages/alembic/script/base.py", line 397, in run_env
      util.load_python_file(self.dir, 'env.py')
    File "/opt/stack/neutron/.tox/py27/local/lib/python2.7/site-packages/alembic/util/pyfiles.py", line 81, in load_python_file
      module = load_module_py(module_id, path)
    File "/opt/stack/neutron/.tox/py27/local/lib/python2.7/site-packages/alembic/util/compat.py", line 79, in load_module_py
      mod = imp.load_source(module_id, path, fp)
    File "/opt/stack/neutron/neutron/db/migration/alembic_migrations/env.py", line 126, in <module>
      run_migrations_online()
    File "/opt/stack/neutron/neutron/db/migration/alembic_migrations/env.py", line 120, in run_migrations_online
      context.run_migrations()
    File "<string>", line 8, in run_migrations
    File "/opt/stack/neutron/.tox/py27/local/lib/python2.7/site-packages/alembic/runtime/environment.py", line 797, in run_migrations
      self.get_context().run_migrations(**kw)
    File "/opt/stack/neutron/.tox/py27/local/lib/python2.7/site-packages/alembic/runtime/migration.py", line 312, in run_migrations
      step.migration_fn(**kw)
    File "/opt/stack/neutron/neutron/db/migration/alembic_migrations/versions/mitaka/expand/13cfb89f881a_add_is_default_to_subnetpool.py", line 36, in upgrade
      nullable=False))
    File "<string>", line 8, in add_column
    File "<string>", line 3, in add_column
    File "/opt/stack/neutron/.tox/py27/local/lib/python2.7/site-packages/alembic/operations/ops.py", line 1535, in add_column
      return operations.invoke(op)
    File "/opt/stack/neutron/.tox/py27/local/lib/python2.7/site-packages/alembic/operations/base.py", line 318, in invoke
      return fn(self, operation)
    File "/opt/stack/neutron/.tox/py27/local/lib/python2.7/site-packages/alembic/operations/toimpl.py", line 123, in add_column
      schema=schema
    File "/opt/stack/neutron/.tox/py27/local/lib/python2.7/site-packages/alembic/ddl/impl.py", line 172, in add_column
      self._exec(base.AddColumn(table_name, column, schema=schema))
    File "/opt/stack/neutron/.tox/py27/local/lib/python2.7/site-packages/alembic/ddl/impl.py", line 118, in _exec
      return conn.execute(construct, *multiparams, **params)
    File "/opt/stack/neutron/.tox/py27/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 914, in execute
      return meth(self, multiparams, params)
    File "/opt/stack/neutron/.tox/py27/local/lib/python2.7/site-packages/sqlalchemy/sql/ddl.py", line 68, in _execute_on_connection
      return connection._execute_ddl(self, multiparams, params)
    File "/opt/stack/neutron/.tox/py27/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 968, in _execute_ddl
      compiled
    File "/opt/stack/neutron/.tox/py27/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1146, in _execute_context
      context)
    File "/opt/stack/neutron/.tox/py27/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1337, in _handle_dbapi_exception
      util.raise_from_cause(newraise, exc_info)
    File "/opt/stack/neutron/.tox/py27/local/lib/python2.7/site-packages/sqlalchemy/util/compat.py", line 200, in raise_from_cause
      reraise(type(exception), exception, tb=exc_tb, cause=cause)
    File "/opt/stack/neutron/.tox/py27/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1139, in _execute_context
      context)
    File "/opt/stack/neutron/.tox/py27/local/lib/python2.7/site-packages/sqlalchemy/engine/default.py", line 450, in do_execute
      cursor.execute(statement, parameters)
  oslo_db.exception.DBError: (psycopg2.IntegrityError) column "is_default" contains null values
   [SQL: 'ALTER TABLE subnetpools ADD COLUMN is_default BOOLEAN NOT NULL']

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1557757/+subscriptions
Follow ups

[Bug 1557757] Re: add_is_default_to_subnetpool migration not postgresql-compliant
From: OpenStack Infra, 2016-03-16