yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1260232] [NEW] db migration on unique constraint should not be applied to all plugins

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Edgar Magana <1260232@xxxxxxxxxxxxxxxxxx>
Date: Thu, 12 Dec 2013 09:04:43 -0000
Reply-to: Bug 1260232 <1260232@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
Public bug reported:

There is an assumption that all "neutron plugins" creates the
plumgrid_neutron.agents table, which is not the case. I just tested big
switch and plumgrid plugins and they are failing:

INFO  [alembic.migration] Running upgrade havana -> e197124d4b9, add unique constraint to members
INFO  [alembic.migration] Running upgrade e197124d4b9 -> 1fcfc149aca4, Add a unique constraint on (agent_type, host) columns to prevent a race
condition when an agent entry is 'upserted'.
Traceback (most recent call last):
  File "/usr/local/bin/neutron-db-manage", line 10, in <module>
    sys.exit(main())
  File "/opt/stack/neutron/neutron/db/migration/cli.py", line 143, in main
    CONF.command.func(config, CONF.command.name)
  File "/opt/stack/neutron/neutron/db/migration/cli.py", line 80, in do_upgrade_downgrade
    do_alembic_command(config, cmd, revision, sql=CONF.command.sql)
  File "/opt/stack/neutron/neutron/db/migration/cli.py", line 59, in do_alembic_command
    getattr(alembic_command, cmd)(config, *args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/alembic/command.py", line 124, in upgrade
    script.run_env()
  File "/usr/local/lib/python2.7/dist-packages/alembic/script.py", line 193, in run_env
    util.load_python_file(self.dir, 'env.py')
  File "/usr/local/lib/python2.7/dist-packages/alembic/util.py", line 177, in load_python_file
    module = load_module(module_id, path)
  File "/usr/local/lib/python2.7/dist-packages/alembic/compat.py", line 39, in load_module
    return imp.load_source(module_id, path, fp)
  File "/opt/stack/neutron/neutron/db/migration/alembic_migrations/env.py", line 105, in <module>
    run_migrations_online()
  File "/opt/stack/neutron/neutron/db/migration/alembic_migrations/env.py", line 89, in run_migrations_online
    options=build_options())
  File "<string>", line 7, in run_migrations
  File "/usr/local/lib/python2.7/dist-packages/alembic/environment.py", line 652, in run_migrations
    self.get_context().run_migrations(**kw)
  File "/usr/local/lib/python2.7/dist-packages/alembic/migration.py", line 224, in run_migrations
    change(**kw)
  File "/opt/stack/neutron/neutron/db/migration/alembic_migrations/versions/1fcfc149aca4_agents_unique_by_type_and_host.py", line 50, in upgrade
    local_cols=['agent_type', 'host']
  File "<string>", line 7, in create_unique_constraint
  File "/usr/local/lib/python2.7/dist-packages/alembic/operations.py", line 539, in create_unique_constraint
    schema=schema, **kw)
  File "/usr/local/lib/python2.7/dist-packages/alembic/ddl/impl.py", line 135, in add_constraint
    self._exec(schema.AddConstraint(const))
  File "/usr/local/lib/python2.7/dist-packages/alembic/ddl/impl.py", line 76, in _exec
    conn.execute(construct, *multiparams, **params)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1449, in execute
    params)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1542, in _execute_ddl
    compiled
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1698, in _execute_context
    context)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1691, in _execute_context
    context)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", line 331, in do_execute
    cursor.execute(statement, parameters)
  File "/usr/lib/python2.7/dist-packages/MySQLdb/cursors.py", line 174, in execute
    self.errorhandler(self, exc, value)
  File "/usr/lib/python2.7/dist-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
    raise errorclass, errorvalue
sqlalchemy.exc.ProgrammingError: (ProgrammingError) (1146, "Table 'plumgrid_neutron.agents' doesn't exist") 'ALTER TABLE agents ADD CONSTRAINT uniq_agents0agent_type0host UNIQUE (agent_type, host)' ()
++ failed
++ local r=1
+++ jobs -p
++ kill
++ set +o xtrace

** Affects: neutron
     Importance: Undecided
     Assignee: Edgar Magana (emagana)
         Status: In Progress

** Changed in: neutron
     Assignee: (unassigned) => Edgar Magana (emagana)

** Changed in: neutron
       Status: New => In Progress

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1260232

Title:
  db migration on unique constraint should not be applied to all plugins

Status in OpenStack Neutron (virtual network service):
  In Progress

Bug description:
  There is an assumption that all "neutron plugins" creates the
  plumgrid_neutron.agents table, which is not the case. I just tested
  big switch and plumgrid plugins and they are failing:

  INFO  [alembic.migration] Running upgrade havana -> e197124d4b9, add unique constraint to members
  INFO  [alembic.migration] Running upgrade e197124d4b9 -> 1fcfc149aca4, Add a unique constraint on (agent_type, host) columns to prevent a race
  condition when an agent entry is 'upserted'.
  Traceback (most recent call last):
    File "/usr/local/bin/neutron-db-manage", line 10, in <module>
      sys.exit(main())
    File "/opt/stack/neutron/neutron/db/migration/cli.py", line 143, in main
      CONF.command.func(config, CONF.command.name)
    File "/opt/stack/neutron/neutron/db/migration/cli.py", line 80, in do_upgrade_downgrade
      do_alembic_command(config, cmd, revision, sql=CONF.command.sql)
    File "/opt/stack/neutron/neutron/db/migration/cli.py", line 59, in do_alembic_command
      getattr(alembic_command, cmd)(config, *args, **kwargs)
    File "/usr/local/lib/python2.7/dist-packages/alembic/command.py", line 124, in upgrade
      script.run_env()
    File "/usr/local/lib/python2.7/dist-packages/alembic/script.py", line 193, in run_env
      util.load_python_file(self.dir, 'env.py')
    File "/usr/local/lib/python2.7/dist-packages/alembic/util.py", line 177, in load_python_file
      module = load_module(module_id, path)
    File "/usr/local/lib/python2.7/dist-packages/alembic/compat.py", line 39, in load_module
      return imp.load_source(module_id, path, fp)
    File "/opt/stack/neutron/neutron/db/migration/alembic_migrations/env.py", line 105, in <module>
      run_migrations_online()
    File "/opt/stack/neutron/neutron/db/migration/alembic_migrations/env.py", line 89, in run_migrations_online
      options=build_options())
    File "<string>", line 7, in run_migrations
    File "/usr/local/lib/python2.7/dist-packages/alembic/environment.py", line 652, in run_migrations
      self.get_context().run_migrations(**kw)
    File "/usr/local/lib/python2.7/dist-packages/alembic/migration.py", line 224, in run_migrations
      change(**kw)
    File "/opt/stack/neutron/neutron/db/migration/alembic_migrations/versions/1fcfc149aca4_agents_unique_by_type_and_host.py", line 50, in upgrade
      local_cols=['agent_type', 'host']
    File "<string>", line 7, in create_unique_constraint
    File "/usr/local/lib/python2.7/dist-packages/alembic/operations.py", line 539, in create_unique_constraint
      schema=schema, **kw)
    File "/usr/local/lib/python2.7/dist-packages/alembic/ddl/impl.py", line 135, in add_constraint
      self._exec(schema.AddConstraint(const))
    File "/usr/local/lib/python2.7/dist-packages/alembic/ddl/impl.py", line 76, in _exec
      conn.execute(construct, *multiparams, **params)
    File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1449, in execute
      params)
    File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1542, in _execute_ddl
      compiled
    File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1698, in _execute_context
      context)
    File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1691, in _execute_context
      context)
    File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", line 331, in do_execute
      cursor.execute(statement, parameters)
    File "/usr/lib/python2.7/dist-packages/MySQLdb/cursors.py", line 174, in execute
      self.errorhandler(self, exc, value)
    File "/usr/lib/python2.7/dist-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
      raise errorclass, errorvalue
  sqlalchemy.exc.ProgrammingError: (ProgrammingError) (1146, "Table 'plumgrid_neutron.agents' doesn't exist") 'ALTER TABLE agents ADD CONSTRAINT uniq_agents0agent_type0host UNIQUE (agent_type, host)' ()
  ++ failed
  ++ local r=1
  +++ jobs -p
  ++ kill
  ++ set +o xtrace

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1260232/+subscriptions
Follow ups

[Bug 1260232] Re: db migration on unique constraint should not be applied to all plugins
From: Thierry Carrez, 2014-01-22
[Bug 1260232] [NEW] db migration on unique constraint should not be applied to all plugins
From: Edgar Magana, 2013-12-12
References

[Bug 1260232] [NEW] db migration on unique constraint should not be applied to all plugins
From: Edgar Magana, 2013-12-12