yahoo-eng-team team mailing list archive

[Thierry Carrez <thierry.carrez+lp@xxxxxxxxx>]

** Changed in: keystone
       Status: Fix Committed => Fix Released

** Changed in: keystone
    Milestone: None => grizzly-3

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to Keystone.
https://bugs.launchpad.net/bugs/1098174

Title:
  MySQL db_sync issue

Status in OpenStack Identity (Keystone):
  Fix Released

Bug description:
  Keystone will not complete its install with MySQL db_sync. Bug
  produced on a new install.

  The first db_sync returns:
  # keystone-manage db_sync
  Traceback (most recent call last):
    File "/usr/bin/keystone-manage", line 5, in <module>
      pkg_resources.run_script('keystone==2013.1', 'keystone-manage')
    File "/usr/lib/python2.6/site-packages/pkg_resources.py", line 461, in run_script
      self.require(requires)[0].run_script(script_name, ns)
    File "/usr/lib/python2.6/site-packages/pkg_resources.py", line 1194, in run_script
      execfile(script_filename, namespace, namespace)
    File "/usr/lib/python2.6/site-packages/keystone-2013.1-py2.6.egg/EGG-INFO/scripts/keystone-manage", line 28, in <module>
      cli.main(argv=sys.argv, config_files=config_files)
    File "/usr/lib/python2.6/site-packages/keystone-2013.1-py2.6.egg/keystone/cli.py", line 144, in main
      CONF.command.cmd_class.main()
    File "/usr/lib/python2.6/site-packages/keystone-2013.1-py2.6.egg/keystone/cli.py", line 49, in main
      driver.db_sync()
    File "/usr/lib/python2.6/site-packages/keystone-2013.1-py2.6.egg/keystone/identity/backends/sql.py", line 167, in db_sync
      migration.db_sync()
    File "/usr/lib/python2.6/site-packages/keystone-2013.1-py2.6.egg/keystone/common/sql/migration.py", line 52, in db_sync
      return versioning_api.upgrade(CONF.sql.connection, repo_path, version)
    File "/usr/lib/python2.6/site-packages/migrate/versioning/api.py", line 186, in upgrade
      return _migrate(url, repository, version, upgrade=True, err=err, **opts)
    File "<string>", line 2, in _migrate
    File "/usr/lib/python2.6/site-packages/migrate/versioning/util/__init__.py", line 159, in with_engine
      return f(*a, **kw)
    File "/usr/lib/python2.6/site-packages/migrate/versioning/api.py", line 366, in _migrate
      schema.runchange(ver, change, changeset.step)
    File "/usr/lib/python2.6/site-packages/migrate/versioning/schema.py", line 91, in runchange
      change.run(self.engine, step)
    File "/usr/lib/python2.6/site-packages/migrate/versioning/script/py.py", line 145, in run
      script_func(engine)
    File "/usr/lib/python2.6/site-packages/keystone-2013.1-py2.6.egg/keystone/common/sql/migrate_repo/versions/010_endpoints_v3.py", line 41, in upgrade
      new_table.create(migrate_engine, checkfirst=True)
    File "/usr/lib64/python2.6/site-packages/sqlalchemy/schema.py", line 593, in create
      checkfirst=checkfirst)
    File "/usr/lib64/python2.6/site-packages/sqlalchemy/engine/base.py", line 2302, in _run_visitor
      conn._run_visitor(visitorcallable, element, **kwargs)
    File "/usr/lib64/python2.6/site-packages/sqlalchemy/engine/base.py", line 1972, in _run_visitor
      **kwargs).traverse_single(element)
    File "/usr/lib64/python2.6/site-packages/sqlalchemy/sql/visitors.py", line 106, in traverse_single
      return meth(obj, **kw)
    File "/usr/lib64/python2.6/site-packages/sqlalchemy/engine/ddl.py", line 86, in visit_table
      self.connection.execute(schema.CreateTable(table))
    File "/usr/lib64/python2.6/site-packages/sqlalchemy/engine/base.py", line 1449, in execute
      params)
    File "/usr/lib64/python2.6/site-packages/sqlalchemy/engine/base.py", line 1536, in _execute_ddl
      compiled = ddl.compile(dialect=dialect)
    File "/usr/lib64/python2.6/site-packages/sqlalchemy/sql/expression.py", line 1778, in compile
      return self._compiler(dialect, bind=bind, **kw)
    File "/usr/lib64/python2.6/site-packages/sqlalchemy/schema.py", line 2908, in _compiler
      return dialect.ddl_compiler(dialect, self, **kw)
    File "/usr/lib64/python2.6/site-packages/sqlalchemy/engine/base.py", line 705, in __init__
      self.string = self.process(self.statement)
    File "/usr/lib64/python2.6/site-packages/sqlalchemy/engine/base.py", line 724, in process
      return obj._compiler_dispatch(self, **kwargs)
    File "/usr/lib64/python2.6/site-packages/sqlalchemy/sql/visitors.py", line 72, in _compiler_dispatch
      return getter(visitor)(self, **kw)
    File "/usr/lib64/python2.6/site-packages/sqlalchemy/sql/compiler.py", line 1612, in visit_create_table
      const = self.create_table_constraints(table)
    File "/usr/lib64/python2.6/site-packages/sqlalchemy/dialects/mysql/base.py", line 1410, in create_table_constraints
      constraint_string = super(MySQLDDLCompiler, self).create_table_constraints(table)
    File "/usr/lib64/python2.6/site-packages/sqlalchemy/sql/compiler.py", line 1632, in create_table_constraints
      for constraint in constraints
    File "/usr/lib64/python2.6/site-packages/sqlalchemy/sql/compiler.py", line 1630, in <genexpr>
      return ", \n\t".join(p for p in
    File "/usr/lib64/python2.6/site-packages/sqlalchemy/sql/compiler.py", line 1638, in <genexpr>
      not getattr(constraint, 'use_alter', False)
    File "/usr/lib64/python2.6/site-packages/sqlalchemy/engine/base.py", line 724, in process
      return obj._compiler_dispatch(self, **kwargs)
    File "/usr/lib64/python2.6/site-packages/sqlalchemy/sql/visitors.py", line 72, in _compiler_dispatch
      return getter(visitor)(self, **kw)
    File "/usr/lib64/python2.6/site-packages/sqlalchemy/sql/compiler.py", line 1777, in visit_foreign_key_constraint
      remote_table = list(constraint._elements.values())[0].column.table
    File "/usr/lib64/python2.6/site-packages/sqlalchemy/util/langhelpers.py", line 494, in __get__
      obj.__dict__[self.__name__] = result = self.fget(obj)
    File "/usr/lib64/python2.6/site-packages/sqlalchemy/schema.py", line 1392, in column
      tname)
  sqlalchemy.exc.NoReferencedTableError: Foreign key associated with column 'endpoint_v3.service_id' could not find table 'service' with which to generate a foreign key to target column 'id'

  
  The second (and subsequent) runs return:
  # keystone-manage db_sync
  Traceback (most recent call last):
    File "/usr/bin/keystone-manage", line 5, in <module>
      pkg_resources.run_script('keystone==2013.1', 'keystone-manage')
    File "/usr/lib/python2.6/site-packages/pkg_resources.py", line 461, in run_script
      self.require(requires)[0].run_script(script_name, ns)
    File "/usr/lib/python2.6/site-packages/pkg_resources.py", line 1194, in run_script
      execfile(script_filename, namespace, namespace)
    File "/usr/lib/python2.6/site-packages/keystone-2013.1-py2.6.egg/EGG-INFO/scripts/keystone-manage", line 28, in <module>
      cli.main(argv=sys.argv, config_files=config_files)
    File "/usr/lib/python2.6/site-packages/keystone-2013.1-py2.6.egg/keystone/cli.py", line 144, in main
      CONF.command.cmd_class.main()
    File "/usr/lib/python2.6/site-packages/keystone-2013.1-py2.6.egg/keystone/cli.py", line 49, in main
      driver.db_sync()
    File "/usr/lib/python2.6/site-packages/keystone-2013.1-py2.6.egg/keystone/identity/backends/sql.py", line 167, in db_sync
      migration.db_sync()
    File "/usr/lib/python2.6/site-packages/keystone-2013.1-py2.6.egg/keystone/common/sql/migration.py", line 52, in db_sync
      return versioning_api.upgrade(CONF.sql.connection, repo_path, version)
    File "/usr/lib/python2.6/site-packages/migrate/versioning/api.py", line 186, in upgrade
      return _migrate(url, repository, version, upgrade=True, err=err, **opts)
    File "<string>", line 2, in _migrate
    File "/usr/lib/python2.6/site-packages/migrate/versioning/util/__init__.py", line 159, in with_engine
      return f(*a, **kw)
    File "/usr/lib/python2.6/site-packages/migrate/versioning/api.py", line 366, in _migrate
      schema.runchange(ver, change, changeset.step)
    File "/usr/lib/python2.6/site-packages/migrate/versioning/schema.py", line 91, in runchange
      change.run(self.engine, step)
    File "/usr/lib/python2.6/site-packages/migrate/versioning/script/py.py", line 145, in run
      script_func(engine)
    File "/usr/lib/python2.6/site-packages/keystone-2013.1-py2.6.egg/keystone/common/sql/migrate_repo/versions/010_endpoints_v3.py", line 25, in upgrade
      legacy_table = sql.Table('endpoint', meta, autoload=True)
    File "/usr/lib64/python2.6/site-packages/sqlalchemy/schema.py", line 318, in __new__
      table._init(name, metadata, *args, **kw)
    File "/usr/lib64/python2.6/site-packages/sqlalchemy/schema.py", line 381, in _init
      self._autoload(metadata, autoload_with, include_columns)
    File "/usr/lib64/python2.6/site-packages/sqlalchemy/schema.py", line 408, in _autoload
      self, include_columns, exclude_columns
    File "/usr/lib64/python2.6/site-packages/sqlalchemy/engine/base.py", line 2425, in run_callable
      return conn.run_callable(callable_, *args, **kwargs)
    File "/usr/lib64/python2.6/site-packages/sqlalchemy/engine/base.py", line 1968, in run_callable
      return callable_(self, *args, **kwargs)
    File "/usr/lib64/python2.6/site-packages/sqlalchemy/engine/default.py", line 260, in reflecttable
      return insp.reflecttable(table, include_columns, exclude_columns)
    File "/usr/lib64/python2.6/site-packages/sqlalchemy/engine/reflection.py", line 350, in reflecttable
      tbl_opts = self.get_table_options(table_name, schema, **table.kwargs)
    File "/usr/lib64/python2.6/site-packages/sqlalchemy/engine/reflection.py", line 178, in get_table_options
      **kw)
    File "<string>", line 1, in <lambda>
    File "/usr/lib64/python2.6/site-packages/sqlalchemy/engine/reflection.py", line 47, in cache
      ret = fn(self, con, *args, **kw)
    File "/usr/lib64/python2.6/site-packages/sqlalchemy/dialects/mysql/base.py", line 2064, in get_table_options
      parsed_state = self._parsed_state_or_create(connection, table_name, schema, **kw)
    File "/usr/lib64/python2.6/site-packages/sqlalchemy/dialects/mysql/base.py", line 2163, in _parsed_state_or_create
      info_cache=kw.get('info_cache', None)
    File "<string>", line 1, in <lambda>
    File "/usr/lib64/python2.6/site-packages/sqlalchemy/engine/reflection.py", line 47, in cache
      ret = fn(self, con, *args, **kw)
    File "/usr/lib64/python2.6/site-packages/sqlalchemy/dialects/mysql/base.py", line 2188, in _setup_parser
      full_name=full_name)
    File "/usr/lib64/python2.6/site-packages/sqlalchemy/dialects/mysql/base.py", line 2275, in _show_create_table
      raise exc.NoSuchTableError(full_name)
  sqlalchemy.exc.NoSuchTableError: `endpoint`

  
  The DB ends up looking like:

  +------------------------+
  | Tables_in_keystone     |
  +------------------------+
  | credential             |
  | domain                 |
  | ec2_credential         |
  | endpoint_v2            |
  | metadata               |
  | migrate_version        |
  | policy                 |
  | role                   |
  | service                |
  | tenant                 |
  | token                  |
  | user                   |
  | user_domain_metadata   |
  | user_tenant_membership |
  +------------------------+

To manage notifications about this bug go to:
https://bugs.launchpad.net/keystone/+bug/1098174/+subscriptions