yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1469029] Re: Migrations fail going from juno -> kilo

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Roman Podoliaka <rpodolyaka+bugs@xxxxxxxxxxxx>
Date: Tue, 21 Jul 2015 13:09:26 -0000
Reply-to: Bug 1469029 <1469029@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
The last Sam's comment means there is a problem with Keystone migration
scripts, or I should really say, they simply do not override the
horrible MySQL default for collation and new tables created are not in
utf8.

** Changed in: oslo.db
       Status: New => Invalid

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to Keystone.
https://bugs.launchpad.net/bugs/1469029

Title:
  Migrations fail going from juno -> kilo

Status in Keystone:
  In Progress
Status in oslo.db:
  Invalid

Bug description:
  Trying to upgrade from Juno -> Kilo

  keystone-manage db_version
  55

  keystone-manage db_sync
  2015-06-26 16:52:47.494 6169 CRITICAL keystone [-] ProgrammingError: (ProgrammingError) (1146, "Table 'keystone_k.identity_provider' doesn't exist") 'ALTER TABLE identity_provider Engine=InnoDB' ()
  2015-06-26 16:52:47.494 6169 TRACE keystone Traceback (most recent call last):
  2015-06-26 16:52:47.494 6169 TRACE keystone   File "/opt/kilo/bin/keystone-manage", line 10, in <module>
  2015-06-26 16:52:47.494 6169 TRACE keystone     execfile(__file__)
  2015-06-26 16:52:47.494 6169 TRACE keystone   File "/opt/keystone/bin/keystone-manage", line 44, in <module>
  2015-06-26 16:52:47.494 6169 TRACE keystone     cli.main(argv=sys.argv, config_files=config_files)
  2015-06-26 16:52:47.494 6169 TRACE keystone   File "/opt/keystone/keystone/cli.py", line 585, in main
  2015-06-26 16:52:47.494 6169 TRACE keystone     CONF.command.cmd_class.main()
  2015-06-26 16:52:47.494 6169 TRACE keystone   File "/opt/keystone/keystone/cli.py", line 76, in main
  2015-06-26 16:52:47.494 6169 TRACE keystone     migration_helpers.sync_database_to_version(extension, version)
  2015-06-26 16:52:47.494 6169 TRACE keystone   File "/opt/keystone/keystone/common/sql/migration_helpers.py", line 247, in sync_database_to_version
  2015-06-26 16:52:47.494 6169 TRACE keystone     _sync_extension_repo(default_extension, version)
  2015-06-26 16:52:47.494 6169 TRACE keystone   File "/opt/keystone/keystone/common/sql/migration_helpers.py", line 232, in _sync_extension_repo
  2015-06-26 16:52:47.494 6169 TRACE keystone     _fix_federation_tables(engine)
  2015-06-26 16:52:47.494 6169 TRACE keystone   File "/opt/keystone/keystone/common/sql/migration_helpers.py", line 167, in _fix_federation_tables
  2015-06-26 16:52:47.494 6169 TRACE keystone     engine.execute("ALTER TABLE identity_provider Engine=InnoDB")
  2015-06-26 16:52:47.494 6169 TRACE keystone   File "/opt/kilo/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1863, in execute
  2015-06-26 16:52:47.494 6169 TRACE keystone     return connection.execute(statement, *multiparams, **params)
  2015-06-26 16:52:47.494 6169 TRACE keystone   File "/opt/kilo/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 833, in execute
  2015-06-26 16:52:47.494 6169 TRACE keystone     return self._execute_text(object, multiparams, params)
  2015-06-26 16:52:47.494 6169 TRACE keystone   File "/opt/kilo/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 982, in _execute_text
  2015-06-26 16:52:47.494 6169 TRACE keystone     statement, parameters
  2015-06-26 16:52:47.494 6169 TRACE keystone   File "/opt/kilo/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1070, in _execute_context
  2015-06-26 16:52:47.494 6169 TRACE keystone     context)
  2015-06-26 16:52:47.494 6169 TRACE keystone   File "/opt/kilo/local/lib/python2.7/site-packages/oslo_db/sqlalchemy/compat/handle_error.py", line 261, in _handle_dbapi_exception
  2015-06-26 16:52:47.494 6169 TRACE keystone     e, statement, parameters, cursor, context)
  2015-06-26 16:52:47.494 6169 TRACE keystone   File "/opt/kilo/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1267, in _handle_dbapi_exception
  2015-06-26 16:52:47.494 6169 TRACE keystone     util.raise_from_cause(newraise, exc_info)
  2015-06-26 16:52:47.494 6169 TRACE keystone   File "/opt/kilo/local/lib/python2.7/site-packages/sqlalchemy/util/compat.py", line 199, in raise_from_cause
  2015-06-26 16:52:47.494 6169 TRACE keystone     reraise(type(exception), exception, tb=exc_tb)
  2015-06-26 16:52:47.494 6169 TRACE keystone   File "/opt/kilo/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1063, in _execute_context
  2015-06-26 16:52:47.494 6169 TRACE keystone     context)
  2015-06-26 16:52:47.494 6169 TRACE keystone   File "/opt/kilo/local/lib/python2.7/site-packages/sqlalchemy/engine/default.py", line 442, in do_execute
  2015-06-26 16:52:47.494 6169 TRACE keystone     cursor.execute(statement, parameters)
  2015-06-26 16:52:47.494 6169 TRACE keystone   File "/opt/kilo/local/lib/python2.7/site-packages/MySQLdb/cursors.py", line 205, in execute
  2015-06-26 16:52:47.494 6169 TRACE keystone     self.errorhandler(self, exc, value)
  2015-06-26 16:52:47.494 6169 TRACE keystone   File "/opt/kilo/local/lib/python2.7/site-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
  2015-06-26 16:52:47.494 6169 TRACE keystone     raise errorclass, errorvalue
  2015-06-26 16:52:47.494 6169 TRACE keystone ProgrammingError: (ProgrammingError) (1146, "Table 'keystone_k.identity_provider' doesn't exist") 'ALTER TABLE identity_provider Engine=InnoDB' ()
  2015-06-26 16:52:47.494 6169 TRACE keystone 

  
  If I run the command again I get:

  keystone-manage db_version
  67

  keystone-manage db_sync
  2015-06-26 16:53:25.489 6186 CRITICAL keystone [-] ValueError: Tables "endpoint_group,project_endpoint,project_endpoint_group" have non utf8 collation, please make sure all tables are CHARSET=utf8
  2015-06-26 16:53:25.489 6186 TRACE keystone Traceback (most recent call last):
  2015-06-26 16:53:25.489 6186 TRACE keystone   File "/opt/kilo/bin/keystone-manage", line 10, in <module>
  2015-06-26 16:53:25.489 6186 TRACE keystone     execfile(__file__)
  2015-06-26 16:53:25.489 6186 TRACE keystone   File "/opt/keystone/bin/keystone-manage", line 44, in <module>
  2015-06-26 16:53:25.489 6186 TRACE keystone     cli.main(argv=sys.argv, config_files=config_files)
  2015-06-26 16:53:25.489 6186 TRACE keystone   File "/opt/keystone/keystone/cli.py", line 585, in main
  2015-06-26 16:53:25.489 6186 TRACE keystone     CONF.command.cmd_class.main()
  2015-06-26 16:53:25.489 6186 TRACE keystone   File "/opt/keystone/keystone/cli.py", line 76, in main
  2015-06-26 16:53:25.489 6186 TRACE keystone     migration_helpers.sync_database_to_version(extension, version)
  2015-06-26 16:53:25.489 6186 TRACE keystone   File "/opt/keystone/keystone/common/sql/migration_helpers.py", line 242, in sync_database_to_version
  2015-06-26 16:53:25.489 6186 TRACE keystone     _sync_common_repo(version)
  2015-06-26 16:53:25.489 6186 TRACE keystone   File "/opt/keystone/keystone/common/sql/migration_helpers.py", line 148, in _sync_common_repo
  2015-06-26 16:53:25.489 6186 TRACE keystone     init_version=init_version)
  2015-06-26 16:53:25.489 6186 TRACE keystone   File "/opt/kilo/local/lib/python2.7/site-packages/oslo_db/sqlalchemy/migration.py", line 77, in db_sync
  2015-06-26 16:53:25.489 6186 TRACE keystone     _db_schema_sanity_check(engine)
  2015-06-26 16:53:25.489 6186 TRACE keystone   File "/opt/kilo/local/lib/python2.7/site-packages/oslo_db/sqlalchemy/migration.py", line 110, in _db_schema_sanity_check
  2015-06-26 16:53:25.489 6186 TRACE keystone     ) % ','.join(table_names))
  2015-06-26 16:53:25.489 6186 TRACE keystone ValueError: Tables "endpoint_group,project_endpoint,project_endpoint_group" have non utf8 collation, please make sure all tables are CHARSET=utf8
  2015-06-26 16:53:25.489 6186 TRACE keystone

To manage notifications about this bug go to:
https://bugs.launchpad.net/keystone/+bug/1469029/+subscriptions
References

[Bug 1469029] [NEW] Migrations fail going from juno -> kilo
From: Sam Morrison, 2015-06-26