yahoo-eng-team team mailing list archive

[Saverio Proto <1684861@xxxxxxxxxxxxxxxxxx>]

Public bug reported:

Upgrading the deployment from Mitaka to Newton.
This bug blocks people from upgrading to Ocata because the database migration for nova fails.

Running nova newton 14.0.5, the database is 334

root@moby:/backups# nova-manage db online_data_migrations
Option "verbose" from group "DEFAULT" is deprecated for removal.  Its value may be silently ignored in the future.
Running batches of 50 until complete
50 rows matched query migrate_flavors, 50 migrated
20 rows matched query migrate_flavors, 20 migrated
Error attempting to run <function migrate_instance_keypairs at 0x7f3373c58b90>
30 rows matched query migrate_instances_add_request_spec, 30 migrated
Error attempting to run <function migrate_instance_keypairs at 0x7f3373c58b90>
50 rows matched query migrate_instances_add_request_spec, 50 migrated
Error attempting to run <function migrate_instance_keypairs at 0x7f3373c58b90>
50 rows matched query migrate_instances_add_request_spec, 50 migrated
Error attempting to run <function migrate_instance_keypairs at 0x7f3373c58b90>
50 rows matched query migrate_instances_add_request_spec, 50 migrated
Error attempting to run <function migrate_instance_keypairs at 0x7f3373c58b90>
50 rows matched query migrate_instances_add_request_spec, 50 migrated
Error attempting to run <function migrate_instance_keypairs at 0x7f3373c58b90>
50 rows matched query migrate_instances_add_request_spec, 50 migrated
Error attempting to run <function migrate_instance_keypairs at 0x7f3373c58b90>
50 rows matched query migrate_instances_add_request_spec, 50 migrated
Error attempting to run <function migrate_instance_keypairs at 0x7f3373c58b90>
50 rows matched query migrate_instances_add_request_spec, 50 migrated
Error attempting to run <function migrate_instance_keypairs at 0x7f3373c58b90>
50 rows matched query migrate_instances_add_request_spec, 50 migrated
Error attempting to run <function migrate_instance_keypairs at 0x7f3373c58b90>
50 rows matched query migrate_instances_add_request_spec, 50 migrated
Error attempting to run <function migrate_instance_keypairs at 0x7f3373c58b90>
50 rows matched query migrate_instances_add_request_spec, 50 migrated
Error attempting to run <function migrate_instance_keypairs at 0x7f3373c58b90>
/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py:188: RuntimeWarning: You have iterated over the result of pkg_resources.parse_version. This is a legacy behavior which is inconsistent with the new version class introduced in setuptools 8.0. In most cases, conversion to a tuple is unnecessary. For comparison of versions, sort the Version instances directly. If you have another use case requiring the tuple, please file a bug with the setuptools project describing that need.
  stacklevel=1,
50 rows matched query migrate_instances_add_request_spec, 5 migrated
2017-04-20 14:48:36.586 396 ERROR nova.objects.keypair [req-565cbe62-030b-4b00-b9db-5ee82117889b - - - - -] Some instances are still missing keypair information. Unable to run keypair migration at this time.
5 rows matched query migrate_aggregates, 5 migrated
5 rows matched query migrate_instance_groups_to_api_db, 5 migrated
2 rows matched query delete_build_requests_with_no_instance_uuid, 2 migrated
Error attempting to run <function migrate_instance_keypairs at 0x7f3373c58b90>
50 rows matched query migrate_instances_add_request_spec, 0 migrated
2017-04-20 14:48:40.620 396 ERROR nova.objects.keypair [req-565cbe62-030b-4b00-b9db-5ee82117889b - - - - -] Some instances are still missing keypair information. Unable to run keypair migration at this time.
root@moby:/backups#

Adding a 'raise' after https://github.com/openstack/nova/blob/stable/newton/nova/cmd/manage.py#L896
you can see:

root@moby:/backups# nova-manage db online_data_migrations
Option "verbose" from group "DEFAULT" is deprecated for removal.  Its value may be silently ignored in the future.
Running batches of 50 until complete
Error attempting to run <function migrate_instance_keypairs at 0x7fb45132ab90>
error: 'NoneType' object has no attribute 'key_name'

** Affects: nova
     Importance: Undecided
         Status: New

** Summary changed:

- Database migrations from mitaka to newton fail due to missing keypairs
+ Database online_data_migrations in newton fail due to missing keypairs

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1684861

Title:
  Database online_data_migrations in newton fail due to missing keypairs

Status in OpenStack Compute (nova):
  New

Bug description:
  Upgrading the deployment from Mitaka to Newton.
  This bug blocks people from upgrading to Ocata because the database migration for nova fails.

  Running nova newton 14.0.5, the database is 334

  root@moby:/backups# nova-manage db online_data_migrations
  Option "verbose" from group "DEFAULT" is deprecated for removal.  Its value may be silently ignored in the future.
  Running batches of 50 until complete
  50 rows matched query migrate_flavors, 50 migrated
  20 rows matched query migrate_flavors, 20 migrated
  Error attempting to run <function migrate_instance_keypairs at 0x7f3373c58b90>
  30 rows matched query migrate_instances_add_request_spec, 30 migrated
  Error attempting to run <function migrate_instance_keypairs at 0x7f3373c58b90>
  50 rows matched query migrate_instances_add_request_spec, 50 migrated
  Error attempting to run <function migrate_instance_keypairs at 0x7f3373c58b90>
  50 rows matched query migrate_instances_add_request_spec, 50 migrated
  Error attempting to run <function migrate_instance_keypairs at 0x7f3373c58b90>
  50 rows matched query migrate_instances_add_request_spec, 50 migrated
  Error attempting to run <function migrate_instance_keypairs at 0x7f3373c58b90>
  50 rows matched query migrate_instances_add_request_spec, 50 migrated
  Error attempting to run <function migrate_instance_keypairs at 0x7f3373c58b90>
  50 rows matched query migrate_instances_add_request_spec, 50 migrated
  Error attempting to run <function migrate_instance_keypairs at 0x7f3373c58b90>
  50 rows matched query migrate_instances_add_request_spec, 50 migrated
  Error attempting to run <function migrate_instance_keypairs at 0x7f3373c58b90>
  50 rows matched query migrate_instances_add_request_spec, 50 migrated
  Error attempting to run <function migrate_instance_keypairs at 0x7f3373c58b90>
  50 rows matched query migrate_instances_add_request_spec, 50 migrated
  Error attempting to run <function migrate_instance_keypairs at 0x7f3373c58b90>
  50 rows matched query migrate_instances_add_request_spec, 50 migrated
  Error attempting to run <function migrate_instance_keypairs at 0x7f3373c58b90>
  50 rows matched query migrate_instances_add_request_spec, 50 migrated
  Error attempting to run <function migrate_instance_keypairs at 0x7f3373c58b90>
  /usr/lib/python2.7/dist-packages/pkg_resources/__init__.py:188: RuntimeWarning: You have iterated over the result of pkg_resources.parse_version. This is a legacy behavior which is inconsistent with the new version class introduced in setuptools 8.0. In most cases, conversion to a tuple is unnecessary. For comparison of versions, sort the Version instances directly. If you have another use case requiring the tuple, please file a bug with the setuptools project describing that need.
    stacklevel=1,
  50 rows matched query migrate_instances_add_request_spec, 5 migrated
  2017-04-20 14:48:36.586 396 ERROR nova.objects.keypair [req-565cbe62-030b-4b00-b9db-5ee82117889b - - - - -] Some instances are still missing keypair information. Unable to run keypair migration at this time.
  5 rows matched query migrate_aggregates, 5 migrated
  5 rows matched query migrate_instance_groups_to_api_db, 5 migrated
  2 rows matched query delete_build_requests_with_no_instance_uuid, 2 migrated
  Error attempting to run <function migrate_instance_keypairs at 0x7f3373c58b90>
  50 rows matched query migrate_instances_add_request_spec, 0 migrated
  2017-04-20 14:48:40.620 396 ERROR nova.objects.keypair [req-565cbe62-030b-4b00-b9db-5ee82117889b - - - - -] Some instances are still missing keypair information. Unable to run keypair migration at this time.
  root@moby:/backups#

  Adding a 'raise' after https://github.com/openstack/nova/blob/stable/newton/nova/cmd/manage.py#L896
  you can see:

  root@moby:/backups# nova-manage db online_data_migrations
  Option "verbose" from group "DEFAULT" is deprecated for removal.  Its value may be silently ignored in the future.
  Running batches of 50 until complete
  Error attempting to run <function migrate_instance_keypairs at 0x7fb45132ab90>
  error: 'NoneType' object has no attribute 'key_name'

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1684861/+subscriptions