yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1794364] Re: 'nova-manage db online_data_migrations' count fail

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Matt Riedemann <mriedem.os@xxxxxxxxx>
Date: Tue, 25 Sep 2018 20:44:45 -0000
Reply-to: Bug 1794364 <1794364@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
(3:05:42 PM) mriedem: that populate_uuids migration was added in queens, and looking at a grenade run from queens, we should see bdms getting migrated to have a uuid, and i'm not seeing any results in the table for a grenade run on queens
(3:05:42 PM) mriedem: http://logs.openstack.org/48/604448/1/check/neutron-grenade/19d8166/logs/grenade.sh.txt.gz
(3:05:46 PM) mriedem: so yeah, likely busted

** Changed in: nova
       Status: New => Confirmed

** Changed in: nova
   Importance: Undecided => High

** Also affects: nova/ocata
   Importance: Undecided
       Status: New

** Also affects: nova/pike
   Importance: Undecided
       Status: New

** Also affects: nova/rocky
   Importance: Undecided
       Status: New

** Also affects: nova/queens
   Importance: Undecided
       Status: New

** Changed in: nova/pike
       Status: New => Confirmed

** Changed in: nova/queens
       Status: New => Confirmed

** Tags added: upgrade

** Changed in: nova/pike
   Importance: Undecided => High

** Changed in: nova/queens
   Importance: Undecided => High

** Changed in: nova/ocata
       Status: New => Confirmed

** Changed in: nova/rocky
       Status: New => Confirmed

** Changed in: nova/ocata
   Importance: Undecided => High

** Changed in: nova/rocky
   Importance: Undecided => High

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1794364

Title:
  'nova-manage db online_data_migrations' count fail

Status in OpenStack Compute (nova):
  Confirmed
Status in OpenStack Compute (nova) ocata series:
  Confirmed
Status in OpenStack Compute (nova) pike series:
  Confirmed
Status in OpenStack Compute (nova) queens series:
  Confirmed
Status in OpenStack Compute (nova) rocky series:
  Confirmed

Bug description:
  
  'nova-manage db online_data_migrations' attempts to display summary counts of migrations "Needed" and "Completed" in a pretty table at the end, but fails to accumulate the totals between successive invocations of _run_migration(), and ends up reporting zeroes.

  # nova-manage db online_data_migrations 
  /usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/enginefacade.py:332: NotSupportedWarning: Configuration option(s) ['use_tpool'] not supported
    exception.NotSupportedWarning
  Running batches of 50 until complete
  /usr/lib/python2.7/site-packages/pymysql/cursors.py:166: Warning: (3090, u"Changing sql mode 'NO_AUTO_CREATE_USER' is deprecated. It will be removed in a future release.")
    result = self._query(query)
  2 rows matched query migrate_instances_add_request_spec, 0 migrated
  13 rows matched query migrate_quota_limits_to_api_db, 13 migrated
  37 rows matched query populate_uuids, 37 migrated
  50 rows matched query populate_uuids, 50 migrated
  50 rows matched query populate_uuids, 50 migrated
  50 rows matched query populate_uuids, 50 migrated
  50 rows matched query populate_uuids, 50 migrated
  50 rows matched query populate_uuids, 50 migrated
  50 rows matched query populate_uuids, 50 migrated
  50 rows matched query populate_uuids, 50 migrated
  50 rows matched query populate_uuids, 50 migrated
  50 rows matched query populate_uuids, 50 migrated
  50 rows matched query populate_uuids, 50 migrated
  50 rows matched query populate_uuids, 50 migrated
  50 rows matched query populate_uuids, 50 migrated
  50 rows matched query populate_uuids, 50 migrated
  50 rows matched query populate_uuids, 50 migrated
  50 rows matched query populate_uuids, 50 migrated
  50 rows matched query populate_uuids, 50 migrated
  50 rows matched query populate_uuids, 50 migrated
  50 rows matched query populate_uuids, 50 migrated
  50 rows matched query populate_uuids, 50 migrated
  50 rows matched query populate_uuids, 50 migrated
  50 rows matched query populate_uuids, 50 migrated
  50 rows matched query populate_uuids, 50 migrated
  50 rows matched query populate_uuids, 50 migrated
  50 rows matched query populate_uuids, 50 migrated
  50 rows matched query populate_uuids, 50 migrated
  50 rows matched query populate_uuids, 50 migrated
  50 rows matched query populate_uuids, 50 migrated
  50 rows matched query populate_uuids, 50 migrated
  50 rows matched query populate_uuids, 50 migrated
  50 rows matched query populate_uuids, 50 migrated
  50 rows matched query populate_uuids, 50 migrated
  50 rows matched query populate_uuids, 50 migrated
  50 rows matched query populate_uuids, 50 migrated
  50 rows matched query populate_uuids, 50 migrated
  50 rows matched query populate_uuids, 50 migrated
  50 rows matched query populate_uuids, 50 migrated
  50 rows matched query populate_uuids, 50 migrated
  50 rows matched query populate_uuids, 50 migrated
  50 rows matched query populate_uuids, 50 migrated
  50 rows matched query populate_uuids, 50 migrated
  50 rows matched query populate_uuids, 50 migrated
  50 rows matched query populate_uuids, 50 migrated
  50 rows matched query populate_uuids, 50 migrated
  50 rows matched query populate_uuids, 50 migrated
  50 rows matched query populate_uuids, 50 migrated
  50 rows matched query populate_uuids, 50 migrated
  50 rows matched query populate_uuids, 50 migrated
  21 rows matched query populate_uuids, 21 migrated
  +---------------------------------------------+--------------+-----------+
  |                  Migration                  | Total Needed | Completed |
  +---------------------------------------------+--------------+-----------+
  | delete_build_requests_with_no_instance_uuid |      0       |     0     |
  |    migrate_aggregate_reset_autoincrement    |      0       |     0     |
  |              migrate_aggregates             |      0       |     0     |
  |      migrate_instance_groups_to_api_db      |      0       |     0     |
  |      migrate_instances_add_request_spec     |      0       |     0     |
  |          migrate_keypairs_to_api_db         |      0       |     0     |
  |       migrate_quota_classes_to_api_db       |      0       |     0     |
  |        migrate_quota_limits_to_api_db       |      0       |     0     |
  |          migration_migrate_to_uuid          |      0       |     0     |
  |                populate_uuids               |      0       |     0     |
  |     service_uuids_online_data_migration     |      0       |     0     |
  +---------------------------------------------+--------------+-----------+

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1794364/+subscriptions
References

[Bug 1794364] [NEW] 'nova-manage db online_data_migrations' count fail
From: iain MacDonnell, 2018-09-25