yahoo-eng-team team mailing list archive

[Matt Riedemann <mriedem.os@xxxxxxxxx>]

Looks like this was the regression:

https://review.openstack.org/#/c/541435/

Because before that the placement_context_manager was configured in the
sqlalchemy DB API code. Now it's only configured in a few select places,
one of which is not the online_data_migrations code.

This is also noticeable in devstack:

http://logs.openstack.org/08/599208/2/check/tempest-
full/608d60a/controller/logs/devstacklog.txt.gz#_2018-09-02_15_04_31_949

But because we try/except Exception in online_data_migrations we ignored
the failure.

** Also affects: nova/rocky
   Importance: Undecided
       Status: New

** Changed in: nova
       Status: New => Triaged

** Changed in: nova
   Importance: Undecided => Critical

** Changed in: nova/rocky
       Status: New => Triaged

** Changed in: nova/rocky
   Importance: Undecided => Critical

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1790701

Title:
  online_data_migrations fail in rocky+

Status in OpenStack Compute (nova):
  Triaged
Status in OpenStack Compute (nova) rocky series:
  Triaged

Bug description:
  # nova-manage --debug db online_data_migrations 
  Running batches of 50 until complete
  2018-09-04 13:39:12.628 15100 ERROR oslo_db.sqlalchemy.exc_filters [req-73eac47f-96f5-4393-8104-80d64e4f281f - - - - -] DBAPIError exception wrapped from (psycopg2.ProgrammingError) relation "projects" does not exist
  LINE 2: FROM projects 
               ^
   [SQL: 'SELECT projects.id \nFROM projects \nWHERE projects.external_id = %(external_id_1)s'] [parameters: {'external_id_1': '00000000-0000-0000-0000-000000000000'}] (Background on this error at: http://sqlalche.me/e/f405): psycopg2.ProgrammingError: relation "projects" does not exist
  LINE 2: FROM projects 
               ^
  2018-09-04 13:39:12.628 15100 ERROR oslo_db.sqlalchemy.exc_filters Traceback (most recent call last):
  2018-09-04 13:39:12.628 15100 ERROR oslo_db.sqlalchemy.exc_filters   File "/usr/lib64/python3.5/site-packages/sqlalchemy/engine/base.py", line 1193, in _execute_context
  2018-09-04 13:39:12.628 15100 ERROR oslo_db.sqlalchemy.exc_filters     context)
  2018-09-04 13:39:12.628 15100 ERROR oslo_db.sqlalchemy.exc_filters   File "/usr/lib64/python3.5/site-packages/sqlalchemy/engine/default.py", line 507, in do_execute
  2018-09-04 13:39:12.628 15100 ERROR oslo_db.sqlalchemy.exc_filters     cursor.execute(statement, parameters)
  2018-09-04 13:39:12.628 15100 ERROR oslo_db.sqlalchemy.exc_filters psycopg2.ProgrammingError: relation "projects" does not exist
  2018-09-04 13:39:12.628 15100 ERROR oslo_db.sqlalchemy.exc_filters LINE 2: FROM projects 
  2018-09-04 13:39:12.628 15100 ERROR oslo_db.sqlalchemy.exc_filters              ^
  2018-09-04 13:39:12.628 15100 ERROR oslo_db.sqlalchemy.exc_filters 
  2018-09-04 13:39:12.628 15100 ERROR oslo_db.sqlalchemy.exc_filters 
  Error attempting to run <function create_incomplete_consumers at 0x7fe62f7d19d8>
  +---------------------------------------------+--------------+-----------+
  |                  Migration                  | Total Needed | Completed |
  +---------------------------------------------+--------------+-----------+
  |         create_incomplete_consumers         |      0       |     0     |
  | delete_build_requests_with_no_instance_uuid |      0       |     0     |
  |      migrate_instances_add_request_spec     |      0       |     0     |
  |          migrate_keypairs_to_api_db         |      0       |     0     |
  |       migrate_quota_classes_to_api_db       |      0       |     0     |
  |        migrate_quota_limits_to_api_db       |      0       |     0     |
  |          migration_migrate_to_uuid          |      0       |     0     |
  |     populate_missing_availability_zones     |      0       |     0     |
  |          populate_queued_for_delete         |      0       |     0     |
  |                populate_uuids               |      0       |     0     |
  |     service_uuids_online_data_migration     |      0       |     0     |
  +---------------------------------------------+--------------+-----------+

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1790701/+subscriptions