yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1365436] [NEW] Database schema differs from migrations

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Oleksii Chuprykov <ochuprykov@xxxxxxxxxxxx>
Date: Thu, 04 Sep 2014 10:58:30 -0000
Reply-to: Bug 1365436 <1365436@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
Public bug reported:

As models and migrations don't have any logical relation in code, so
differences are possible. Furthermore in most of cases differences
exists.  Possible solution -- adding a new migration and altering a
model to remove differences.  Next, tests for differences between models
and migrations should be added.

Patch that removes diff https://review.openstack.org/#/c/113916/
Adding the sync tests https://review.openstack.org/#/c/110683/

Diff example from glance:
AssertionError: Models and migration scripts aren't in sync:
[ ( 'remove_table',
    Table('migrate_version', MetaData(bind=None), Column('repository_id', VARCHAR(length=250), table=<migrate_version>, primary_key=True, nullable=False), Column('repository_path', TEXT(), table=<migrate_version>), Column('version', INTEGER(), table=<migrate_version>), schema=None)),
  [ ( 'modify_default',
      None,
      'image_locations',
      'status',
      { 'existing_nullable': False,
        'existing_type': VARCHAR(length=30)},
      u"'active'::character varying",
      None)],
  [ ( 'modify_nullable',
      None,
      'image_locations',
      'updated_at',
      { 'existing_server_default': None,
        'existing_type': TIMESTAMP()},
      True,
      False)],
  [ ( 'modify_nullable',
      None,
      'image_members',
      'status',
      { 'existing_server_default': None,
        'existing_type': VARCHAR(length=20)},
      True,
      False)],
  [ ( 'modify_nullable',
      None,
      'image_members',
      'updated_at',
      { 'existing_server_default': None,
        'existing_type': TIMESTAMP()},
      True,
      False)],
  [ ( 'modify_nullable',
      None,
      'image_properties',
      'updated_at',
      { 'existing_server_default': None,
        'existing_type': TIMESTAMP()},
      True,
      False)],
  ( 'remove_index',
    Index('ix_image_properties_image_id_name', Column('image_id', VARCHAR(length=36), ForeignKey(u'images.id'), table=<image_properties>, nullable=False), Column('name', VARCHAR(length=255), table=<image_properties>, nullable=False))),
  ( 'add_constraint',
    UniqueConstraint(Column('image_id', String(length=36), ForeignKey('images.id'), table=<image_properties>, nullable=False), Column('name', String(length=255), table=<image_properties>, nullable=False))),
  [ ( 'modify_nullable',
      None,
      'image_tags',
      'updated_at',
      { 'existing_server_default': None,
        'existing_type': TIMESTAMP()},
      True,
      False)],
  [ ( 'modify_default',
      None,
      'images',
      'id',
      { 'existing_nullable': False,
        'existing_type': VARCHAR(length=36)},
      u"nextval('images_id_seq'::regclass)",
      None)],
  [ ( 'modify_nullable',
      None,
      'images',
      'protected',
      { 'existing_server_default': None,
        'existing_type': BOOLEAN()},
      True,
      False)],
  [ ( 'modify_nullable',
      None,
      'images',
      'updated_at',
      { 'existing_server_default': None,
        'existing_type': TIMESTAMP()},
      True,
      False)],
  [ ( 'modify_nullable',
      None,
      'tasks',
      'status',
      { 'existing_server_default': None,
        'existing_type': VARCHAR(length=30)},
      False,
      True)],
  [ ( 'modify_nullable',
      None,
      'tasks',
      'type',
      { 'existing_server_default': None,
        'existing_type': VARCHAR(length=30)},
      False,
      True)],
  [ ( 'modify_nullable',
      None,
      'tasks',
      'updated_at',
      { 'existing_server_default': None,
        'existing_type': TIMESTAMP()},
      True,
      False)]]

** Affects: glance
     Importance: Undecided
     Assignee: Oleksii Chuprykov (ochuprykov)
         Status: New


** Tags: db

** Changed in: glance
     Assignee: (unassigned) => Oleksii Chuprykov (ochuprykov)

** Tags added: db

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to Glance.
https://bugs.launchpad.net/bugs/1365436

Title:
  Database schema differs from migrations

Status in OpenStack Image Registry and Delivery Service (Glance):
  New

Bug description:
  As models and migrations don't have any logical relation in code, so
  differences are possible. Furthermore in most of cases differences
  exists.  Possible solution -- adding a new migration and altering a
  model to remove differences.  Next, tests for differences between
  models and migrations should be added.

  Patch that removes diff https://review.openstack.org/#/c/113916/
  Adding the sync tests https://review.openstack.org/#/c/110683/

  Diff example from glance:
  AssertionError: Models and migration scripts aren't in sync:
  [ ( 'remove_table',
      Table('migrate_version', MetaData(bind=None), Column('repository_id', VARCHAR(length=250), table=<migrate_version>, primary_key=True, nullable=False), Column('repository_path', TEXT(), table=<migrate_version>), Column('version', INTEGER(), table=<migrate_version>), schema=None)),
    [ ( 'modify_default',
        None,
        'image_locations',
        'status',
        { 'existing_nullable': False,
          'existing_type': VARCHAR(length=30)},
        u"'active'::character varying",
        None)],
    [ ( 'modify_nullable',
        None,
        'image_locations',
        'updated_at',
        { 'existing_server_default': None,
          'existing_type': TIMESTAMP()},
        True,
        False)],
    [ ( 'modify_nullable',
        None,
        'image_members',
        'status',
        { 'existing_server_default': None,
          'existing_type': VARCHAR(length=20)},
        True,
        False)],
    [ ( 'modify_nullable',
        None,
        'image_members',
        'updated_at',
        { 'existing_server_default': None,
          'existing_type': TIMESTAMP()},
        True,
        False)],
    [ ( 'modify_nullable',
        None,
        'image_properties',
        'updated_at',
        { 'existing_server_default': None,
          'existing_type': TIMESTAMP()},
        True,
        False)],
    ( 'remove_index',
      Index('ix_image_properties_image_id_name', Column('image_id', VARCHAR(length=36), ForeignKey(u'images.id'), table=<image_properties>, nullable=False), Column('name', VARCHAR(length=255), table=<image_properties>, nullable=False))),
    ( 'add_constraint',
      UniqueConstraint(Column('image_id', String(length=36), ForeignKey('images.id'), table=<image_properties>, nullable=False), Column('name', String(length=255), table=<image_properties>, nullable=False))),
    [ ( 'modify_nullable',
        None,
        'image_tags',
        'updated_at',
        { 'existing_server_default': None,
          'existing_type': TIMESTAMP()},
        True,
        False)],
    [ ( 'modify_default',
        None,
        'images',
        'id',
        { 'existing_nullable': False,
          'existing_type': VARCHAR(length=36)},
        u"nextval('images_id_seq'::regclass)",
        None)],
    [ ( 'modify_nullable',
        None,
        'images',
        'protected',
        { 'existing_server_default': None,
          'existing_type': BOOLEAN()},
        True,
        False)],
    [ ( 'modify_nullable',
        None,
        'images',
        'updated_at',
        { 'existing_server_default': None,
          'existing_type': TIMESTAMP()},
        True,
        False)],
    [ ( 'modify_nullable',
        None,
        'tasks',
        'status',
        { 'existing_server_default': None,
          'existing_type': VARCHAR(length=30)},
        False,
        True)],
    [ ( 'modify_nullable',
        None,
        'tasks',
        'type',
        { 'existing_server_default': None,
          'existing_type': VARCHAR(length=30)},
        False,
        True)],
    [ ( 'modify_nullable',
        None,
        'tasks',
        'updated_at',
        { 'existing_server_default': None,
          'existing_type': TIMESTAMP()},
        True,
        False)]]

To manage notifications about this bug go to:
https://bugs.launchpad.net/glance/+bug/1365436/+subscriptions
Follow ups

[Bug 1365436] Re: Database schema differs from migrations
From: Thierry Carrez, 2015-03-19
[Bug 1365436] [NEW] Database schema differs from migrations
From: Oleksii Chuprykov, 2014-09-04
References

[Bug 1365436] [NEW] Database schema differs from migrations
From: Oleksii Chuprykov, 2014-09-04