yahoo-eng-team team mailing list archive

[Slawek Kaplonski <2100770@xxxxxxxxxxxxxxxxxx>]

Public bug reported:

There are some (corner) cases, like e.g. we have seen recently while
doing the RHOSP 17.1 to the RHOSO 18 adoption, then revert and then
adoption again where something fishy happened with the db migration and
when it was run second time, it failed due to the fact that table it
tried to create was already created.

I know that the issue wasn't really in Neutron in this case but it could
be somehow "workarounded" in Neutron by making our alembic migration
scripts to be idempotent.

We do something like that in some of them already, like e.g.
https://github.com/openstack/neutron/blob/1ddcb3e8901b87b914ef6c2819b1674190f067d7/neutron/db/migration/alembic_migrations/versions/2023.2/expand/c33da356b165_security_group_default_rules.py#L90
and it usually was done like that for the patches that we knew we will
be backporting downstream to some stable version. Making all the alembic
migration scripts to be idempotent should have few benefits:

- make the whole db upgrade process a bit more robust in some corner cases,
- make it easier for the people who are maintaing their own version downstream to backport some specific features to their version if that's needed by them.

This RFE proposes to do couple of things:
1. Update existing alembic migration scripts to be idempotent,
2. Propose functional test which would make sure that every existing and every new alembic migration script is idempotent,
3. Update alembic migration documentation in Neutron: https://docs.openstack.org/neutron/latest/contributor/alembic_migrations.html so that it will mention that scripts should be idempotent always.

** Affects: neutron
     Importance: Wishlist
     Assignee: Slawek Kaplonski (slaweq)
         Status: New


** Tags: db

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/2100770

Title:
  [RFE] Make alembic_migration scripts in neutron idempotent

Status in neutron:
  New

Bug description:
  There are some (corner) cases, like e.g. we have seen recently while
  doing the RHOSP 17.1 to the RHOSO 18 adoption, then revert and then
  adoption again where something fishy happened with the db migration
  and when it was run second time, it failed due to the fact that table
  it tried to create was already created.

  I know that the issue wasn't really in Neutron in this case but it
  could be somehow "workarounded" in Neutron by making our alembic
  migration scripts to be idempotent.

  We do something like that in some of them already, like e.g.
  https://github.com/openstack/neutron/blob/1ddcb3e8901b87b914ef6c2819b1674190f067d7/neutron/db/migration/alembic_migrations/versions/2023.2/expand/c33da356b165_security_group_default_rules.py#L90
  and it usually was done like that for the patches that we knew we will
  be backporting downstream to some stable version. Making all the
  alembic migration scripts to be idempotent should have few benefits:

  - make the whole db upgrade process a bit more robust in some corner cases,
  - make it easier for the people who are maintaing their own version downstream to backport some specific features to their version if that's needed by them.

  This RFE proposes to do couple of things:
  1. Update existing alembic migration scripts to be idempotent,
  2. Propose functional test which would make sure that every existing and every new alembic migration script is idempotent,
  3. Update alembic migration documentation in Neutron: https://docs.openstack.org/neutron/latest/contributor/alembic_migrations.html so that it will mention that scripts should be idempotent always.

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/2100770/+subscriptions