yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1758786] [NEW] neutron db migration failed in subproject neutron-fwaas

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Zhenmei <zma@xxxxxxxxxx>
Date: Mon, 26 Mar 2018 02:09:36 -0000
Reply-to: Bug 1758786 <1758786@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
Public bug reported:

We use command “neutron-db-manage --config-file
/etc/neutron/neutron.conf --config-file
/etc/neutron/plugins/vmware/dvs.ini upgrade  head” to do db migration
from oscar to queens. In this process, each sub-component like neutron-
fwaas, neutron-lbaas do their own db migration with expand branch first
and then contract branch. However in neutron-fwaas db migration, the
script "876782258a43_create_default_firewall_groups_table" under the
expand branch depend on the script
"fd38cd995cc0_shared_attribute_for_firewall_resources" under the
contract branch. It caused the following error:

\"/usr/lib/python2.7/dist-packages/neutron_fwaas/db/migration/alembic_migrations/versions/queens/expand/876782258a43_create_default_firewall_groups_table.py\", line 65, in check_sanity
 
    resources.FIREWALL_GROUP.name == const.DEFAULT_FWG).first()
 
  File \"/usr/lib/python2.7/dist-packages/sqlalchemy/orm/query.py\", line 2755, in first
 
    ret = list(self[0:1])
 
  File \"/usr/lib/python2.7/dist-packages/sqlalchemy/orm/query.py\", line 2547, in __getitem__
 
    return list(res)
 
  File \"/usr/lib/python2.7/dist-packages/sqlalchemy/orm/query.py\", line 2855, in __iter__
 
    return self._execute_and_instances(context)
 
  File \"/usr/lib/python2.7/dist-packages/sqlalchemy/orm/query.py\", line 2878, in _execute_and_instances
 
    result = conn.execute(querycontext.statement, self._params)
 
  File \"/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py\", line 945, in execute
 
    return meth(self, multiparams, params)
 
  File \"/usr/lib/python2.7/dist-packages/sqlalchemy/sql/elements.py\", line 263, in _execute_on_connection
 
    return connection._execute_clauseelement(self, multiparams, params)
 
  File \"/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py\", line 1053, in _execute_clauseelement
 
    compiled_sql, distilled_params
 
  File \"/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py\", line 1189, in _execute_context
 
    context)
 
  File \"/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py\", line 1398, in _handle_dbapi_exception
 
    util.raise_from_cause(newraise, exc_info)
 
  File \"/usr/lib/python2.7/dist-packages/sqlalchemy/util/compat.py\", line 203, in raise_from_cause
 
    reraise(type(exception), exception, tb=exc_tb, cause=cause)
 
  File \"/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py\", line 1182, in _execute_context
 
    context)
 
  File \"/usr/lib/python2.7/dist-packages/sqlalchemy/engine/default.py\", line 470, in do_execute
 
    cursor.execute(statement, parameters)
 
  File \"/usr/lib/python2.7/dist-packages/pymysql/cursors.py\", line 165, in execute
 
    result = self._query(query)
 
  File \"/usr/lib/python2.7/dist-packages/pymysql/cursors.py\", line 321, in _query
 
    conn.query(q)
 
  File \"/usr/lib/python2.7/dist-packages/pymysql/connections.py\", line 860, in query
 
    self._affected_rows = self._read_query_result(unbuffered=unbuffered)
 
  File \"/usr/lib/python2.7/dist-packages/pymysql/connections.py\", line 1061, in _read_query_result
 
    result.read()
 
  File \"/usr/lib/python2.7/dist-packages/pymysql/connections.py\", line 1349, in read
 
    first_packet = self.connection._read_packet()
 
  File \"/usr/lib/python2.7/dist-packages/pymysql/connections.py\", line 1018, in _read_packet
 
    packet.check_error()
 
  File \"/usr/lib/python2.7/dist-packages/pymysql/connections.py\", line 384, in check_error
 
    err.raise_mysql_exception(self._data)
 
  File \"/usr/lib/python2.7/dist-packages/pymysql/err.py\", line 107, in raise_mysql_exception
 
    raise errorclass(errno, errval)
 
oslo_db.exception.DBError: (pymysql.err.InternalError) (1054, u\"Unknown column 'firewall_groups_v2.shared' in 'field list'\") [SQL: u'SELECT firewall_groups_v2.project_id AS firewall_groups_v2_project_id, firewall_groups_v2.id AS firewall_groups_v2_id, firewall_groups_v2.name AS firewall_groups_v2_name, firewall_groups_v2.description AS firewall_groups_v2_description, firewall_groups_v2.ingress_firewall_policy_id AS firewall_groups_v2_ingress_firewall_policy_id, firewall_groups_v2.egress_firewall_policy_id AS firewall_groups_v2_egress_firewall_policy_id, firewall_groups_v2.admin_state_up AS firewall_groups_v2_admin_state_up, firewall_groups_v2.status AS firewall_groups_v2_status, firewall_groups_v2.shared AS firewall_groups_v2_shared \
 
FROM firewall_groups_v2 \
 
WHERE firewall_groups_v2.name = %(name_1)s \
 
LIMIT %(param_1)s'

** Affects: neutron
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1758786

Title:
  neutron db migration failed in subproject neutron-fwaas

Status in neutron:
  New

Bug description:
  We use command “neutron-db-manage --config-file
  /etc/neutron/neutron.conf --config-file
  /etc/neutron/plugins/vmware/dvs.ini upgrade  head” to do db migration
  from oscar to queens. In this process, each sub-component like
  neutron-fwaas, neutron-lbaas do their own db migration with expand
  branch first and then contract branch. However in neutron-fwaas db
  migration, the script
  "876782258a43_create_default_firewall_groups_table" under the expand
  branch depend on the script
  "fd38cd995cc0_shared_attribute_for_firewall_resources" under the
  contract branch. It caused the following error:

  \"/usr/lib/python2.7/dist-packages/neutron_fwaas/db/migration/alembic_migrations/versions/queens/expand/876782258a43_create_default_firewall_groups_table.py\", line 65, in check_sanity
   
      resources.FIREWALL_GROUP.name == const.DEFAULT_FWG).first()
   
    File \"/usr/lib/python2.7/dist-packages/sqlalchemy/orm/query.py\", line 2755, in first
   
      ret = list(self[0:1])
   
    File \"/usr/lib/python2.7/dist-packages/sqlalchemy/orm/query.py\", line 2547, in __getitem__
   
      return list(res)
   
    File \"/usr/lib/python2.7/dist-packages/sqlalchemy/orm/query.py\", line 2855, in __iter__
   
      return self._execute_and_instances(context)
   
    File \"/usr/lib/python2.7/dist-packages/sqlalchemy/orm/query.py\", line 2878, in _execute_and_instances
   
      result = conn.execute(querycontext.statement, self._params)
   
    File \"/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py\", line 945, in execute
   
      return meth(self, multiparams, params)
   
    File \"/usr/lib/python2.7/dist-packages/sqlalchemy/sql/elements.py\", line 263, in _execute_on_connection
   
      return connection._execute_clauseelement(self, multiparams, params)
   
    File \"/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py\", line 1053, in _execute_clauseelement
   
      compiled_sql, distilled_params
   
    File \"/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py\", line 1189, in _execute_context
   
      context)
   
    File \"/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py\", line 1398, in _handle_dbapi_exception
   
      util.raise_from_cause(newraise, exc_info)
   
    File \"/usr/lib/python2.7/dist-packages/sqlalchemy/util/compat.py\", line 203, in raise_from_cause
   
      reraise(type(exception), exception, tb=exc_tb, cause=cause)
   
    File \"/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py\", line 1182, in _execute_context
   
      context)
   
    File \"/usr/lib/python2.7/dist-packages/sqlalchemy/engine/default.py\", line 470, in do_execute
   
      cursor.execute(statement, parameters)
   
    File \"/usr/lib/python2.7/dist-packages/pymysql/cursors.py\", line 165, in execute
   
      result = self._query(query)
   
    File \"/usr/lib/python2.7/dist-packages/pymysql/cursors.py\", line 321, in _query
   
      conn.query(q)
   
    File \"/usr/lib/python2.7/dist-packages/pymysql/connections.py\", line 860, in query
   
      self._affected_rows = self._read_query_result(unbuffered=unbuffered)
   
    File \"/usr/lib/python2.7/dist-packages/pymysql/connections.py\", line 1061, in _read_query_result
   
      result.read()
   
    File \"/usr/lib/python2.7/dist-packages/pymysql/connections.py\", line 1349, in read
   
      first_packet = self.connection._read_packet()
   
    File \"/usr/lib/python2.7/dist-packages/pymysql/connections.py\", line 1018, in _read_packet
   
      packet.check_error()
   
    File \"/usr/lib/python2.7/dist-packages/pymysql/connections.py\", line 384, in check_error
   
      err.raise_mysql_exception(self._data)
   
    File \"/usr/lib/python2.7/dist-packages/pymysql/err.py\", line 107, in raise_mysql_exception
   
      raise errorclass(errno, errval)
   
  oslo_db.exception.DBError: (pymysql.err.InternalError) (1054, u\"Unknown column 'firewall_groups_v2.shared' in 'field list'\") [SQL: u'SELECT firewall_groups_v2.project_id AS firewall_groups_v2_project_id, firewall_groups_v2.id AS firewall_groups_v2_id, firewall_groups_v2.name AS firewall_groups_v2_name, firewall_groups_v2.description AS firewall_groups_v2_description, firewall_groups_v2.ingress_firewall_policy_id AS firewall_groups_v2_ingress_firewall_policy_id, firewall_groups_v2.egress_firewall_policy_id AS firewall_groups_v2_egress_firewall_policy_id, firewall_groups_v2.admin_state_up AS firewall_groups_v2_admin_state_up, firewall_groups_v2.status AS firewall_groups_v2_status, firewall_groups_v2.shared AS firewall_groups_v2_shared \
   
  FROM firewall_groups_v2 \
   
  WHERE firewall_groups_v2.name = %(name_1)s \
   
  LIMIT %(param_1)s'

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1758786/+subscriptions
Follow ups

[Bug 1758786] Re: neutron db migration failed in subproject neutron-fwaas
From: OpenStack Infra, 2018-03-30