yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1716321] [NEW] StaleDataError: UPDATE statement on table 'standardattributes' expected to update 1 row(s); 0 were matched.

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Jens Offenbach <wolle5050@xxxxxx>
Date: Mon, 11 Sep 2017 06:01:28 -0000
Reply-to: Bug 1716321 <1716321@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
Public bug reported:

I have manually setup an HA/DVR deployment of OpenStack Pike on Ubuntu
16.04. The neutron.log gives me periodallcy the following exception:

2017-09-11 07:50:34.655 5830 ERROR oslo_db.api [req-ed871530-109c-475a-8b52-9422b156d24b - - - - -] DB exceeded retry limit.: StaleDataError: UPDATE statement on table 'standardattributes' expected to update 1 row(s); 0 were matched.
2017-09-11 07:50:34.655 5830 ERROR oslo_db.api Traceback (most recent call last):
2017-09-11 07:50:34.655 5830 ERROR oslo_db.api   File "/usr/lib/python2.7/dist-packages/oslo_db/api.py", line 138, in wrapper
2017-09-11 07:50:34.655 5830 ERROR oslo_db.api     return f(*args, **kwargs)
2017-09-11 07:50:34.655 5830 ERROR oslo_db.api   File "/usr/lib/python2.7/dist-packages/neutron/db/api.py", line 128, in wrapped
2017-09-11 07:50:34.655 5830 ERROR oslo_db.api     LOG.debug("Retry wrapper got retriable exception: %s", e)
2017-09-11 07:50:34.655 5830 ERROR oslo_db.api   File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2017-09-11 07:50:34.655 5830 ERROR oslo_db.api     self.force_reraise()
2017-09-11 07:50:34.655 5830 ERROR oslo_db.api   File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2017-09-11 07:50:34.655 5830 ERROR oslo_db.api     six.reraise(self.type_, self.value, self.tb)
2017-09-11 07:50:34.655 5830 ERROR oslo_db.api   File "/usr/lib/python2.7/dist-packages/neutron/db/api.py", line 124, in wrapped
2017-09-11 07:50:34.655 5830 ERROR oslo_db.api     return f(*dup_args, **dup_kwargs)
2017-09-11 07:50:34.655 5830 ERROR oslo_db.api   File "/usr/lib/python2.7/dist-packages/neutron/plugins/ml2/plugin.py", line 1699, in update_port_statuses
2017-09-11 07:50:34.655 5830 ERROR oslo_db.api     context, port_dbs_by_id[port_id], status, host)
2017-09-11 07:50:34.655 5830 ERROR oslo_db.api   File "/usr/lib/python2.7/dist-packages/neutron/plugins/ml2/plugin.py", line 1711, in _safe_update_individual_port_db_status
2017-09-11 07:50:34.655 5830 ERROR oslo_db.api     ectx.reraise = bool(db.get_port(context, port_id))
2017-09-11 07:50:34.655 5830 ERROR oslo_db.api   File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2017-09-11 07:50:34.655 5830 ERROR oslo_db.api     self.force_reraise()
2017-09-11 07:50:34.655 5830 ERROR oslo_db.api   File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2017-09-11 07:50:34.655 5830 ERROR oslo_db.api     six.reraise(self.type_, self.value, self.tb)
2017-09-11 07:50:34.655 5830 ERROR oslo_db.api   File "/usr/lib/python2.7/dist-packages/neutron/plugins/ml2/plugin.py", line 1707, in _safe_update_individual_port_db_status
2017-09-11 07:50:34.655 5830 ERROR oslo_db.api     context, port, status, host)
2017-09-11 07:50:34.655 5830 ERROR oslo_db.api   File "/usr/lib/python2.7/dist-packages/neutron/plugins/ml2/plugin.py", line 1750, in _update_individual_port_db_status
2017-09-11 07:50:34.655 5830 ERROR oslo_db.api     updated = True
2017-09-11 07:50:34.655 5830 ERROR oslo_db.api   File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__
2017-09-11 07:50:34.655 5830 ERROR oslo_db.api     self.gen.next()
2017-09-11 07:50:34.655 5830 ERROR oslo_db.api   File "/usr/lib/python2.7/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1029, in _transaction_scope
2017-09-11 07:50:34.655 5830 ERROR oslo_db.api     yield resource
2017-09-11 07:50:34.655 5830 ERROR oslo_db.api   File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__
2017-09-11 07:50:34.655 5830 ERROR oslo_db.api     self.gen.next()
2017-09-11 07:50:34.655 5830 ERROR oslo_db.api   File "/usr/lib/python2.7/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 641, in _session
2017-09-11 07:50:34.655 5830 ERROR oslo_db.api     self.session.rollback()
2017-09-11 07:50:34.655 5830 ERROR oslo_db.api   File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2017-09-11 07:50:34.655 5830 ERROR oslo_db.api     self.force_reraise()
2017-09-11 07:50:34.655 5830 ERROR oslo_db.api   File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2017-09-11 07:50:34.655 5830 ERROR oslo_db.api     six.reraise(self.type_, self.value, self.tb)
2017-09-11 07:50:34.655 5830 ERROR oslo_db.api   File "/usr/lib/python2.7/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 638, in _session
2017-09-11 07:50:34.655 5830 ERROR oslo_db.api     self._end_session_transaction(self.session)
2017-09-11 07:50:34.655 5830 ERROR oslo_db.api   File "/usr/lib/python2.7/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 666, in _end_session_transaction
2017-09-11 07:50:34.655 5830 ERROR oslo_db.api     session.commit()
2017-09-11 07:50:34.655 5830 ERROR oslo_db.api   File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 874, in commit
2017-09-11 07:50:34.655 5830 ERROR oslo_db.api     self.transaction.commit()
2017-09-11 07:50:34.655 5830 ERROR oslo_db.api   File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 461, in commit
2017-09-11 07:50:34.655 5830 ERROR oslo_db.api     self._prepare_impl()
2017-09-11 07:50:34.655 5830 ERROR oslo_db.api   File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 441, in _prepare_impl
2017-09-11 07:50:34.655 5830 ERROR oslo_db.api     self.session.flush()
2017-09-11 07:50:34.655 5830 ERROR oslo_db.api   File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 2139, in flush
2017-09-11 07:50:34.655 5830 ERROR oslo_db.api     self._flush(objects)
2017-09-11 07:50:34.655 5830 ERROR oslo_db.api   File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 2259, in _flush
2017-09-11 07:50:34.655 5830 ERROR oslo_db.api     transaction.rollback(_capture_exception=True)
2017-09-11 07:50:34.655 5830 ERROR oslo_db.api   File "/usr/lib/python2.7/dist-packages/sqlalchemy/util/langhelpers.py", line 66, in __exit__
2017-09-11 07:50:34.655 5830 ERROR oslo_db.api     compat.reraise(exc_type, exc_value, exc_tb)
2017-09-11 07:50:34.655 5830 ERROR oslo_db.api   File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 2223, in _flush
2017-09-11 07:50:34.655 5830 ERROR oslo_db.api     flush_context.execute()
2017-09-11 07:50:34.655 5830 ERROR oslo_db.api   File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/unitofwork.py", line 389, in execute
2017-09-11 07:50:34.655 5830 ERROR oslo_db.api     rec.execute(self)
2017-09-11 07:50:34.655 5830 ERROR oslo_db.api   File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/unitofwork.py", line 548, in execute
2017-09-11 07:50:34.655 5830 ERROR oslo_db.api     uow
2017-09-11 07:50:34.655 5830 ERROR oslo_db.api   File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/persistence.py", line 177, in save_obj
2017-09-11 07:50:34.655 5830 ERROR oslo_db.api     mapper, table, update)
2017-09-11 07:50:34.655 5830 ERROR oslo_db.api   File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/persistence.py", line 760, in _emit_update_statements
2017-09-11 07:50:34.655 5830 ERROR oslo_db.api     (table.description, len(records), rows))
2017-09-11 07:50:34.655 5830 ERROR oslo_db.api StaleDataError: UPDATE statement on table 'standardattributes' expected to update 1 row(s); 0 were matched.
2017-09-11 07:50:34.655 5830 ERROR oslo_db.api 
2017-09-11 07:50:34.657 5830 ERROR neutron.plugins.ml2.rpc [req-ed871530-109c-475a-8b52-9422b156d24b - - - - -] Failed to update device c34b99f0-fdf4-4cb4-b218-33d01de26af4 up: StaleDataError: UPDATE statement on table 'standardattributes' expected to update 1 row(s); 0 were matched.

I am using MariaDB:
$ mysql --version
mysql  Ver 15.1 Distrib 10.1.26-MariaDB, for debian-linux-gnu (x86_64) using readline 5.2

I can also offer you an abstract from the database showing that all
relevant information is available:

INSERT INTO `ports` (`project_id`, `id`, `name`, `network_id`,
`mac_address`, `admin_state_up`, `status`, `device_id`, `device_owner`,
`standard_attr_id`, `ip_allocation`) VALUES
('ed2d2efd86dd40e7a45491d8502318d3',
'c34b99f0-fdf4-4cb4-b218-33d01de26af4', '', '18e47141-f3f3-4071-a51d-
f93d80a2bacd', '02:05:69:e7:e7:ba', 1, 'ACTIVE', 'f7f8682e-ae8a-49f4
-a00f-5530cb8865bc', 'network:router_interface_distributed', 258,
'immediate');

INSERT INTO `standardattributes` (`id`, `resource_type`, `created_at`,
`updated_at`, `description`, `revision_number`) VALUES (258, 'ports',
'2017-09-07 08:29:28', '2017-09-11 05:39:22', '', 1150832);

Can someone please have a look on this issue.

** Affects: neutron
     Importance: Undecided
         Status: New


** Tags: l3-dvr-backlog l3-ha

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1716321

Title:
  StaleDataError: UPDATE statement on table 'standardattributes'
  expected to update 1 row(s); 0 were matched.

Status in neutron:
  New

Bug description:
  I have manually setup an HA/DVR deployment of OpenStack Pike on Ubuntu
  16.04. The neutron.log gives me periodallcy the following exception:

  2017-09-11 07:50:34.655 5830 ERROR oslo_db.api [req-ed871530-109c-475a-8b52-9422b156d24b - - - - -] DB exceeded retry limit.: StaleDataError: UPDATE statement on table 'standardattributes' expected to update 1 row(s); 0 were matched.
  2017-09-11 07:50:34.655 5830 ERROR oslo_db.api Traceback (most recent call last):
  2017-09-11 07:50:34.655 5830 ERROR oslo_db.api   File "/usr/lib/python2.7/dist-packages/oslo_db/api.py", line 138, in wrapper
  2017-09-11 07:50:34.655 5830 ERROR oslo_db.api     return f(*args, **kwargs)
  2017-09-11 07:50:34.655 5830 ERROR oslo_db.api   File "/usr/lib/python2.7/dist-packages/neutron/db/api.py", line 128, in wrapped
  2017-09-11 07:50:34.655 5830 ERROR oslo_db.api     LOG.debug("Retry wrapper got retriable exception: %s", e)
  2017-09-11 07:50:34.655 5830 ERROR oslo_db.api   File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
  2017-09-11 07:50:34.655 5830 ERROR oslo_db.api     self.force_reraise()
  2017-09-11 07:50:34.655 5830 ERROR oslo_db.api   File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
  2017-09-11 07:50:34.655 5830 ERROR oslo_db.api     six.reraise(self.type_, self.value, self.tb)
  2017-09-11 07:50:34.655 5830 ERROR oslo_db.api   File "/usr/lib/python2.7/dist-packages/neutron/db/api.py", line 124, in wrapped
  2017-09-11 07:50:34.655 5830 ERROR oslo_db.api     return f(*dup_args, **dup_kwargs)
  2017-09-11 07:50:34.655 5830 ERROR oslo_db.api   File "/usr/lib/python2.7/dist-packages/neutron/plugins/ml2/plugin.py", line 1699, in update_port_statuses
  2017-09-11 07:50:34.655 5830 ERROR oslo_db.api     context, port_dbs_by_id[port_id], status, host)
  2017-09-11 07:50:34.655 5830 ERROR oslo_db.api   File "/usr/lib/python2.7/dist-packages/neutron/plugins/ml2/plugin.py", line 1711, in _safe_update_individual_port_db_status
  2017-09-11 07:50:34.655 5830 ERROR oslo_db.api     ectx.reraise = bool(db.get_port(context, port_id))
  2017-09-11 07:50:34.655 5830 ERROR oslo_db.api   File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
  2017-09-11 07:50:34.655 5830 ERROR oslo_db.api     self.force_reraise()
  2017-09-11 07:50:34.655 5830 ERROR oslo_db.api   File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
  2017-09-11 07:50:34.655 5830 ERROR oslo_db.api     six.reraise(self.type_, self.value, self.tb)
  2017-09-11 07:50:34.655 5830 ERROR oslo_db.api   File "/usr/lib/python2.7/dist-packages/neutron/plugins/ml2/plugin.py", line 1707, in _safe_update_individual_port_db_status
  2017-09-11 07:50:34.655 5830 ERROR oslo_db.api     context, port, status, host)
  2017-09-11 07:50:34.655 5830 ERROR oslo_db.api   File "/usr/lib/python2.7/dist-packages/neutron/plugins/ml2/plugin.py", line 1750, in _update_individual_port_db_status
  2017-09-11 07:50:34.655 5830 ERROR oslo_db.api     updated = True
  2017-09-11 07:50:34.655 5830 ERROR oslo_db.api   File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__
  2017-09-11 07:50:34.655 5830 ERROR oslo_db.api     self.gen.next()
  2017-09-11 07:50:34.655 5830 ERROR oslo_db.api   File "/usr/lib/python2.7/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1029, in _transaction_scope
  2017-09-11 07:50:34.655 5830 ERROR oslo_db.api     yield resource
  2017-09-11 07:50:34.655 5830 ERROR oslo_db.api   File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__
  2017-09-11 07:50:34.655 5830 ERROR oslo_db.api     self.gen.next()
  2017-09-11 07:50:34.655 5830 ERROR oslo_db.api   File "/usr/lib/python2.7/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 641, in _session
  2017-09-11 07:50:34.655 5830 ERROR oslo_db.api     self.session.rollback()
  2017-09-11 07:50:34.655 5830 ERROR oslo_db.api   File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
  2017-09-11 07:50:34.655 5830 ERROR oslo_db.api     self.force_reraise()
  2017-09-11 07:50:34.655 5830 ERROR oslo_db.api   File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
  2017-09-11 07:50:34.655 5830 ERROR oslo_db.api     six.reraise(self.type_, self.value, self.tb)
  2017-09-11 07:50:34.655 5830 ERROR oslo_db.api   File "/usr/lib/python2.7/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 638, in _session
  2017-09-11 07:50:34.655 5830 ERROR oslo_db.api     self._end_session_transaction(self.session)
  2017-09-11 07:50:34.655 5830 ERROR oslo_db.api   File "/usr/lib/python2.7/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 666, in _end_session_transaction
  2017-09-11 07:50:34.655 5830 ERROR oslo_db.api     session.commit()
  2017-09-11 07:50:34.655 5830 ERROR oslo_db.api   File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 874, in commit
  2017-09-11 07:50:34.655 5830 ERROR oslo_db.api     self.transaction.commit()
  2017-09-11 07:50:34.655 5830 ERROR oslo_db.api   File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 461, in commit
  2017-09-11 07:50:34.655 5830 ERROR oslo_db.api     self._prepare_impl()
  2017-09-11 07:50:34.655 5830 ERROR oslo_db.api   File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 441, in _prepare_impl
  2017-09-11 07:50:34.655 5830 ERROR oslo_db.api     self.session.flush()
  2017-09-11 07:50:34.655 5830 ERROR oslo_db.api   File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 2139, in flush
  2017-09-11 07:50:34.655 5830 ERROR oslo_db.api     self._flush(objects)
  2017-09-11 07:50:34.655 5830 ERROR oslo_db.api   File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 2259, in _flush
  2017-09-11 07:50:34.655 5830 ERROR oslo_db.api     transaction.rollback(_capture_exception=True)
  2017-09-11 07:50:34.655 5830 ERROR oslo_db.api   File "/usr/lib/python2.7/dist-packages/sqlalchemy/util/langhelpers.py", line 66, in __exit__
  2017-09-11 07:50:34.655 5830 ERROR oslo_db.api     compat.reraise(exc_type, exc_value, exc_tb)
  2017-09-11 07:50:34.655 5830 ERROR oslo_db.api   File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 2223, in _flush
  2017-09-11 07:50:34.655 5830 ERROR oslo_db.api     flush_context.execute()
  2017-09-11 07:50:34.655 5830 ERROR oslo_db.api   File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/unitofwork.py", line 389, in execute
  2017-09-11 07:50:34.655 5830 ERROR oslo_db.api     rec.execute(self)
  2017-09-11 07:50:34.655 5830 ERROR oslo_db.api   File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/unitofwork.py", line 548, in execute
  2017-09-11 07:50:34.655 5830 ERROR oslo_db.api     uow
  2017-09-11 07:50:34.655 5830 ERROR oslo_db.api   File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/persistence.py", line 177, in save_obj
  2017-09-11 07:50:34.655 5830 ERROR oslo_db.api     mapper, table, update)
  2017-09-11 07:50:34.655 5830 ERROR oslo_db.api   File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/persistence.py", line 760, in _emit_update_statements
  2017-09-11 07:50:34.655 5830 ERROR oslo_db.api     (table.description, len(records), rows))
  2017-09-11 07:50:34.655 5830 ERROR oslo_db.api StaleDataError: UPDATE statement on table 'standardattributes' expected to update 1 row(s); 0 were matched.
  2017-09-11 07:50:34.655 5830 ERROR oslo_db.api 
  2017-09-11 07:50:34.657 5830 ERROR neutron.plugins.ml2.rpc [req-ed871530-109c-475a-8b52-9422b156d24b - - - - -] Failed to update device c34b99f0-fdf4-4cb4-b218-33d01de26af4 up: StaleDataError: UPDATE statement on table 'standardattributes' expected to update 1 row(s); 0 were matched.

  I am using MariaDB:
  $ mysql --version
  mysql  Ver 15.1 Distrib 10.1.26-MariaDB, for debian-linux-gnu (x86_64) using readline 5.2

  I can also offer you an abstract from the database showing that all
  relevant information is available:

  INSERT INTO `ports` (`project_id`, `id`, `name`, `network_id`,
  `mac_address`, `admin_state_up`, `status`, `device_id`,
  `device_owner`, `standard_attr_id`, `ip_allocation`) VALUES
  ('ed2d2efd86dd40e7a45491d8502318d3',
  'c34b99f0-fdf4-4cb4-b218-33d01de26af4', '', '18e47141-f3f3-4071-a51d-
  f93d80a2bacd', '02:05:69:e7:e7:ba', 1, 'ACTIVE', 'f7f8682e-ae8a-49f4
  -a00f-5530cb8865bc', 'network:router_interface_distributed', 258,
  'immediate');

  INSERT INTO `standardattributes` (`id`, `resource_type`, `created_at`,
  `updated_at`, `description`, `revision_number`) VALUES (258, 'ports',
  '2017-09-07 08:29:28', '2017-09-11 05:39:22', '', 1150832);

  Can someone please have a look on this issue.

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1716321/+subscriptions
Follow ups

[Bug 1716321] Re: StaleDataError: UPDATE statement on table 'standardattributes' expected to update 1 row(s); 0 were matched.
From: Christoph Fiehe, 2017-10-18