← Back to team overview

yahoo-eng-team team mailing list archive

[Bug 1805132] [NEW] bulk creation of security group rules fails StaleDataError

 

Public bug reported:

Concurrently creating (many) security group rules in a security group,
either posting concurrently directly to the api, or by using heat fails
(on queens) with:

Retry wrapper got retriable exception: UPDATE statement on table 'standardattributes' expected to update 1 row(s); 0 were matched. wrapped /usr/lib/python2.7/site-packages/neutron/db/api.py:126
2018-11-26 12:01:51.917 4605 ERROR oslo_db.api [req-459cb962-2cc8-403a-af68-411efb7eb39c 2fe7c01cdd724059bb17e4cf38bc44b5 d45bb2772ddd4eda8f9bec432238eda1 - default default] DB exceeded retry limit.: StaleDataError: UPDATE statement on table 'standardattributes' expected to update 1 row(s); 0 were matched.


StaleDataError: UPDATE statement on table 'standardattributes' expected to update 1 row(s); 0 were matched.

I've written a small script to reproduce this.

https://gist.github.com/jsm222/2b7db15eaebe69111793359c9a585823

Example usages:
python secruletest.py concurrent --auth-url http://127.0.0.1:5000/v3 --username demo --project-name demo --domain-id default --rules-amount 200
or
python secruletest.py heat --rules-amount 200 --file-out /tmp/heat.yaml

The heat template is just written not applied, the first example makes a
security group named NeutronSecurityGroupRuleTest (can be set with
--security-group-name ) and begins posting rules concurrently.

It is reproducible on rdo packstack queens, and also fails on production
environment (Ubuntu based queens)

** Affects: neutron
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1805132

Title:
  bulk creation of security group rules fails StaleDataError

Status in neutron:
  New

Bug description:
  Concurrently creating (many) security group rules in a security group,
  either posting concurrently directly to the api, or by using heat
  fails (on queens) with:

  Retry wrapper got retriable exception: UPDATE statement on table 'standardattributes' expected to update 1 row(s); 0 were matched. wrapped /usr/lib/python2.7/site-packages/neutron/db/api.py:126
  2018-11-26 12:01:51.917 4605 ERROR oslo_db.api [req-459cb962-2cc8-403a-af68-411efb7eb39c 2fe7c01cdd724059bb17e4cf38bc44b5 d45bb2772ddd4eda8f9bec432238eda1 - default default] DB exceeded retry limit.: StaleDataError: UPDATE statement on table 'standardattributes' expected to update 1 row(s); 0 were matched.

  
  StaleDataError: UPDATE statement on table 'standardattributes' expected to update 1 row(s); 0 were matched.

  I've written a small script to reproduce this.

  https://gist.github.com/jsm222/2b7db15eaebe69111793359c9a585823

  Example usages:
  python secruletest.py concurrent --auth-url http://127.0.0.1:5000/v3 --username demo --project-name demo --domain-id default --rules-amount 200
  or
  python secruletest.py heat --rules-amount 200 --file-out /tmp/heat.yaml

  The heat template is just written not applied, the first example makes
  a security group named NeutronSecurityGroupRuleTest (can be set with
  --security-group-name ) and begins posting rules concurrently.

  It is reproducible on rdo packstack queens, and also fails on
  production environment (Ubuntu based queens)

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1805132/+subscriptions


Follow ups