yahoo-eng-team team mailing list archive

[Angus Lees <1350149@xxxxxxxxxxxxxxxxxx>]

Yep, bug 1337690 is an example of this showing up in neutron.

** Also affects: neutron
   Importance: Undecided
       Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1350149

Title:
  mysqldb + eventlet = potential deadlocks

Status in OpenStack Neutron (virtual network service):
  New
Status in Oslo - a Library of Common OpenStack Code:
  New

Bug description:
  mysqldb is a C library, and thus can't be monkeypatched by eventlet.

  With eventlet, this series of events may lead to a deadlock:
  1. greenlet A opens a mysql DB transaction, does some stuff
  2. greenlet A yields, perhaps by making another unrelated blocking operation (eg: sending a network request)
  3. greenlet B is scheduled and opens a mysql DB transaction that conflicts with A's
  4. greenlet B enters mysqldb and the C call blocks, waiting until A completes

  Note that this is a deadlock on the client side, since A will never be
  scheduled again - and not a typical example of waiting for a
  transaction lock.  The python process is now wedged until a mysqldb
  deadlock timer expires and B raises an exception.

  The mysql deadlock timer has a distinctive "Lock wait timeout
  exceeded; try restarting transaction" error message.

  There are bugs relating to specific instances of this, like bug 1298355 and bug 1313794.
  .. but fundamentally this issue is generic across any use of mysqldb + eventlet + any code that yields the greenlet while holding a lock inside the database server.  This bug is to track the generic issue across all such instances of the above pattern.  Note it is not specific to oslo.db.

  See also this blueprint for a proposed fix (switch to a different mysql driver):
  - https://blueprints.launchpad.net/oslo/+spec/switch-to-mysql-connector

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1350149/+subscriptions