yahoo-eng-team team mailing list archive

[Sean Dague <sean@xxxxxxxxx>]

Public bug reported:

When we changed the default db driver from python-mysql (links to c lib,
not eventlet aware) to pymysql (pure python, does not block greenlets)
the Neutron job failure rate went way up.

http://logs.openstack.org/66/190166/1/gate/gate-tempest-dsvm-neutron-
full/d39af17/logs/screen-q-svc.txt.gz?level=TRACE#_2015-06-12_00_14_36_581
is an example

2015-06-12 00:14:36.581 4696 ERROR neutron.plugins.ml2.plugin
DBDeadlock: (pymysql.err.InternalError) (1213, u'Deadlock found when
trying to get lock; try restarting transaction') [SQL: u'DELETE FROM
ipallocationpools WHERE ipallocationpools.id = %s'] [parameters:
(u'09572654-65fd-4cf1-999b-3193b405f111',)]

I believe this is because the DB layer in neutron was depending on the
implicit synchronization that was created by python-mysql blocking it's
worker on the database. With that removed in pymysql, there is a bunch
of racing going on that needs explicit synchronization.

** Affects: neutron
     Importance: Critical
         Status: New

** Changed in: neutron
   Importance: Undecided => Critical

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1464612

Title:
  Neutron server races when using the pymysql driver

Status in OpenStack Neutron (virtual network service):
  New

Bug description:
  When we changed the default db driver from python-mysql (links to c
  lib, not eventlet aware) to pymysql (pure python, does not block
  greenlets) the Neutron job failure rate went way up.

  http://logs.openstack.org/66/190166/1/gate/gate-tempest-dsvm-neutron-
  full/d39af17/logs/screen-q-svc.txt.gz?level=TRACE#_2015-06-12_00_14_36_581
  is an example

  2015-06-12 00:14:36.581 4696 ERROR neutron.plugins.ml2.plugin
  DBDeadlock: (pymysql.err.InternalError) (1213, u'Deadlock found when
  trying to get lock; try restarting transaction') [SQL: u'DELETE FROM
  ipallocationpools WHERE ipallocationpools.id = %s'] [parameters:
  (u'09572654-65fd-4cf1-999b-3193b405f111',)]

  I believe this is because the DB layer in neutron was depending on the
  implicit synchronization that was created by python-mysql blocking
  it's worker on the database. With that removed in pymysql, there is a
  bunch of racing going on that needs explicit synchronization.

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1464612/+subscriptions