yahoo-eng-team team mailing list archive

[iain MacDonnell <1825584@xxxxxxxxxxxxxxxxxx>]

Public bug reported:

Stein nova-api running under uWSGI presents an AMQP issue. The first API
call that requires RPC creates an AMQP connection and successfully
completes. Normally regular heartbeats would be sent from this point on,
to maintain the connection. This is not happening. After a few minutes,
the AMQP server (rabbitmq, in my case) notices that there have been no
heartbeats, and drops the connection. A later nova API call that
requires RPC tries to use the old connection, and throws a "connection
reset by peer" exception and the API call fails. A mailing-list reponse
suggests that this is affecting mod_wsgi also:

http://lists.openstack.org/pipermail/openstack-
discuss/2019-April/005310.html

I've discovered that this problem seems to be caused by eventlet monkey-
patching, which was introduced in:

https://github.com/openstack/nova/commit/23ba1c690652832c655d57476630f02c268c87ae

It was later rearranged in:

https://github.com/openstack/nova/commit/3c5e2b0e9fac985294a949852bb8c83d4ed77e04

but this problem remains.

If I comment out the import of nova.monkey_patch in
nova/api/openstack/__init__.py the problem goes away.

Seems that eventlet monkey-patching and uWSGI are not getting along for
some reason...

** Affects: nova
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1825584

Title:
  eventlet monkey-patching breaks AMQP heartbeat on uWSGI

Status in OpenStack Compute (nova):
  New

Bug description:
  Stein nova-api running under uWSGI presents an AMQP issue. The first
  API call that requires RPC creates an AMQP connection and successfully
  completes. Normally regular heartbeats would be sent from this point
  on, to maintain the connection. This is not happening. After a few
  minutes, the AMQP server (rabbitmq, in my case) notices that there
  have been no heartbeats, and drops the connection. A later nova API
  call that requires RPC tries to use the old connection, and throws a
  "connection reset by peer" exception and the API call fails. A
  mailing-list reponse suggests that this is affecting mod_wsgi also:

  http://lists.openstack.org/pipermail/openstack-
  discuss/2019-April/005310.html

  I've discovered that this problem seems to be caused by eventlet
  monkey-patching, which was introduced in:

  https://github.com/openstack/nova/commit/23ba1c690652832c655d57476630f02c268c87ae

  It was later rearranged in:

  https://github.com/openstack/nova/commit/3c5e2b0e9fac985294a949852bb8c83d4ed77e04

  but this problem remains.

  If I comment out the import of nova.monkey_patch in
  nova/api/openstack/__init__.py the problem goes away.

  Seems that eventlet monkey-patching and uWSGI are not getting along
  for some reason...

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1825584/+subscriptions