yahoo-eng-team team mailing list archive

[Matt Riedemann <mriedem.os@xxxxxxxxx>]

Public bug reported:

I noticed this while debugging this change:

https://review.openstack.org/#/c/524710/

When we enable guestfs.debug, we're receiving events from guestfs and
logging them.

However, guestfs.launch is run in an eventlet thread pool, and running
things within an eventlet thread pool for native apps is a no-no for
doing logging because it can cause a greenthread switch that makes us
lose the return from the guestfs.launch call, which eventually means we
hang when trying to create the instance.

So we need to just run guestfs.launch non-threaded if guestfs.debug is
enabled.

>From the logs, you can see the guestfs.launch thread switch here:

http://logs.openstack.org/50/524750/1/check/legacy-tempest-dsvm-neutron-
full-centos-7/a7f051e/logs/screen-n-cpu.txt.gz#_Dec_04_13_43_19_941869

And the logging callback here:

http://logs.openstack.org/50/524750/1/check/legacy-tempest-dsvm-neutron-
full-centos-7/a7f051e/logs/screen-n-cpu.txt.gz#_Dec_04_15_25_29_320753

** Affects: nova
     Importance: High
         Status: Confirmed


** Tags: eventlet guestfs

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1737214

Title:
  Enabling guestfs.debug can hang guestfs.launch

Status in OpenStack Compute (nova):
  Confirmed

Bug description:
  I noticed this while debugging this change:

  https://review.openstack.org/#/c/524710/

  When we enable guestfs.debug, we're receiving events from guestfs and
  logging them.

  However, guestfs.launch is run in an eventlet thread pool, and running
  things within an eventlet thread pool for native apps is a no-no for
  doing logging because it can cause a greenthread switch that makes us
  lose the return from the guestfs.launch call, which eventually means
  we hang when trying to create the instance.

  So we need to just run guestfs.launch non-threaded if guestfs.debug is
  enabled.

  From the logs, you can see the guestfs.launch thread switch here:

  http://logs.openstack.org/50/524750/1/check/legacy-tempest-dsvm-
  neutron-full-
  centos-7/a7f051e/logs/screen-n-cpu.txt.gz#_Dec_04_13_43_19_941869

  And the logging callback here:

  http://logs.openstack.org/50/524750/1/check/legacy-tempest-dsvm-
  neutron-full-
  centos-7/a7f051e/logs/screen-n-cpu.txt.gz#_Dec_04_15_25_29_320753

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1737214/+subscriptions