yahoo-eng-team team mailing list archive

[OpenStack Infra <1994913@xxxxxxxxxxxxxxxxxx>]

Reviewed:  https://review.opendev.org/c/openstack/nova/+/862769
Committed: https://opendev.org/openstack/nova/commit/ecb11043e9c2a798950d63b55cd08684b1d77c4a
Submitter: "Zuul (22348)"
Branch:    master

commit ecb11043e9c2a798950d63b55cd08684b1d77c4a
Author: melanie witt <melwittt@xxxxxxxxx>
Date:   Thu Oct 27 01:43:55 2022 +0000

    Add mock to avoid loading guestfs in unit test
    
    We recently discovered that when the perfect conditions are present
    where:
    
      * libguestfs-dev/el and guestfs python bindings are installed
    
    and
    
      * unit tests are not being run in a venv or guestfs python bindings
        are installed in the tox venv
    
    the test will end up loading the guestfs module and try to call the
    real guestfs and possibly libvirt and fail because of it.
    
    Our unit tests shouldn't be loading modules like guestfs, so this adds
    proper mocking to the test along with a poison fixture that will
    prevent future accidental imports of such modules.
    
    Closes-Bug: #1994913
    
    Change-Id: I676ee1fd33cf053681a07448759c28f0f2ad79d1


** Changed in: nova
       Status: In Progress => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1994913

Title:
  Unit tests can fail if python guestfs is installed in the test
  environment

Status in OpenStack Compute (nova):
  Fix Released

Bug description:
  Ran into this downstream [1] where a unit test failed because it
  actually tried to call libvirt, which of course failed:

  libguestfs: error: could not create appliance through libvirt.

  Try running qemu directly without libvirt using this environment variable:
  export LIBGUESTFS_BACKEND=direct

  Original error from libvirt: internal error: qemu unexpectedly closed the monitor: 2022-10-22T13:20:42.059709Z qemu-kvm: -blockdev {"node-name":"libvirt-2-format","read-only":false,"cache":{"direct":false,"no-flush":false},"driver":"qcow2","file":"libvirt-2-storage","backing":null}: Image is not in qcow2 format [code=1 int1=-1]
  {5} nova.tests.unit.virt.disk.test_api.APITestCase.test_can_resize_need_fs_type_specified [191.116248s] ... FAILED

  Captured traceback:
  ~~~~~~~~~~~~~~~~~~~
      Traceback (most recent call last):
        File "/home/zuul/src/code.engineering.redhat.com/nova/nova/tests/unit/virt/disk/test_api.py", line 47, in test_can_resize_need_fs_type_specified
          self.assertFalse(api.is_image_extendable(image))
        File "/home/zuul/src/code.engineering.redhat.com/nova/nova/virt/disk/api.py", line 190, in is_image_extendable
          fs.setup(mount=False)
        File "/home/zuul/src/code.engineering.redhat.com/nova/nova/virt/disk/vfs/guestfs.py", line 235, in setup
          self.handle.launch()
        File "/usr/lib/python3.9/site-packages/eventlet/tpool.py", line 190, in doit
          result = proxy_call(self._autowrap, f, *args, **kwargs)
        File "/usr/lib/python3.9/site-packages/eventlet/tpool.py", line 148, in proxy_call
          rv = execute(f, *args, **kwargs)
        File "/usr/lib/python3.9/site-packages/eventlet/tpool.py", line 121, in execute
          rv = e.wait()
        File "/usr/lib/python3.9/site-packages/eventlet/event.py", line 125, in wait
          result = hub.switch()
        File "/usr/lib/python3.9/site-packages/eventlet/hubs/hub.py", line 313, in switch
          return self.greenlet.switch()
        File "/usr/lib/python3.9/site-packages/eventlet/hubs/hub.py", line 365, in run
          self.wait(sleep_time)
        File "/usr/lib/python3.9/site-packages/eventlet/hubs/poll.py", line 80, in wait
          presult = self.do_poll(seconds)
        File "/usr/lib/python3.9/site-packages/eventlet/hubs/epolls.py", line 31, in do_poll
          return self.poll.poll(seconds)
        File "/usr/lib/python3.9/site-packages/fixtures/_fixtures/timeout.py", line 52, in signal_handler
          raise TimeoutException()
      fixtures._fixtures.timeout.TimeoutException

  That ^ was with libguestfs 1.46.1.

  Repro steps (not identical traceback but close):

  1. Make sure you have libguestfs header files

  $ sudo apt install libguestfs-dev

  2. Enter the unit test venv

  $ source .tox/py38/bin/activate

  3. Install the libguestfs python bindings into the venv. This has to
  be done by tarball [2]. Note that the bindings were not built for
  python 3 until version 1.40 [3]

  $ pip install
  http://libguestfs.org/download/python/guestfs-1.40.2.tar.gz

  4. Run the unit test

  $ tox -epy38 test_can_resize_need_fs_type_specified

  5. The test should fail

  The code in nova will call libvirt if guestfs is installed [4] and if
  it is not installed, it will emit a warning and move on [5]. In my
  repro environment it didn't get as far as a call to libvirt though.

  Upstream the test doesn't fail because neither libvirt nor guestfs are
  installed in the CI environment for unit test jobs.

  A unit test shouldn't be able to call out to real guestfs or libvirt
  and it wasn't mocked out properly.

  We can address it by adding mocking to the unit test and create a
  fixture that will poison future imports by unit or func tests.

  [1] https://bugzilla.redhat.com/show_bug.cgi?id=2135856
  [2] https://libguestfs.org/guestfs-python.3.html#using-python-bindings-in-a-virtualenv
  [3] https://bugzilla.redhat.com/show_bug.cgi?id=1627964#c15
  [4] https://github.com/openstack/nova/blob/b1958b7cfa6b8aca5b76b3f133627bb733d29f00/nova/virt/disk/vfs/guestfs.py#L66-L70
  [5] https://github.com/openstack/nova/blob/b1958b7cfa6b8aca5b76b3f133627bb733d29f00/nova/virt/disk/api.py#L189-L201

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1994913/+subscriptions