yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1949716] [NEW] nova.exception.NovaException: No operating system found in <LocalFileImage:{'format': 'qcow2', 'path': '/opt/stack/data/nova/instances/905e3060-4eb2-4777-b1c3-c37634c29cde/disk'}>

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Lee Yarwood <1949716@xxxxxxxxxxxxxxxxxx>
Date: Thu, 04 Nov 2021 09:02:16 -0000
Reply-to: Bug 1949716 <1949716@xxxxxxxxxxxxxxxxxx>
Sender: noreply@xxxxxxxxxxxxx
Public bug reported:

Description
===========

I'm trying to re-enable ENABLE_FILE_INJECTION based testing in the
tempest-all job in the following change after previously resolving
https://launchpad.net/bugs/1882421:

https://review.opendev.org/c/openstack/tempest/+/787714

However now the server personality tests that inject files into an
instance disk at launch are failing:

https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_3bf/787714/3/experimental/tempest-
all/3bfe4b1/testr_results.html

https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_3bf/787714/3/experimental/tempest-
all/3bfe4b1/controller/logs/screen-n-cpu.txt

Nov 02 00:29:07.705470 ubuntu-focal-ovh-gra1-0027180141 nova-compute[117578]: ERROR nova.compute.manager [None req-0c6b658a-1c00-4c77-92f9-6238716e539e tempest-ServerPersonalityTestJSON-1918276831 tempest-ServerPersonalityTestJSON-1918276831-project] [instance: 905e3060-4eb2-4777-b1c3-c37634c29cde] Instance failed to spawn: nova.exception.NovaException: No operating system found in <LocalFileImage:{'format': 'qcow2', 'path': '/opt/stack/data/nova/instances/905e3060-4eb2-4777-b1c3-c37634c29cde/disk'}>
Nov 02 00:29:07.705470 ubuntu-focal-ovh-gra1-0027180141 nova-compute[117578]: ERROR nova.compute.manager [instance: 905e3060-4eb2-4777-b1c3-c37634c29cde] Traceback (most recent call last):
Nov 02 00:29:07.705470 ubuntu-focal-ovh-gra1-0027180141 nova-compute[117578]: ERROR nova.compute.manager [instance: 905e3060-4eb2-4777-b1c3-c37634c29cde]   File "/opt/stack/nova/nova/compute/manager.py", line 2638, in _build_resources
Nov 02 00:29:07.705470 ubuntu-focal-ovh-gra1-0027180141 nova-compute[117578]: ERROR nova.compute.manager [instance: 905e3060-4eb2-4777-b1c3-c37634c29cde]     yield resources
Nov 02 00:29:07.705470 ubuntu-focal-ovh-gra1-0027180141 nova-compute[117578]: ERROR nova.compute.manager [instance: 905e3060-4eb2-4777-b1c3-c37634c29cde]   File "/opt/stack/nova/nova/compute/manager.py", line 2398, in _build_and_run_instance
Nov 02 00:29:07.705470 ubuntu-focal-ovh-gra1-0027180141 nova-compute[117578]: ERROR nova.compute.manager [instance: 905e3060-4eb2-4777-b1c3-c37634c29cde]     self.driver.spawn(context, instance, image_meta,
Nov 02 00:29:07.705470 ubuntu-focal-ovh-gra1-0027180141 nova-compute[117578]: ERROR nova.compute.manager [instance: 905e3060-4eb2-4777-b1c3-c37634c29cde]   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 4172, in spawn
Nov 02 00:29:07.705470 ubuntu-focal-ovh-gra1-0027180141 nova-compute[117578]: ERROR nova.compute.manager [instance: 905e3060-4eb2-4777-b1c3-c37634c29cde]     created_instance_dir, created_disks = self._create_image(
Nov 02 00:29:07.705470 ubuntu-focal-ovh-gra1-0027180141 nova-compute[117578]: ERROR nova.compute.manager [instance: 905e3060-4eb2-4777-b1c3-c37634c29cde]   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 4562, in _create_image
Nov 02 00:29:07.705470 ubuntu-focal-ovh-gra1-0027180141 nova-compute[117578]: ERROR nova.compute.manager [instance: 905e3060-4eb2-4777-b1c3-c37634c29cde]     created_disks = self._create_and_inject_local_root(
Nov 02 00:29:07.705470 ubuntu-focal-ovh-gra1-0027180141 nova-compute[117578]: ERROR nova.compute.manager [instance: 905e3060-4eb2-4777-b1c3-c37634c29cde]   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 4701, in _create_and_inject_local_root
Nov 02 00:29:07.705470 ubuntu-focal-ovh-gra1-0027180141 nova-compute[117578]: ERROR nova.compute.manager [instance: 905e3060-4eb2-4777-b1c3-c37634c29cde]     self._inject_data(backend, instance, injection_info)
Nov 02 00:29:07.705470 ubuntu-focal-ovh-gra1-0027180141 nova-compute[117578]: ERROR nova.compute.manager [instance: 905e3060-4eb2-4777-b1c3-c37634c29cde]   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 4471, in _inject_data
Nov 02 00:29:07.705470 ubuntu-focal-ovh-gra1-0027180141 nova-compute[117578]: ERROR nova.compute.manager [instance: 905e3060-4eb2-4777-b1c3-c37634c29cde]     LOG.error('Error injecting data into image '
Nov 02 00:29:07.705470 ubuntu-focal-ovh-gra1-0027180141 nova-compute[117578]: ERROR nova.compute.manager [instance: 905e3060-4eb2-4777-b1c3-c37634c29cde]   File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__
Nov 02 00:29:07.705470 ubuntu-focal-ovh-gra1-0027180141 nova-compute[117578]: ERROR nova.compute.manager [instance: 905e3060-4eb2-4777-b1c3-c37634c29cde]     self.force_reraise()
Nov 02 00:29:07.705470 ubuntu-focal-ovh-gra1-0027180141 nova-compute[117578]: ERROR nova.compute.manager [instance: 905e3060-4eb2-4777-b1c3-c37634c29cde]   File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise
Nov 02 00:29:07.705470 ubuntu-focal-ovh-gra1-0027180141 nova-compute[117578]: ERROR nova.compute.manager [instance: 905e3060-4eb2-4777-b1c3-c37634c29cde]     raise self.value
Nov 02 00:29:07.705470 ubuntu-focal-ovh-gra1-0027180141 nova-compute[117578]: ERROR nova.compute.manager [instance: 905e3060-4eb2-4777-b1c3-c37634c29cde]   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 4464, in _inject_data
Nov 02 00:29:07.705470 ubuntu-focal-ovh-gra1-0027180141 nova-compute[117578]: ERROR nova.compute.manager [instance: 905e3060-4eb2-4777-b1c3-c37634c29cde]     disk_api.inject_data(disk.get_model(self._conn),
Nov 02 00:29:07.705470 ubuntu-focal-ovh-gra1-0027180141 nova-compute[117578]: ERROR nova.compute.manager [instance: 905e3060-4eb2-4777-b1c3-c37634c29cde]   File "/opt/stack/nova/nova/virt/disk/api.py", line 355, in inject_data
Nov 02 00:29:07.705470 ubuntu-focal-ovh-gra1-0027180141 nova-compute[117578]: ERROR nova.compute.manager [instance: 905e3060-4eb2-4777-b1c3-c37634c29cde]     fs.setup()
Nov 02 00:29:07.705470 ubuntu-focal-ovh-gra1-0027180141 nova-compute[117578]: ERROR nova.compute.manager [instance: 905e3060-4eb2-4777-b1c3-c37634c29cde]   File "/opt/stack/nova/nova/virt/disk/vfs/guestfs.py", line 238, in setup
Nov 02 00:29:07.705470 ubuntu-focal-ovh-gra1-0027180141 nova-compute[117578]: ERROR nova.compute.manager [instance: 905e3060-4eb2-4777-b1c3-c37634c29cde]     self.setup_os()
Nov 02 00:29:07.705470 ubuntu-focal-ovh-gra1-0027180141 nova-compute[117578]: ERROR nova.compute.manager [instance: 905e3060-4eb2-4777-b1c3-c37634c29cde]   File "/opt/stack/nova/nova/virt/disk/vfs/guestfs.py", line 121, in setup_os
Nov 02 00:29:07.705470 ubuntu-focal-ovh-gra1-0027180141 nova-compute[117578]: ERROR nova.compute.manager [instance: 905e3060-4eb2-4777-b1c3-c37634c29cde]     self.setup_os_inspect()
Nov 02 00:29:07.705470 ubuntu-focal-ovh-gra1-0027180141 nova-compute[117578]: ERROR nova.compute.manager [instance: 905e3060-4eb2-4777-b1c3-c37634c29cde]   File "/opt/stack/nova/nova/virt/disk/vfs/guestfs.py", line 139, in setup_os_inspect
Nov 02 00:29:07.705470 ubuntu-focal-ovh-gra1-0027180141 nova-compute[117578]: ERROR nova.compute.manager [instance: 905e3060-4eb2-4777-b1c3-c37634c29cde]     raise exception.NovaException(_("No operating system found in %s")
Nov 02 00:29:07.705470 ubuntu-focal-ovh-gra1-0027180141 nova-compute[117578]: ERROR nova.compute.manager [instance: 905e3060-4eb2-4777-b1c3-c37634c29cde] nova.exception.NovaException: No operating system found in <LocalFileImage:{'format': 'qcow2', 'path': '/opt/stack/data/nova/instances/905e3060-4eb2-4777-b1c3-c37634c29cde/disk'}>

Steps to reproduce
==================

recheck https://review.opendev.org/c/openstack/tempest/+/787714 for now.

Expected result
===============

Personality tests pass.

Actual result
=============

Personality tests fail.

Environment
===========
1. Exact version of OpenStack you are running. See the following
  list for all releases: http://docs.openstack.org/releases/

   master/HEAD

2. Which hypervisor did you use?
   (For example: Libvirt + KVM, Libvirt + XEN, Hyper-V, PowerKVM, ...)
   What's the version of that?

   Libvirt + QEMU

2. Which storage type did you use?
   (For example: Ceph, LVM, GPFS, ...)
   What's the version of that?

   N/A

3. Which networking type did you use?
   (For example: nova-network, Neutron with OpenVSwitch, ...)

   N/A

Logs & Configs
==============

As above.

** Affects: nova
     Importance: Undecided
     Assignee: Lee Yarwood (lyarwood)
         Status: New


** Tags: gate-failure libvirt

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1949716

Title:
  nova.exception.NovaException: No operating system found in
  <LocalFileImage:{'format': 'qcow2', 'path':
  '/opt/stack/data/nova/instances/905e3060-4eb2-4777-b1c3-c37634c29cde/disk'}>

Status in OpenStack Compute (nova):
  New

Bug description:
  Description
  ===========

  I'm trying to re-enable ENABLE_FILE_INJECTION based testing in the
  tempest-all job in the following change after previously resolving
  https://launchpad.net/bugs/1882421:

  https://review.opendev.org/c/openstack/tempest/+/787714

  However now the server personality tests that inject files into an
  instance disk at launch are failing:

  https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_3bf/787714/3/experimental/tempest-
  all/3bfe4b1/testr_results.html

  https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_3bf/787714/3/experimental/tempest-
  all/3bfe4b1/controller/logs/screen-n-cpu.txt

  Nov 02 00:29:07.705470 ubuntu-focal-ovh-gra1-0027180141 nova-compute[117578]: ERROR nova.compute.manager [None req-0c6b658a-1c00-4c77-92f9-6238716e539e tempest-ServerPersonalityTestJSON-1918276831 tempest-ServerPersonalityTestJSON-1918276831-project] [instance: 905e3060-4eb2-4777-b1c3-c37634c29cde] Instance failed to spawn: nova.exception.NovaException: No operating system found in <LocalFileImage:{'format': 'qcow2', 'path': '/opt/stack/data/nova/instances/905e3060-4eb2-4777-b1c3-c37634c29cde/disk'}>
  Nov 02 00:29:07.705470 ubuntu-focal-ovh-gra1-0027180141 nova-compute[117578]: ERROR nova.compute.manager [instance: 905e3060-4eb2-4777-b1c3-c37634c29cde] Traceback (most recent call last):
  Nov 02 00:29:07.705470 ubuntu-focal-ovh-gra1-0027180141 nova-compute[117578]: ERROR nova.compute.manager [instance: 905e3060-4eb2-4777-b1c3-c37634c29cde]   File "/opt/stack/nova/nova/compute/manager.py", line 2638, in _build_resources
  Nov 02 00:29:07.705470 ubuntu-focal-ovh-gra1-0027180141 nova-compute[117578]: ERROR nova.compute.manager [instance: 905e3060-4eb2-4777-b1c3-c37634c29cde]     yield resources
  Nov 02 00:29:07.705470 ubuntu-focal-ovh-gra1-0027180141 nova-compute[117578]: ERROR nova.compute.manager [instance: 905e3060-4eb2-4777-b1c3-c37634c29cde]   File "/opt/stack/nova/nova/compute/manager.py", line 2398, in _build_and_run_instance
  Nov 02 00:29:07.705470 ubuntu-focal-ovh-gra1-0027180141 nova-compute[117578]: ERROR nova.compute.manager [instance: 905e3060-4eb2-4777-b1c3-c37634c29cde]     self.driver.spawn(context, instance, image_meta,
  Nov 02 00:29:07.705470 ubuntu-focal-ovh-gra1-0027180141 nova-compute[117578]: ERROR nova.compute.manager [instance: 905e3060-4eb2-4777-b1c3-c37634c29cde]   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 4172, in spawn
  Nov 02 00:29:07.705470 ubuntu-focal-ovh-gra1-0027180141 nova-compute[117578]: ERROR nova.compute.manager [instance: 905e3060-4eb2-4777-b1c3-c37634c29cde]     created_instance_dir, created_disks = self._create_image(
  Nov 02 00:29:07.705470 ubuntu-focal-ovh-gra1-0027180141 nova-compute[117578]: ERROR nova.compute.manager [instance: 905e3060-4eb2-4777-b1c3-c37634c29cde]   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 4562, in _create_image
  Nov 02 00:29:07.705470 ubuntu-focal-ovh-gra1-0027180141 nova-compute[117578]: ERROR nova.compute.manager [instance: 905e3060-4eb2-4777-b1c3-c37634c29cde]     created_disks = self._create_and_inject_local_root(
  Nov 02 00:29:07.705470 ubuntu-focal-ovh-gra1-0027180141 nova-compute[117578]: ERROR nova.compute.manager [instance: 905e3060-4eb2-4777-b1c3-c37634c29cde]   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 4701, in _create_and_inject_local_root
  Nov 02 00:29:07.705470 ubuntu-focal-ovh-gra1-0027180141 nova-compute[117578]: ERROR nova.compute.manager [instance: 905e3060-4eb2-4777-b1c3-c37634c29cde]     self._inject_data(backend, instance, injection_info)
  Nov 02 00:29:07.705470 ubuntu-focal-ovh-gra1-0027180141 nova-compute[117578]: ERROR nova.compute.manager [instance: 905e3060-4eb2-4777-b1c3-c37634c29cde]   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 4471, in _inject_data
  Nov 02 00:29:07.705470 ubuntu-focal-ovh-gra1-0027180141 nova-compute[117578]: ERROR nova.compute.manager [instance: 905e3060-4eb2-4777-b1c3-c37634c29cde]     LOG.error('Error injecting data into image '
  Nov 02 00:29:07.705470 ubuntu-focal-ovh-gra1-0027180141 nova-compute[117578]: ERROR nova.compute.manager [instance: 905e3060-4eb2-4777-b1c3-c37634c29cde]   File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__
  Nov 02 00:29:07.705470 ubuntu-focal-ovh-gra1-0027180141 nova-compute[117578]: ERROR nova.compute.manager [instance: 905e3060-4eb2-4777-b1c3-c37634c29cde]     self.force_reraise()
  Nov 02 00:29:07.705470 ubuntu-focal-ovh-gra1-0027180141 nova-compute[117578]: ERROR nova.compute.manager [instance: 905e3060-4eb2-4777-b1c3-c37634c29cde]   File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise
  Nov 02 00:29:07.705470 ubuntu-focal-ovh-gra1-0027180141 nova-compute[117578]: ERROR nova.compute.manager [instance: 905e3060-4eb2-4777-b1c3-c37634c29cde]     raise self.value
  Nov 02 00:29:07.705470 ubuntu-focal-ovh-gra1-0027180141 nova-compute[117578]: ERROR nova.compute.manager [instance: 905e3060-4eb2-4777-b1c3-c37634c29cde]   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 4464, in _inject_data
  Nov 02 00:29:07.705470 ubuntu-focal-ovh-gra1-0027180141 nova-compute[117578]: ERROR nova.compute.manager [instance: 905e3060-4eb2-4777-b1c3-c37634c29cde]     disk_api.inject_data(disk.get_model(self._conn),
  Nov 02 00:29:07.705470 ubuntu-focal-ovh-gra1-0027180141 nova-compute[117578]: ERROR nova.compute.manager [instance: 905e3060-4eb2-4777-b1c3-c37634c29cde]   File "/opt/stack/nova/nova/virt/disk/api.py", line 355, in inject_data
  Nov 02 00:29:07.705470 ubuntu-focal-ovh-gra1-0027180141 nova-compute[117578]: ERROR nova.compute.manager [instance: 905e3060-4eb2-4777-b1c3-c37634c29cde]     fs.setup()
  Nov 02 00:29:07.705470 ubuntu-focal-ovh-gra1-0027180141 nova-compute[117578]: ERROR nova.compute.manager [instance: 905e3060-4eb2-4777-b1c3-c37634c29cde]   File "/opt/stack/nova/nova/virt/disk/vfs/guestfs.py", line 238, in setup
  Nov 02 00:29:07.705470 ubuntu-focal-ovh-gra1-0027180141 nova-compute[117578]: ERROR nova.compute.manager [instance: 905e3060-4eb2-4777-b1c3-c37634c29cde]     self.setup_os()
  Nov 02 00:29:07.705470 ubuntu-focal-ovh-gra1-0027180141 nova-compute[117578]: ERROR nova.compute.manager [instance: 905e3060-4eb2-4777-b1c3-c37634c29cde]   File "/opt/stack/nova/nova/virt/disk/vfs/guestfs.py", line 121, in setup_os
  Nov 02 00:29:07.705470 ubuntu-focal-ovh-gra1-0027180141 nova-compute[117578]: ERROR nova.compute.manager [instance: 905e3060-4eb2-4777-b1c3-c37634c29cde]     self.setup_os_inspect()
  Nov 02 00:29:07.705470 ubuntu-focal-ovh-gra1-0027180141 nova-compute[117578]: ERROR nova.compute.manager [instance: 905e3060-4eb2-4777-b1c3-c37634c29cde]   File "/opt/stack/nova/nova/virt/disk/vfs/guestfs.py", line 139, in setup_os_inspect
  Nov 02 00:29:07.705470 ubuntu-focal-ovh-gra1-0027180141 nova-compute[117578]: ERROR nova.compute.manager [instance: 905e3060-4eb2-4777-b1c3-c37634c29cde]     raise exception.NovaException(_("No operating system found in %s")
  Nov 02 00:29:07.705470 ubuntu-focal-ovh-gra1-0027180141 nova-compute[117578]: ERROR nova.compute.manager [instance: 905e3060-4eb2-4777-b1c3-c37634c29cde] nova.exception.NovaException: No operating system found in <LocalFileImage:{'format': 'qcow2', 'path': '/opt/stack/data/nova/instances/905e3060-4eb2-4777-b1c3-c37634c29cde/disk'}>

  Steps to reproduce
  ==================

  recheck https://review.opendev.org/c/openstack/tempest/+/787714 for
  now.

  Expected result
  ===============

  Personality tests pass.

  Actual result
  =============

  Personality tests fail.

  Environment
  ===========
  1. Exact version of OpenStack you are running. See the following
    list for all releases: http://docs.openstack.org/releases/

     master/HEAD

  2. Which hypervisor did you use?
     (For example: Libvirt + KVM, Libvirt + XEN, Hyper-V, PowerKVM, ...)
     What's the version of that?

     Libvirt + QEMU

  2. Which storage type did you use?
     (For example: Ceph, LVM, GPFS, ...)
     What's the version of that?

     N/A

  3. Which networking type did you use?
     (For example: nova-network, Neutron with OpenVSwitch, ...)

     N/A

  Logs & Configs
  ==============

  As above.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1949716/+subscriptions