yahoo-eng-team team mailing list archive

[Lars Erik Pedersen <2125730@xxxxxxxxxxxxxxxxxx>]

Public bug reported:

Description
===========
In Epoxy, after glance began to default the image_format/require_image_format_match to True, snapshotting a volume-backed instance fails with a http 415 from glance when we're trying to store the empty metadata-holding image. The error message states "Image disk_format qcow2 does not match stream data raw".

My current workaround, is simply setting require_image_format_match to
false, and then everything works as expected. I've also tried setting
snapshot_image_format to raw (rather that default "determine the correct
format automatically") in nova, but that did not help.

It seems to me that "something" (presumeably nova?) is telling glance
that the empty image is qcow2, but the format validation thinks it is
raw, and then fails.

Talked to melwitt on IRC, and she figured this probably should be a nova
bug, as nova should create the metadata-holding image with the correct
disk_format set.

Steps to reproduce
==================
1. Create an instance that boots from a volume
2. Create a snapshot, either through horizon, or through "openstack server image create" on the CLI
3. Observe that volume snapshot is created, and the glance image/snapshot fails with http 415 and ends up stuck in the "queued" state.

Expected result
===============
The empty metadata-holding image should be created successfully.

Actual result
=============
The empty metadata-holding image fails creation, and is left stucked in the queued state.

Environment
===========
- Nova, Caracal, 3:29.2.0-0ubuntu1, Ubuntu 24.04
- Glance, Epoxy, 2:30.0.0-0ubuntu1~cloud0, Ubuntu 24.04 (UCA)

We are running this mix because we needed a fix for this glance bug
https://bugs.launchpad.net/glance/+bug/2086675 and since it's not
backported to caracal, we're running glance on epoxy, and everything
else on caracal.

Logs & Configs
==============
Error log from glance WSGI
https://paste.openstack.org/show/bN5aDv2k2EF9FudCEGI1/

When disabling require_image_format_match this no longer an ERROR, but a WARNING
https://paste.openstack.org/show/b2Ib7Xk4GT6zPs7GuQdg/

Running glance with the RBD/ceph backend. Ceph Squid, 19.2.3.

** Affects: nova
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/2125730

Title:
  Snapshot of volume-backed instances fails with http 415 in Epoxy

Status in OpenStack Compute (nova):
  New

Bug description:
  Description
  ===========
  In Epoxy, after glance began to default the image_format/require_image_format_match to True, snapshotting a volume-backed instance fails with a http 415 from glance when we're trying to store the empty metadata-holding image. The error message states "Image disk_format qcow2 does not match stream data raw".

  My current workaround, is simply setting require_image_format_match to
  false, and then everything works as expected. I've also tried setting
  snapshot_image_format to raw (rather that default "determine the
  correct format automatically") in nova, but that did not help.

  It seems to me that "something" (presumeably nova?) is telling glance
  that the empty image is qcow2, but the format validation thinks it is
  raw, and then fails.

  Talked to melwitt on IRC, and she figured this probably should be a
  nova bug, as nova should create the metadata-holding image with the
  correct disk_format set.

  Steps to reproduce
  ==================
  1. Create an instance that boots from a volume
  2. Create a snapshot, either through horizon, or through "openstack server image create" on the CLI
  3. Observe that volume snapshot is created, and the glance image/snapshot fails with http 415 and ends up stuck in the "queued" state.

  Expected result
  ===============
  The empty metadata-holding image should be created successfully.

  Actual result
  =============
  The empty metadata-holding image fails creation, and is left stucked in the queued state.

  Environment
  ===========
  - Nova, Caracal, 3:29.2.0-0ubuntu1, Ubuntu 24.04
  - Glance, Epoxy, 2:30.0.0-0ubuntu1~cloud0, Ubuntu 24.04 (UCA)

  We are running this mix because we needed a fix for this glance bug
  https://bugs.launchpad.net/glance/+bug/2086675 and since it's not
  backported to caracal, we're running glance on epoxy, and everything
  else on caracal.

  Logs & Configs
  ==============
  Error log from glance WSGI
  https://paste.openstack.org/show/bN5aDv2k2EF9FudCEGI1/

  When disabling require_image_format_match this no longer an ERROR, but a WARNING
  https://paste.openstack.org/show/b2Ib7Xk4GT6zPs7GuQdg/

  Running glance with the RBD/ceph backend. Ceph Squid, 19.2.3.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/2125730/+subscriptions