yahoo-eng-team team mailing list archive

[Pierre Libeau <2040986@xxxxxxxxxxxxxxxxxx>]

Public bug reported:

Description
===========
Request_specs is not updated during rebuild if we raise an exception during the scheduling of the instance. we can found update of image_ref but request_specs missing so next action on the instance failed and the instance is put in error state.

Steps to reproduce
==================
* create an instance with image "a"
* set an image "b" with specific properties like hypervisor_type='baremetal'
* rebuild instance with specific image "b" -> instance go in ERROR
* rebuild instance without specific image (instance rebuild with image "a")

$ openstack server create --image 82ef29e2-e49b-4021-96d8-00f161c1a987
--network 851acde6-6c03-43e5-b42b-4058599d0247 --flavor m1.small test-
rebuild

$ openstack image set 14ff5912-d98c-43b1-a13a-0617c1de7a09 --property
hypervisor_type='baremetal'

$ openstack server rebuild bd94ba1a-1bb6-40bf-905a-43cb49503cdc --image
14ff5912-d98c-43b1-a13a-0617c1de7a09

$ openstack server rebuild bd94ba1a-1bb6-40bf-905a-43cb49503cdc

Expected result
===============
request_specs table in nova_api and instances tables in nova_cell1 need to have same info about image in the DB. The actual bug break the action like live-migration on the instance.

Actual result
=============
request_specs table in nova_api and instances tables in nova_cell1 are unsync

stack@devstack1 ~/devstack % openstack server show bd94ba1a-1bb6-40bf-905a-43cb49503cdc -c image
+-------+--------------------------------------------------+
| Field | Value                                            |
+-------+--------------------------------------------------+
| image | Debian 12 (82ef29e2-e49b-4021-96d8-00f161c1a987) |
+-------+--------------------------------------------------+

MySQL root@127.0.0.1:nova_api> select * from request_specs \G
***************************[ 1. row ]***************************
created_at    | 2023-09-26 14:39:48
updated_at    | 2023-09-26 15:30:20
id            | 1
instance_uuid | bd94ba1a-1bb6-40bf-905a-43cb49503cdc
spec          | {"nova_object.name": "RequestSpec", "nova_object.namespace": "nova", "nova_object.version": "1.14", "nova_object.data": {"id": 1, "image": {"nova_object.name": "ImageMeta", "nova_object.namespace": "nova", "nova_object.version": "1.8", "nova_object.data": {"id": "14ff5912-d98c-43b1-a13a-0617c1de7a09", "name": "cirros-0.6.2-x86_64-disk", "status": "active", "checksum": "c8fc807773e5354afe61636071771906", "owner": "ba0a4db0b823489f9c85c9c8e5379dea", "size": 21430272, "container_format": "bare", "disk_format": "qcow2", "created_at": "2023-09-26T13:18:36Z", "updated_at": "2023-09-26T15:28:00Z", "min_ram": 0, "min_disk": 0, "properties": {"nova_object.name": "ImageMetaProps", "nova_object.namespace": "nova", "nova_object.version": "1.34", "nova_object.data": {"hw_rng_model": "virtio", "img_hv_type": "baremetal"}, "nova_object.changes": ["hw_rng_model", "img_hv_type"]}}, "nova_object.changes": ["disk_format", "min_ram", "id", "container_format", "updated_at", "status", "name", "min_disk", "properties", "created_at", "checksum", "owner", "size"]}, "numa_topology": null, "pci_requests": {"nova_object.name": "InstancePCIRequests", "nova_object.namespace": "nova", "nova_object.version": "1.1", "nova_object.data": {"requests": []}, "nova_object.changes": ["requests"]}, "project_id": "26386575ede0452d8c19ac63f3633966", "user_id": "cf2eef961513436fbf256b5ec03863ed", "availability_zone": null, "flavor": {"nova_object.name": "Flavor", "nova_object.namespace": "nova", "nova_object.version": "1.2", "nova_object.data": {"id": 7, "name": "m1.small", "memory_mb": 2048, "vcpus": 1, "root_gb": 20, "ephemeral_gb": 0, "flavorid": "2", "swap": 0, "rxtx_factor": 1.0, "vcpu_weight": 0, "disabled": false, "is_public": true, "extra_specs": {"hw_rng:allowed": "True"}, "description": null, "created_at": "2023-09-26T13:18:30Z", "updated_at": null, "deleted_at": null, "deleted": false}, "nova_object.changes": ["extra_specs"]}, "num_instances": 1, "ignore_hosts": null, "force_hosts": null, "force_nodes": null, "retry": null, "limits": {"nova_object.name": "SchedulerLimits", "nova_object.namespace": "nova", "nova_object.version": "1.0", "nova_object.data": {"numa_topology": null, "vcpu": null, "disk_gb": null, "memory_mb": null}, "nova_object.changes": ["memory_mb", "disk_gb", "vcpu", "numa_topology"]}, "instance_group": null, "scheduler_hints": {}, "instance_uuid": "bd94ba1a-1bb6-40bf-905a-43cb49503cdc", "security_groups": {"nova_object.name": "SecurityGroupList", "nova_object.namespace": "nova", "nova_object.version": "1.1", "nova_object.data": {"objects": [{"nova_object.name": "SecurityGroup", "nova_object.namespace": "nova", "nova_object.version": "1.2", "nova_object.data": {"name": "default"}, "nova_object.changes": ["name"]}]}, "nova_object.changes": ["objects"]}, "is_bfv": false}, "nova_object.changes": ["limits", "flavor", "image", "security_groups", "pci_requests"]}

Environment
===========
on devstack and master branch

** Affects: nova
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/2040986

Title:
  request_specs not updated after failed rebuild

Status in OpenStack Compute (nova):
  New

Bug description:
  Description
  ===========
  Request_specs is not updated during rebuild if we raise an exception during the scheduling of the instance. we can found update of image_ref but request_specs missing so next action on the instance failed and the instance is put in error state.

  Steps to reproduce
  ==================
  * create an instance with image "a"
  * set an image "b" with specific properties like hypervisor_type='baremetal'
  * rebuild instance with specific image "b" -> instance go in ERROR
  * rebuild instance without specific image (instance rebuild with image "a")

  $ openstack server create --image 82ef29e2-e49b-4021-96d8-00f161c1a987
  --network 851acde6-6c03-43e5-b42b-4058599d0247 --flavor m1.small test-
  rebuild

  $ openstack image set 14ff5912-d98c-43b1-a13a-0617c1de7a09 --property
  hypervisor_type='baremetal'

  $ openstack server rebuild bd94ba1a-1bb6-40bf-905a-43cb49503cdc
  --image 14ff5912-d98c-43b1-a13a-0617c1de7a09

  $ openstack server rebuild bd94ba1a-1bb6-40bf-905a-43cb49503cdc

  Expected result
  ===============
  request_specs table in nova_api and instances tables in nova_cell1 need to have same info about image in the DB. The actual bug break the action like live-migration on the instance.

  Actual result
  =============
  request_specs table in nova_api and instances tables in nova_cell1 are unsync

  stack@devstack1 ~/devstack % openstack server show bd94ba1a-1bb6-40bf-905a-43cb49503cdc -c image
  +-------+--------------------------------------------------+
  | Field | Value                                            |
  +-------+--------------------------------------------------+
  | image | Debian 12 (82ef29e2-e49b-4021-96d8-00f161c1a987) |
  +-------+--------------------------------------------------+

  MySQL root@127.0.0.1:nova_api> select * from request_specs \G
  ***************************[ 1. row ]***************************
  created_at    | 2023-09-26 14:39:48
  updated_at    | 2023-09-26 15:30:20
  id            | 1
  instance_uuid | bd94ba1a-1bb6-40bf-905a-43cb49503cdc
  spec          | {"nova_object.name": "RequestSpec", "nova_object.namespace": "nova", "nova_object.version": "1.14", "nova_object.data": {"id": 1, "image": {"nova_object.name": "ImageMeta", "nova_object.namespace": "nova", "nova_object.version": "1.8", "nova_object.data": {"id": "14ff5912-d98c-43b1-a13a-0617c1de7a09", "name": "cirros-0.6.2-x86_64-disk", "status": "active", "checksum": "c8fc807773e5354afe61636071771906", "owner": "ba0a4db0b823489f9c85c9c8e5379dea", "size": 21430272, "container_format": "bare", "disk_format": "qcow2", "created_at": "2023-09-26T13:18:36Z", "updated_at": "2023-09-26T15:28:00Z", "min_ram": 0, "min_disk": 0, "properties": {"nova_object.name": "ImageMetaProps", "nova_object.namespace": "nova", "nova_object.version": "1.34", "nova_object.data": {"hw_rng_model": "virtio", "img_hv_type": "baremetal"}, "nova_object.changes": ["hw_rng_model", "img_hv_type"]}}, "nova_object.changes": ["disk_format", "min_ram", "id", "container_format", "updated_at", "status", "name", "min_disk", "properties", "created_at", "checksum", "owner", "size"]}, "numa_topology": null, "pci_requests": {"nova_object.name": "InstancePCIRequests", "nova_object.namespace": "nova", "nova_object.version": "1.1", "nova_object.data": {"requests": []}, "nova_object.changes": ["requests"]}, "project_id": "26386575ede0452d8c19ac63f3633966", "user_id": "cf2eef961513436fbf256b5ec03863ed", "availability_zone": null, "flavor": {"nova_object.name": "Flavor", "nova_object.namespace": "nova", "nova_object.version": "1.2", "nova_object.data": {"id": 7, "name": "m1.small", "memory_mb": 2048, "vcpus": 1, "root_gb": 20, "ephemeral_gb": 0, "flavorid": "2", "swap": 0, "rxtx_factor": 1.0, "vcpu_weight": 0, "disabled": false, "is_public": true, "extra_specs": {"hw_rng:allowed": "True"}, "description": null, "created_at": "2023-09-26T13:18:30Z", "updated_at": null, "deleted_at": null, "deleted": false}, "nova_object.changes": ["extra_specs"]}, "num_instances": 1, "ignore_hosts": null, "force_hosts": null, "force_nodes": null, "retry": null, "limits": {"nova_object.name": "SchedulerLimits", "nova_object.namespace": "nova", "nova_object.version": "1.0", "nova_object.data": {"numa_topology": null, "vcpu": null, "disk_gb": null, "memory_mb": null}, "nova_object.changes": ["memory_mb", "disk_gb", "vcpu", "numa_topology"]}, "instance_group": null, "scheduler_hints": {}, "instance_uuid": "bd94ba1a-1bb6-40bf-905a-43cb49503cdc", "security_groups": {"nova_object.name": "SecurityGroupList", "nova_object.namespace": "nova", "nova_object.version": "1.1", "nova_object.data": {"objects": [{"nova_object.name": "SecurityGroup", "nova_object.namespace": "nova", "nova_object.version": "1.2", "nova_object.data": {"name": "default"}, "nova_object.changes": ["name"]}]}, "nova_object.changes": ["objects"]}, "is_bfv": false}, "nova_object.changes": ["limits", "flavor", "image", "security_groups", "pci_requests"]}

  Environment
  ===========
  on devstack and master branch

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/2040986/+subscriptions