yahoo-eng-team team mailing list archive
-
yahoo-eng-team team
-
Mailing list archive
-
Message #96543
[Bug 2126065] [NEW] migration stuck in state accepted after nova-compute restart on source compute
Public bug reported:
environement:
nova version : 2025.1
Libvirt + KVM + qemu
local storage on compute
Bug Description:
We have observed that when the nova-compute service restarts on a compute node with an ongoing instance migration on that compute and a migration recorded in the database as "accepted,".
The instance remains stuck in the "migrating" state, and the migration remains "accepted" indefinitely.
The only solution is to manually reset the instance's state and cancel the migration to restore the instance to a functional state.
How to Reproduce:
Setup:
- 2 compute nodes
- 1 instance
- A functioning control plane
1. Initiate the instance migration.
2. Wait for the migration to reach "accepted" status and the instance to enter the "migrating" state.
3. Once these states are achieved, restart the nova-compute service on the source compute node of the instance.
4. After the restart, nova-compute performs checks and skips the instance in the "migrating" state.
scenario:
nova log after restart nova-compute service :
Setting instance vm_state to ERROR:
nova.exception_Remote.UnexpectedTaskStateError_Remote: Conflict updating
instance 76fa79d9-0690-441b-921b-5a69911c8d21. Expected: {'task_state':
['resize_prep']}. Actual: {'task_state': None}
migration state in DB:
created_at: 2025-09-25 10:31:12
updated_at: 2025-09-25 10:31:34
deleted_at: NULL
id: 2
source_compute: default-compute-0
dest_compute: default-compute-1
dest_host: 162.19.52.171
status: error
instance_uuid: 76fa79d9-0690-441b-921b-5a69911c8d21
old_instance_type_id: 1
new_instance_type_id: 1
source_node: default-compute-0
dest_node: default-compute-1
deleted: 0
migration_type: migration
hidden: 0
memory_total: NULL
memory_processed: NULL
memory_remaining: NULL
disk_total: NULL
disk_processed: NULL
disk_remaining: NULL
uuid: 61673b36-9fc2-431f-8c03-8a148013f35c
cross_cell_move: 0
user_id: 39c05d6f875b440982365842a86effdf
project_id: c3fb2dcff53047a88af1867dfa433b81
dest_compute_id: 2
when trying to reboot :
openstack server reboot --hard 76fa79d9-0690-441b-921b-5a69911c8d21
ConflictException: 409: Client Error for url:
http://nova.146.59.204.36.xip.opensteak.fr/v2.1/servers/76fa79d9-0690-441b-921b-5a69911c8d21/action,
Cannot 'reboot' instance 76fa79d9-0690-441b-921b-5a69911c8d21 while it
is in task_state resize_migrating
virsh list --all
Id Name State
-----------------------------------
1 instance-00000001 running
** Affects: nova
Importance: Undecided
Status: New
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/2126065
Title:
migration stuck in state accepted after nova-compute restart on source
compute
Status in OpenStack Compute (nova):
New
Bug description:
environement:
nova version : 2025.1
Libvirt + KVM + qemu
local storage on compute
Bug Description:
We have observed that when the nova-compute service restarts on a compute node with an ongoing instance migration on that compute and a migration recorded in the database as "accepted,".
The instance remains stuck in the "migrating" state, and the migration remains "accepted" indefinitely.
The only solution is to manually reset the instance's state and cancel the migration to restore the instance to a functional state.
How to Reproduce:
Setup:
- 2 compute nodes
- 1 instance
- A functioning control plane
1. Initiate the instance migration.
2. Wait for the migration to reach "accepted" status and the instance to enter the "migrating" state.
3. Once these states are achieved, restart the nova-compute service on the source compute node of the instance.
4. After the restart, nova-compute performs checks and skips the instance in the "migrating" state.
scenario:
nova log after restart nova-compute service :
Setting instance vm_state to ERROR:
nova.exception_Remote.UnexpectedTaskStateError_Remote: Conflict
updating instance 76fa79d9-0690-441b-921b-5a69911c8d21. Expected:
{'task_state': ['resize_prep']}. Actual: {'task_state': None}
migration state in DB:
created_at: 2025-09-25 10:31:12
updated_at: 2025-09-25 10:31:34
deleted_at: NULL
id: 2
source_compute: default-compute-0
dest_compute: default-compute-1
dest_host: 162.19.52.171
status: error
instance_uuid: 76fa79d9-0690-441b-921b-5a69911c8d21
old_instance_type_id: 1
new_instance_type_id: 1
source_node: default-compute-0
dest_node: default-compute-1
deleted: 0
migration_type: migration
hidden: 0
memory_total: NULL
memory_processed: NULL
memory_remaining: NULL
disk_total: NULL
disk_processed: NULL
disk_remaining: NULL
uuid: 61673b36-9fc2-431f-8c03-8a148013f35c
cross_cell_move: 0
user_id: 39c05d6f875b440982365842a86effdf
project_id: c3fb2dcff53047a88af1867dfa433b81
dest_compute_id: 2
when trying to reboot :
openstack server reboot --hard 76fa79d9-0690-441b-921b-5a69911c8d21
ConflictException: 409: Client Error for url:
http://nova.146.59.204.36.xip.opensteak.fr/v2.1/servers/76fa79d9-0690-441b-921b-5a69911c8d21/action,
Cannot 'reboot' instance 76fa79d9-0690-441b-921b-5a69911c8d21 while it
is in task_state resize_migrating
virsh list --all
Id Name State
-----------------------------------
1 instance-00000001 running
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/2126065/+subscriptions
