← Back to team overview

yahoo-eng-team team mailing list archive

[Bug 1905224] [NEW] bfv instance create failed due to insufficient resources, the volume is always attaching

 

Public bug reported:

When using a volume to create a instance, the instance failed and in
ERROR state due to insufficient resources, and the volume is always in
attaching state. I must reset volume to available status through cinder
reset-state and continue use it.

I am in pike version, i see https://review.opendev.org/#/c/335358/,
nova-api reserve volume before attach volume to a instance. If scheduler
failed for No ValidHost, the volume is always not available unless reset
state. When delete the ERROR instance,the volume is still in attaching
status.

Now, we should restore the state of the volume instead of resetting it
every time due to scheduling failure.

Reproduce the problem through the following steps

1)cinder create 1 --image-id 7b379b4c-ad6e-4ade-81f2-94a26e3afa3a --name
fake_test

()[root@busybox-openstack-556bf6df94-kkcr2 /]# cinder list
+--------------------------------------+-----------+-----------+------+-------------+----------+-------------+
| ID                                   | Status    | Name      | Size | Volume Type | Bootable | Attached to |
+--------------------------------------+-----------+-----------+------+-------------+----------+-------------+
| 8ca2c77b-120a-41c1-b9f6-e8ac0f23a0f2 | available | fake_test | 1    | -           | true     |             |
+--------------------------------------+-----------+-----------+------+-------------+----------+-------------+
2)nova boot --boot-volume 8ca2c77b-120a-41c1-b9f6-e8ac0f23a0f2 --flavor 200 --nic net-name=share_net faker_vm

()[root@busybox-openstack-556bf6df94-kkcr2 /]# nova list --all-t --fields name,host,status,task_state
+--------------------------------------+----------+------+--------+------------+
| ID                                   | Name     | Host | Status | Task State |
+--------------------------------------+----------+------+--------+------------+
| 256754e0-ed48-4acb-8293-4ac6cdae7384 | faker_vm | None | ERROR  | None       |
+--------------------------------------+----------+------+--------+------------+
()[root@busybox-openstack-556bf6df94-kkcr2 /]# cinder list
+--------------------------------------+-----------+----------+------+-------------+----------+-------------+
| ID                                   | Status    | Name     | Size | Volume Type | Bootable | Attached to |
+--------------------------------------+-----------+----------+------+-------------+----------+-------------+
| 84534871-e327-46d4-af3f-ac347da69213 | attaching | fake_sys | 1    | -           | true     |             |
+--------------------------------------+-----------+----------+------+-------------+----------+-------------+

3)nova delete 256754e0-ed48-4acb-8293-4ac6cdae7384

()[root@busybox-openstack-556bf6df94-kkcr2 /]# nova list --all-t
+----+------+-----------+--------+------------+-------------+----------+
| ID | Name | Tenant ID | Status | Task State | Power State | Networks |
+----+------+-----------+--------+------------+-------------+----------+
+----+------+-----------+--------+------------+-------------+----------+
()[root@busybox-openstack-556bf6df94-kkcr2 /]# cinder list --all
+--------------------------------------+----------------------------------+-----------+----------+------+-------------+----------+-------------+
| ID                                   | Tenant ID                        | Status    | Name     | Size | Volume Type | Bootable | Attached to |
+--------------------------------------+----------------------------------+-----------+----------+------+-------------+----------+-------------+
| 84534871-e327-46d4-af3f-ac347da69213 | c7bc6a73869541c7a65b49b06fa9e21a | attaching | fake_sys | 1    | -           | true     |             |
+--------------------------------------+----------------------------------+-----------+----------+------+-------------+----------+-------------+

** Affects: nova
     Importance: Undecided
         Status: New

** Description changed:

  When using a volume to create a instance, the instance failed and in
  ERROR state due to insufficient resources, and the volume is always in
  attaching state. I must reset volume to available status through cinder
  reset-state and continue use it.
  
  now i am in pike version, i see https://review.opendev.org/#/c/335358/,
  nova-api reserve volume before attach volume to a instance. If scheduler
  failed for No ValidHost, the volume is always not available unless reset
- state.When delete the ERROR instance,the volume is still in attaching
- status。
+ state. When delete the ERROR instance,the volume is still in attaching
+ status.
  
  Now, we should restore the state of the volume instead of resetting it
  every time due to scheduling failure.
- 
  
  Reproduce the problem through the following steps
  
  1)cinder create 1 --image-id 7b379b4c-ad6e-4ade-81f2-94a26e3afa3a --name
  fake_test
  
  ()[root@busybox-openstack-556bf6df94-kkcr2 /]# cinder list
  +--------------------------------------+-----------+-----------+------+-------------+----------+-------------+
  | ID                                   | Status    | Name      | Size | Volume Type | Bootable | Attached to |
  +--------------------------------------+-----------+-----------+------+-------------+----------+-------------+
  | 8ca2c77b-120a-41c1-b9f6-e8ac0f23a0f2 | available | fake_test | 1    | -           | true     |             |
  +--------------------------------------+-----------+-----------+------+-------------+----------+-------------+
  2)nova boot --boot-volume 8ca2c77b-120a-41c1-b9f6-e8ac0f23a0f2 --flavor 200 --nic net-name=share_net faker_vm
  
  ()[root@busybox-openstack-556bf6df94-kkcr2 /]# nova list --all-t --fields name,host,status,task_state
  +--------------------------------------+----------+------+--------+------------+
  | ID                                   | Name     | Host | Status | Task State |
  +--------------------------------------+----------+------+--------+------------+
  | 256754e0-ed48-4acb-8293-4ac6cdae7384 | faker_vm | None | ERROR  | None       |
  +--------------------------------------+----------+------+--------+------------+
- ()[root@busybox-openstack-556bf6df94-kkcr2 /]# cinder list 
+ ()[root@busybox-openstack-556bf6df94-kkcr2 /]# cinder list
  +--------------------------------------+-----------+----------+------+-------------+----------+-------------+
  | ID                                   | Status    | Name     | Size | Volume Type | Bootable | Attached to |
  +--------------------------------------+-----------+----------+------+-------------+----------+-------------+
  | 84534871-e327-46d4-af3f-ac347da69213 | attaching | fake_sys | 1    | -           | true     |             |
  +--------------------------------------+-----------+----------+------+-------------+----------+-------------+
  
  3)nova delete 256754e0-ed48-4acb-8293-4ac6cdae7384
  
  ()[root@busybox-openstack-556bf6df94-kkcr2 /]# nova list --all-t
  +----+------+-----------+--------+------------+-------------+----------+
  | ID | Name | Tenant ID | Status | Task State | Power State | Networks |
  +----+------+-----------+--------+------------+-------------+----------+
  +----+------+-----------+--------+------------+-------------+----------+
  ()[root@busybox-openstack-556bf6df94-kkcr2 /]# cinder list --all
  +--------------------------------------+----------------------------------+-----------+----------+------+-------------+----------+-------------+
  | ID                                   | Tenant ID                        | Status    | Name     | Size | Volume Type | Bootable | Attached to |
  +--------------------------------------+----------------------------------+-----------+----------+------+-------------+----------+-------------+
  | 84534871-e327-46d4-af3f-ac347da69213 | c7bc6a73869541c7a65b49b06fa9e21a | attaching | fake_sys | 1    | -           | true     |             |
  +--------------------------------------+----------------------------------+-----------+----------+------+-------------+----------+-------------+

** Description changed:

  When using a volume to create a instance, the instance failed and in
  ERROR state due to insufficient resources, and the volume is always in
  attaching state. I must reset volume to available status through cinder
  reset-state and continue use it.
  
- now i am in pike version, i see https://review.opendev.org/#/c/335358/,
+ I am in pike version, i see https://review.opendev.org/#/c/335358/,
  nova-api reserve volume before attach volume to a instance. If scheduler
  failed for No ValidHost, the volume is always not available unless reset
  state. When delete the ERROR instance,the volume is still in attaching
  status.
  
  Now, we should restore the state of the volume instead of resetting it
  every time due to scheduling failure.
  
  Reproduce the problem through the following steps
  
  1)cinder create 1 --image-id 7b379b4c-ad6e-4ade-81f2-94a26e3afa3a --name
  fake_test
  
  ()[root@busybox-openstack-556bf6df94-kkcr2 /]# cinder list
  +--------------------------------------+-----------+-----------+------+-------------+----------+-------------+
  | ID                                   | Status    | Name      | Size | Volume Type | Bootable | Attached to |
  +--------------------------------------+-----------+-----------+------+-------------+----------+-------------+
  | 8ca2c77b-120a-41c1-b9f6-e8ac0f23a0f2 | available | fake_test | 1    | -           | true     |             |
  +--------------------------------------+-----------+-----------+------+-------------+----------+-------------+
  2)nova boot --boot-volume 8ca2c77b-120a-41c1-b9f6-e8ac0f23a0f2 --flavor 200 --nic net-name=share_net faker_vm
  
  ()[root@busybox-openstack-556bf6df94-kkcr2 /]# nova list --all-t --fields name,host,status,task_state
  +--------------------------------------+----------+------+--------+------------+
  | ID                                   | Name     | Host | Status | Task State |
  +--------------------------------------+----------+------+--------+------------+
  | 256754e0-ed48-4acb-8293-4ac6cdae7384 | faker_vm | None | ERROR  | None       |
  +--------------------------------------+----------+------+--------+------------+
  ()[root@busybox-openstack-556bf6df94-kkcr2 /]# cinder list
  +--------------------------------------+-----------+----------+------+-------------+----------+-------------+
  | ID                                   | Status    | Name     | Size | Volume Type | Bootable | Attached to |
  +--------------------------------------+-----------+----------+------+-------------+----------+-------------+
  | 84534871-e327-46d4-af3f-ac347da69213 | attaching | fake_sys | 1    | -           | true     |             |
  +--------------------------------------+-----------+----------+------+-------------+----------+-------------+
  
  3)nova delete 256754e0-ed48-4acb-8293-4ac6cdae7384
  
  ()[root@busybox-openstack-556bf6df94-kkcr2 /]# nova list --all-t
  +----+------+-----------+--------+------------+-------------+----------+
  | ID | Name | Tenant ID | Status | Task State | Power State | Networks |
  +----+------+-----------+--------+------------+-------------+----------+
  +----+------+-----------+--------+------------+-------------+----------+
  ()[root@busybox-openstack-556bf6df94-kkcr2 /]# cinder list --all
  +--------------------------------------+----------------------------------+-----------+----------+------+-------------+----------+-------------+
  | ID                                   | Tenant ID                        | Status    | Name     | Size | Volume Type | Bootable | Attached to |
  +--------------------------------------+----------------------------------+-----------+----------+------+-------------+----------+-------------+
  | 84534871-e327-46d4-af3f-ac347da69213 | c7bc6a73869541c7a65b49b06fa9e21a | attaching | fake_sys | 1    | -           | true     |             |
  +--------------------------------------+----------------------------------+-----------+----------+------+-------------+----------+-------------+

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1905224

Title:
  bfv instance create failed due to insufficient resources, the volume
  is always attaching

Status in OpenStack Compute (nova):
  New

Bug description:
  When using a volume to create a instance, the instance failed and in
  ERROR state due to insufficient resources, and the volume is always in
  attaching state. I must reset volume to available status through
  cinder reset-state and continue use it.

  I am in pike version, i see https://review.opendev.org/#/c/335358/,
  nova-api reserve volume before attach volume to a instance. If
  scheduler failed for No ValidHost, the volume is always not available
  unless reset state. When delete the ERROR instance,the volume is still
  in attaching status.

  Now, we should restore the state of the volume instead of resetting it
  every time due to scheduling failure.

  Reproduce the problem through the following steps

  1)cinder create 1 --image-id 7b379b4c-ad6e-4ade-81f2-94a26e3afa3a
  --name fake_test

  ()[root@busybox-openstack-556bf6df94-kkcr2 /]# cinder list
  +--------------------------------------+-----------+-----------+------+-------------+----------+-------------+
  | ID                                   | Status    | Name      | Size | Volume Type | Bootable | Attached to |
  +--------------------------------------+-----------+-----------+------+-------------+----------+-------------+
  | 8ca2c77b-120a-41c1-b9f6-e8ac0f23a0f2 | available | fake_test | 1    | -           | true     |             |
  +--------------------------------------+-----------+-----------+------+-------------+----------+-------------+
  2)nova boot --boot-volume 8ca2c77b-120a-41c1-b9f6-e8ac0f23a0f2 --flavor 200 --nic net-name=share_net faker_vm

  ()[root@busybox-openstack-556bf6df94-kkcr2 /]# nova list --all-t --fields name,host,status,task_state
  +--------------------------------------+----------+------+--------+------------+
  | ID                                   | Name     | Host | Status | Task State |
  +--------------------------------------+----------+------+--------+------------+
  | 256754e0-ed48-4acb-8293-4ac6cdae7384 | faker_vm | None | ERROR  | None       |
  +--------------------------------------+----------+------+--------+------------+
  ()[root@busybox-openstack-556bf6df94-kkcr2 /]# cinder list
  +--------------------------------------+-----------+----------+------+-------------+----------+-------------+
  | ID                                   | Status    | Name     | Size | Volume Type | Bootable | Attached to |
  +--------------------------------------+-----------+----------+------+-------------+----------+-------------+
  | 84534871-e327-46d4-af3f-ac347da69213 | attaching | fake_sys | 1    | -           | true     |             |
  +--------------------------------------+-----------+----------+------+-------------+----------+-------------+

  3)nova delete 256754e0-ed48-4acb-8293-4ac6cdae7384

  ()[root@busybox-openstack-556bf6df94-kkcr2 /]# nova list --all-t
  +----+------+-----------+--------+------------+-------------+----------+
  | ID | Name | Tenant ID | Status | Task State | Power State | Networks |
  +----+------+-----------+--------+------------+-------------+----------+
  +----+------+-----------+--------+------------+-------------+----------+
  ()[root@busybox-openstack-556bf6df94-kkcr2 /]# cinder list --all
  +--------------------------------------+----------------------------------+-----------+----------+------+-------------+----------+-------------+
  | ID                                   | Tenant ID                        | Status    | Name     | Size | Volume Type | Bootable | Attached to |
  +--------------------------------------+----------------------------------+-----------+----------+------+-------------+----------+-------------+
  | 84534871-e327-46d4-af3f-ac347da69213 | c7bc6a73869541c7a65b49b06fa9e21a | attaching | fake_sys | 1    | -           | true     |             |
  +--------------------------------------+----------------------------------+-----------+----------+------+-------------+----------+-------------+

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1905224/+subscriptions