yahoo-eng-team team mailing list archive

[Joe Talerico <jtaleric@xxxxxxxxxx>]

Public bug reported:

To recreate  :
Run :  nova boot --image <image> --flavor m1.small --num-instances 40 test
Then: for id in `nova list | grep "test" | awk '{print $2}'`; do nova delete $id & done

Guest is removed from the hypervisor node:
[root@pcloud16 ~]# virsh list --all
 Id    Name                           State
----------------------------------------------------

[root@pcloud16 ~]#

However, nova list, shows a guest being deleted:
+--------------------------------------+-------------------------------------------+--------+------------+-------------+------------------+
| 2917e808-2de0-4266-9de5-3fd5d171eb46 | test-2917e808-2de0-4266-9de5-3fd5d171eb46 | BUILD  | deleting   | NOSTATE     | 10Net-3=10.0.3.2 |
+--------------------------------------+-------------------------------------------+--------+------------+-------------+------------------+

So I try to reset the state:
[root@pcloud18 ~(keystone_admin)]# nova reset-state --active 2917e808-2de0-4266-9de5-3fd5d171eb46
[root@pcloud18 ~(keystone_admin)]# nova list
+--------------------------------------+-------------------------------------------+--------+------------+-------------+------------------+
| ID                                   | Name                                      | Status | Task State | Power State | Networks         |
+--------------------------------------+-------------------------------------------+--------+------------+-------------+------------------+
| 2917e808-2de0-4266-9de5-3fd5d171eb46 | test-2917e808-2de0-4266-9de5-3fd5d171eb46 | ACTIVE | None       | NOSTATE     | 10Net-3=10.0.3.2 |
+--------------------------------------+-------------------------------------------+--------+------------+-------------+------------------+
[root@pcloud18 ~(keystone_admin)]# nova list
+--------------------------------------+-------------------------------------------+--------+------------+-------------+------------------+
| ID                                   | Name                                      | Status | Task State | Power State | Networks         |
+--------------------------------------+-------------------------------------------+--------+------------+-------------+------------------+
| 2917e808-2de0-4266-9de5-3fd5d171eb46 | test-2917e808-2de0-4266-9de5-3fd5d171eb46 | ACTIVE | deleting   | NOSTATE     | 10Net-3=10.0.3.2 |
+--------------------------------------+-------------------------------------------+--------+------------+-------------+------------------+

reap doesn't seem to clean things up either: 
Before resetting the state: 
2014-03-06 06:33:00.481 11409 WARNING nova.compute.manager [req-20be6b2d-7d78-40ee-bab4-8fcd61f67290 None None] Found 1 in the database and 0 on the hypervisor.
2014-03-06 06:33:00.482 11409 INFO nova.compute.manager [req-20be6b2d-7d78-40ee-bab4-8fcd61f67290 None None] [instance: 2917e808-2de0-4266-9de5-3fd5d171eb46] During sync_power_state the instance has a pending task. Skip.

When I reset the state: 
2014-03-06 06:43:02.985 11409 WARNING nova.compute.manager [-] Found 1 in the database and 0 on the hypervisor.
2014-03-06 06:43:03.090 11409 WARNING nova.compute.manager [-] [instance: 2917e808-2de0-4266-9de5-3fd5d171eb46] Instance is unexpectedly not found. Ignore.

** Affects: nova
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1288958

Title:
  nova out of sync with the hypervisor

Status in OpenStack Compute (Nova):
  New

Bug description:
  To recreate  :
  Run :  nova boot --image <image> --flavor m1.small --num-instances 40 test
  Then: for id in `nova list | grep "test" | awk '{print $2}'`; do nova delete $id & done

  Guest is removed from the hypervisor node:
  [root@pcloud16 ~]# virsh list --all
   Id    Name                           State
  ----------------------------------------------------

  [root@pcloud16 ~]#

  However, nova list, shows a guest being deleted:
  +--------------------------------------+-------------------------------------------+--------+------------+-------------+------------------+
  | 2917e808-2de0-4266-9de5-3fd5d171eb46 | test-2917e808-2de0-4266-9de5-3fd5d171eb46 | BUILD  | deleting   | NOSTATE     | 10Net-3=10.0.3.2 |
  +--------------------------------------+-------------------------------------------+--------+------------+-------------+------------------+

  So I try to reset the state:
  [root@pcloud18 ~(keystone_admin)]# nova reset-state --active 2917e808-2de0-4266-9de5-3fd5d171eb46
  [root@pcloud18 ~(keystone_admin)]# nova list
  +--------------------------------------+-------------------------------------------+--------+------------+-------------+------------------+
  | ID                                   | Name                                      | Status | Task State | Power State | Networks         |
  +--------------------------------------+-------------------------------------------+--------+------------+-------------+------------------+
  | 2917e808-2de0-4266-9de5-3fd5d171eb46 | test-2917e808-2de0-4266-9de5-3fd5d171eb46 | ACTIVE | None       | NOSTATE     | 10Net-3=10.0.3.2 |
  +--------------------------------------+-------------------------------------------+--------+------------+-------------+------------------+
  [root@pcloud18 ~(keystone_admin)]# nova list
  +--------------------------------------+-------------------------------------------+--------+------------+-------------+------------------+
  | ID                                   | Name                                      | Status | Task State | Power State | Networks         |
  +--------------------------------------+-------------------------------------------+--------+------------+-------------+------------------+
  | 2917e808-2de0-4266-9de5-3fd5d171eb46 | test-2917e808-2de0-4266-9de5-3fd5d171eb46 | ACTIVE | deleting   | NOSTATE     | 10Net-3=10.0.3.2 |
  +--------------------------------------+-------------------------------------------+--------+------------+-------------+------------------+

  reap doesn't seem to clean things up either: 
  Before resetting the state: 
  2014-03-06 06:33:00.481 11409 WARNING nova.compute.manager [req-20be6b2d-7d78-40ee-bab4-8fcd61f67290 None None] Found 1 in the database and 0 on the hypervisor.
  2014-03-06 06:33:00.482 11409 INFO nova.compute.manager [req-20be6b2d-7d78-40ee-bab4-8fcd61f67290 None None] [instance: 2917e808-2de0-4266-9de5-3fd5d171eb46] During sync_power_state the instance has a pending task. Skip.

  When I reset the state: 
  2014-03-06 06:43:02.985 11409 WARNING nova.compute.manager [-] Found 1 in the database and 0 on the hypervisor.
  2014-03-06 06:43:03.090 11409 WARNING nova.compute.manager [-] [instance: 2917e808-2de0-4266-9de5-3fd5d171eb46] Instance is unexpectedly not found. Ignore.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1288958/+subscriptions