yahoo-eng-team team mailing list archive

[Thierry Carrez <thierry.carrez+lp@xxxxxxxxx>]

** Changed in: nova
       Status: Fix Committed => Fix Released

** Changed in: nova
    Milestone: None => grizzly-3

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1099729

Title:
  quota_usages in_use value change incorrectly when delete a resizing
  instance

Status in OpenStack Compute (Nova):
  Fix Released

Bug description:
  reproduce steps in devstack:
  1. create an instance with flavor m1.small using demo user
  2. resize it from m1.small to m1.tiny
  3. when the task_state of instance is 'resize_finish', delete it by demo user
  4. query the nova db like below:
  mysql> select * from quota_usages ;
  +---------------------+---------------------+------------+---------+----+----------------------------------+-----------+--------+----------+---------------+
  | created_at          | updated_at          | deleted_at | deleted | id | project_id                       | resource  | in_use | reserved | until_refresh |
  +---------------------+---------------------+------------+---------+----+----------------------------------+-----------+--------+----------+---------------+
  | 2013-01-08 04:43:21 | 2013-01-15 07:32:20 | NULL       |       0 |  1 | c0fb216d13cc46ca9abe7261ab9ab0d5 | instances |      0 |        0 |          NULL |
  | 2013-01-08 04:43:21 | 2013-01-15 07:32:20 | NULL       |       0 |  2 | c0fb216d13cc46ca9abe7261ab9ab0d5 | ram       |   1536 |        0 |          NULL |
  | 2013-01-08 04:43:21 | 2013-01-15 07:32:20 | NULL       |       0 |  3 | c0fb216d13cc46ca9abe7261ab9ab0d5 | cores     |      0 |        0 |          NULL |
  +---------------------+---------------------+------------+---------+----+----------------------------------+-----------+--------+----------+---------------+

  this is a race condition issue, so it is probabilistic to reproduce it, but you can add a time.sleep(10) to  nova/compute/manager.py:_finish_resize(), just like this:
   LOG.debug(_("-----------------------sleep 10 start-------------------------"))
   time.sleep(10)
   self.network_api.setup_networks_on_host(context, instance,
                                                  migration['dest_compute'])
  when you see the debug log during resize, you delete the instance, and this issue will occur almost every time.

  the reason is that, when we delete an instance, we use
  instance['memory_mb'], instance['vcpus'] to create quota's
  reservations, if the instance is resizing but not 'finished', the
  in_use value in quota_usages is NOT commited(2048M), but the
  instance['memory_mb'], instance['vcpus'] is updated(512M), so the
  deleting operation will commit a wrong reservations(-512M), and the
  in_use value changes to 2048-512=1536.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1099729/+subscriptions