yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1558532] [NEW] AntiAffinity policy problems with resize and migrate

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Zengzhi Wang <wang.zengzhi@xxxxxxxxxx>
Date: Thu, 17 Mar 2016 12:39:24 -0000
Reply-to: Bug 1558532 <1558532@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
Public bug reported:

1. version
kilo 2015.1.0

2.Relevant log files:
nova-scheduler.log
（Log contains “========”  is added by myself）

2016-03-17 17:54:23.092 15208 DEBUG nova.scheduler.filters.affinity_filter [req-3612987d-bef8-4727-811f-1dd668ac3328 eae626536b034af1a3635a36645dc54a 4d054614f49644c1935bdade9703c99b - - -] ================group_hosts: '[u'SBCJNailSlot3', u'sbcj-chenling-02-slot11']' host_passes /usr/lib/python2.7/site-packages/nova/scheduler/filters/affinity_filter.py:98
2016-03-17 17:54:23.093 15208 DEBUG nova.scheduler.filters.affinity_filter [req-3612987d-bef8-4727-811f-1dd668ac3328 eae626536b034af1a3635a36645dc54a 4d054614f49644c1935bdade9703c99b - - -] Group anti affinity: check if sbcj-chenling-02-slot11 not in [u'SBCJNailSlot3', u'sbcj-chenling-02-slot11'] host_passes /usr/lib/python2.7/site-packages/nova/scheduler/filters/affinity_filter.py:101
2016-03-17 17:54:23.094 15208 DEBUG nova.scheduler.filters.affinity_filter [req-3612987d-bef8-4727-811f-1dd668ac3328 eae626536b034af1a3635a36645dc54a 4d054614f49644c1935bdade9703c99b - - -] ================group_hosts: '[u'SBCJNailSlot3', u'sbcj-chenling-02-slot11']' host_passes /usr/lib/python2.7/site-packages/nova/scheduler/filters/affinity_filter.py:98
2016-03-17 17:54:23.094 15208 DEBUG nova.scheduler.filters.affinity_filter [req-3612987d-bef8-4727-811f-1dd668ac3328 eae626536b034af1a3635a36645dc54a 4d054614f49644c1935bdade9703c99b - - -] Group anti affinity: check if SBCJNailSlot3 not in [u'SBCJNailSlot3', u'sbcj-chenling-02-slot11'] host_passes /usr/lib/python2.7/site-packages/nova/scheduler/filters/affinity_filter.py:101
2016-03-17 17:54:23.094 15208 INFO nova.filters [req-3612987d-bef8-4727-811f-1dd668ac3328 eae626536b034af1a3635a36645dc54a 4d054614f49644c1935bdade9703c99b - - -] Filter ServerGroupAntiAffinityFilter returned 0 hosts for instance c48ee7d6-451e-4b96-ac39-261082f8950e
2016-03-17 17:54:23.095 15208 DEBUG nova.scheduler.filter_scheduler [req-3612987d-bef8-4727-811f-1dd668ac3328 eae626536b034af1a3635a36645dc54a 4d054614f49644c1935bdade9703c99b - - -] There are 0 hosts available but 1 instances requested to build. select_destinations /usr/lib/python2.7/site-packages/nova/scheduler/filter_scheduler.py:77

3.Reproduce steps:
*3.1
create a anti-affinity group with policy 'anti-affinity' named "antiaffinitygroup"
boot two vms using "antiaffinitygroup",named "test_wzz1" and "test_wzz2"

results:
[root@sbcj-chenling-02-slot11 ~(keystone_admin)]# nova list
+--------------------------------------+-----------+--------+------------+-------------+-------------------+
| ID                                   | Name      | Status | Task State | Power State | Networks          |
+--------------------------------------+-----------+--------+------------+-------------+-------------------+
| c48ee7d6-451e-4b96-ac39-261082f8950e | test_wzz1 | ACTIVE | -          | Running     | test0301=3.3.3.60 |
| 55107de8-38bc-4b3a-8681-df17746e4f11 | test_wzz2 | ACTIVE | -          | Running     | test0301=3.3.3.61 |
+--------------------------------------+-----------+--------+------------+-------------+-------------------+

[root@sbcj-chenling-02-slot11 ~(keystone_admin)]# nova server-group-list
+--------------------------------------+-------------------+--------------------+------------------------------------------------------------------------------------+----------+
| Id                                   | Name              | Policies           | Members                                                                            | Metadata |
+--------------------------------------+-------------------+--------------------+------------------------------------------------------------------------------------+----------+
| 79ebfbfc-6e7c-4bf0-97c5-14d7fca22e96 | antiaffinitygroup | [u'anti-affinity'] | [u'55107de8-38bc-4b3a-8681-df17746e4f11', u'c48ee7d6-451e-4b96-ac39-261082f8950e'] | {}       |
+--------------------------------------+-------------------+--------------------+------------------------------------------------------------------------------------+----------+

*3.2
I have two compute nodes：sbcj-chenling-02-slot11，SBCJNailSlot3.
The vm test_wzz1 in SBCJNailSlot3,and the test_wzz2 in sbcj-chenling-02-slot11,because of the anti-affinity group
 Then set allow_resize_to_same_host=true,and add ServerGroupAntiAffinityFilter in scheduler_default_filters in nova.conf,and restart services.

*3.3
 now，execute the command : nova resize test_wzz1 m1.small

Example：
[root@sbcj-chenling-02-slot11 ~(keystone_admin)]# nova resize test_wzz1 m1.small

Expected result:
vm resize to the same host successfully.

Actual result:
ERROR (BadRequest): No valid host was found. No valid host found for resize: No valid host was found. There are not enough hosts available. (HTTP 400) (Request-ID: req-3612987d-bef8-4727-811f-1dd668ac3328)
Relevant log files: See section 2

4.Bonus points for reproducing shell script / test:
Operation cold-migration has the same problem.
It seems  test_wzz1 anti-affinity against to itself,so resize failed although we allow vm resize to same host.
This is unreasonable.

** Affects: nova
     Importance: Undecided
     Assignee: Zengzhi Wang (wang-zengzhi)
         Status: New

** Changed in: nova
     Assignee: (unassigned) => Zengzhi Wang (wang-zengzhi)

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1558532

Title:
  AntiAffinity policy problems with resize and migrate

Status in OpenStack Compute (nova):
  New

Bug description:
  1. version
  kilo 2015.1.0

  2.Relevant log files:
  nova-scheduler.log
  （Log contains “========”  is added by myself）

  2016-03-17 17:54:23.092 15208 DEBUG nova.scheduler.filters.affinity_filter [req-3612987d-bef8-4727-811f-1dd668ac3328 eae626536b034af1a3635a36645dc54a 4d054614f49644c1935bdade9703c99b - - -] ================group_hosts: '[u'SBCJNailSlot3', u'sbcj-chenling-02-slot11']' host_passes /usr/lib/python2.7/site-packages/nova/scheduler/filters/affinity_filter.py:98
  2016-03-17 17:54:23.093 15208 DEBUG nova.scheduler.filters.affinity_filter [req-3612987d-bef8-4727-811f-1dd668ac3328 eae626536b034af1a3635a36645dc54a 4d054614f49644c1935bdade9703c99b - - -] Group anti affinity: check if sbcj-chenling-02-slot11 not in [u'SBCJNailSlot3', u'sbcj-chenling-02-slot11'] host_passes /usr/lib/python2.7/site-packages/nova/scheduler/filters/affinity_filter.py:101
  2016-03-17 17:54:23.094 15208 DEBUG nova.scheduler.filters.affinity_filter [req-3612987d-bef8-4727-811f-1dd668ac3328 eae626536b034af1a3635a36645dc54a 4d054614f49644c1935bdade9703c99b - - -] ================group_hosts: '[u'SBCJNailSlot3', u'sbcj-chenling-02-slot11']' host_passes /usr/lib/python2.7/site-packages/nova/scheduler/filters/affinity_filter.py:98
  2016-03-17 17:54:23.094 15208 DEBUG nova.scheduler.filters.affinity_filter [req-3612987d-bef8-4727-811f-1dd668ac3328 eae626536b034af1a3635a36645dc54a 4d054614f49644c1935bdade9703c99b - - -] Group anti affinity: check if SBCJNailSlot3 not in [u'SBCJNailSlot3', u'sbcj-chenling-02-slot11'] host_passes /usr/lib/python2.7/site-packages/nova/scheduler/filters/affinity_filter.py:101
  2016-03-17 17:54:23.094 15208 INFO nova.filters [req-3612987d-bef8-4727-811f-1dd668ac3328 eae626536b034af1a3635a36645dc54a 4d054614f49644c1935bdade9703c99b - - -] Filter ServerGroupAntiAffinityFilter returned 0 hosts for instance c48ee7d6-451e-4b96-ac39-261082f8950e
  2016-03-17 17:54:23.095 15208 DEBUG nova.scheduler.filter_scheduler [req-3612987d-bef8-4727-811f-1dd668ac3328 eae626536b034af1a3635a36645dc54a 4d054614f49644c1935bdade9703c99b - - -] There are 0 hosts available but 1 instances requested to build. select_destinations /usr/lib/python2.7/site-packages/nova/scheduler/filter_scheduler.py:77

  3.Reproduce steps:
  *3.1
  create a anti-affinity group with policy 'anti-affinity' named "antiaffinitygroup"
  boot two vms using "antiaffinitygroup",named "test_wzz1" and "test_wzz2"

  results:
  [root@sbcj-chenling-02-slot11 ~(keystone_admin)]# nova list
  +--------------------------------------+-----------+--------+------------+-------------+-------------------+
  | ID                                   | Name      | Status | Task State | Power State | Networks          |
  +--------------------------------------+-----------+--------+------------+-------------+-------------------+
  | c48ee7d6-451e-4b96-ac39-261082f8950e | test_wzz1 | ACTIVE | -          | Running     | test0301=3.3.3.60 |
  | 55107de8-38bc-4b3a-8681-df17746e4f11 | test_wzz2 | ACTIVE | -          | Running     | test0301=3.3.3.61 |
  +--------------------------------------+-----------+--------+------------+-------------+-------------------+

  [root@sbcj-chenling-02-slot11 ~(keystone_admin)]# nova server-group-list
  +--------------------------------------+-------------------+--------------------+------------------------------------------------------------------------------------+----------+
  | Id                                   | Name              | Policies           | Members                                                                            | Metadata |
  +--------------------------------------+-------------------+--------------------+------------------------------------------------------------------------------------+----------+
  | 79ebfbfc-6e7c-4bf0-97c5-14d7fca22e96 | antiaffinitygroup | [u'anti-affinity'] | [u'55107de8-38bc-4b3a-8681-df17746e4f11', u'c48ee7d6-451e-4b96-ac39-261082f8950e'] | {}       |
  +--------------------------------------+-------------------+--------------------+------------------------------------------------------------------------------------+----------+

  *3.2
  I have two compute nodes：sbcj-chenling-02-slot11，SBCJNailSlot3.
  The vm test_wzz1 in SBCJNailSlot3,and the test_wzz2 in sbcj-chenling-02-slot11,because of the anti-affinity group
   Then set allow_resize_to_same_host=true,and add ServerGroupAntiAffinityFilter in scheduler_default_filters in nova.conf,and restart services.

  *3.3
   now，execute the command : nova resize test_wzz1 m1.small

  Example：
  [root@sbcj-chenling-02-slot11 ~(keystone_admin)]# nova resize test_wzz1 m1.small

  Expected result:
  vm resize to the same host successfully.

  Actual result:
  ERROR (BadRequest): No valid host was found. No valid host found for resize: No valid host was found. There are not enough hosts available. (HTTP 400) (Request-ID: req-3612987d-bef8-4727-811f-1dd668ac3328)
  Relevant log files: See section 2

  4.Bonus points for reproducing shell script / test:
  Operation cold-migration has the same problem.
  It seems  test_wzz1 anti-affinity against to itself,so resize failed although we allow vm resize to same host.
  This is unreasonable.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1558532/+subscriptions
Follow ups

[Bug 1558532] Re: AntiAffinity policy problems with resize and migrate to same host
From: OpenStack Infra, 2016-05-23