yahoo-eng-team team mailing list archive

[OpenStack Infra <1793747@xxxxxxxxxxxxxxxxxx>]

Reviewed:  https://review.openstack.org/604898
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=c9448cbdbf96e7436b13ac5c3a92addfc0f2f5a2
Submitter: Zuul
Branch:    master

commit c9448cbdbf96e7436b13ac5c3a92addfc0f2f5a2
Author: Dan Smith <dansmith@xxxxxxxxxx>
Date:   Mon Sep 24 13:28:29 2018 -0700

    Revert "Make host_aggregate_map dictionary case-insensitive"
    
    This reverts commit 0dc0db932e3ad5ad911f2072015cb9854f6e4e23.
    
    The original change caused our host state processing to be inconsistent
    with our own hypervisors API. Automation tooling that used our API
    to add hosts to aggregates would fail silently. We are reverting
    this and will propose a check on the aggregate host add action
    which will confirm the case-sensitive mapping of the host being
    added, which is what we should have done in the first place.
    
    Change-Id: Ibd44ba9de5680958f55f0ae6325cfc33dabadc4c
    Closes-Bug: #1793747


** Changed in: nova
       Status: In Progress => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1793747

Title:
  Fails to boot instance using Blazar flavor if compute host names are
  in uppercase

Status in OpenStack Compute (nova):
  Fix Released
Status in OpenStack Compute (nova) pike series:
  In Progress
Status in OpenStack Compute (nova) queens series:
  In Progress
Status in OpenStack Compute (nova) rocky series:
  In Progress

Bug description:
  Bug Description:
  ================

  Steps to reproduce:
  ====================

  $ nova hypervisor-list
  +--------------------------------------+----------------------+--------------------+----------------------+
  | ID                                   | Hypervisor hostname  | State              | Status               |
  +--------------------------------------+----------------------+--------------------+----------------------+
  | e517e75b-d57c-45b2-af41-6be2fed536c6 | Openstack-VirtualBox | up                 | enabled              |
  +--------------------------------------+----------------------+--------------------+----------------------+

  Step 1: Add host into freepool
  $ blazar host-create Openstack-VirtualBox

  $ blazar host-list
  +----+---------------------+-------+-----------+----------+
  | id | hypervisor_hostname | vcpus | memory_mb | local_gb |          
  +----+---------------------+-------+-----------+----------+
  | 1  | Openstack-VirtualBox|     4 |     11941 |       91 |             
  +----+---------------------+-------+-----------+----------+

  
  Step 2: Create a lease
  $ blazar lease-create --reservation resource_type=virtual:instance,vcpus=1,memory_mb=1024,disk_gb=20,amount=1,affinity=False --start-date "2018-08-27 12:59" --end-date "2018-08-27 13:55" lease-1

  nova_api database entries related to aggregates:

  Blazar creates aggregate with id=18
  mysql> select * from aggregates;
  +---------------------+------------+----+--------------------------------------+--------------------------------------+
  | created_at          | updated_at | id | uuid                                 | name                                 |
  +---------------------+------------+----+--------------------------------------+--------------------------------------+
  | 2018-08-13 06:49:37 | NULL       |  1 | 2a7d838f-4e42-48af-a9a9-faf3f29e3c96 | freepool                             |
  | 2018-08-29 13:43:15 | NULL       | 18 | 88c37cc5-373a-4da5-820f-508d25f00903 | 8c85522a-cc39-4a0d-a5ca-4de92e4d2c1f |
  +---------------------+------------+----+--------------------------------------+--------------------------------------+

  Blazar adds aggregate with aggregate_id=18, to host at the time of
  lease-start event.

  mysql> select * from aggregate_hosts;
  +---------------------+------------+----+-----------------------+--------------+
  | created_at          | updated_at | id | host                  | aggregate_id |
  +---------------------+------------+----+-----------------------+--------------+
  | 2018-08-29 13:34:46 | NULL       | 32 | Openstack-VirtualBox  |            1 |
  | 2018-08-29 13:39:05 | NULL       | 34 | Openstack-VirtualBox  |           18 |
  +---------------------+------------+----+-----------------------+--------------+

  Step 3: Create a server: Please specify the flavor of the reservation and group_id as a scheduler hint.
  $ openstack server create --flavor 03067174-2a5e-43f7-baf7-037aac23b4ef --image cirros-0.4.0-x86_64-disk --network 42d6f419-b445-40a6-b542-e8a502c6ae64 --hint group=09389292-6639-48d6-9709-045061f42ebf instance-1

  For more details regarding instance reservation please refer:
  https://docs.openstack.org/blazar/latest/cli/instance-reservation.html

  Logs
  =====

  Service logs of n-sch:
  =======================
  Aug 27 13:00:31 Openstack-VirtualBox nova-scheduler[22177]: INFO nova.filters [None req-c1891de9-d8b9-4d3f-ae78-9b8ad848a5ec admin admin] Filter AggregateInstanceExtraSpecsFilter returned 0 hosts Aug 27 13:00:31 Openstack-VirtualBox nova-scheduler[22177]: DEBUG nova.filters [None req-c1891de9-d8b9-4d3f-ae78-9b8ad848a5ec admin admin] Filtering removed all hosts for the request with instance ID '7e5aef57-fec6-4e25-8219-0e01aab4b9f3'. Filter results: [('RetryFilter', [(u'Openstack-VirtualBox', u'Openstack-VirtualBox')]), ('AvailabilityZoneFilter', [(u'Openstack-VirtualBox', u'Openstack-VirtualBox')]), ('ComputeFilter', [(u'Openstack-VirtualBox', u'Openstack-VirtualBox')]), ('ComputeCapabilitiesFilter', [(u'Openstack-VirtualBox', u'Openstack-VirtualBox')]), ('ImagePropertiesFilter', [(u'Openstack-VirtualBox', u'Openstack-VirtualBox')]), ('ServerGroupAntiAffinityFilter', [(u'Openstack-VirtualBox', u'Openstack-VirtualBox')]), ('ServerGroupAffinityFilter', [(u'Openstack-VirtualBox', u'Openstack-VirtualBox')]), ('SameHostFilter', [(u'Openstack-VirtualBox', u'Openstack-VirtualBox')]), ('DifferentHostFilter', [(u'Openstack-VirtualBox', u'Openstack-VirtualBox')]), ('AggregateInstanceExtraSpecsFilter', None)] {{(pid=22198) get_filtered_objects /opt/stack/nova/nova/filters.py:129}}
  Aug 27 13:00:31 Openstack-VirtualBox nova-scheduler[22177]: INFO nova.filters [None req-c1891de9-d8b9-4d3f-ae78-9b8ad848a5ec admin admin] Filtering removed all hosts for the request with instance ID '7e5aef57-fec6-4e25-8219-0e01aab4b9f3'. Filter results: ['RetryFilter: (start: 1, end: 1)', 'AvailabilityZoneFilter: (start: 1, end: 1)', 'ComputeFilter: (start: 1, end: 1)', 'ComputeCapabilitiesFilter: (start: 1, end: 1)', 'ImagePropertiesFilter: (start: 1, end: 1)', 'ServerGroupAntiAffinityFilter: (start: 1, end: 1)', 'ServerGroupAffinityFilter: (start: 1, end: 1)', 'SameHostFilter: (start: 1, end: 1)', 'DifferentHostFilter: (start: 1, end: 1)', 'AggregateInstanceExtraSpecsFilter: (start: 1, end: 0)']

  Service logs of n-super-cond:
  ===============================
  Aug 27 13:00:31 Openstack-VirtualBox nova-conductor[22131]: WARNING nova.scheduler.utils [None req-c1891de9-d8b9-4d3f-ae78-9b8ad848a5ec admin admin] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available.
  Aug 27 13:00:31 Openstack-VirtualBox nova-conductor[22131]: Traceback (most recent call last):
  Aug 27 13:00:31 Openstack-VirtualBox nova-conductor[22131]:   File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/server.py", line 226, in inner
  Aug 27 13:00:31 Openstack-VirtualBox nova-conductor[22131]:     return func(*args, **kwargs)
  Aug 27 13:00:31 Openstack-VirtualBox nova-conductor[22131]:   File "/opt/stack/nova/nova/scheduler/manager.py", line 169, in select_destinations
  Aug 27 13:00:31 Openstack-VirtualBox nova-conductor[22131]:     allocation_request_version, return_alternates)
  Aug 27 13:00:31 Openstack-VirtualBox nova-conductor[22131]:   File "/opt/stack/nova/nova/scheduler/filter_scheduler.py", line 91, in select_destinations
  Aug 27 13:00:31 Openstack-VirtualBox nova-conductor[22131]:     allocation_request_version, return_alternates)
  Aug 27 13:00:31 Openstack-VirtualBox nova-conductor[22131]:   File "/opt/stack/nova/nova/scheduler/filter_scheduler.py", line 256, in _schedule
  Aug 27 13:00:31 Openstack-VirtualBox nova-conductor[22131]:     claimed_instance_uuids)
  Aug 27 13:00:31 Openstack-VirtualBox nova-conductor[22131]:   File "/opt/stack/nova/nova/scheduler/filter_scheduler.py", line 293, in _ensure_sufficient_hosts
  Aug 27 13:00:31 Openstack-VirtualBox nova-conductor[22131]:     raise exception.NoValidHost(reason=reason)
  Aug 27 13:00:31 Openstack-VirtualBox nova-conductor[22131]: NoValidHost: No valid host was found. There are not enough hosts available.
  Aug 27 13:00:31 Openstack-VirtualBox nova-conductor[22131]: : NoValidHost_Remote: No valid host was found. There are not enough hosts available.
  Aug 27 13:00:31 Openstack-VirtualBox nova-conductor[22131]: WARNING nova.scheduler.utils [None req-c1891de9-d8b9-4d3f-ae78-9b8ad848a5ec admin admin] [instance: 7e5aef57-fec6-4e25-8219-0e01aab4b9f3] Setting instance to ERROR state.: NoValidHost_Remote: No valid host was found. There are not enough hosts available.

  
  'AggregateInstanceExtraSpecsFilter' returns 0 hosts because at[1] host_state.aggregates is not returning aggregate with metadata created by Blazar. Lately, this patch [2] was merged in which HostManager._get_aggregates_info method, the host is converted into lower case and then it finds aggregates for that host in host_aggregates_map. But if the compute host name is in uppercase, it doesn’t add the host in host_aggregates_map in lowercase in _update_aggregates method hence _get_aggregates_info method doesn’t return the aggregate associated with the requested host.

  
  Expected Result: Instance should be booted on the host "Openstack-VirtualBox"
  Actual Result: NoValidHost_Remote: No valid host was found

  
  [1] : https://github.com/openstack/nova/blob/master/nova/scheduler/filters/utils.py#L38
  [2] : https://review.openstack.org/#/c/498334

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1793747/+subscriptions