yahoo-eng-team team mailing list archive

[José Miguel Guzmán <1687345@xxxxxxxxxxxxxxxxxx>]

Public bug reported:

Kolla (stable/ocata, 4.0.2) was deployed as multinode (virtualized on
Openstack/KVM) environment, but fails to provision instances.

First symptom was 
... Starting with 0 host(s) get_filtered_objects /var/lib/kolla/venv/local/lib/python2.7/site-packages/nova/filters.py:70
... Filtering removed all hosts for the request with instance ID '...'. Filter results: ['RetryFilter: (start: 0, end: 0)']
... NoValidHost: No valid host was found. There are not enough hosts available.

Placement API does not return any resource provider

GET /resource_providers
{
  "resource_providers": []
}

Following the request flow, I reach a point where I confirmed the
resource_providers table is empty.

root@pod09-control01:~# docker exec -ti mariadb mysql -unova_api -p ... nova_api
...
MariaDB [nova_api]> select * from resource_providers;
Empty set (0.00 sec)

The only unusual thing, is that initially I deployed with QEMU, as
compute node was going to be virtualized over KVM. Later, I enabled KVM
nested virtualization in the host, so, now compute is virtualization
capable.

(openstack) hypervisor show 1 -fjson
{
  "status": "enabled",
  "cpu_info": "{\"vendor\": \"Intel\", \"model\": \"Skylake-Client\", \"arch\": \"x86_64\", \"features\": [\"ssse3\", \"pge\", \"avx\", \"xsaveopt\", \"clflush\", \"sep\", \"rtm\", \"tsc_adjust\", \"vme\", \"invpcid\", \"tsc\", \"sse\", \"xsave\", \"smap\", \"vmx\", \"erms\", \"hle\", \"cmov\", \"smep\", \"fpu\", \"clflushopt\", \"pat\", \"osxsave\", \"lm\", \"msr\", \"adx\", \"3dnowprefetch\", \"nx\", \"fxsr\", \"syscall\", \"sse4.1\", \"pae\", \"sse4.2\", \"pclmuldq\", \"xgetbv1\", \"fma\", \"tsc-deadline\", \"mmx\", \"arat\", \"cx8\", \"mce\", \"de\", \"aes\", \"mca\", \"pse\", \"pni\", \"abm\", \"rdseed\", \"popcnt\", \"pdpe1gb\", \"apic\", \"fsgsbase\", \"f16c\", \"mpx\", \"xsavec\", \"lahf_lm\", \"rdtscp\", \"avx2\", \"sse2\", \"ss\", \"hypervisor\", \"bmi1\", \"bmi2\", \"pcid\", \"cx16\", \"pse36\", \"mtrr\", \"movbe\", \"rdrand\", \"x2apic\"], \"topology\": {\"cores\": 1, \"cells\": 1, \"threads\": 1, \"sockets\": 4}}",
  "free_disk_gb": 38,
  "memory_mb_used": 540,
  "uptime": "4:25",
  "host_time": "01:46:19",
  "local_gb_used": 5,
  "id": 1,
  "current_workload": 0,
  "state": "up",
  "load_average": "0.02, 0.02, 0.00",
  "users": "0",
  "aggregates": [],
  "host_ip": "10.0.0.12",
  "hypervisor_hostname": "pod9-compute01",
  "hypervisor_version": 2008000,
  "disk_available_least": 33,
  "local_gb": 38,
  "free_ram_mb": 7471,
  "vcpus_used": 0,
  "hypervisor_type": "QEMU",
  "memory_mb": 7983,
  "vcpus": 4,
  "running_vms": 0,
  "service_id": 6,
  "service_host": "pod9-compute01"
}

Any ideas, how can I make resource providers work?

** Affects: kolla
     Importance: Undecided
         Status: New

** Affects: nova
     Importance: Undecided
         Status: New


** Tags: nova-scheduler ocata placement

** Also affects: nova
   Importance: Undecided
       Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1687345

Title:
  NoValidHosts (resource_providers table is empty)

Status in kolla:
  New
Status in OpenStack Compute (nova):
  New

Bug description:
  Kolla (stable/ocata, 4.0.2) was deployed as multinode (virtualized on
  Openstack/KVM) environment, but fails to provision instances.

  First symptom was 
  ... Starting with 0 host(s) get_filtered_objects /var/lib/kolla/venv/local/lib/python2.7/site-packages/nova/filters.py:70
  ... Filtering removed all hosts for the request with instance ID '...'. Filter results: ['RetryFilter: (start: 0, end: 0)']
  ... NoValidHost: No valid host was found. There are not enough hosts available.

  Placement API does not return any resource provider

  GET /resource_providers
  {
    "resource_providers": []
  }

  Following the request flow, I reach a point where I confirmed the
  resource_providers table is empty.

  root@pod09-control01:~# docker exec -ti mariadb mysql -unova_api -p ... nova_api
  ...
  MariaDB [nova_api]> select * from resource_providers;
  Empty set (0.00 sec)

  The only unusual thing, is that initially I deployed with QEMU, as
  compute node was going to be virtualized over KVM. Later, I enabled
  KVM nested virtualization in the host, so, now compute is
  virtualization capable.

  (openstack) hypervisor show 1 -fjson
  {
    "status": "enabled",
    "cpu_info": "{\"vendor\": \"Intel\", \"model\": \"Skylake-Client\", \"arch\": \"x86_64\", \"features\": [\"ssse3\", \"pge\", \"avx\", \"xsaveopt\", \"clflush\", \"sep\", \"rtm\", \"tsc_adjust\", \"vme\", \"invpcid\", \"tsc\", \"sse\", \"xsave\", \"smap\", \"vmx\", \"erms\", \"hle\", \"cmov\", \"smep\", \"fpu\", \"clflushopt\", \"pat\", \"osxsave\", \"lm\", \"msr\", \"adx\", \"3dnowprefetch\", \"nx\", \"fxsr\", \"syscall\", \"sse4.1\", \"pae\", \"sse4.2\", \"pclmuldq\", \"xgetbv1\", \"fma\", \"tsc-deadline\", \"mmx\", \"arat\", \"cx8\", \"mce\", \"de\", \"aes\", \"mca\", \"pse\", \"pni\", \"abm\", \"rdseed\", \"popcnt\", \"pdpe1gb\", \"apic\", \"fsgsbase\", \"f16c\", \"mpx\", \"xsavec\", \"lahf_lm\", \"rdtscp\", \"avx2\", \"sse2\", \"ss\", \"hypervisor\", \"bmi1\", \"bmi2\", \"pcid\", \"cx16\", \"pse36\", \"mtrr\", \"movbe\", \"rdrand\", \"x2apic\"], \"topology\": {\"cores\": 1, \"cells\": 1, \"threads\": 1, \"sockets\": 4}}",
    "free_disk_gb": 38,
    "memory_mb_used": 540,
    "uptime": "4:25",
    "host_time": "01:46:19",
    "local_gb_used": 5,
    "id": 1,
    "current_workload": 0,
    "state": "up",
    "load_average": "0.02, 0.02, 0.00",
    "users": "0",
    "aggregates": [],
    "host_ip": "10.0.0.12",
    "hypervisor_hostname": "pod9-compute01",
    "hypervisor_version": 2008000,
    "disk_available_least": 33,
    "local_gb": 38,
    "free_ram_mb": 7471,
    "vcpus_used": 0,
    "hypervisor_type": "QEMU",
    "memory_mb": 7983,
    "vcpus": 4,
    "running_vms": 0,
    "service_id": 6,
    "service_host": "pod9-compute01"
  }

  Any ideas, how can I make resource providers work?

To manage notifications about this bug go to:
https://bugs.launchpad.net/kolla/+bug/1687345/+subscriptions