yahoo-eng-team team mailing list archive
-
yahoo-eng-team team
-
Mailing list archive
-
Message #49764
[Bug 1572555] [NEW] Nova reports memory_mb=0 for available Ironic node
Public bug reported:
this is on latest devstack master and might be related to bug 1572472
Reproduce
1. deploy Ironic+Nova in DevStack as usual, 3 VMs x 1cpu,1024MB RAM,10GB disk posing as Ironic nodes
ironic node-list
+--------------------------------------+--------+---------------+-------------+--------------------+-------------+
| UUID | Name | Instance UUID | Power State | Provisioning State | Maintenance |
+--------------------------------------+--------+---------------+-------------+--------------------+-------------+
| bb785191-e548-4a6d-820e-bf2c5cdba922 | node-0 | None | power off | available | False |
| 2a508e69-08ab-4b0e-abaa-39e4e268bd47 | node-1 | None | power off | available | False |
| 9e0763d9-6f96-4327-a189-3bf12f5856ac | node-2 | None | power off | available | False |
+--------------------------------------+--------+---------------+-------------+--------------------+-------------+
2. check nova hypervisor-stats and nova hypervisor-show <node-uuid>
for all hypervisors nova reports memory_mb=1024, local_gb=10
nova hypervisor-stats
+----------------------+-------+
| Property | Value |
+----------------------+-------+
| count | 3 |
| current_workload | 0 |
| disk_available_least | 30 |
| free_disk_gb | 30 |
| free_ram_mb | 3072 |
| local_gb | 30 |
| local_gb_used | 0 |
| memory_mb | 3072 |
| memory_mb_used | 0 |
| running_vms | 0 |
| vcpus | 3 |
| vcpus_used | 0 |
+----------------------+-------+
3. put two ironic nodes into maintenance
ironic node-set-maintenance node-1 on
ironic node-set-maintenance node-2 on
ironic node-list
+--------------------------------------+--------+---------------+-------------+--------------------+-------------+
| UUID | Name | Instance UUID | Power State | Provisioning State | Maintenance |
+--------------------------------------+--------+---------------+-------------+--------------------+-------------+
| bb785191-e548-4a6d-820e-bf2c5cdba922 | node-0 | None | power off | available | False |
| 2a508e69-08ab-4b0e-abaa-39e4e268bd47 | node-1 | None | power off | available | True |
| 9e0763d9-6f96-4327-a189-3bf12f5856ac | node-2 | None | power off | available | True |
+--------------------------------------+--------+---------------+-------------+--------------------+-------------+
4. wait for nova hypervisor-stats to be updated
Expected result:
total memory_mb is 1024, total local_gb is 10
Actual result:
total memory_mb is 0, total local_gb is 0
nova hypervisor-stats
+----------------------+-------+
| Property | Value |
+----------------------+-------+
| count | 3 |
| current_workload | 0 |
| disk_available_least | 10 |
| free_disk_gb | 10 |
| free_ram_mb | 1024 |
| local_gb | 0 |
| local_gb_used | 0 |
| memory_mb | 0 |
| memory_mb_used | 0 |
| running_vms | 0 |
| vcpus | 0 |
| vcpus_used | 0 |
+----------------------+-------+
also hypervisor-show shows these values as 0:
nova hypervisor-show bb785191-e548-4a6d-820e-bf2c5cdba922
+-------------------------+--------------------------------------+
| Property | Value |
+-------------------------+--------------------------------------+
| cpu_info | |
| current_workload | 0 |
| disk_available_least | 10 |
| free_disk_gb | 10 |
| free_ram_mb | 1024 |
| host_ip | 192.168.100.12 |
| hypervisor_hostname | bb785191-e548-4a6d-820e-bf2c5cdba922 |
| hypervisor_type | ironic |
| hypervisor_version | 1 |
| id | 1 |
| local_gb | 0 |
| local_gb_used | 0 |
| memory_mb | 0 |
| memory_mb_used | 0 |
| running_vms | 0 |
| service_disabled_reason | None |
| service_host | ironic |
| service_id | 8 |
| state | up |
| status | enabled |
| vcpus | 0 |
| vcpus_used | 0 |
+-------------------------+--------------------------------------+
Also, when "watch"ing hypervisor-stats I had a glimpse at correct values
that immediately jumped to zero on next query.
Due to this bug the instance is failing to be scheduled to the actually
available Ironic node due to Nova scheduler's RamFilter removing all hosts
** Affects: nova
Importance: Undecided
Status: New
** Tags: ironic
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1572555
Title:
Nova reports memory_mb=0 for available Ironic node
Status in OpenStack Compute (nova):
New
Bug description:
this is on latest devstack master and might be related to bug 1572472
Reproduce
1. deploy Ironic+Nova in DevStack as usual, 3 VMs x 1cpu,1024MB RAM,10GB disk posing as Ironic nodes
ironic node-list
+--------------------------------------+--------+---------------+-------------+--------------------+-------------+
| UUID | Name | Instance UUID | Power State | Provisioning State | Maintenance |
+--------------------------------------+--------+---------------+-------------+--------------------+-------------+
| bb785191-e548-4a6d-820e-bf2c5cdba922 | node-0 | None | power off | available | False |
| 2a508e69-08ab-4b0e-abaa-39e4e268bd47 | node-1 | None | power off | available | False |
| 9e0763d9-6f96-4327-a189-3bf12f5856ac | node-2 | None | power off | available | False |
+--------------------------------------+--------+---------------+-------------+--------------------+-------------+
2. check nova hypervisor-stats and nova hypervisor-show <node-uuid>
for all hypervisors nova reports memory_mb=1024, local_gb=10
nova hypervisor-stats
+----------------------+-------+
| Property | Value |
+----------------------+-------+
| count | 3 |
| current_workload | 0 |
| disk_available_least | 30 |
| free_disk_gb | 30 |
| free_ram_mb | 3072 |
| local_gb | 30 |
| local_gb_used | 0 |
| memory_mb | 3072 |
| memory_mb_used | 0 |
| running_vms | 0 |
| vcpus | 3 |
| vcpus_used | 0 |
+----------------------+-------+
3. put two ironic nodes into maintenance
ironic node-set-maintenance node-1 on
ironic node-set-maintenance node-2 on
ironic node-list
+--------------------------------------+--------+---------------+-------------+--------------------+-------------+
| UUID | Name | Instance UUID | Power State | Provisioning State | Maintenance |
+--------------------------------------+--------+---------------+-------------+--------------------+-------------+
| bb785191-e548-4a6d-820e-bf2c5cdba922 | node-0 | None | power off | available | False |
| 2a508e69-08ab-4b0e-abaa-39e4e268bd47 | node-1 | None | power off | available | True |
| 9e0763d9-6f96-4327-a189-3bf12f5856ac | node-2 | None | power off | available | True |
+--------------------------------------+--------+---------------+-------------+--------------------+-------------+
4. wait for nova hypervisor-stats to be updated
Expected result:
total memory_mb is 1024, total local_gb is 10
Actual result:
total memory_mb is 0, total local_gb is 0
nova hypervisor-stats
+----------------------+-------+
| Property | Value |
+----------------------+-------+
| count | 3 |
| current_workload | 0 |
| disk_available_least | 10 |
| free_disk_gb | 10 |
| free_ram_mb | 1024 |
| local_gb | 0 |
| local_gb_used | 0 |
| memory_mb | 0 |
| memory_mb_used | 0 |
| running_vms | 0 |
| vcpus | 0 |
| vcpus_used | 0 |
+----------------------+-------+
also hypervisor-show shows these values as 0:
nova hypervisor-show bb785191-e548-4a6d-820e-bf2c5cdba922
+-------------------------+--------------------------------------+
| Property | Value |
+-------------------------+--------------------------------------+
| cpu_info | |
| current_workload | 0 |
| disk_available_least | 10 |
| free_disk_gb | 10 |
| free_ram_mb | 1024 |
| host_ip | 192.168.100.12 |
| hypervisor_hostname | bb785191-e548-4a6d-820e-bf2c5cdba922 |
| hypervisor_type | ironic |
| hypervisor_version | 1 |
| id | 1 |
| local_gb | 0 |
| local_gb_used | 0 |
| memory_mb | 0 |
| memory_mb_used | 0 |
| running_vms | 0 |
| service_disabled_reason | None |
| service_host | ironic |
| service_id | 8 |
| state | up |
| status | enabled |
| vcpus | 0 |
| vcpus_used | 0 |
+-------------------------+--------------------------------------+
Also, when "watch"ing hypervisor-stats I had a glimpse at correct
values that immediately jumped to zero on next query.
Due to this bug the instance is failing to be scheduled to the actually
available Ironic node due to Nova scheduler's RamFilter removing all hosts
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1572555/+subscriptions
Follow ups
