yahoo-eng-team team mailing list archive
-
yahoo-eng-team team
-
Mailing list archive
-
Message #85009
[Bug 1913641] [NEW] Incorrect Shelved_offloaded instance metrics on openstack usage show output
Public bug reported:
env: bionic-ussuri and bionic-wallaby (devstack)
When running "openstack usage show --project <project>", having only
shelved_offloaded instances in the project, it continues to track
metrics as if the instance was running, even though it is not. See
output below:
$ openstack server list
+--------------------------------------+------+-------------------+--------------------------------------------------------+--------------------------+-----------+
| ID | Name | Status | Networks | Image | Flavor |
+--------------------------------------+------+-------------------+--------------------------------------------------------+--------------------------+-----------+
| a8d3fbb6-1734-4e3f-81db-b1c42a462bf7 | ins1 | SHELVED_OFFLOADED | private=10.0.0.30, fd6b:5cf:38bb:0:f816:3eff:fe66:c5b0 | cirros-0.5.1-x86_64-disk | cirros256 |
+--------------------------------------+------+-------------------+--------------------------------------------------------+--------------------------+-----------+
$ openstack usage show --project admin
Usage from 2020-12-31 to 2021-01-29 on project 1bfc9c13d7da4a4183c0b16cfa80020f:
+---------------+-------+
| Field | Value |
+---------------+-------+
| CPU Hours | 0.04 |
| Disk GB-Hours | 0.04 |
| RAM MB-Hours | 9.43 |
| Servers | 1 |
+---------------+-------+
$ openstack server show ins1
+-------------------------------------+-----------------------------------------------------------------+
| Field | Value |
+-------------------------------------+-----------------------------------------------------------------+
| OS-DCF:diskConfig | MANUAL |
| OS-EXT-AZ:availability_zone | |
| OS-EXT-SRV-ATTR:host | None |
| OS-EXT-SRV-ATTR:hypervisor_hostname | None |
| OS-EXT-SRV-ATTR:instance_name | instance-00000001 |
| OS-EXT-STS:power_state | Shutdown |
| OS-EXT-STS:task_state | None |
| OS-EXT-STS:vm_state | shelved_offloaded |
| OS-SRV-USG:launched_at | 2021-01-28T19:33:34.000000 |
| OS-SRV-USG:terminated_at | None |
| accessIPv4 | |
| accessIPv6 | |
| addresses | private=10.0.0.30, fd6b:5cf:38bb:0:f816:3eff:fe66:c5b0 |
| config_drive | |
| created | 2021-01-28T19:33:25Z |
| flavor | cirros256 (c1) |
| hostId | |
| id | a8d3fbb6-1734-4e3f-81db-b1c42a462bf7 |
| image | cirros-0.5.1-x86_64-disk (9e09f573-99f7-4f7c-bf16-47d475320207) |
| key_name | None |
| name | ins1 |
| project_id | 1bfc9c13d7da4a4183c0b16cfa80020f |
| properties | |
| security_groups | name='default' |
| status | SHELVED_OFFLOADED |
| updated | 2021-01-28T19:34:10Z |
| user_id | 1330bb68cfe147a580cc6085705ea319 |
| volumes_attached | |
+-------------------------------------+-----------------------------------------------------------------+
The reason it happens is because of the logic in [0], where it relies on the terminated_at field to make stop tracking the metrics. As can be seen in the "openstack server show" output, shelving the instance doesn't fill in the terminated_at field.
[0] https://github.com/openstack/nova/blob/6c0ceda3659405149b7c0b5c283275ef0a896269/nova/api/openstack/compute/simple_tenant_usage.py#L74
Steps to reproduce:
1) create an instance
2) shelve it, wait for shelved_offloaded state
3) run "openstack usage show --project <project>" multiple times and notice the metrics increasing
Expected result: Metrics for the shelved_offloaded instances shouldn't
be accounted for while it is in that state.
** Affects: nova
Importance: Undecided
Status: New
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1913641
Title:
Incorrect Shelved_offloaded instance metrics on openstack usage show
output
Status in OpenStack Compute (nova):
New
Bug description:
env: bionic-ussuri and bionic-wallaby (devstack)
When running "openstack usage show --project <project>", having only
shelved_offloaded instances in the project, it continues to track
metrics as if the instance was running, even though it is not. See
output below:
$ openstack server list
+--------------------------------------+------+-------------------+--------------------------------------------------------+--------------------------+-----------+
| ID | Name | Status | Networks | Image | Flavor |
+--------------------------------------+------+-------------------+--------------------------------------------------------+--------------------------+-----------+
| a8d3fbb6-1734-4e3f-81db-b1c42a462bf7 | ins1 | SHELVED_OFFLOADED | private=10.0.0.30, fd6b:5cf:38bb:0:f816:3eff:fe66:c5b0 | cirros-0.5.1-x86_64-disk | cirros256 |
+--------------------------------------+------+-------------------+--------------------------------------------------------+--------------------------+-----------+
$ openstack usage show --project admin
Usage from 2020-12-31 to 2021-01-29 on project 1bfc9c13d7da4a4183c0b16cfa80020f:
+---------------+-------+
| Field | Value |
+---------------+-------+
| CPU Hours | 0.04 |
| Disk GB-Hours | 0.04 |
| RAM MB-Hours | 9.43 |
| Servers | 1 |
+---------------+-------+
$ openstack server show ins1
+-------------------------------------+-----------------------------------------------------------------+
| Field | Value |
+-------------------------------------+-----------------------------------------------------------------+
| OS-DCF:diskConfig | MANUAL |
| OS-EXT-AZ:availability_zone | |
| OS-EXT-SRV-ATTR:host | None |
| OS-EXT-SRV-ATTR:hypervisor_hostname | None |
| OS-EXT-SRV-ATTR:instance_name | instance-00000001 |
| OS-EXT-STS:power_state | Shutdown |
| OS-EXT-STS:task_state | None |
| OS-EXT-STS:vm_state | shelved_offloaded |
| OS-SRV-USG:launched_at | 2021-01-28T19:33:34.000000 |
| OS-SRV-USG:terminated_at | None |
| accessIPv4 | |
| accessIPv6 | |
| addresses | private=10.0.0.30, fd6b:5cf:38bb:0:f816:3eff:fe66:c5b0 |
| config_drive | |
| created | 2021-01-28T19:33:25Z |
| flavor | cirros256 (c1) |
| hostId | |
| id | a8d3fbb6-1734-4e3f-81db-b1c42a462bf7 |
| image | cirros-0.5.1-x86_64-disk (9e09f573-99f7-4f7c-bf16-47d475320207) |
| key_name | None |
| name | ins1 |
| project_id | 1bfc9c13d7da4a4183c0b16cfa80020f |
| properties | |
| security_groups | name='default' |
| status | SHELVED_OFFLOADED |
| updated | 2021-01-28T19:34:10Z |
| user_id | 1330bb68cfe147a580cc6085705ea319 |
| volumes_attached | |
+-------------------------------------+-----------------------------------------------------------------+
The reason it happens is because of the logic in [0], where it relies on the terminated_at field to make stop tracking the metrics. As can be seen in the "openstack server show" output, shelving the instance doesn't fill in the terminated_at field.
[0] https://github.com/openstack/nova/blob/6c0ceda3659405149b7c0b5c283275ef0a896269/nova/api/openstack/compute/simple_tenant_usage.py#L74
Steps to reproduce:
1) create an instance
2) shelve it, wait for shelved_offloaded state
3) run "openstack usage show --project <project>" multiple times and notice the metrics increasing
Expected result: Metrics for the shelved_offloaded instances shouldn't
be accounted for while it is in that state.
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1913641/+subscriptions
Follow ups
