yahoo-eng-team team mailing list archive
-
yahoo-eng-team team
-
Mailing list archive
-
Message #95684
[Bug 2106380] [NEW] Slow server list response with API version 2.96
Public bug reported:
Description:
The accumulation of instance_group_member records in the nova-api database causes the server list to slow down.
Details:
Information about instance group members is currently stored in the instance_group_member table of the nova-api database.
Even if an instance belonging to a particular instance group is deleted, it is not removed from the instance_group_member table. There is a mechanism to delete records of deleted instances from instance_group_member using the nova-manage db archive_deleted_rows command. [1]
In os-compute-api-version 2.96, it was updated to obtain and display the AZ information from request_specs when listing (showing) servers. [2]
At this time, instance_group information is also retrieved. [3]
If the records in instance_group_member have grown significantly, the
response time for the server list becomes extremely slow. In our
environment, there were nearly 20,000 instance_group_member records
linked to a single instance group (the cause of which was a bug in the
Kubernetes managed service on our OpenStack).
References:
[1] https://github.com/openstack/nova/blob/9d910ec4bf2a12baf3b5f0ec3bc41686413538fb/nova/cmd/manage.py#L493-L495
[2] https://review.opendev.org/c/openstack/nova/+/904568/17
[3] https://github.com/openstack/nova/blob/9d910ec4bf2a12baf3b5f0ec3bc41686413538fb/nova/objects/request_spec.py#L747-L763
Environment:
By os-compute-api-version 2.96
Our Reproduction Environment:
Dalmatial release (2024.2)
** Affects: nova
Importance: Undecided
Status: New
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/2106380
Title:
Slow server list response with API version 2.96
Status in OpenStack Compute (nova):
New
Bug description:
Description:
The accumulation of instance_group_member records in the nova-api database causes the server list to slow down.
Details:
Information about instance group members is currently stored in the instance_group_member table of the nova-api database.
Even if an instance belonging to a particular instance group is deleted, it is not removed from the instance_group_member table. There is a mechanism to delete records of deleted instances from instance_group_member using the nova-manage db archive_deleted_rows command. [1]
In os-compute-api-version 2.96, it was updated to obtain and display the AZ information from request_specs when listing (showing) servers. [2]
At this time, instance_group information is also retrieved. [3]
If the records in instance_group_member have grown significantly, the
response time for the server list becomes extremely slow. In our
environment, there were nearly 20,000 instance_group_member records
linked to a single instance group (the cause of which was a bug in the
Kubernetes managed service on our OpenStack).
References:
[1] https://github.com/openstack/nova/blob/9d910ec4bf2a12baf3b5f0ec3bc41686413538fb/nova/cmd/manage.py#L493-L495
[2] https://review.opendev.org/c/openstack/nova/+/904568/17
[3] https://github.com/openstack/nova/blob/9d910ec4bf2a12baf3b5f0ec3bc41686413538fb/nova/objects/request_spec.py#L747-L763
Environment:
By os-compute-api-version 2.96
Our Reproduction Environment:
Dalmatial release (2024.2)
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/2106380/+subscriptions