yahoo-eng-team team mailing list archive
-
yahoo-eng-team team
-
Mailing list archive
-
Message #81799
[Bug 1863201] Re: stein regression listing security group rules
Reviewed: https://review.opendev.org/708695
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=d874c46bff7045ba25f5dd6e790f7ddb209cb224
Submitter: Zuul
Branch: master
commit d874c46bff7045ba25f5dd6e790f7ddb209cb224
Author: Rodolfo Alonso Hernandez <ralonsoh@xxxxxxxxxx>
Date: Tue Feb 18 17:08:22 2020 +0000
Filter by owner SGs when retrieving the SG rules
Retrieving the SG rules now is used the admin context. This allows to
get all possible rules, independently of the user calling. The filters
passed and the RBAC policies filter those results, returning only:
- The SG rules belonging to the user.
- The SG rules belonging to a SG owned by the user.
However, if the SG list is too long, the query can take a lot of time.
Instead of this, the filtering is done in the DB query. If no filters
are passed to "get_security_group_rules" and the context is not the
admin context, only the rules specified in the first paragraph will
be retrieved.
Because overwriting the method "get_objects" is too complex, an
intermediate query is done to retrieve the SG rule IDs. Those IDs
will be used as a filter in the "get_objects" call.
Closes-Bug: #1863201
Change-Id: I25d3da929f8d0b6ee15d7b90ec59b9d58a4ae6a5
** Changed in: neutron
Status: In Progress => Fix Released
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1863201
Title:
stein regression listing security group rules
Status in neutron:
Fix Released
Bug description:
Upgrading neutron from rocky -> stein and get a considerable slow down when listing all security groups for a project. Goes from ~2 seconds to almost 2 minutes. Looking into the code it looks like it is very inefficient because it gets all rules from the DB and then filters after the fact.
We have around 7000 rules in our QA env.
Very keen to get this sorted but don't know the neutron code base that
well so can offer testing of patches if there are any out there
already.
It looks like this happened with listing ports too for stein and found
this https://bugzilla.redhat.com/show_bug.cgi?id=1737012 so wonder if
this is related?
With Rocky:
time openstack security group rule list
+--------------------------------------+-------------+-----------+--------------------+------------+--------------------------------------+--------------------------------------+
| ID | IP Protocol | Ethertype | IP Range | Port Range | Remote Security Group | Security Group |
+--------------------------------------+-------------+-----------+--------------------+------------+--------------------------------------+--------------------------------------+
| 01b877cc-1621-44cd-8e69-1345ab01a1ef | None | IPv4 | 0.0.0.0/0 | | None | 3dcbd4fa-d017-4361-b0b0-b7508e923087 |
| 0c744788-6319-42e5-931a-5e7b0df166c4 | None | IPv6 | ::/0 | | None | 3dcbd4fa-d017-4361-b0b0-b7508e923087 |
| 0fc6b79d-d211-4201-ac76-60fb8ea40c9c | None | IPv4 | 0.0.0.0/0 | | None | 8f55c18b-cd8c-4d84-afef-f8b83d5eb128 |
| 17d6c8a3-7894-42a6-92f2-1bd56a30ef1d | tcp | IPv4 | 0.0.0.0/0 | 80:80 | None | ed257fd7-d825-4014-96a8-c16adfea70f0 |
| 19d3ba79-65f1-4c89-a1c2-b32049ceb25a | None | IPv6 | ::/0 | | None | 008510a7-d176-4ee5-87e2-e74da06c55ba |
| 21f1d173-b99f-47a7-9983-6926f7bc58f3 | icmp | IPv4 | 0.0.0.0/0 | | None | 008510a7-d176-4ee5-87e2-e74da06c55ba |
| 3321d5ff-11c3-4104-be13-107c789e4bf8 | None | IPv6 | ::/0 | | None | 57cb14de-dd5f-4f0c-b0cf-a7effc36fca5 |
| 381c6816-9b5c-42b7-9dd3-dae12a49c08b | None | IPv4 | 0.0.0.0/0 | | None | 3f63cfbb-87ee-4aa2-8193-7e86cb542881 |
| 3886ad10-99ea-4f60-a36c-ffbe80d92907 | None | IPv6 | ::/0 | | None | ed257fd7-d825-4014-96a8-c16adfea70f0 |
| 5be4853a-75d1-435c-87ca-56c54a243f70 | None | IPv4 | 0.0.0.0/0 | | None | 57cb14de-dd5f-4f0c-b0cf-a7effc36fca5 |
| 71656249-4454-410e-8e7d-24910df127ba | None | IPv6 | ::/0 | | None | 8f55c18b-cd8c-4d84-afef-f8b83d5eb128 |
| 783324ac-6844-4d4d-985c-936015bcb66e | icmp | IPv4 | 0.0.0.0/0 | | None | 3f63cfbb-87ee-4aa2-8193-7e86cb542881 |
| 7ca7f0cc-b4df-401f-aaa4-662f17afcfb0 | None | IPv4 | 0.0.0.0/0 | | None | 008510a7-d176-4ee5-87e2-e74da06c55ba |
| 825a33ff-b693-456d-811e-a0b494e8e308 | None | IPv6 | ::/0 | | 008510a7-d176-4ee5-87e2-e74da06c55ba | 008510a7-d176-4ee5-87e2-e74da06c55ba |
| 89fd2d18-45d3-4a86-a020-09d240912e5c | tcp | IPv4 | 128.250.116.173/32 | 22:22 | None | 008510a7-d176-4ee5-87e2-e74da06c55ba |
| 8a1f45f1-e4c8-41e4-b6f3-80ab48b7e38d | None | IPv6 | ::/0 | | None | bf7abb53-e5ca-428d-9fce-6a2e37a25ee0 |
| 9ebc6d15-e3eb-4d20-88d4-6737367ffc08 | None | IPv4 | 0.0.0.0/0 | | None | ed257fd7-d825-4014-96a8-c16adfea70f0 |
| 9f29f539-a80a-4a8d-89cc-f714224b5f8c | icmp | IPv4 | 0.0.0.0/0 | | None | 8f55c18b-cd8c-4d84-afef-f8b83d5eb128 |
| a1bc8f05-3a20-48c2-bae5-a60f4ffed514 | None | IPv4 | 0.0.0.0/0 | | 008510a7-d176-4ee5-87e2-e74da06c55ba | 008510a7-d176-4ee5-87e2-e74da06c55ba |
| bef999d6-669a-47f6-988c-e69bab6df87a | tcp | IPv4 | 0.0.0.0/0 | 22:22 | 57cb14de-dd5f-4f0c-b0cf-a7effc36fca5 | bf7abb53-e5ca-428d-9fce-6a2e37a25ee0 |
| c5ce339b-cd92-492c-9af4-6eab875027ce | tcp | IPv4 | 0.0.0.0/0 | 80:80 | 008510a7-d176-4ee5-87e2-e74da06c55ba | 008510a7-d176-4ee5-87e2-e74da06c55ba |
| d9ec0eba-d80d-4331-a588-e4f8c1c75533 | None | IPv6 | ::/0 | | None | 3f63cfbb-87ee-4aa2-8193-7e86cb542881 |
| de760e03-92a9-4183-8acc-1d82addc3604 | None | IPv4 | 0.0.0.0/0 | | None | bf7abb53-e5ca-428d-9fce-6a2e37a25ee0 |
| f4bc4616-1d18-4488-84bb-546516c053bc | tcp | IPv4 | 0.0.0.0/0 | 443:443 | None | ed257fd7-d825-4014-96a8-c16adfea70f0 |
+--------------------------------------+-------------+-----------+--------------------+------------+--------------------------------------+--------------------------------------+
real 0m2.499s
user 0m0.642s
sys 0m0.053s
With Stein:
time openstack security group rule list
+--------------------------------------+-------------+-----------+--------------------+------------+--------------------------------------+--------------------------------------+
| ID | IP Protocol | Ethertype | IP Range | Port Range | Remote Security Group | Security Group |
+--------------------------------------+-------------+-----------+--------------------+------------+--------------------------------------+--------------------------------------+
| 01b877cc-1621-44cd-8e69-1345ab01a1ef | None | IPv4 | 0.0.0.0/0 | | None | 3dcbd4fa-d017-4361-b0b0-b7508e923087 |
| 0c744788-6319-42e5-931a-5e7b0df166c4 | None | IPv6 | ::/0 | | None | 3dcbd4fa-d017-4361-b0b0-b7508e923087 |
| 0fc6b79d-d211-4201-ac76-60fb8ea40c9c | None | IPv4 | 0.0.0.0/0 | | None | 8f55c18b-cd8c-4d84-afef-f8b83d5eb128 |
| 17d6c8a3-7894-42a6-92f2-1bd56a30ef1d | tcp | IPv4 | 0.0.0.0/0 | 80:80 | None | ed257fd7-d825-4014-96a8-c16adfea70f0 |
| 19d3ba79-65f1-4c89-a1c2-b32049ceb25a | None | IPv6 | ::/0 | | None | 008510a7-d176-4ee5-87e2-e74da06c55ba |
| 21f1d173-b99f-47a7-9983-6926f7bc58f3 | icmp | IPv4 | 0.0.0.0/0 | | None | 008510a7-d176-4ee5-87e2-e74da06c55ba |
| 3321d5ff-11c3-4104-be13-107c789e4bf8 | None | IPv6 | ::/0 | | None | 57cb14de-dd5f-4f0c-b0cf-a7effc36fca5 |
| 381c6816-9b5c-42b7-9dd3-dae12a49c08b | None | IPv4 | 0.0.0.0/0 | | None | 3f63cfbb-87ee-4aa2-8193-7e86cb542881 |
| 3886ad10-99ea-4f60-a36c-ffbe80d92907 | None | IPv6 | ::/0 | | None | ed257fd7-d825-4014-96a8-c16adfea70f0 |
| 5be4853a-75d1-435c-87ca-56c54a243f70 | None | IPv4 | 0.0.0.0/0 | | None | 57cb14de-dd5f-4f0c-b0cf-a7effc36fca5 |
| 71656249-4454-410e-8e7d-24910df127ba | None | IPv6 | ::/0 | | None | 8f55c18b-cd8c-4d84-afef-f8b83d5eb128 |
| 783324ac-6844-4d4d-985c-936015bcb66e | icmp | IPv4 | 0.0.0.0/0 | | None | 3f63cfbb-87ee-4aa2-8193-7e86cb542881 |
| 7ca7f0cc-b4df-401f-aaa4-662f17afcfb0 | None | IPv4 | 0.0.0.0/0 | | None | 008510a7-d176-4ee5-87e2-e74da06c55ba |
| 825a33ff-b693-456d-811e-a0b494e8e308 | None | IPv6 | ::/0 | | 008510a7-d176-4ee5-87e2-e74da06c55ba | 008510a7-d176-4ee5-87e2-e74da06c55ba |
| 89fd2d18-45d3-4a86-a020-09d240912e5c | tcp | IPv4 | 128.250.116.173/32 | 22:22 | None | 008510a7-d176-4ee5-87e2-e74da06c55ba |
| 8a1f45f1-e4c8-41e4-b6f3-80ab48b7e38d | None | IPv6 | ::/0 | | None | bf7abb53-e5ca-428d-9fce-6a2e37a25ee0 |
| 9ebc6d15-e3eb-4d20-88d4-6737367ffc08 | None | IPv4 | 0.0.0.0/0 | | None | ed257fd7-d825-4014-96a8-c16adfea70f0 |
| 9f29f539-a80a-4a8d-89cc-f714224b5f8c | icmp | IPv4 | 0.0.0.0/0 | | None | 8f55c18b-cd8c-4d84-afef-f8b83d5eb128 |
| a1bc8f05-3a20-48c2-bae5-a60f4ffed514 | None | IPv4 | 0.0.0.0/0 | | 008510a7-d176-4ee5-87e2-e74da06c55ba | 008510a7-d176-4ee5-87e2-e74da06c55ba |
| bef999d6-669a-47f6-988c-e69bab6df87a | tcp | IPv4 | 0.0.0.0/0 | 22:22 | 57cb14de-dd5f-4f0c-b0cf-a7effc36fca5 | bf7abb53-e5ca-428d-9fce-6a2e37a25ee0 |
| c5ce339b-cd92-492c-9af4-6eab875027ce | tcp | IPv4 | 0.0.0.0/0 | 80:80 | 008510a7-d176-4ee5-87e2-e74da06c55ba | 008510a7-d176-4ee5-87e2-e74da06c55ba |
| d9ec0eba-d80d-4331-a588-e4f8c1c75533 | None | IPv6 | ::/0 | | None | 3f63cfbb-87ee-4aa2-8193-7e86cb542881 |
| de760e03-92a9-4183-8acc-1d82addc3604 | None | IPv4 | 0.0.0.0/0 | | None | bf7abb53-e5ca-428d-9fce-6a2e37a25ee0 |
| f4bc4616-1d18-4488-84bb-546516c053bc | tcp | IPv4 | 0.0.0.0/0 | 443:443 | None | ed257fd7-d825-4014-96a8-c16adfea70f0 |
+--------------------------------------+-------------+-----------+--------------------+------------+--------------------------------------+--------------------------------------+
real 1m51.921s
user 0m0.624s
sys 0m0.077s
To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1863201/+subscriptions
References
