yahoo-eng-team team mailing list archive

[norman shen <1909160@xxxxxxxxxxxxxxxxxx>]

Public bug reported:

I saw listing security group is slow and causing cpu spikes
unexpectedly,

I run a Rock neutron-server with api worker set to 1, when executing
command like

```console
root@mgt01:~# time curl -H "x-auth-token: $token" http://neutron-server.openstack.svc.region-stackdev.myinspurcloud.com:9696/v2.0/security_groups -owide -s

real	0m2.328s
user	0m0.016s
sys	0m0.012s


root@mgt01:~# curl -H "x-auth-token: $token" http://neutron-server.openstack.svc.region-stackdev.myinspurcloud.com:9696/v2.0/security_groups | jq '.security_groups | length'
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  497k  100  497k    0     0   219k      0  0:00:02  0:00:02 --:--:--  219k
225
```

It will return in around 2 seconds. There are around 200 security groups, so maybe it is
not extremely slow, but what is interesting is that calling this rest api seems making cpu
spikes for neutorn-server pod,

```console
CONTAINER ID        NAME                                                                                                  CPU %               MEM USAGE / LIMIT    MEM %               NET I/O             BLOCK I/O           PIDS
8a30733e3932        k8s_neutron-server_neutron-server-787dcd7964-2zxt5_openstack_71cbb9bc-4530-11eb-bcc6-525400d22fc9_0   92.83%              1020MiB / 2.441GiB   40.81%              0B / 0B             0B / 16.4kB         8
```

I am wondering why security group listing is cpu bound?

** Affects: neutron
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1909160

Title:
  high cpu usage when listing security groups

Status in neutron:
  New

Bug description:
  I saw listing security group is slow and causing cpu spikes
  unexpectedly,

  I run a Rock neutron-server with api worker set to 1, when executing
  command like

  ```console
  root@mgt01:~# time curl -H "x-auth-token: $token" http://neutron-server.openstack.svc.region-stackdev.myinspurcloud.com:9696/v2.0/security_groups -owide -s

  real	0m2.328s
  user	0m0.016s
  sys	0m0.012s

  
  root@mgt01:~# curl -H "x-auth-token: $token" http://neutron-server.openstack.svc.region-stackdev.myinspurcloud.com:9696/v2.0/security_groups | jq '.security_groups | length'
    % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                   Dload  Upload   Total   Spent    Left  Speed
  100  497k  100  497k    0     0   219k      0  0:00:02  0:00:02 --:--:--  219k
  225
  ```

  It will return in around 2 seconds. There are around 200 security groups, so maybe it is
  not extremely slow, but what is interesting is that calling this rest api seems making cpu
  spikes for neutorn-server pod,

  ```console
  CONTAINER ID        NAME                                                                                                  CPU %               MEM USAGE / LIMIT    MEM %               NET I/O             BLOCK I/O           PIDS
  8a30733e3932        k8s_neutron-server_neutron-server-787dcd7964-2zxt5_openstack_71cbb9bc-4530-11eb-bcc6-525400d22fc9_0   92.83%              1020MiB / 2.441GiB   40.81%              0B / 0B             0B / 16.4kB         8
  ```

  I am wondering why security group listing is cpu bound?

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1909160/+subscriptions