yahoo-eng-team team mailing list archive
-
yahoo-eng-team team
-
Mailing list archive
-
Message #89561
[Bug 1987377] [NEW] neutron-metadata-agent the memory usage is increasing
Public bug reported:
env:
branch: stable/victoria
The memory footprint becomes smaller after restarting the metadata-
agent, but as it runs longer, the memory footprint becomes larger and
larger until it is killed by oom
kubectl top pod neutron-metadata-agent-default-6nz79 -nopenstack
NAME CPU(cores) MEMORY(bytes)
neutron-metadata-agent-default-6nz79 4m 7121Mi
kubectl top pod -nopenstack neutron-metadata-agent-default-7znzp
NAME CPU(cores) MEMORY(bytes)
neutron-metadata-agent-default-7znzp 3m 24321Mi
Tasks: 12 total, 1 running, 11 sleeping, 0 stopped, 0 zombie
%Cpu(s): 3.5 us, 1.3 sy, 0.0 ni, 94.2 id, 0.0 wa, 0.0 hi, 0.3 si, 0.7 st
KiB Mem : 32885820 total, 3087452 free, 28965316 used, 833052 buff/cache
KiB Swap: 0 total, 0 free, 0 used. 3446688 avail Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
1 neutron 20 0 1020 4 0 S 0.0 0.0 0:00.07 pause
314636 neutron 20 0 193348 83160 4328 S 0.0 0.3 14:47.42 neutron-metadat
314649 neutron 20 0 3246420 2.988g 972 S 0.0 9.5 8:37.78 neutron-metadat
314650 neutron 20 0 3225648 2.970g 3184 S 0.0 9.5 8:36.11 neutron-metadat
314651 neutron 20 0 3228576 2.970g 0 S 0.0 9.5 8:37.24 neutron-metadat
314652 neutron 20 0 3223508 2.966g 1316 S 0.0 9.5 8:35.71 neutron-metadat
314653 neutron 20 0 3216512 2.959g 844 S 0.0 9.4 8:37.38 neutron-metadat
314654 neutron 20 0 3265104 3.006g 976 S 0.0 9.6 8:40.20 neutron-metadat
314655 neutron 20 0 3180172 2.924g 280 S 0.0 9.3 8:33.43 neutron-metadat
377345 neutron 20 0 193348 83388 4556 S 0.0 0.3 0:00.01 neutron-metadat
** Affects: neutron
Importance: Undecided
Status: New
** Description changed:
+ env:
+ branch: stable/victoria
+
+ The memory footprint becomes smaller after restarting the metadata-
+ agent, but as it runs longer, the memory footprint becomes larger and
+ larger until it is killed by oom
+
+
kubectl top pod neutron-metadata-agent-default-6nz79 -nopenstack
- NAME CPU(cores) MEMORY(bytes)
- neutron-metadata-agent-default-6nz79 4m 7121Mi
+ NAME CPU(cores) MEMORY(bytes)
+ neutron-metadata-agent-default-6nz79 4m 7121Mi
kubectl top pod -nopenstack neutron-metadata-agent-default-7znzp
- NAME CPU(cores) MEMORY(bytes)
- neutron-metadata-agent-default-7znzp 3m 24321Mi
+ NAME CPU(cores) MEMORY(bytes)
+ neutron-metadata-agent-default-7znzp 3m 24321Mi
- neutron-metadata-agent-default-7znzp 1/1
- Running 14 103d 10.200.4.54 mgt14 <none>
- <none>
Tasks: 12 total, 1 running, 11 sleeping, 0 stopped, 0 zombie
%Cpu(s): 3.5 us, 1.3 sy, 0.0 ni, 94.2 id, 0.0 wa, 0.0 hi, 0.3 si, 0.7 st
KiB Mem : 32885820 total, 3087452 free, 28965316 used, 833052 buff/cache
- KiB Swap: 0 total, 0 free, 0 used. 3446688 avail Mem
+ KiB Swap: 0 total, 0 free, 0 used. 3446688 avail Mem
- PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
- 1 neutron 20 0 1020 4 0 S 0.0 0.0 0:00.07 pause
- 314636 neutron 20 0 193348 83160 4328 S 0.0 0.3 14:47.42 neutron-metadat
- 314649 neutron 20 0 3246420 2.988g 972 S 0.0 9.5 8:37.78 neutron-metadat
- 314650 neutron 20 0 3225648 2.970g 3184 S 0.0 9.5 8:36.11 neutron-metadat
- 314651 neutron 20 0 3228576 2.970g 0 S 0.0 9.5 8:37.24 neutron-metadat
- 314652 neutron 20 0 3223508 2.966g 1316 S 0.0 9.5 8:35.71 neutron-metadat
- 314653 neutron 20 0 3216512 2.959g 844 S 0.0 9.4 8:37.38 neutron-metadat
- 314654 neutron 20 0 3265104 3.006g 976 S 0.0 9.6 8:40.20 neutron-metadat
- 314655 neutron 20 0 3180172 2.924g 280 S 0.0 9.3 8:33.43 neutron-metadat
- 377345 neutron 20 0 193348 83388 4556 S 0.0 0.3 0:00.01 neutron-metadat
+ PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
+ 1 neutron 20 0 1020 4 0 S 0.0 0.0 0:00.07 pause
+ 314636 neutron 20 0 193348 83160 4328 S 0.0 0.3 14:47.42 neutron-metadat
+ 314649 neutron 20 0 3246420 2.988g 972 S 0.0 9.5 8:37.78 neutron-metadat
+ 314650 neutron 20 0 3225648 2.970g 3184 S 0.0 9.5 8:36.11 neutron-metadat
+ 314651 neutron 20 0 3228576 2.970g 0 S 0.0 9.5 8:37.24 neutron-metadat
+ 314652 neutron 20 0 3223508 2.966g 1316 S 0.0 9.5 8:35.71 neutron-metadat
+ 314653 neutron 20 0 3216512 2.959g 844 S 0.0 9.4 8:37.38 neutron-metadat
+ 314654 neutron 20 0 3265104 3.006g 976 S 0.0 9.6 8:40.20 neutron-metadat
+ 314655 neutron 20 0 3180172 2.924g 280 S 0.0 9.3 8:33.43 neutron-metadat
+ 377345 neutron 20 0 193348 83388 4556 S 0.0 0.3 0:00.01 neutron-metadat
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1987377
Title:
neutron-metadata-agent the memory usage is increasing
Status in neutron:
New
Bug description:
env:
branch: stable/victoria
The memory footprint becomes smaller after restarting the metadata-
agent, but as it runs longer, the memory footprint becomes larger and
larger until it is killed by oom
kubectl top pod neutron-metadata-agent-default-6nz79 -nopenstack
NAME CPU(cores) MEMORY(bytes)
neutron-metadata-agent-default-6nz79 4m 7121Mi
kubectl top pod -nopenstack neutron-metadata-agent-default-7znzp
NAME CPU(cores) MEMORY(bytes)
neutron-metadata-agent-default-7znzp 3m 24321Mi
Tasks: 12 total, 1 running, 11 sleeping, 0 stopped, 0 zombie
%Cpu(s): 3.5 us, 1.3 sy, 0.0 ni, 94.2 id, 0.0 wa, 0.0 hi, 0.3 si, 0.7 st
KiB Mem : 32885820 total, 3087452 free, 28965316 used, 833052 buff/cache
KiB Swap: 0 total, 0 free, 0 used. 3446688 avail Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
1 neutron 20 0 1020 4 0 S 0.0 0.0 0:00.07 pause
314636 neutron 20 0 193348 83160 4328 S 0.0 0.3 14:47.42 neutron-metadat
314649 neutron 20 0 3246420 2.988g 972 S 0.0 9.5 8:37.78 neutron-metadat
314650 neutron 20 0 3225648 2.970g 3184 S 0.0 9.5 8:36.11 neutron-metadat
314651 neutron 20 0 3228576 2.970g 0 S 0.0 9.5 8:37.24 neutron-metadat
314652 neutron 20 0 3223508 2.966g 1316 S 0.0 9.5 8:35.71 neutron-metadat
314653 neutron 20 0 3216512 2.959g 844 S 0.0 9.4 8:37.38 neutron-metadat
314654 neutron 20 0 3265104 3.006g 976 S 0.0 9.6 8:40.20 neutron-metadat
314655 neutron 20 0 3180172 2.924g 280 S 0.0 9.3 8:33.43 neutron-metadat
377345 neutron 20 0 193348 83388 4556 S 0.0 0.3 0:00.01 neutron-metadat
To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1987377/+subscriptions
Follow ups
