linuxdcpp-team team mailing list archive

[beefstick <951217@xxxxxxxxxxxxxxxxxx>]

Okay, I did a test for all 4 options. I hope I will give enough info, if
you need more I am happy to supply but I am not available for the coming
1.5 weeks.

A quick recap of what is in the log below: I see the issue in all 4 scenarios
Matching a queue with a smaller file list does not decrease memory usage.

Here is my log:
***********
Fresh reboot of the system
linuxDC++ Idle 1 adcs hub connected, no queue, no uploads, autosearch for alternates=off, max file list size set to 300MB otherwise lists of more than 20MB will not open at all.
PID  USER PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND  
3198 me   21   1  906m 164m  15m S    1  8.2   0:25.83 linuxdcpp

1) You manually load a file list from disk by opening it in the file
menu and it opens up the tab to browse the share

Filelist = 64.4MB
PID  USER PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
3198 me   21   1 1324m 502m  15m S    1 25.1   1:25.04 linuxdcpp

Close file list + wait 5 minutes to make sure that memory can be given
back

PID  USER PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
3198 me   21   1 1324m 502m  15m S    2 25.1   1:31.14 linuxdcpp

Nothing given back


2) You download someone's list and it opens up the tab to browse the share when the download is complete.

Started from scratch with freshly started linuxdcpp (%mem=8.2)
downloaded the same file list (64.4MB)

PID  USER PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
4777 me   21   1 1370m 535m  15m S    1 26.7   1:40.66 linuxdcpp

Note that because of the download the memory increased 1.6% more than is the case for 1), I have no idea why but it might be causing something.
Close file list and wait 5 minutes

PID  USER PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
4777 me   21   1 1370m 536m  15m S    1 26.7   1:50.26 linuxdcpp

Nothing given back


3) You download a directory and the list is downloaded by default

Started from scratch with freshly started linuxdcpp (%mem=8.2)
downloaded file list 9.0MB

PID  USER PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
5376 me   21   1 1185m 222m  15m S    1 11.1   0:38.33 linuxdcpp

Memory increase even though file list is not opened by the program to
show it on screen

4) You choose the match queue option in search, transfers, hub, or
favorite users window.

Match queue on 1 user (filelist 3.91MB) after 3)

PID  USER PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
5376 me   21   1 1185m 249m  15m S    1 12.4   0:48.92 linuxdcpp

Memory increase even though file list is not opened
Match again (without waiting) on different user (filelist 3.16MB)

PID  USER PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
5376 me   21   1 1185m 255m  16m S    1 12.8   1:10.23 linuxdcpp

Again memory increase while file list is smaller
Match again on different user (filelist 11.98MB)

PID  USER PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
5376 me   21   1 1249m 289m  16m S    1 14.4   1:23.87 linuxdcpp

Memory increase on big file list, as expected.
Waiting for 10 minutes
Match again on different user (filelist 2.98MB, smallest I could find)

PID  USER PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
5376 me   21   1 1250m 289m  16m S    1 14.5   1:41.21 linuxdcpp

Small effect, but again memory increase while file list is smaller
********

-- 
You received this bug notification because you are a member of LinuxDC++
Team, which is subscribed to LinuxDC++.
https://bugs.launchpad.net/bugs/951217

Title:
  excessive memory usage on file list download

Status in Linux DC++:
  Incomplete

Bug description:
  Description of the issue

  I have just downloaded a number of file lists ranging from several kB
  to several MB and this is what happens

  Mem:   2053260k total,  1891236k used,   162024k free,     2496k buffers
  Swap:  2095100k total,   255924k used,  1839176k free,   384880k cached

    PID USER      PR  NI  VIRT  RES  SHR S   %CPU %MEM    TIME+  COMMAND                                                                                                              
  27935 me      21   1 2250m 1.1g 8540 S      2    56.1   1:20.87 linuxdcpp 

  The filelists are all parsed correctly as they should but the memory usage increases to <insane> and stays that way forever.
  Since filelists are never re-used by the program in any way they should be unloaded from memory as well or at least placed into cached so that it can be overwritten.
  An other option is to create the possibility for downloading partial file lists to limit the use of RAM.

  Steps to reproduce

  Download several filelists of up to 60 MB or even more

  Expected result

  Increase of RAM usage to accommodate for parsing filelist, take out
  info needed, dump filelist to reduce RAM usage.

  Actual result

  Increase or RAM usage to accommodate for parsing filelist, take out
  info needed, leave unneeded info in RAM

  Version

  Compiled from latest BZR

  OS

  Opensuse 11.3 to 12.1

  No backtrace since no crash

To manage notifications about this bug go to:
https://bugs.launchpad.net/linuxdcpp/+bug/951217/+subscriptions