registry team mailing list archive
-
registry team
-
Mailing list archive
-
Message #15379
[Bug 484521] Re: Nautilus causes memory leak in Xorg?
Launchpad has imported 10 comments from the remote bug at
http://bugs.freedesktop.org/show_bug.cgi?id=22011.
If you reply to an imported comment from within Launchpad, your comment
will be sent to the remote bug automatically. Read more about
Launchpad's inter-bugtracker facilities at
https://help.launchpad.net/InterBugTracking.
------------------------------------------------------------------------
On 2009-05-30T22:31:20+00:00 cfox wrote:
My system has 8GB of ram, but that's not enough to prevent X from putting 6.3 gigabytes of data into resident memory, with another 1.3GB sitting in my swap file. I am in a situation where I am running:
1. a firefox window with 3 tabs
2. one terminal
and X is using 8 Gigabytes of RAM in total.... not acceptable.
Here are the results of top:
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
9149 root 20 0 8258m 6.3g 7524 S 1 80.9 28:18.08 X
And to prove it's not some magical "caching" algorithm, here's what xrestop says:
xrestop - Display: localhost:0
Monitoring 19 clients. XErrors: 0
Pixmaps: 54073K total, Other: 109K total, All: 54182K total
res-base Wins GCs Fnts Pxms Misc Pxm mem Other Total PID Identifier
1a00000 28 11 1 327 530 24680K 14K 24695K 9304 Qt-subapplication
2200000 7 5 1 257 312 9725K 8K 9733K 9331 usr/b : python
1200000 61 3 1 324 437 9340K 12K 9353K 9298 kwin
0000000 1 0 2 0 113 8437K 4K 8442K ? <unknown>
0400000 7 2 1 256 297 661K 8K 669K 9327 Qt-subapplication
1000000 225 119 1 89 244 341K 14K 356K 21439 Enter Bug: xorg - Minefield
2000000 7 1 1 236 260 226K 7K 233K 9345 klipper
0e00000 6 1 1 233 384 201K 10K 211K 9289 kded4
1c00000 5 1 1 243 269 188K 7K 195K 9329 kmix
2600000 3 1 1 226 250 105K 6K 112K 9343 Document Print Status
1400000 5 1 1 226 252 81K 7K 88K 9296 Qt-subapplication
1800000 0 1 0 223 245 81K 5K 87K ? <unknown>
0200000 0 1 1 0 0 0B 1K 1K ? <unknown>
1e00000 1 1 0 0 0 0B 48B 48B ? xrestop
1600000 1 1 0 0 0 0B 48B 48B ? <unknown>
0c00000 1 1 0 0 0 0B 48B 48B ? <unknown>
0600000 1 1 0 0 0 0B 48B 48B ? <unknown>
0a00000 0 1 0 0 0 0B 24B 24B ? <unknown>
0800000 0 1 0 0 0 0B 24B 24B ? <unknown>
Before going any further, I know what caching is... and there is no way in hell that you can justify using 8 GB of memory to "cache" a few trivial applications. This is a memory leak, pure and simple. I've seen it using both open source drivers supplied by Intel, and closed source ones from Nvidia. The current example is using the Nvidia driver. Before blaming everything on the binary drivers, I've noticed the same problem on multiple machines running different video cards with different drivers, but the leak remains constant... in fact it is often SLOWER with the Nvidia driver than with the open-source Intel ones.
Also, before telling me that X will magically release this memory at
some indeterminate point in the future... I tried to turn my swap off to
see what X was actually swapping out to disk.. and I can't turn it off
because there isn't enough physical memory left to store everything! If
the "caching" was so smart, it would have released some memory when the
entire system with 8GB of RAM is filled, would it not?
I'm running the xorg 1.6.1 server, and I'm starting to wonder if anybody
really cares about this bug. The sick sad truth is that you guys seem
to want to make Windows Vista look like a lean, resource-efficient
desktop with easy to reproduce leaks like this.
Reply at: https://bugs.launchpad.net/gnome-system-
monitor/+bug/484521/comments/0
------------------------------------------------------------------------
On 2009-06-02T12:01:59+00:00 Alan-coopersmith-oracle wrote:
If Xorg leaked 6 gb of memory for all users, it would have been fixed already.
That it's not fixed is not a sign that no one cares about Xorg memory usage,
but that there's some bug you're hitting that the developers haven't seen before.
Any information you could provide about what's unique in your environment or
output from a leak checker such as valgrind would help us find it.
Reply at: https://bugs.launchpad.net/gnome-system-
monitor/+bug/484521/comments/1
------------------------------------------------------------------------
On 2009-06-03T16:28:56+00:00 cfox wrote:
I think that many "normal" users are not seeing 6GB of memory usage
because many of them don't have that much RAM available, but that's
besides the point. Here are the stupidly simple ways able to get Xorg
1.6.1 to leak memory in a repeatable fashion:
1. On my Intel x4500 based laptop using KDE 4.2, the newest 2.6.29 kernel and 2.7.1 Intel drivers, with or without compositing turned on, all I have to do is run a graphical program, then kill that program. I chose the "kcalc" utility since it was simple to run repeatedly. The following python program will cause a nice, monotonic increase in memory usage that had ballooned to about 2.6GB after 24 hours:
from subprocess import Popen
import time
RUN_CMD = "/usr//bin/kcalc"
# test program that automatically executes, then kills simple x programs.
# Designed to show that x.org will memory leak simply by running & then
# killing trivial programs.
while (True): # continue until the user kills us
process_list = []
# launch then kill & wait 10 at a time:
for II in xrange (0, 10):
process_list.append (Popen (("kcalc%d" % II), executable=RUN_CMD,
close_fds=True))
time.sleep (10) # give programs time to open fully, adjust for your
machine
for nextProc in process_list:
nextProc.kill ()
nextProc.wait () # no zombies!
This is a nice, simple leak... I can have 2 windows open and still have X using over 50% of system memory on a 4GB laptop.
2. On my desktop that is using an 8800GT and the 185.18.14 proprietary drivers, the memory handling is... BETTER than the "pure" open-source Intel drivers, so the above method will not cause a leak (or at least leak too slowly to notice). However, the X server still cannot handle memory, and once it gets it's grimy manacles onto a memory page, that page will NOT ever be given back to the system.
Here's how I got my desktop's X server to consume 2.9GB of RAM that it will not release in about 5 minutes:
1. Dump a couple of hundred pdf files into a directory. They don't have to be huge, mine were just collected at random from my home directory.
2. Run the following script to open them all up at once:
#!/bin/sh
pdfList=`ls *.pdf`
for nextPdf in ${pdfList}
do
okular ${nextPdf} &
done;
3. Wait a few minutes to let as many of them open until X actually refuses to spawn new windows.
4. Do a simple "pkill okular" that kills off all the processes.
5. Now: X was using 2.9GB of memory with all those windows open...
and I'm not complaining about it USING memory when there are over 200
windows open. I AM complaining about it releasing ZERO resources back
to the system after ALL those windows close. Before you try to tell me
this is "caching" or some type of "feature" let me tell you that X is in
no way shape or form more responsive using 2.9GB of RAM than it is when
I first spawn it only using 36MB, and additionally, if this is a
"feature" I should be able to CONFIGURE AND CONTROL the maximum amount
of "cache" that X will use. Since we both know this is not the case, we
both know this is a memory leak. By the way, the 6GB of usage I had
before was NOT from opening hundreds of windows at once, it was actually
from leaked memory that occurred when I was running programs via WINE
for several hours, but I knew that if I said I was using WINE you'd
ignore the report and shift blame elsewhere even though WINE had exited
100% but X still used 6GB of memory.
A few more things to prevent you from shifting blame away: After all
of the tests I describe above, I killed & restarted both kwin and
plasma, so their memory was completely freed and then reinitialized.
This did exactly nothing to lower X's memory usage so don't try to say
they are the real culprits.
I'll stop posting these bugs when you personally run all of these
tests on commodity, 64 bit hardware using any major graphics chip and
current driver from Intel, Nvidia or ATI, and can show me that the X
server is using the same amount of memory before the test as after the
test. If you want to blame KDE, I'm warning you that I've seen X behave
badly on GNOME desktops too, so run a comprehensive test suite. I've
put up concrete numbers and very simple ways to hammer the X server, so
I hope you can tell me how to stop this from happening.
Reply at: https://bugs.launchpad.net/gnome-system-
monitor/+bug/484521/comments/2
------------------------------------------------------------------------
On 2009-06-09T23:20:31+00:00 Peregrinebf wrote:
Just confirming, during normal use (firefox, gmusicbrowser, keepassx
open, KDE 4, ubuntu 9.04 amd64. Closed firefox, waited 2 hours, mem
usage of X stayed the same. Reopened to post this.)
Top:
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
2903 root 20 0 1124m 297m 4928 S 11 15.0 365:06.42 Xorg
xrestop - Display: :0.0
Monitoring 37 clients. XErrors: 0
Pixmaps: 76603K total, Other: 186K total, All: 76789K total
297MB is rather more than 77MB. By 220MB.
Later, will try using Xmonad. I think we can be confident that that will
ensure it is not simply a KDE bug of some sort.
Reply at: https://bugs.launchpad.net/gnome-system-
monitor/+bug/484521/comments/3
------------------------------------------------------------------------
On 2009-06-11T15:40:07+00:00 Peregrinebf wrote:
It leaks with XMonad, but not as much: I used the PDF script foxcm
provided.
Before:
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
21290 root 20 0 385m 22m 11m S 1 1.1 0:01.43 Xorg
After:
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
21290 root 20 0 502m 116m 4504 S 1 5.9 2:17.23 Xorg
xrestop - Display: :0
Monitoring 14 clients. XErrors: 357
Pixmaps: 6871K total, Other: 22K total, All: 6893K total
That indicates that at least part of the problem (though not all) comes
from KDE.
Reply at: https://bugs.launchpad.net/gnome-system-
monitor/+bug/484521/comments/4
------------------------------------------------------------------------
On 2009-07-02T16:34:25+00:00 cfox wrote:
OK, I've done even more tests to isolate where this leak is coming from:
1. I tested using fluxbox to eliminate as many possible leaking parties as possible.
2. I tested GTK and Qt based applications, each one using a program that had as few dependencies as I could find.
3. I disabled GLX and used the open source NV driver so if the leaks here are driver related, it's technically open source (although I get leaks using either proprietary or open source drivers, and in some ways NV is worse that the proprietary driver).
Once again, the test is simple: Open up an X program with many
instances, then close all the instances completely and see what X.org
decides to do with memory. I'm using Xorg 1.6.1.901 and the 2.1.13
release of the NV driver.
Xorg's original resident memory when I did a fresh restart (from a direct startx, not using kdm/xdm/gdm) is 35MB.
I opened up to the maximum allowed instances of gvim, which uses GTK2, but is not otherwise dependant upon GNOME desktop components: Result: X maxed out at 450 MB resident.. BUT it dropped back to 65 MB after all gvim windows were closed, which I am not going to complain about.
The next test ran Qt 4.5.2 based Scribus. Scribus is a Qt based application, but it is not really dependant upon KDE so none of the KDE components or even libraries are being loaded, just Qt. In this case, 128 instances (to keep from running out of memory) pushed X.org to 1.4GB of resident usage.... where it stayed after all the scribus processes were shut down.
So does this mean the problem is fully diagnosed? I actually think
there could be more than one leak I'm seeing, but this one is at least
reproducible. However, this really should be fixed in the X server.
The Qt libraries may be misbehaving, but that is no excuse for X to not
release memory when the programs that reserved the memory come to an
end. If a kernel leaked memory whenever a user process didn't clean up
perfectly, we'd call it Windows 3.1. The same principle applies here,
the X server has the responsibility to clean up memory after a user
application terminates, even if the user application is not operating
perfectly.
Here's the result of xrestop while X.org is using 1.4GB of RAM:
0 - Fluxbox ( PID: ? ):
res_base : oxc00000
res_mask : ox1fffff
windows : 184
GCs : 21
fonts : 1
pixmaps : 182
pictures : 4
glyphsets : 5
colormaps : 1
passive grabs : 293
cursors : 19
unknowns : 13
pixmap bytes : 6629830
other bytes : ~13984
total bytes : ~6643814
1 - <unknown> ( PID: ? ):
res_base : ox0
res_mask : ox1fffff
windows : 1
GCs : 0
fonts : 2
pixmaps : 0
pictures : 0
glyphsets : 0
colormaps : 0
passive grabs : 0
cursors : 1
unknowns : 45
pixmap bytes : 4608000
other bytes : ~3176
total bytes : ~4611176
2 - <unknown> ( PID: ? ):
res_base : oxe00000
res_mask : ox1fffff
windows : 0
GCs : 0
fonts : 0
pixmaps : 1
pictures : 0
glyphsets : 0
colormaps : 1
passive grabs : 0
cursors : 0
unknowns : 0
pixmap bytes : 4608000
other bytes : ~24
total bytes : ~4608024
3 - Bug 22011 – X really does hog memory - Shiretoko ( PID: 8471 ):
res_base : ox1000000
res_mask : ox1fffff
windows : 321
GCs : 173
fonts : 1
pixmaps : 107
pictures : 164
glyphsets : 44
colormaps : 0
passive grabs : 0
cursors : 7
unknowns : 42
pixmap bytes : 538243
other bytes : ~19048
total bytes : ~557291
4 - login ( PID: 6103 ):
res_base : oxa00000
res_mask : ox1fffff
windows : 3
GCs : 34
fonts : 8
pixmaps : 3
pictures : 0
glyphsets : 0
colormaps : 1
passive grabs : 12
cursors : 9
unknowns : 2
pixmap bytes : 15
other bytes : ~9656
total bytes : ~9671
5 - xterm ( PID: 6107 ):
res_base : ox800000
res_mask : ox1fffff
windows : 2
GCs : 20
fonts : 6
pixmaps : 1
pictures : 0
glyphsets : 0
colormaps : 1
passive grabs : 12
cursors : 8
unknowns : 2
pixmap bytes : 0
other bytes : ~7224
total bytes : ~7224
6 - xterm ( PID: 8466 ):
res_base : ox400000
res_mask : ox1fffff
windows : 2
GCs : 12
fonts : 6
pixmaps : 1
pictures : 0
glyphsets : 0
colormaps : 1
passive grabs : 12
cursors : 8
unknowns : 2
pixmap bytes : 0
other bytes : ~7032
total bytes : ~7032
7 - xterm ( PID: 6106 ):
res_base : ox600000
res_mask : ox1fffff
windows : 2
GCs : 9
fonts : 6
pixmaps : 1
pictures : 0
glyphsets : 0
colormaps : 1
passive grabs : 12
cursors : 8
unknowns : 2
pixmap bytes : 0
other bytes : ~6960
total bytes : ~6960
8 - xrestop ( PID: ? ):
res_base : ox1600000
res_mask : ox1fffff
windows : 1
GCs : 1
fonts : 0
pixmaps : 0
pictures : 0
glyphsets : 0
colormaps : 0
passive grabs : 0
cursors : 0
unknowns : 0
pixmap bytes : 0
other bytes : ~48
total bytes : ~48
9 - <unknown> ( PID: ? ):
res_base : ox1200000
res_mask : ox1fffff
windows : 1
GCs : 1
fonts : 0
pixmaps : 0
pictures : 0
glyphsets : 0
colormaps : 0
passive grabs : 0
cursors : 0
unknowns : 0
pixmap bytes : 0
other bytes : ~48
total bytes : ~48
10 - <unknown> ( PID: ? ):
res_base : ox1400000
res_mask : ox1fffff
windows : 0
GCs : 1
fonts : 0
pixmaps : 0
pictures : 0
glyphsets : 0
colormaps : 0
passive grabs : 0
cursors : 0
unknowns : 0
pixmap bytes : 0
other bytes : ~24
total bytes : ~24
11 - <unknown> ( PID: ? ):
res_base : ox200000
res_mask : ox1fffff
windows : 0
GCs : 1
fonts : 0
pixmaps : 0
pictures : 0
glyphsets : 0
colormaps : 0
passive grabs : 0
cursors : 0
unknowns : 0
pixmap bytes : 0
other bytes : ~24
total bytes : ~2
Reply at: https://bugs.launchpad.net/gnome-system-
monitor/+bug/484521/comments/5
------------------------------------------------------------------------
On 2009-07-02T19:21:59+00:00 cfox wrote:
As another followup, using my laptop with Xorg 1.6.1.901 and Intel 2.7.1
drivers, I get similar results: gvim will release most of its memory
when I kill it, and Qt based applications will not release memory when
they exit.
As an added bonus, Qt based applications will hang or crash the X server
when too many are opened at once. This occurs well below the normal 255
client limit, and the X server normally crashes out entirely. X's
memory usage does not have to be particularly huge for this to occur,
even small windows like a kdialog can cause the crash while there is
still free memory.
Reply at: https://bugs.launchpad.net/gnome-system-
monitor/+bug/484521/comments/6
------------------------------------------------------------------------
On 2009-07-08T21:10:17+00:00 cfox wrote:
I went back in time with an Ubuntu 8.10 installation to see how it
behaved, and got the same basic memory bug with the X.org 1.5 series and
Qt 4.4.3 using the Intel driver without compositing. The older software
leaks, but overall RAM usage was not as great as with the newer versions
which use more RAM and therefore leak a lot faster.
Reply at: https://bugs.launchpad.net/gnome-system-
monitor/+bug/484521/comments/7
------------------------------------------------------------------------
On 2009-07-09T00:23:30+00:00 Michel-daenzer wrote:
If valgrind --leak-check=full doesn't show where the apparent leaks are
coming from, you could try running the X server in something like odin
(http://cgit.freedesktop.org/~ickle/odin), which shows where memory was
allocated at any time.
Reply at: https://bugs.launchpad.net/gnome-system-
monitor/+bug/484521/comments/8
------------------------------------------------------------------------
On 2009-09-29T03:23:26+00:00 Alex Pyattaev wrote:
OK, 1 more drop into endless sea:
Hardware: GF6150 GO (mobile integrated chip)
2GB RAM.
no swapfile (why would i need it with 2 GB RAM?)
Software:
gentoo linux 2.6.31
nvidia-driver 185.31 - 195.25, nv driver from kernel (does not really matter)
X.Org X Server 1.6.3.901 (1.6.4 RC 1)
Release Date: 2009-8-25
X Protocol Version 11, Revision 0
Build Operating System: Linux 2.6.30-gentoo-r6-localbuild x86_64
Current Operating System: Linux headhunter.homelinux.org 2.6.31-gentoo-localbuild #2 SMP PREEMPT Sat Sep 19 19:44:00 EEST 2009 x86_64
Build Date: 17 September 2009 02:29:14AM
Of course, it is somehow beta-based system, but anyway, X seems to gradually consume RAM and CPU resources. After night of doing nothing X consumes about 200 MB, with active usage (opening and closing windows mostly affects) RAM usage gradually grows up to 1.6 GB and maybe more. CPU usage on memory allocation grows as well, so basically at 1.6 GB usage kernel has big troubles allocating big chunks of RAM (I have tried some 3D apps known to allocate big chunks (spring RTS) and they refused to start (ram allocation failed)). So basically X consumes and fragments hell amount of RAM without reason. It started with recent versions of X, but I can't tell which exactly since I don't usually leave my computer up for long. I'll do valgrind debug ASAP.
Reply at: https://bugs.launchpad.net/gnome-system-
monitor/+bug/484521/comments/9
** Changed in: xorg-server
Importance: Unknown => High
--
Nautilus causes memory leak in Xorg?
https://bugs.launchpad.net/bugs/484521
You received this bug notification because you are a member of Registry
Administrators, which is the registrant for Gnome System Monitor.
