ac100 team mailing list archive
Mailing list archive
[Bug 655831] Re: update-apt-xapian-index bogs down system
A weekly fatal unable to fork error message from /etc/cron.weekly/apt-
xapian-index on a server with 512MB memory with a web server and
database server running leads to this bug. That cron job calls /usr/sbin
/update-apt-xapian-index, which is a Python script that imports axi and
axi.indexer. Those Python packages are the heart of the problem.
On an idle system with 512M memory and 768M swap, while /etc/cron.weekly
/apt-xapian-index was running, I used free to see memory and swap used.
It rose to then peaked at this:
total used free
Mem: 503528 497736 5792
Swap: 786428 231196 555232
Total used memory is 728M.
After ending, it immediately dropped back to this:
total used free
Mem: 503528 259160 244368
Swap: 786428 224484 561944
Total used memory is 483M.
I conclude that apt-xapian-index consumes the difference, which is 245M.
Running "apt-cache stats" I see at the end "Total space accounted for:
Therefore, it takes 245M to sort and index 26M of information. This
seems conclusive that the algorithms, containers, and/or functions
chosen are very inefficient. It should definitely not require 10 times
the memory space of what is being indexed.
The solution is to change the sorting algorithm in the python axi and
axi.indexer modules. The first priority is to switch to an algorithm
that consumes a whole lot less memory (i.e., each step of the algorithm
keeps less objects in memory), and it will stop crashing and stop
thrashing (memory swapping to disk).
The second priority is of lesser importance (because renice can solve a
lot of the effect), which is to switch to an algorithm that takes a lot
less time to run (i.e., takes fewer steps to complete), and it will stop
consuming so much CPU for so long.
You received this bug notification because you are a member of AC100
Team, which is subscribed to AC100_enablement.
update-apt-xapian-index bogs down system
Status in Enablement project for the Toshiba AC100 NetBook:
Status in Apt Xapian Index:
Status in “apt-xapian-index” package in Ubuntu:
Binary package hint: apt-xapian-index
This is related to bug 363695 but is not a dupe - that has been
every time the update-apt-xapian-index occurs, the system crawls.
Possibly related to the I/O responsiveness bug of bug 131094.
The process is extremely intrusive, taking simple things such as flash
video and throwing the performance down the toilet.
While this apport bug was collected via a Lucid machine this also
DistroRelease: Ubuntu 10.04
Package: apt-xapian-index 0.25ubuntu2
ProcVersionSignature: Ubuntu 2.6.32-25.44-generic 18.104.22.168+drm33.7
Uname: Linux 2.6.32-25-generic i686
Date: Wed Oct 6 12:32:35 2010
InstallationMedia: Ubuntu 10.04 LTS "Lucid Lynx" - Release i386 (20100429)
To manage notifications about this bug go to: