touch-packages team mailing list archive

[Jack Miller <jack@xxxxxxxxxxx>]

Public bug reported:

We have a few ppc64el machines in our lab running Ubuntu 15.04 and the
daily apt-get cronjob that runs will intermittently use  all of its
memory and fill its disk.

Looking at the disk, it appears that the sources and packages files it's
downloading are unreasonably huge.

$ ls -l /var/lib/apt/lists/partial
...
-rw-r--r-- 1 root root 480265872092 Jul 28 06:15 archive.ubuntu.com_ubuntu_dists_vivid-updates_main_source_Sources.bz2
-rw-r--r-- 1 root root 462813724672 Jul 28 03:04 ports.ubuntu.com_ubuntu-ports_dists_vivid-updates_universe_binary-ppc64el_Packages.bz2
...

All of the other files look normal, maybe a few megabytes at most (as
you would expect from apt) but these two are almost half a terabyte in
size apiece which is clearly wrong and may only be limited by our disk
space (it's got a 1TB disk).

It appears that /usr/lib/apt/methods/http is still running.

$ ps aux | grep apt
root      1675  0.0  0.0   3136  1536 ?        S    06:25   0:00 /bin/sh /etc/cron.daily/apt
root      1749  0.0  0.0  12864  4544 ?        S    06:28   0:00 apt-get -qq -y update
root      1752 79.9  0.0  12032  3840 ?        R    06:28 102:30 /usr/lib/apt/methods/http
root      1753 77.7  0.0  12032  3840 ?        R    06:28  99:39 /usr/lib/apt/methods/http
root      1755  0.0  0.0  11328  3136 ?        S    06:28   0:00 /usr/lib/apt/methods/gpgv
root      1763  0.0  0.0  14912  6848 ?        S    06:28   0:05 /usr/lib/apt/methods/bzip2
root      1766  0.0  0.0  11328  3072 ?        S    06:28   0:00 /usr/lib/apt/methods/copy

The system has almost all of its 32G of RAM filled as well, looking at
the rest of the ps output it doesn't look like the used memory is
actually attached to a process, but it's in the cache, so I assume it's
just a side effect of trying to work with the gigantic files.

$ free -m
             total       used       free     shared    buffers     cached
Mem:         32482      30571       1911         19         90      28860
-/+ buffers/cache:       1620      30861
Swap:        38607          0      38607

We haven't seen any similar trouble with 14.04 or 14.10.

Here's the version information

$ lsb_release -rd
Description:    Ubuntu 15.04
Release:        15.04

$ apt-get -v
apt 1.0.9.7ubuntu4 for ppc64el compiled on Apr  7 2015 14:44:13
Supported modules:
*Ver: Standard .deb
*Pkg:  Debian dpkg interface (Priority 30)
 Pkg:  Debian APT solver interface (Priority -1000)
 S.L: 'deb' Standard Debian binary tree
 S.L: 'deb-src' Standard Debian source tree
 Idx: Debian Source Index
 Idx: Debian Package Index
 Idx: Debian Translation Index
 Idx: Debian dpkg status file
 Idx: EDSP scenario file

** Affects: apt (Ubuntu)
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to apt in Ubuntu.
https://bugs.launchpad.net/bugs/1479455

Title:
  apt-get leaking memory and filling disk

Status in apt package in Ubuntu:
  New

Bug description:
  We have a few ppc64el machines in our lab running Ubuntu 15.04 and the
  daily apt-get cronjob that runs will intermittently use  all of its
  memory and fill its disk.

  Looking at the disk, it appears that the sources and packages files
  it's downloading are unreasonably huge.

  $ ls -l /var/lib/apt/lists/partial
  ...
  -rw-r--r-- 1 root root 480265872092 Jul 28 06:15 archive.ubuntu.com_ubuntu_dists_vivid-updates_main_source_Sources.bz2
  -rw-r--r-- 1 root root 462813724672 Jul 28 03:04 ports.ubuntu.com_ubuntu-ports_dists_vivid-updates_universe_binary-ppc64el_Packages.bz2
  ...

  All of the other files look normal, maybe a few megabytes at most (as
  you would expect from apt) but these two are almost half a terabyte in
  size apiece which is clearly wrong and may only be limited by our disk
  space (it's got a 1TB disk).

  It appears that /usr/lib/apt/methods/http is still running.

  $ ps aux | grep apt
  root      1675  0.0  0.0   3136  1536 ?        S    06:25   0:00 /bin/sh /etc/cron.daily/apt
  root      1749  0.0  0.0  12864  4544 ?        S    06:28   0:00 apt-get -qq -y update
  root      1752 79.9  0.0  12032  3840 ?        R    06:28 102:30 /usr/lib/apt/methods/http
  root      1753 77.7  0.0  12032  3840 ?        R    06:28  99:39 /usr/lib/apt/methods/http
  root      1755  0.0  0.0  11328  3136 ?        S    06:28   0:00 /usr/lib/apt/methods/gpgv
  root      1763  0.0  0.0  14912  6848 ?        S    06:28   0:05 /usr/lib/apt/methods/bzip2
  root      1766  0.0  0.0  11328  3072 ?        S    06:28   0:00 /usr/lib/apt/methods/copy

  The system has almost all of its 32G of RAM filled as well, looking at
  the rest of the ps output it doesn't look like the used memory is
  actually attached to a process, but it's in the cache, so I assume
  it's just a side effect of trying to work with the gigantic files.

  $ free -m
               total       used       free     shared    buffers     cached
  Mem:         32482      30571       1911         19         90      28860
  -/+ buffers/cache:       1620      30861
  Swap:        38607          0      38607

  We haven't seen any similar trouble with 14.04 or 14.10.

  Here's the version information

  $ lsb_release -rd
  Description:    Ubuntu 15.04
  Release:        15.04

  $ apt-get -v
  apt 1.0.9.7ubuntu4 for ppc64el compiled on Apr  7 2015 14:44:13
  Supported modules:
  *Ver: Standard .deb
  *Pkg:  Debian dpkg interface (Priority 30)
   Pkg:  Debian APT solver interface (Priority -1000)
   S.L: 'deb' Standard Debian binary tree
   S.L: 'deb-src' Standard Debian source tree
   Idx: Debian Source Index
   Idx: Debian Package Index
   Idx: Debian Translation Index
   Idx: Debian dpkg status file
   Idx: EDSP scenario file

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/apt/+bug/1479455/+subscriptions