touch-packages team mailing list archive

[Kyle Beauchamp <kyleabeauchamp@xxxxxxxxx>]

I have a fully updated 15.04 machine and I'm still having issues with
apt cache eating up my disk.  I get into a situation where `http` is my
most active process and my apt partial cache is growing quickly.  See
the logs below, which show the top processes, the large file sizes in
the partial cache, and the file size growth in the partial cache.  When
the disk fills completely, the system is then no longer able to boot
into unity and I am required to manually delete files with a command
line login.  ..

top:
 2581 root      20   0   40260   5012   4660 R  97.0  0.0  11:04.46 http                                                                                                                                            
 2582 root      20   0   40260   4992   4636 R  97.0  0.0  10:40.69 http    

:~$ ls /var/lib/apt/lists/partial/ -lS|head -n 5
total 93832500
-rw-r--r-- 1 root root 49198669906 May 29 10:41 security.ubuntu.com_ubuntu_dists_vivid-security_universe_binary-i386_Packages.bz2
-rw-r--r-- 1 root root 46855568434 May 29 10:41 us.archive.ubuntu.com_ubuntu_dists_vivid-updates_main_binary-i386_Packages.bz2
-rw-r--r-- 1 root root     7017495 Apr 24 14:45 us.archive.ubuntu.com_ubuntu_dists_vivid_universe_source_Sources.bz2
-rw-r--r-- 1 root root     6486447 Apr 24 14:40 us.archive.ubuntu.com_ubuntu_dists_vivid_universe_binary-i386_Packages.bz2
:~$ ls /var/lib/apt/lists/partial/ -lS|head -n 5
total 94526900
-rw-r--r-- 1 root root 49566396416 May 29 10:41 security.ubuntu.com_ubuntu_dists_vivid-security_universe_binary-i386_Packages.bz2
-rw-r--r-- 1 root root 47198911538 May 29 10:41 us.archive.ubuntu.com_ubuntu_dists_vivid-updates_main_binary-i386_Packages.bz2
-rw-r--r-- 1 root root     7017495 Apr 24 14:45 us.archive.ubuntu.com_ubuntu_dists_vivid_universe_source_Sources.bz2
-rw-r--r-- 1 root root     6486447 Apr 24 14:40 us.archive.ubuntu.com_ubuntu_dists_vivid_universe_binary-i386_Packages.bz2

-- 
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to apt in Ubuntu.
https://bugs.launchpad.net/bugs/1445239

Title:
  apt's lists/partial fills disk

Status in APT:
  New
Status in apt package in Ubuntu:
  Fix Released
Status in apt source package in Vivid:
  Confirmed

Bug description:
  Since I upgraded to vivid, apt fills the disk space at
  /var/lib/apt/lists/partial. This was triggered by the automatic update
  every day, and as a solution I killed the processes named `http`
  spawned by apt, and removed the offending file from lists/partial
  manually. Until now, `apt-get update` and `apt-get upgrade` had been
  working. However, today, when I tried `apt-get update`, it got stuck
  in some file, continuously increasing its size when at 100%.

  I tried inspecting the file to see if there is a repetition of data,
  and there seems to be, as I have suspected.

  First of all, the file is it.archive.ubuntu
  .com_ubuntu_dists_vivid_main_binary-i386_Packages.bz2 this time, but
  I'm not sure if the previous time it was also this file.

  I used `tail` and `hexdump` to figure out whether a certain pattern
  shows up in the file. Here are some examples:

  $ tail -c 200000 it.archive.ubuntu.com_ubuntu_dists_vivid_main_binary-i386_Packages.bz2 | hexdump | grep '\<bbdf\>'
  0000c80 8d9e 63ad bbdf 40a7 3f3e 2b4a 9b95 aa5a
  0010c80 8d9e 63ad bbdf 40a7 3f3e 2b4a 9b95 aa5a
  0020c80 8d9e 63ad bbdf 40a7 3f3e 2b4a 9b95 aa5a
  0030c80 8d9e 63ad bbdf 40a7 3f3e 2b4a 9b95 aa5a

  $ tail -c 200000 it.archive.ubuntu.com_ubuntu_dists_vivid_main_binary-i386_Packages.bz2 | hexdump | grep '\<9b95\>'
  0000c80 8d9e 63ad bbdf 40a7 3f3e 2b4a 9b95 aa5a
  0010c80 8d9e 63ad bbdf 40a7 3f3e 2b4a 9b95 aa5a
  0020c80 8d9e 63ad bbdf 40a7 3f3e 2b4a 9b95 aa5a
  0030c80 8d9e 63ad bbdf 40a7 3f3e 2b4a 9b95 aa5a

  $ tail -c 200000 it.archive.ubuntu.com_ubuntu_dists_vivid_main_binary-i386_Packages.bz2 | hexdump | grep '\<1234\>'
  0002630 fd50 b08d 7082 1234 85b3 a61e 2921 e0cd
  0012630 fd50 b08d 7082 1234 85b3 a61e 2921 e0cd
  0022630 fd50 b08d 7082 1234 85b3 a61e 2921 e0cd

  $ tail -c 200000 it.archive.ubuntu.com_ubuntu_dists_vivid_main_binary-i386_Packages.bz2 | hexdump | grep '\<76a8\>'
  000fa90 747d 76a8 078a 67b6 60d9 83f5 5ff3 787c
  001fa90 747d 76a8 078a 67b6 60d9 83f5 5ff3 787c
  002fa90 747d 76a8 078a 67b6 60d9 83f5 5ff3 787c

  As you can see, it seems that after the error, a certain pattern seems
  to repeat every exactly 0x10000 bytes. I attached a copy of those
  0x10000 bytes that keep repeating (from an arbitrary offset).

  ---

  $ lsb_release -rd
  Description:	Ubuntu Vivid Vervet (development branch)
  Release:	15.04

  $ apt-cache policy apt
  apt:
    Installed: 1.0.9.7ubuntu4
    Candidate: 1.0.9.7ubuntu4
    Version table:
   *** 1.0.9.7ubuntu4 0
          500 http://it.archive.ubuntu.com/ubuntu/ vivid/main amd64 Packages
          100 /var/lib/dpkg/status

  ---

  Do let me know if I can test further to see what could be wrong. I'll
  keep an eye next times the behavior repeat to make sure the file is
  the same / the pattern is the same and I'll report back. If you think
  the problem could be the Italy server, let me know and I'll try with a
  different one.

  ProblemType: Bug
  DistroRelease: Ubuntu 15.04
  Package: apt 1.0.9.7ubuntu4
  ProcVersionSignature: Ubuntu 3.19.0-13.13-generic 3.19.3
  Uname: Linux 3.19.0-13-generic x86_64
  NonfreeKernelModules: nvidia
  ApportVersion: 2.17-0ubuntu2
  Architecture: amd64
  CurrentDesktop: Unity
  Date: Fri Apr 17 00:04:05 2015
  SourcePackage: apt
  UpgradeStatus: Upgraded to vivid on 2015-03-28 (19 days ago)

To manage notifications about this bug go to:
https://bugs.launchpad.net/apt/+bug/1445239/+subscriptions