touch-packages team mailing list archive

[Michael Vogt <michael.vogt@xxxxxxxxxxxxx>]

** Description changed:

- Since I upgraded to vivid, apt fills the disk space at
- /var/lib/apt/lists/partial. This was triggered by the automatic update
- every day, and as a solution I killed the processes named `http` spawned
- by apt, and removed the offending file from lists/partial manually.
- Until now, `apt-get update` and `apt-get upgrade` had been working.
- However, today, when I tried `apt-get update`, it got stuck in some
- file, continuously increasing its size when at 100%.
+ Test case:
+ - there is a artificial test in test/integration now
+ - as this is server side dependent a regression test should be enough
+  
+ Since I upgraded to vivid, apt fills the disk space at /var/lib/apt/lists/partial. This was triggered by the automatic update every day, and as a solution I killed the processes named `http` spawned by apt, and removed the offending file from lists/partial manually. Until now, `apt-get update` and `apt-get upgrade` had been working. However, today, when I tried `apt-get update`, it got stuck in some file, continuously increasing its size when at 100%.
  
  I tried inspecting the file to see if there is a repetition of data, and
  there seems to be, as I have suspected.
  
  First of all, the file is it.archive.ubuntu
  .com_ubuntu_dists_vivid_main_binary-i386_Packages.bz2 this time, but I'm
  not sure if the previous time it was also this file.
  
  I used `tail` and `hexdump` to figure out whether a certain pattern
  shows up in the file. Here are some examples:
  
  $ tail -c 200000 it.archive.ubuntu.com_ubuntu_dists_vivid_main_binary-i386_Packages.bz2 | hexdump | grep '\<bbdf\>'
  0000c80 8d9e 63ad bbdf 40a7 3f3e 2b4a 9b95 aa5a
  0010c80 8d9e 63ad bbdf 40a7 3f3e 2b4a 9b95 aa5a
  0020c80 8d9e 63ad bbdf 40a7 3f3e 2b4a 9b95 aa5a
  0030c80 8d9e 63ad bbdf 40a7 3f3e 2b4a 9b95 aa5a
  
  $ tail -c 200000 it.archive.ubuntu.com_ubuntu_dists_vivid_main_binary-i386_Packages.bz2 | hexdump | grep '\<9b95\>'
  0000c80 8d9e 63ad bbdf 40a7 3f3e 2b4a 9b95 aa5a
  0010c80 8d9e 63ad bbdf 40a7 3f3e 2b4a 9b95 aa5a
  0020c80 8d9e 63ad bbdf 40a7 3f3e 2b4a 9b95 aa5a
  0030c80 8d9e 63ad bbdf 40a7 3f3e 2b4a 9b95 aa5a
  
  $ tail -c 200000 it.archive.ubuntu.com_ubuntu_dists_vivid_main_binary-i386_Packages.bz2 | hexdump | grep '\<1234\>'
  0002630 fd50 b08d 7082 1234 85b3 a61e 2921 e0cd
  0012630 fd50 b08d 7082 1234 85b3 a61e 2921 e0cd
  0022630 fd50 b08d 7082 1234 85b3 a61e 2921 e0cd
  
  $ tail -c 200000 it.archive.ubuntu.com_ubuntu_dists_vivid_main_binary-i386_Packages.bz2 | hexdump | grep '\<76a8\>'
  000fa90 747d 76a8 078a 67b6 60d9 83f5 5ff3 787c
  001fa90 747d 76a8 078a 67b6 60d9 83f5 5ff3 787c
  002fa90 747d 76a8 078a 67b6 60d9 83f5 5ff3 787c
  
  As you can see, it seems that after the error, a certain pattern seems
  to repeat every exactly 0x10000 bytes. I attached a copy of those
  0x10000 bytes that keep repeating (from an arbitrary offset).
  
  ---
  
  $ lsb_release -rd
  Description:	Ubuntu Vivid Vervet (development branch)
  Release:	15.04
  
  $ apt-cache policy apt
  apt:
-   Installed: 1.0.9.7ubuntu4
-   Candidate: 1.0.9.7ubuntu4
-   Version table:
-  *** 1.0.9.7ubuntu4 0
-         500 http://it.archive.ubuntu.com/ubuntu/ vivid/main amd64 Packages
-         100 /var/lib/dpkg/status
+   Installed: 1.0.9.7ubuntu4
+   Candidate: 1.0.9.7ubuntu4
+   Version table:
+  *** 1.0.9.7ubuntu4 0
+         500 http://it.archive.ubuntu.com/ubuntu/ vivid/main amd64 Packages
+         100 /var/lib/dpkg/status
  
  ---
  
  Do let me know if I can test further to see what could be wrong. I'll
  keep an eye next times the behavior repeat to make sure the file is the
  same / the pattern is the same and I'll report back. If you think the
  problem could be the Italy server, let me know and I'll try with a
  different one.
  
  ProblemType: Bug
  DistroRelease: Ubuntu 15.04
  Package: apt 1.0.9.7ubuntu4
  ProcVersionSignature: Ubuntu 3.19.0-13.13-generic 3.19.3
  Uname: Linux 3.19.0-13-generic x86_64
  NonfreeKernelModules: nvidia
  ApportVersion: 2.17-0ubuntu2
  Architecture: amd64
  CurrentDesktop: Unity
  Date: Fri Apr 17 00:04:05 2015
  SourcePackage: apt
  UpgradeStatus: Upgraded to vivid on 2015-03-28 (19 days ago)

** Changed in: apt (Ubuntu Vivid)
       Status: Confirmed => In Progress

-- 
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to apt in Ubuntu.
https://bugs.launchpad.net/bugs/1445239

Title:
  apt's lists/partial fills disk

Status in APT:
  New
Status in apt package in Ubuntu:
  Fix Released
Status in apt source package in Vivid:
  In Progress

Bug description:
  Test case:
  - there is a artificial test in test/integration now
  - as this is server side dependent a regression test should be enough
   
  Since I upgraded to vivid, apt fills the disk space at /var/lib/apt/lists/partial. This was triggered by the automatic update every day, and as a solution I killed the processes named `http` spawned by apt, and removed the offending file from lists/partial manually. Until now, `apt-get update` and `apt-get upgrade` had been working. However, today, when I tried `apt-get update`, it got stuck in some file, continuously increasing its size when at 100%.

  I tried inspecting the file to see if there is a repetition of data,
  and there seems to be, as I have suspected.

  First of all, the file is it.archive.ubuntu
  .com_ubuntu_dists_vivid_main_binary-i386_Packages.bz2 this time, but
  I'm not sure if the previous time it was also this file.

  I used `tail` and `hexdump` to figure out whether a certain pattern
  shows up in the file. Here are some examples:

  $ tail -c 200000 it.archive.ubuntu.com_ubuntu_dists_vivid_main_binary-i386_Packages.bz2 | hexdump | grep '\<bbdf\>'
  0000c80 8d9e 63ad bbdf 40a7 3f3e 2b4a 9b95 aa5a
  0010c80 8d9e 63ad bbdf 40a7 3f3e 2b4a 9b95 aa5a
  0020c80 8d9e 63ad bbdf 40a7 3f3e 2b4a 9b95 aa5a
  0030c80 8d9e 63ad bbdf 40a7 3f3e 2b4a 9b95 aa5a

  $ tail -c 200000 it.archive.ubuntu.com_ubuntu_dists_vivid_main_binary-i386_Packages.bz2 | hexdump | grep '\<9b95\>'
  0000c80 8d9e 63ad bbdf 40a7 3f3e 2b4a 9b95 aa5a
  0010c80 8d9e 63ad bbdf 40a7 3f3e 2b4a 9b95 aa5a
  0020c80 8d9e 63ad bbdf 40a7 3f3e 2b4a 9b95 aa5a
  0030c80 8d9e 63ad bbdf 40a7 3f3e 2b4a 9b95 aa5a

  $ tail -c 200000 it.archive.ubuntu.com_ubuntu_dists_vivid_main_binary-i386_Packages.bz2 | hexdump | grep '\<1234\>'
  0002630 fd50 b08d 7082 1234 85b3 a61e 2921 e0cd
  0012630 fd50 b08d 7082 1234 85b3 a61e 2921 e0cd
  0022630 fd50 b08d 7082 1234 85b3 a61e 2921 e0cd

  $ tail -c 200000 it.archive.ubuntu.com_ubuntu_dists_vivid_main_binary-i386_Packages.bz2 | hexdump | grep '\<76a8\>'
  000fa90 747d 76a8 078a 67b6 60d9 83f5 5ff3 787c
  001fa90 747d 76a8 078a 67b6 60d9 83f5 5ff3 787c
  002fa90 747d 76a8 078a 67b6 60d9 83f5 5ff3 787c

  As you can see, it seems that after the error, a certain pattern seems
  to repeat every exactly 0x10000 bytes. I attached a copy of those
  0x10000 bytes that keep repeating (from an arbitrary offset).

  ---

  $ lsb_release -rd
  Description:	Ubuntu Vivid Vervet (development branch)
  Release:	15.04

  $ apt-cache policy apt
  apt:
    Installed: 1.0.9.7ubuntu4
    Candidate: 1.0.9.7ubuntu4
    Version table:
   *** 1.0.9.7ubuntu4 0
          500 http://it.archive.ubuntu.com/ubuntu/ vivid/main amd64 Packages
          100 /var/lib/dpkg/status

  ---

  Do let me know if I can test further to see what could be wrong. I'll
  keep an eye next times the behavior repeat to make sure the file is
  the same / the pattern is the same and I'll report back. If you think
  the problem could be the Italy server, let me know and I'll try with a
  different one.

  ProblemType: Bug
  DistroRelease: Ubuntu 15.04
  Package: apt 1.0.9.7ubuntu4
  ProcVersionSignature: Ubuntu 3.19.0-13.13-generic 3.19.3
  Uname: Linux 3.19.0-13-generic x86_64
  NonfreeKernelModules: nvidia
  ApportVersion: 2.17-0ubuntu2
  Architecture: amd64
  CurrentDesktop: Unity
  Date: Fri Apr 17 00:04:05 2015
  SourcePackage: apt
  UpgradeStatus: Upgraded to vivid on 2015-03-28 (19 days ago)

To manage notifications about this bug go to:
https://bugs.launchpad.net/apt/+bug/1445239/+subscriptions