touch-packages team mailing list archive

[Rolf Leggewie <948461@xxxxxxxxxxxxxxxxxx>]

Hardy has seen the end of its life and is no longer receiving any
updates. Marking the Hardy task for this ticket as "Won't Fix".

** Changed in: cloud-init (Ubuntu Hardy)
       Status: Triaged => Won't Fix

-- 
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to apt in Ubuntu.
https://bugs.launchpad.net/bugs/948461

Title:
  apt-get hashsum/size mismatch because s3 mirrors don't support http
  pipelining correctly

Status in “apt” package in Ubuntu:
  Confirmed
Status in “cloud-init” package in Ubuntu:
  Fix Released
Status in “apt” source package in Lucid:
  Won't Fix
Status in “cloud-init” source package in Lucid:
  Fix Released
Status in “apt” source package in Maverick:
  Won't Fix
Status in “cloud-init” source package in Maverick:
  Fix Released
Status in “apt” source package in Natty:
  Won't Fix
Status in “cloud-init” source package in Natty:
  Fix Released
Status in “apt” source package in Oneiric:
  Won't Fix
Status in “cloud-init” source package in Oneiric:
  Fix Released
Status in “apt” source package in Precise:
  Won't Fix
Status in “cloud-init” source package in Precise:
  Fix Released
Status in “apt” source package in Hardy:
  Won't Fix
Status in “cloud-init” source package in Hardy:
  Won't Fix

Bug description:
  [Problem]
  When using S3 mirrors, the apt client requests files A,B,C,D in the same request, but S3 responds with files B,C,D,A or some other arbitrary ordering.  Apt then saves the files under the wrong file names.

  This is due to a bug in Amazon AWS S3, where http pipelining does not
  work in violation of RFC 2068.  apt uses http pipelining currently, so
  this prevents apt from using mirrors hosted on S3 cloud instances.

  [Impact]
  Blocks use of S3 mirrors for providing ubuntu updates.  Affects Hardy, Lucid, Maverick, Oneiric, Natty, and Precise.

  Use of S3 mirrors is considered as a way of enhancing our mirror
  services.  Currently mirrors have moments of instability, so having
  the option of switching to S3 mirrors is very important for update
  reliability.

  [Development Fix]
  Fixed in Precise's cloud-init as of these comments, which adds an apt_pipelining option, turned off by default but configurable.
  http://bazaar.launchpad.net/~cloud-init-dev/cloud-init/trunk/revision/536
  http://bazaar.launchpad.net/~cloud-init-dev/cloud-init/trunk/revision/537

  [Stable Fix]
  TODO

  [Text Case]
  1) Launch an instance in EC2
  2) Run: sed -i "s,ec2.archive.ubuntu.com,ec2.archive.ubuntu.com.s3.amazonaws.com,g" /etc/apt/sources.list
  3.) Set your apt debug levels to whatever you want
  4) Run "apt-get -y update"
  5) Run "apt-get -y install firefox xorg libreoffice > /tmp/install.log 2>&1"
  Broken Behavior:  Cached files are being misnamed
  Fixed Behavior:  Cached files are properly named

  The S3 EC2 mirrors seem to reproduce it most reliably.   The
  downloaded files are whole and match the proper checksums, but they
  have the wrong content.

  [Regression Potential]
  http pipelining is an optimization technique in apt which allows multiple requests to be made at the same time.  When it works, it provides a small performance boost, but when it doesn't the failures are arbitrary and not adequately explained.

  Thus, the expected regression is slightly slower apt behavior for
  non-S3 cloud users.  However, testing shows the impact is less than a
  few seconds so will likely be unnoticed.  In the off chance that it
  does inadvertently cause non-trivial performance impact, the change is
  configurable on a per-host instance.

  [Original Report]
  [SRU: Cloud-Init]
  r536 and r537 contains a fix that:
    1.) Disables APT pipelining on first boot for new Cloud Images instances
    2.) Disables APT pipelining on installation for existing cloud-images.

  The rational for this SRU is due to the Mirror situation for EC2 Cloud
  images. Currently, the mirrors are proving to have moments of
  instability that is causing both end-user and development pain. In
  order to address this, we built out Amazon AWS S3 mirrors.
  Unfortantly, due to a bug in S3, apt http pipeling does not work with
  S3. As a result, in order to provide enhanced availability for the
  Cloud Image repositories in EC2, apt http pipelining needs to be
  disabled ahead of flipping the new S3 mirrors as the active mirrors.

  The performance impact of this change is unnoticable and in testing,
  has proven to provide no ill effect. http pipeplining allows for
  multiple requests to be made at the same time. The apt change log
  states that http pipelining is a "micro-enhancement" and within the
  EC2 environment shaves a few seconds of many requests. When apt http
  pipeling works, at best, it saves a few seconds, when it fails against
  a buggy HTTP1.1 server or behind a proxy that does not support it, the
  failures are arbitrary and not adiquetly explained. This change will
  enhance the Cloud Images by making apt request packages sequentially
  instead of in batches and prevent arbitrary failures caused by non-
  compliant web servers. Cloud image users, both on EC2 and outside,
  will benefit from this fix.

  http://bazaar.launchpad.net/~cloud-init-dev/cloud-init/trunk/revision/536
  http://bazaar.launchpad.net/~cloud-init-dev/cloud-init/trunk/revision/537

  _______________________________________________________________________________________
  apt-get is appears to be mangling the local caching file names, which is appearing as a hashsum or size mismatch.

  Evidence below:

  _______________________________________________________________________________________
  Output of "apt-get -y install firefox":
  Failed to fetch http://us-east-1.ec2.archive.ubuntu.com.s3.amazonaws.com/ubuntu/pool/main/p/python-defer/python-defer_1.0.2+bzr481-1_all.deb  Size mismatch
  Failed to fetch http://us-east-1.ec2.archive.ubuntu.com.s3.amazonaws.com/ubuntu/pool/main/a/aptdaemon/python-aptdaemon_0.43+bzr769-0ubuntu1_all.deb  Size mismatch
  Failed to fetch http://us-east-1.ec2.archive.ubuntu.com.s3.amazonaws.com/ubuntu/pool/main/a/aptdaemon/aptdaemon_0.43+bzr769-0ubuntu1_all.deb  Size mismatch
  Failed to fetch http://us-east-1.ec2.archive.ubuntu.com.s3.amazonaws.com/ubuntu/pool/main/u/ubufox/xul-ext-ubufox_2.0-0ubuntu1_all.deb  Size mismatch

  _______________________________________________________________________________________

  apt-get -y install firefox with debug for HTTP turned on:
  Get:63 http://us-east-1.ec2.archive.ubuntu.com.s3.amazonaws.com/ubuntu/ precise/main python-defer all 1.0.2+bzr481-1 [10.9 kB]
  HTTP/1.1 200 OK^M
  x-amz-id-2: EXccd6GVMfE6ly4SYdwy313VK42d/gI3ncyxmaotuVIMbBBi6FJkuDYWrzLw7vXE^M
  x-amz-request-id: 7572BAB5E6FFFAC0^M
  Date: Tue, 06 Mar 2012 20:06:13 GMT^M
  Last-Modified: Fri, 03 Feb 2012 08:54:27 GMT^M
  ETag: "6c07f8db615cc32657b1cc57450a1608"^M
  Accept-Ranges: bytes^M
  Content-Type: application/x-debian-package^M
  Content-Length: 15142^M
  Server: AmazonS3^M
  ^M
  Get:64 http://us-east-1.ec2.archive.ubuntu.com.s3.amazonaws.com/ubuntu/ precise/main python-aptdaemon all 0.43+bzr769-0ubuntu1 [80.3 kB]
  HTTP/1.1 200 OK^M
  x-amz-id-2: IcNECY1OtzHOXnTnTW3fqoJGyfdODq+qRXXGYTDEOJxfw0CP9UH8EUSllVR3Gf3F^M
  x-amz-request-id: 5DEEEE629E3BA012^M
  Date: Tue, 06 Mar 2012 20:06:13 GMT^M
  Last-Modified: Fri, 02 Mar 2012 20:55:40 GMT^M
  ETag: "501373631a241847cf76d13aea9264b3"^M
  Accept-Ranges: bytes^M
  Content-Type: application/x-debian-package^M
  Content-Length: 56824^M
  Server: AmazonS3^M
  ^M
  Get:65 http://us-east-1.ec2.archive.ubuntu.com.s3.amazonaws.com/ubuntu/ precise/main aptdaemon all 0.43+bzr769-0ubuntu1 [15.1 kB]
  HTTP/1.1 200 OK^M
  x-amz-id-2: HHNSOZKH74D4I1XCaOiwDOzIWxIVJ88ibfKxvIp/4NpkczzPmsOhMqxrTj2T02qO^M
  x-amz-request-id: A6FD3143E6A0BA1A^M
  Date: Tue, 06 Mar 2012 20:06:34 GMT^M
  Last-Modified: Sat, 21 Jan 2012 00:48:55 GMT^M
  ETag: "3653165af1f20a437a14632ce0a2e6c2"^M
  Accept-Ranges: bytes^M
  Content-Type: application/x-debian-package^M
  Content-Length: 10902^M
  Server: AmazonS3^M

  _______________________________________________________________________________________
  Evaluation of the MD5sums of downloaded failed items:
  root@ip-10-6-85-206:/var/cache/apt/archives/partial# md5sum *
  501373631a241847cf76d13aea9264b3  aptdaemon_0.43+bzr769-0ubuntu1_all.deb
  6c07f8db615cc32657b1cc57450a1608  python-aptdaemon_0.43+bzr769-0ubuntu1_all.deb
  566f7c92a9fcd0c5c40d08a977176291  python-defer_1.0.2+bzr481-1_all.deb
  3653165af1f20a437a14632ce0a2e6c2  xul-ext-ubufox_2.0-0ubuntu1_all.deb

  _______________________________________________________________________________________
  Snipets from Packages.bz2
  From Meta-data:
    Package: aptdaemon
    MD5sum: 6c07f8db615cc32657b1cc57450a1608

    Package: xul-ext-ubufox
    MD5sum: 501373631a241847cf76d13aea9264b3

    Package: python-aptdaemon
    MD5sum: 566f7c92a9fcd0c5c40d08a977176291

    Package: python-defer
    MD5sum: 3653165af1f20a437a14632ce0a2e6c2

  _______________________________________________________________________________________
  "dpkg -i" on pacakges downloaded

  root@ip-10-6-85-206:/var/cache/apt/archives/partial# dpkg -I aptdaemon_0.43+bzr769-0ubuntu1_all.deb
   new debian package, version 2.0.
   size 56824 bytes: control archive= 1520 bytes.
        23 bytes,     1 lines      conffiles
      1023 bytes,    29 lines      control
      1804 bytes,    21 lines      md5sums
   Package: xul-ext-ubufox
   Source: ubufox
   Version: 2.0-0ubuntu1
   Architecture: all
   Maintainer: Ubuntu Mozilla Team <ubuntu-mozillateam@xxxxxxxxxxxxxxxx>
   Installed-Size: 383
   Depends: aptdaemon, libglib2.0-0 (>= 2.26)
   Recommends: firefox (>= 4.0~b6)
   Enhances: firefox
   Breaks: ubufox (<< 0.9~rc2-0ubuntu3)a
   Replaces: ubufox (<< 0.9~rc2-0ubuntu3)
   Provides: firefox-ubufox, ubufox
   Section: web
   Priority: optional
   Homepage: https://launchpad.net/ubufox
   Description: Ubuntu-specific configuration defaults and apt support for Firefox
    Adds Ubuntu-specific modifications to Firefox.
    .
    Integrates the browser with Ubuntu to:
     * Enable searching for missing plugins from Ubuntu software catalog
     * Add the following options to the Help menu
       - Get help on-line
       - Help translating Firefox
       - Ubuntu Release Notes
     * Set homepage to Ubuntu Start Page
     * Display a restart notification after upgrading Firefox
     * Add ask.com to the search engines.
    .
    You can uninstall this if you prefer to use a pristine Firefox install.

  root@ip-10-6-85-206:/var/cache/apt/archives/partial# dpkg -I python-aptdaemon_0.43+bzr769-0ubuntu1_all.deb
   new debian package, version 2.0.
   size 15142 bytes: control archive= 1508 bytes.
        68 bytes,     2 lines      conffiles
      1390 bytes,    30 lines      control
      1088 bytes,    15 lines      md5sums
   Package: aptdaemon
   Version: 0.43+bzr769-0ubuntu1
   Architecture: all
   Maintainer: Ubuntu Developers <ubuntu-devel-discuss@xxxxxxxxxxxxxxxx>
   Installed-Size: 188
   Depends: python, python-aptdaemon (= 0.43+bzr769-0ubuntu1), python-gi, gir1.2-glib-2.0
   Breaks: software-center (<< 1.1.21)
   Section: admin
   Priority: extra
   Homepage: https://launchpad.net/aptdaemon
   Description: transaction based package management service
    Aptdaemon allows normal users to perform package management tasks, e.g.
    refreshing the cache, upgrading the system, installing or removing software
    packages.
    .
    Currently it comes with the following main features:
    .
     - Programming language independent D-Bus interface, which allows one to
       write clients in several languages
     - Runs only if required (D-Bus activation)
     - Fine grained privilege management using PolicyKit, e.g. allowing all
       desktop user to query for updates without entering a password
     - Support for media changes during installation from DVD/CDROM
     - Support for debconf (Debian's package configuration system)
     - Support for attaching a terminal to the underlying dpkg call
    .
    This package contains the aptd script and all the data files required to run
    the daemon. Moreover it contains the aptdcon script, which is a command
    line client for aptdaemon. The API is not stable yet.
   Original-Maintainer: Julian Andres Klode <jak@xxxxxxxxxx>

  ProblemType: Bug
  DistroRelease: Ubuntu 12.04
  Package: apt 0.8.16~exp12ubuntu4
  ProcVersionSignature: Ubuntu 3.2.0-18.28-virtual 3.2.9
  Uname: Linux 3.2.0-18-virtual x86_64
  ApportVersion: 1.94-0ubuntu1
  Architecture: amd64
  Date: Tue Mar  6 21:21:17 2012
  Ec2AMI: ami-0400dd6d
  Ec2AMIManifest: (unknown)
  Ec2AvailabilityZone: us-east-1c
  Ec2InstanceType: m1.large
  Ec2Kernel: aki-825ea7eb
  Ec2Ramdisk: unavailable
  SourcePackage: apt
  UpgradeStatus: No upgrade log present (probably fresh install)

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/apt/+bug/948461/+subscriptions