touch-packages team mailing list archive

[Theodore Ts'o <tytso@xxxxxxx>]

The change was deliberate,  because under some extreme circumstances you
could end up corrupting the file system image when resize2fs aborted in
the middle of a shrink operation.  This tends to make some users cranky
when they lose their data.

If you had bothered to read the entire git commit description, you would
have gotten a lot more context:

    resize2fs: refine minimum required blocks for flex_bg file systems
    
    The previous commit exposed bugs in the calculation for flex_bg file
    systems.  The problem is that since (by default) we keep the metadata
    blocks for the flex_bg in the first block group of the flex_bg, and
    because we don't want to overwrite metadata blocks used by the
    original file system with data blocks make life easier in case the
    resize is aborted for some reason, we need to treat all of the
    metadata blocks in the existing flex_bg has in use for the purposes of
    calculate_minimum_resize_size().
    
    Even though this means we need to reserve more data blocks to avoid
    running out of space, the net result of these two commits is a net
    savings in how much we can shrink a file system.
    
    Using the following test sequence:
    
        mke2fs -F -t ext4 /tmp/foo.img 2T
        resize2fs -M /tmp/foo.img
        resize2fs -M /tmp/foo.img
        resize2fs -M /tmp/foo.img
    
    Here is the comparison in the resulting file systems between the old
    and new resize2fs (units are in 4k blocks):
    
                    resize #1  resize #2   resize #3
    old resize2fs    1117186     45679       43536
    new resize2fs      48784     37413       37392
    
    Signed-off-by: "Theodore Ts'o" <tytso@xxxxxxx>

Note that using resize2fs -M multiple times will result in a file system
which is *not* optimized for speed.   In fact, it can be really bad for
slow devices, such as CD-ROM's and Amazon Elastic Block devices, since
files and directories will end up getting fragmented and where the data
blocks can end up getting placed farther than optimal from the inode
table locations.

-- 
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to e2fsprogs in Ubuntu.
https://bugs.launchpad.net/bugs/1415077

Title:
  resize2fs  -P minimum size  differs greatly between v1.42.10 and
  v1.42.9

Status in e2fsprogs package in Ubuntu:
  Invalid

Bug description:
  I have a disk image created as part of lp:maas-images.
  This process takes a cloud image (http://cloud-images.ubuntu.com) such as
  http://cloud-images.ubuntu.com/trusty/current/trusty-server-cloudimg-amd64.tar.gz .
  Inside that .tar.gz is a .img file.  The .img file is operated on
  by loop mount, chroot, apt-get install and other changes.

  Immediately upon download and extraction of a pristine image, resize2fs output
  on trusty, utopic and vivid differ.  A summary of differences:
   a.) 'resize2fs -P' on
       trusty (1.42.9) : 230329
       utopic (1.42.10): 236103
       vivid  (1.42.12): 236103
   b.) vivid resize2fs requires running 'e2fsck -f' first.

  I do realize that 'resize2fs -P' reports "Estimated minumum size" and that a
  difference of ~6k blocks doesn't seem like too big of a deal, but I'm seeing
  this behavior on a much grander scale after doing some loop mount operations
  on the disk image as part of the maas-images build process (lp:maas-images).
  On that derived image, I see 'resize2fs -P' numbers like:
   trusty: 274500
   utopic: 358400
   vivid:  358400
  The 83900 blocks out of a 358400 seems significant (23% of the fs size),
  and results in the image appearing full on utopic/vivid, even though
  a mount and df show only 74% full.

  See below for more information.

  ## The original image as downloaded.
  $ wget http://cloud-images.ubuntu.com/releases/14.04/release-20150123/ubuntu-14.04-server-cloudimg-amd64.tar.gz
  $ tar -Sxvzf ubuntu-14.04-server-cloudimg-amd64.tar.gz
  $ md5sum trusty-server-cloudimg-amd64.img
  9fd4790c920aaeb0b503ebd2bf7f383c  trusty-server-cloudimg-amd64.img

  ## Trusty
  $ resize2fs -P trusty-server-cloudimg-amd64.img
  resize2fs 1.42.9 (4-Feb-2014)
  Estimated minimum size of the filesystem: 230329

  $ dumpe2fs trusty-server-cloudimg-amd64.img | pastebinit
  dumpe2fs 1.42.9 (4-Feb-2014)
  http://paste.ubuntu.com/9898743/

  ## Utopic
  $ resize2fs -P trusty-server-cloudimg-amd64.img
  resize2fs 1.42.10 (18-May-2014)
  Estimated minimum size of the filesystem: 236103

  $ dumpe2fs trusty-server-cloudimg-amd64.img | pastebinit
  dumpe2fs 1.42.10 (18-May-2014)
  http://paste.ubuntu.com/9898789/

  ## Vivid
  $ resize2fs -P trusty-server-cloudimg-amd64.img
  resize2fs 1.42.12 (29-Aug-2014)
  Please run 'e2fsck -f trusty-server-cloudimg-amd64.img' first.

  $ cp trusty-server-cloudimg-amd64.img trusty-server-cloudimg-amd64.img.dist
  $ e2fsck -f trusty-server-cloudimg-amd64.img
  e2fsck 1.42.12 (29-Aug-2014)
  Pass 1: Checking inodes, blocks, and sizes
  Pass 2: Checking directory structure
  Pass 3: Checking directory connectivity
  Pass 4: Checking reference counts
  Pass 5: Checking group summary information
  cloudimg-rootfs: 57885/90112 files (0.0% non-contiguous), 207837/360448 blocks

  $ resize2fs -P trusty-server-cloudimg-amd64.img
  resize2fs 1.42.12 (29-Aug-2014)
  Estimated minimum size of the filesystem: 236103

  $ md5sum trusty-server-cloudimg-amd64.img.dist trusty-server-cloudimg-amd64.img
  9fd4790c920aaeb0b503ebd2bf7f383c  trusty-server-cloudimg-amd64.img.dist
  c38a3ca8cba7996c85365e122175d013  trusty-server-cloudimg-amd64.img

  $ dumpe2fs trusty-server-cloudimg-amd64.img | pastebinit
  dumpe2fs 1.42.12 (29-Aug-2014)
  http://paste.ubuntu.com/9898831/

  $ dumpe2fs trusty-server-cloudimg-amd64.img.dist | pastebinit
  dumpe2fs 1.42.12 (29-Aug-2014)
  http://paste.ubuntu.com/9898832/

  
  ## maas images image
  ## output below is for the maas build process output image
  ## This one specifically is an arm64 image, and the operations
  ## done (apt-get update) and such were done with the help chroot via
  ## qemu-static.

  ## build the image (works on trusty, fails due to resizing issues on utopic)
  $ time maas-cloudimg2eph2 -vv --kernel=linux-generic --arch=arm64 \
    http://cloud-images.ubuntu.com/daily/server/vivid/20150125/vivid-server-cloudimg-arm64.tar.gz \
    /tmp/maas_final/vivid/arm64/20150125/root-image.gz \
    --krd-pack=linux-generic,/tmp/maas_final/vivid/arm64/20150125/vivid/generic/boot-kernel,/tmp/maas_final/vivid/arm64/20150125/vivid/generic/boot-initrd 2>&1 | tee out.log

  $ pastebinit out.log
  http://paste.ubuntu.com/9899165/

  $ zcat /tmp/maas_final/vivid/arm64/20150125/root-image.gz > /tmp/root-
  image

  $ ls -l /tmp/root-image
  -rw-rw-r-- 1 ubuntu ubuntu 1468006400 Jan 27 14:56 /tmp/root-image

  ## see attached root-image.e2image.xz
  $ e2image /tmp/root-image /tmp/root-image.e2image
  e2image 1.42.9 (4-Feb-2014)
  $ xz /tmp/root-image.e2image

  
  ## Now, the above 'root-image' file on utopic, vivid and trusty.
  ##
  ## Trusty
  $ resize2fs -P /tmp/root-image
  resize2fs 1.42.9 (4-Feb-2014)
  Estimated minimum size of the filesystem: 274500

  $ mkdir -p /tmp/mp && sudo mount -o loop,ro /tmp/root-image /tmp/mp &&
    df /tmp/mp && sudo umount /tmp/mp
  Filesystem     1K-blocks   Used Available Use% Mounted on
  /dev/loop0       1356344 980284    359676  74% /tmp/mp

  $ dumpe2fs /tmp/root-image | pastebinit
  dumpe2fs 1.42.9 (4-Feb-2014)
  http://paste.ubuntu.com/9899267/

  ## Utopic
  $ resize2fs -P /tmp/root-image
  resize2fs 1.42.10 (18-May-2014)
  Estimated minimum size of the filesystem: 358400

  $ mkdir -p /tmp/mp && sudo mount -o loop,ro /tmp/root-image /tmp/mp &&
    df /tmp/mp && sudo umount /tmp/mp
  Filesystem     1K-blocks   Used Available Use% Mounted on
  /dev/loop0       1356344 980284    359676  74% /tmp/mp

  $ dumpe2fs /tmp/root-image | pastebinit
  dumpe2fs 1.42.10 (18-May-2014)
  http://paste.ubuntu.com/9899313/

  ## Vivid
  $ resize2fs -P /tmp/root-image
  resize2fs 1.42.12 (29-Aug-2014)
  Estimated minimum size of the filesystem: 358400

  $ mkdir -p /tmp/mp && sudo mount -o loop,ro /tmp/root-image /tmp/mp &&
    df /tmp/mp && sudo umount /tmp/mp
  Filesystem     1K-blocks   Used Available Use% Mounted on
  /dev/loop0       1356344 980284    359676  74% /tmp/mp

  $ dumpe2fs /tmp/root-image | pastebinit
  dumpe2fs 1.42.12 (29-Aug-2014)
  http://paste.ubuntu.com/9899328/

  ProblemType: Bug
  DistroRelease: Ubuntu 15.04
  Package: e2fsprogs 1.42.12-1ubuntu1
  ProcVersionSignature: User Name 3.18.0-9.10-generic 3.18.2
  Uname: Linux 3.18.0-9-generic x86_64
  ApportVersion: 2.15.1-0ubuntu2
  Architecture: amd64
  Date: Tue Jan 27 14:13:26 2015
  Ec2AMI: ami-000000f5
  Ec2AMIManifest: FIXME
  Ec2AvailabilityZone: nova
  Ec2InstanceType: m1.small
  Ec2Kernel: aki-00000002
  Ec2Ramdisk: ari-00000002
  ProcEnviron:
   TERM=xterm
   PATH=(custom, no user)
   XDG_RUNTIME_DIR=<set>
   LANG=en_US.UTF-8
   SHELL=/bin/bash
  SourcePackage: e2fsprogs
  UpgradeStatus: No upgrade log present (probably fresh install)

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/e2fsprogs/+bug/1415077/+subscriptions