kernel-packages team mailing list archive

[René Bleisch <accounts@xxxxxxxxxxxxxxxx>]

Having a look on the source, I'm confirmed, that the assembling problem
is related to the change indicated as:

   0cc3a8b md: Fix handling for devices from 2TB to 4TB in 0.90
metadata.

resp. to the following lines in drivers/md/md.c (Line 850ff):

--------------
850 	rdev->sectors = rdev->sb_start;
851 	/* Limit to 4TB as metadata cannot record more than that */
852 	if (rdev->sectors >= (2ULL << 32))
853 		rdev->sectors = (2ULL << 32) - 2;

--------------

Lines 851-853 where introduced in version 2.6.32-37.80


A further hint:
I operate also another data server with the same raid setup (two hardware raids merged to a raid0 md-raid), but with Ubuntu 12.04  on it (v 3.2.0-29). With this raid I didnt have any problems up to now.
Checking the source code of md.c for the respective part, I find the following:

-------------
	rdev->sectors = rdev->sb_start;
	/* Limit to 4TB as metadata cannot record more than that.
	 * (not needed for Linear and RAID0 as metadata doesn't
	 * record this size)
	 */
	if (rdev->sectors >= (2ULL << 32) && sb->level >= 1)
		rdev->sectors = (2ULL << 32) - 2;
-------------

here, the 4 TB limit is only valid for raid levels above 1 ("&&
sb->level >= 1")

In summary:

-> introducing the 4 TB limit in general leads to problems for my large raids
-> limiting the 4 TB limit to raid levels above 1 seems to solve the problem (as I use raid level 0)

-- 
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1247064

Title:
  mdraid not assembled correctly when booting from kernel newer than
  2.6.32-33

Status in “linux” package in Ubuntu:
  Incomplete
Status in “mdadm” package in Ubuntu:
  New

Bug description:
  I operate a raid server consisting of two hardware RAID6 arrays (11 2 TB disks each), which are combined in a MD-raid (Raid0).
  When booting Kernel 2.6.32-33, the Raid is assembled correctly:

  cat /proc/mdstat:
  --------------------
  Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10] 
  md0 : active raid0 sdb1[0] sdc1[1]
        35156244352 blocks 64k chunks
        
  unused devices: <none>

  cat /proc/partitions:
  ------------------------
  major minor  #blocks  name

     8        0  244140288 sda
     8        1     248832 sda1
     8        2    1952768 sda2
     8        3  241937408 sda3
     8       16 17578123776 sdb
     8       17 17578122240 sdb1
     8       32 17578123776 sdc
     8       33 17578122240 sdc1
     9        0 35156244352 md0

  
  When booting any newer kernel (eg. 2.6.32-38 or 2.6.32-52), the RAID is assembled incorrectly, its size being far too low (8.5 TB instead of 35 TB). In the following, the raid array is not usable.

  cat /proc/mdstat:
  ---------------------
  Personalities : [linear] [multipath] [raid0] 
  md0 : active raid0 sdb1[0] sdc1[1]
        8589934464 blocks 64k chunks
        
  unused devices: <none>

  cat /proc/partitions:
  ------------------------
  major minor  #blocks  name

     8        0  244140288 sda
     8        1     248832 sda1
     8        2    1952768 sda2
     8        3  241937408 sda3
     8       16 17578123776 sdb
     8       17 17578122240 sdb1
     8       32 17578123776 sdc
     8       33 17578122240 sdc1

  
  The two RAID6-hardware RAID arrays are recognized correctly by the system, further the Areca controllers are also working correctly.

  I suspect there is a kernel bug, preventing mdadm to get the correct
  device sizes for large size raid arrays.

  ProblemType: Bug
  DistroRelease: Ubuntu 10.04
  Package: mdadm 2.6.7.1-1ubuntu15
  ProcVersionSignature: Ubuntu 2.6.32-33.72-generic 2.6.32.41+drm33.18
  Uname: Linux 2.6.32-33-generic x86_64
  Architecture: amd64
  Date: Fri Nov  1 12:09:02 2013
  MDadmExamine.dev.sda: Error: command ['/sbin/mdadm', '-E', '/dev/sda'] failed with exit code 1: mdadm: No md superblock detected on /dev/sda.
  MDadmExamine.dev.sda1: Error: command ['/sbin/mdadm', '-E', '/dev/sda1'] failed with exit code 1: mdadm: No md superblock detected on /dev/sda1.
  MDadmExamine.dev.sda2: Error: command ['/sbin/mdadm', '-E', '/dev/sda2'] failed with exit code 1: mdadm: No md superblock detected on /dev/sda2.
  MDadmExamine.dev.sda3: Error: command ['/sbin/mdadm', '-E', '/dev/sda3'] failed with exit code 1: mdadm: No md superblock detected on /dev/sda3.
  MDadmExamine.dev.sdb: Error: command ['/sbin/mdadm', '-E', '/dev/sdb'] failed with exit code 1: mdadm: No md superblock detected on /dev/sdb.
  MDadmExamine.dev.sdc: Error: command ['/sbin/mdadm', '-E', '/dev/sdc'] failed with exit code 1: mdadm: No md superblock detected on /dev/sdc.
  MachineType: Supermicro X8DTU
  ProcCmdLine: BOOT_IMAGE=/vmlinuz-2.6.32-33-generic root=UUID=fdee92a7-11e6-4999-8d24-473a8c7986b4 ro quiet splash
  ProcEnviron:
   LC_TIME=
   PATH=(custom, no user)
   LANG=en_US.UTF-8
   SHELL=/bin/bash
  ProcMDstat:
   Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10] 
   md0 : active raid0 sdb1[0] sdc1[1]
         35156244352 blocks 64k chunks
         
   unused devices: <none>
  SourcePackage: mdadm
  dmi.bios.date: 08/03/2012
  dmi.bios.vendor: American Megatrends Inc.
  dmi.bios.version: 2.1c
  dmi.board.asset.tag: To Be Filled By O.E.M.
  dmi.board.name: X8DTU
  dmi.board.vendor: Supermicro
  dmi.board.version: 1234567890
  dmi.chassis.asset.tag: To Be Filled By O.E.M.
  dmi.chassis.type: 17
  dmi.chassis.vendor: Supermicro
  dmi.chassis.version: 1234567890
  dmi.modalias: dmi:bvnAmericanMegatrendsInc.:bvr2.1c:bd08/03/2012:svnSupermicro:pnX8DTU:pvr1234567890:rvnSupermicro:rnX8DTU:rvr1234567890:cvnSupermicro:ct17:cvr1234567890:
  dmi.product.name: X8DTU
  dmi.product.version: 1234567890
  dmi.sys.vendor: Supermicro
  etc.blkid.tab: Error: [Errno 2] No such file or directory: '/etc/blkid.tab'

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1247064/+subscriptions