sslug-teknik team mailing list archive
-
sslug-teknik team
-
Mailing list archive
-
Message #26086
Raid 5 disk failure
Hej
Vi koerer Raid 5 paa 5 diske.
Men her til aften ser det ud til at den ene disk stod af (/dev/sdd1)
fra /var/log/messages :
""" kernel: rdev sdd1: O:sdd1, SZ:00000000 F:1 DN:3 no rdev superblock!
"""
Der bliver skrevet masser af info til /var/log/messages (syslogd koerer
afsted for fuld tryk)
Jeg kan ikke helt greje om diskene simpelthen er staaet eller hvad ?
Jeg troede vi koerte RAID 5 med 1 'redundant' disk der altid var klar
til at tage over ??
Jeg kan heller ikke lide linjen " kernel: md: bug in file raid5.c, line
659 " i sys-loggen.
Jeg haaber nogen kan hjaelpe lidt paa vej til om vi skal have skiftet en
disk, eller det generelt er RAID opsaetningen den er gal med.
Vi koerer RedHat 6.2 kernel 2.2.14-5.0smp, og RAID 5 paa 5 SCSI diske.
MVH Thomas
Her er lidt info :
---
[root@ooo /root]# cat /proc/mdstat
Personalities : [raid5]
read_ahead 1024 sectors
md0 : active raid5 sdd1[3](F) sdc1[2] sdb1[1] sda1[0] 141339392 blocks
level 5, 32k chunk, algorithm 2 [5/3] [UUU__]
unused devices: <none>
---
Her info fra /var/log/messages
---
kernel: raid5: Disk failure on sdd1, disabling device. Operation
continuing on 3 devices
kernel: md: updating md0 RAID superblock on device
kernel: (skipping faulty sdd1 )
kernel: sdc1 [events: 00000022](write) sdc1's sb offset: 35334848
kernel: md: recovery thread got woken up ...
kernel: md0: no spare disk to reconstruct array! -- continuing in
degraded mode
kernel: md: recovery thread finished ...
kernel: SCSI disk error : host 0 channel 0 id 3 lun 0 return code =
26030000
kernel: scsidisk I/O error: dev 08:31, sector 43817080
kernel: md: bug in file raid5.c, line 659
kernel:
kernel: **********************************
kernel: * <COMPLETE RAID STATE PRINTOUT> *
kernel: **********************************
kernel: md0: <sdd1><sdc1><sdb1><sda1> array superblock:
kernel: SB: (V:0.90.0) ID:<cc0ba769.871814fd.419d6c03.df7575b9>
CT:394e4c1e
kernel: L5 S35334848 ND:4 RD:5 md0 LO:2 CS:32768
kernel: UT:3992dd7b ST:0 AD:3 WD:3 FD:1 SD:0 CSUM:925a221d
E:00000022
kernel: D 0: DISK<N:0,sda1(8,1),R:0,S:6>
kernel: D 1: DISK<N:1,sdb1(8,17),R:1,S:6>
kernel: D 2: DISK<N:2,sdc1(8,33),R:2,S:6>
kernel: D 3: DISK<N:3,sdd1(8,49),R:3,S:1>
kernel: D 4: DISK<N:4,[dev 00:00](0,0),R:4,S:9>
kernel: D 5: DISK<N:0,[dev 00:00](0,0),R:0,S:4>
kernel: D 6: DISK<N:0,[dev 00:00](0,0),R:0,S:4>
kernel: D 7: DISK<N:0,[dev 00:00](0,0),R:0,S:4>
kernel: D 8: DISK<N:0,[dev 00:00](0,0),R:0,S:4>
kernel: D 9: DISK<N:0,[dev 00:00](0,0),R:0,S:4>
kernel: D 10: DISK<N:0,[dev 00:00](0,0),R:0,S:4>
kernel: D 11: DISK<N:0,[dev 00:00](0,0),R:0,S:4>
kernel: THIS: DISK<N:3,sdd1(8,49),R:3,S:6>
kernel: rdev sdd1: O:sdd1, SZ:00000000 F:1 DN:3 no rdev superblock!
kernel: rdev sdc1: O:sdc1, SZ:35334848 F:0 DN:2 rdev superblock:
kernel: SB: (V:0.90.0) ID:<cc0ba769.871814fd.419d6c03.df7575b9>
CT:394e4c1e
kernel: L5 S35334848 ND:4 RD:5 md0 LO:2 CS:32768
kernel: UT:3992dd7b ST:0 AD:3 WD:3 FD:1 SD:0 CSUM:925ec310
E:00000022
kernel: D 0: DISK<N:0,sda1(8,1),R:0,S:6>
kernel: D 1: DISK<N:1,sdb1(8,17),R:1,S:6>
kernel: D 2: DISK<N:2,sdc1(8,33),R:2,S:6>
kernel: D 3: DISK<N:3,sdd1(8,49),R:3,S:1>
kernel: D 4: DISK<N:4,[dev 00:00](0,0),R:4,S:9>
kernel: D 5: DISK<N:0,[dev 00:00](0,0),R:0,S:4>
kernel: D 6: DISK<N:0,[dev 00:00](0,0),R:0,S:4>
kernel: D 7: DISK<N:0,[dev 00:00](0,0),R:0,S:4>
kernel: D 8: DISK<N:0,[dev 00:00](0,0),R:0,S:4>
kernel: D 9: DISK<N:0,[dev 00:00](0,0),R:0,S:4>
kernel: D 10: DISK<N:0,[dev 00:00](0,0),R:0,S:4>
kernel: D 11: DISK<N:0,[dev 00:00](0,0),R:0,S:4>
kernel: THIS: DISK<N:2,sdc1(8,33),R:2,S:6>
kernel: rdev sdb1: O:sdb1, SZ:35334848 F:0 DN:1 rdev superblock:
kernel: SB: (V:0.90.0) ID:<cc0ba769.871814fd.419d6c03.df7575b9>
CT:394e4c1e
kernel: L5 S35334848 ND:4 RD:5 md0 LO:2 CS:32768
kernel: UT:3992dd7b ST:0 AD:3 WD:3 FD:1 SD:0 CSUM:925ec2fe
E:00000022
kernel: D 0: DISK<N:0,sda1(8,1),R:0,S:6>
kernel: D 1: DISK<N:1,sdb1(8,17),R:1,S:6>
kernel: D 2: DISK<N:2,sdc1(8,33),R:2,S:6>
kernel: D 3: DISK<N:3,sdd1(8,49),R:3,S:1>
kernel: D 4: DISK<N:4,[dev 00:00](0,0),R:4,S:9>
kernel: D 5: DISK<N:0,[dev 00:00](0,0),R:0,S:4>
kernel: D 6: DISK<N:0,[dev 00:00](0,0),R:0,S:4>
kernel: D 7: DISK<N:0,[dev 00:00](0,0),R:0,S:4>
kernel: D 8: DISK<N:0,[dev 00:00](0,0),R:0,S:4>
kernel: D 9: DISK<N:0,[dev 00:00](0,0),R:0,S:4>
kernel: D 10: DISK<N:0,[dev 00:00](0,0),R:0,S:4>
kernel: D 11: DISK<N:0,[dev 00:00](0,0),R:0,S:4>
kernel: THIS: DISK<N:1,sdb1(8,17),R:1,S:6>
kernel: rdev sda1: O:sda1, SZ:35334848 F:0 DN:0 rdev superblock:
kernel: SB: (V:0.90.0) ID:<cc0ba769.871814fd.419d6c03.df7575b9>
CT:394e4c1e
kernel: L5 S35334848 ND:4 RD:5 md0 LO:2 CS:32768
kernel: UT:3992dd7b ST:0 AD:3 WD:3 FD:1 SD:0 CSUM:925ec2ec
E:00000022
kernel: D 0: DISK<N:0,sda1(8,1),R:0,S:6>
kernel: D 1: DISK<N:1,sdb1(8,17),R:1,S:6>
kernel: D 2: DISK<N:2,sdc1(8,33),R:2,S:6>
kernel: D 3: DISK<N:3,sdd1(8,49),R:3,S:1>
kernel: D 4: DISK<N:4,[dev 00:00](0,0),R:4,S:9>
kernel: D 5: DISK<N:0,[dev 00:00](0,0),R:0,S:4>
kernel: D 6: DISK<N:0,[dev 00:00](0,0),R:0,S:4>
kernel: D 7: DISK<N:0,[dev 00:00](0,0),R:0,S:4>
kernel: D 8: DISK<N:0,[dev 00:00](0,0),R:0,S:4>
kernel: D 9: DISK<N:0,[dev 00:00](0,0),R:0,S:4>
kernel: D 10: DISK<N:0,[dev 00:00](0,0),R:0,S:4>
kernel: D 11: DISK<N:0,[dev 00:00](0,0),R:0,S:4>
kernel: THIS: DISK<N:1,sdb1(8,17),R:1,S:6>
kernel: rdev sda1: O:sda1, SZ:35334848 F:0 DN:0 rdev superblock:
kernel: SB: (V:0.90.0) ID:<cc0ba769.871814fd.419d6c03.df7575b9>
CT:394e4c1e
kernel: L5 S35334848 ND:4 RD:5 md0 LO:2 CS:32768
kernel: UT:3992dd7b ST:0 AD:3 WD:3 FD:1 SD:0 CSUM:925ec2ec
E:00000022
kernel: D 0: DISK<N:0,sda1(8,1),R:0,S:6>
kernel: D 1: DISK<N:1,sdb1(8,17),R:1,S:6>
kernel: D 2: DISK<N:2,sdc1(8,33),R:2,S:6>
kernel: D 3: DISK<N:3,sdd1(8,49),R:3,S:1>
kernel: D 4: DISK<N:4,[dev 00:00](0,0),R:4,S:9>
kernel: D 5: DISK<N:0,[dev 00:00](0,0),R:0,S:4>
kernel: D 6: DISK<N:0,[dev 00:00](0,0),R:0,S:4>
kernel: D 7: DISK<N:0,[dev 00:00](0,0),R:0,S:4>
kernel: D 8: DISK<N:0,[dev 00:00](0,0),R:0,S:4>
kernel: D 9: DISK<N:0,[dev 00:00](0,0),R:0,S:4>
kernel: D 10: DISK<N:0,[dev 00:00](0,0),R:0,S:4>
kernel: D 11: DISK<N:0,[dev 00:00](0,0),R:0,S:4>
kernel: THIS: DISK<N:0,sda1(8,1),R:0,S:6>
kernel: **********************************
kernel:
kernel: md: recovery thread got woken up ...
kernel: md0: no spare disk to reconstruct array! -- continuing in
degraded mode
kernel: md: recovery thread finished ...
kernel: SCSI disk error : host 0 channel 0 id 3 lun 0 return code =
26030000
kernel: scsidisk I/O error: dev 08:31, sector 43817112
kernel: md: bug in file raid5.c, line 659
kernel:
kernel: **********************************
kernel: * <COMPLETE RAID STATE PRINTOUT> *
kernel: **********************************
kernel: md0: <sdd1><sdc1><sdb1><sda1> array superblock:
kernel: SB: (V:0.90.0) ID:<cc0ba769.871814fd.419d6c03.df7575b9>
CT:394e4c1e
kernel: L5 S35334848 ND:4 RD:5 md0 LO:2 CS:32768
kernel: UT:3992dd7b ST:0 AD:3 WD:3 FD:1 SD:0 CSUM:925a221d
E:00000022
kernel: D 0: DISK<N:0,sda1(8,1),R:0,S:6>
kernel: D 1: DISK<N:1,sdb1(8,17),R:1,S:6>
kernel: D 2: DISK<N:2,sdc1(8,33),R:2,S:6>
kernel: D 3: DISK<N:3,sdd1(8,49),R:3,S:1>
kernel: D 4: DISK<N:4,[dev 00:00](0,0),R:4,S:9>
kernel: D 5: DISK<N:0,[dev 00:00](0,0),R:0,S:4>
kernel: D 6: DISK<N:0,[dev 00:00](0,0),R:0,S:4>
kernel: D 7: DISK<N:0,[dev 00:00](0,0),R:0,S:4>
kernel: D 8: DISK<N:0,[dev 00:00](0,0),R:0,S:4>
kernel: D 9: DISK<N:0,[dev 00:00](0,0),R:0,S:4>
kernel: D 10: DISK<N:0,[dev 00:00](0,0),R:0,S:4>
kernel: D 11: DISK<N:0,[dev 00:00](0,0),R:0,S:4>
kernel: THIS: DISK<N:3,sdd1(8,49),R:3,S:6>
kernel: rdev sdd1: O:sdd1, SZ:00000000 F:1 DN:3 no rdev superblock!
kernel: rdev sdc1: O:sdc1, SZ:35334848 F:0 DN:2 rdev superblock:
kernel: SB: (V:0.90.0) ID:<cc0ba769.871814fd.419d6c03.df7575b9>
CT:394e4c1e
kernel: L5 S35334848 ND:4 RD:5 md0 LO:2 CS:32768
kernel: UT:3992dd7b ST:0 AD:3 WD:3 FD:1 SD:0 CSUM:925ec310
E:00000022
kernel: D 0: DISK<N:0,sda1(8,1),R:0,S:6>
kernel: D 1: DISK<N:1,sdb1(8,17),R:1,S:6>
kernel: D 2: DISK<N:2,sdc1(8,33),R:2,S:6>
kernel: D 3: DISK<N:3,sdd1(8,49),R:3,S:1>
kernel: D 4: DISK<N:4,[dev 00:00](0,0),R:4,S:9>
kernel: D 5: DISK<N:0,[dev 00:00](0,0),R:0,S:4>
kernel: D 6: DISK<N:0,[dev 00:00](0,0),R:0,S:4>
kernel: D 7: DISK<N:0,[dev 00:00](0,0),R:0,S:4>
kernel: D 8: DISK<N:0,[dev 00:00](0,0),R:0,S:4>
kernel: D 9: DISK<N:0,[dev 00:00](0,0),R:0,S:4>
kernel: D 10: DISK<N:0,[dev 00:00](0,0),R:0,S:4>
kernel: D 11: DISK<N:0,[dev 00:00](0,0),R:0,S:4>
kernel: THIS: DISK<N:2,sdc1(8,33),R:2,S:6>
kernel: rdev sdb1: O:sdb1, SZ:35334848 F:0 DN:1 rdev superblock:
kernel: SB: (V:0.90.0) ID:<cc0ba769.871814fd.419d6c03.df7575b9>
CT:394e4c1e
kernel: L5 S35334848 ND:4 RD:5 md0 LO:2 CS:32768
kernel: UT:3992dd7b ST:0 AD:3 WD:3 FD:1 SD:0 CSUM:925ec2fe
E:00000022
kernel: D 0: DISK<N:0,sda1(8,1),R:0,S:6>
kernel: D 1: DISK<N:1,sdb1(8,17),R:1,S:6>
kernel: D 2: DISK<N:2,sdc1(8,33),R:2,S:6>
kernel: D 3: DISK<N:3,sdd1(8,49),R:3,S:1>
kernel: D 4: DISK<N:4,[dev 00:00](0,0),R:4,S:9>
kernel: D 5: DISK<N:0,[dev 00:00](0,0),R:0,S:4>
kernel: D 6: DISK<N:0,[dev 00:00](0,0),R:0,S:4>
kernel: D 7: DISK<N:0,[dev 00:00](0,0),R:0,S:4>
kernel: D 8: DISK<N:0,[dev 00:00](0,0),R:0,S:4>
kernel: D 9: DISK<N:0,[dev 00:00](0,0),R:0,S:4>
kernel: D 10: DISK<N:0,[dev 00:00](0,0),R:0,S:4>
kernel: D 11: DISK<N:0,[dev 00:00](0,0),R:0,S:4>
kernel: THIS: DISK<N:1,sdb1(8,17),R:1,S:6>
kernel: rdev sda1: O:sda1, SZ:35334848 F:0 DN:0 rdev superblock:
kernel: SB: (V:0.90.0) ID:<cc0ba769.871814fd.419d6c03.df7575b9>
CT:394e4c1e
kernel: L5 S35334848 ND:4 RD:5 md0 LO:2 CS:32768
kernel: UT:3992dd7b ST:0 AD:3 WD:3 FD:1 SD:0 CSUM:925ec2ec
E:00000022
kernel: D 0: DISK<N:0,sda1(8,1),R:0,S:6>
kernel: D 1: DISK<N:1,sdb1(8,17),R:1,S:6>
kernel: D 2: DISK<N:2,sdc1(8,33),R:2,S:6>
kernel: D 3: DISK<N:3,sdd1(8,49),R:3,S:1>
kernel: D 4: DISK<N:4,[dev 00:00](0,0),R:4,S:9>
kernel: D 5: DISK<N:0,[dev 00:00](0,0),R:0,S:4>
kernel: D 6: DISK<N:0,[dev 00:00](0,0),R:0,S:4>
kernel: D 7: DISK<N:0,[dev 00:00](0,0),R:0,S:4>
kernel: D 8: DISK<N:0,[dev 00:00](0,0),R:0,S:4>
kernel: D 9: DISK<N:0,[dev 00:00](0,0),R:0,S:4>
kernel: D 10: DISK<N:0,[dev 00:00](0,0),R:0,S:4>
kernel: D 11: DISK<N:0,[dev 00:00](0,0),R:0,S:4>
kernel: THIS: DISK<N:0,sda1(8,1),R:0,S:6>
kernel: **********************************
kernel:
kernel: (scsi0:0:2:0) Synchronous at 40.0 Mbyte/sec, offset 63.
kernel: md: recovery thread got woken up ...
kernel: md0: no spare disk to reconstruct array! -- continuing in
degraded mode
kernel: md: recovery thread finished ...
kernel: sdb1 [events: 00000022](write) sdb1's sb offset: 35334848
kernel: (scsi0:0:1:0) Synchronous at 40.0 Mbyte/sec, offset 63.
kernel: sda1 [events: 00000022](write) sda1's sb offset: 35334848
kernel: scsi0 channel 0 : resetting for second half of retries.
kernel: SCSI bus is being reset for host 0 channel 0.
kernel: (scsi0:0:0:0) Synchronous at 40.0 Mbyte/sec, offset 63.
kernel: .
kernel: raid5: restarting stripe 43817048
kernel: raid5: md0: unrecoverable I/O error for block 21908523
kernel: raid5: md0: unrecoverable I/O error for block 21908515
kernel: raid5: restarting stripe 43817080
kernel: raid5: md0: unrecoverable I/O error for block 21908519
kernel: raid5: restarting stripe 43817112
kernel: raid5: md0: unrecoverable I/O error for block 21908563
kernel: raid5: bug: sector 43817112, new 00000000, copy cf81f040
kernel: raid5: md0: unrecoverable I/O error for block 27362148
.
.
---
De ovensattaende IO fejl kommer nu bare i en lind stroem
RAIDTAB
---
[root@ooo /etc]# more raidtab
# Sample raid-5 configuration
raiddev /dev/md0
raid-level 5
nr-raid-disks 5
chunk-size 32
parity-algorithm left-symmetric
nr-spare-disks 0
device /dev/sda1
raid-disk 0
device /dev/sdb1
raid-disk 1
device /dev/sdc1
raid-disk 2
device /dev/sdd1
raid-disk 3
device /dev/sde1
raid-disk 4
---
Follow ups
