kernel-packages team mailing list archive

[Chris Read <chris.read@xxxxxxxxx>]

Public bug reported:

I've got 2 NFS servers that have been running Ubuntu 12.04 for a number
of years. For the last 9 months or so they've been happily running
kernel 3.8.0-44 with no problem. As soon as I upgraded them to 3.13.0-53
the both experienced the same problem on an XFS filesystem when it
started to have a heavy write load:

[267127.795322] XFS (sda1): Internal error xfs_attr3_leaf_write_verify at line 216 of file /build/buildd/linux-lts-trusty-3.13.0/fs/xfs/xfs_attr_leaf.c.  Caller 0xffffffffa0250ada
[267127.811202] CPU: 10 PID: 28744 Comm: xfsaild/sda1 Tainted: G          I   3.13.0-53-generic #89~precise1-Ubuntu
[267127.811206] Hardware name: IBM System x3690 X5 -[7148AC1]-/Node 1, System Card, BIOS -[MLE170CUS-1.70]- 09/23/2011
[267127.811211]  0000000000000001 ffff881016b7bb38 ffffffff8175a3e4 ffff88103f34f030
[267127.811221]  ffff881023d9c000 ffff881016b7bb58 ffffffffa025317f ffffffffa0250ada
[267127.811228]  ffffffff00000001 ffff881016b7bb98 ffffffffa02531ee 00000fe000200000
[267127.811236] Call Trace:
[267127.811251]  [<ffffffff8175a3e4>] dump_stack+0x46/0x58
[267127.811311]  [<ffffffffa025317f>] xfs_error_report+0x3f/0x50 [xfs]
[267127.811337]  [<ffffffffa0250ada>] ? _xfs_buf_ioapply+0x6a/0x140 [xfs]
[267127.811363]  [<ffffffffa02531ee>] xfs_corruption_error+0x5e/0x90 [xfs]
[267127.811396]  [<ffffffffa02711b0>] xfs_attr3_leaf_write_verify+0x100/0x120 [xfs]
[267127.811421]  [<ffffffffa0250ada>] ? _xfs_buf_ioapply+0x6a/0x140 [xfs]
[267127.811445]  [<ffffffffa0250e05>] ? xfs_bdstrat_cb+0x65/0xf0 [xfs]
[267127.811470]  [<ffffffffa0250e05>] ? xfs_bdstrat_cb+0x65/0xf0 [xfs]
[267127.811494]  [<ffffffffa0250ada>] _xfs_buf_ioapply+0x6a/0x140 [xfs]
[267127.811505]  [<ffffffff8109fba0>] ? try_to_wake_up+0x210/0x210
[267127.811530]  [<ffffffffa0250d29>] xfs_buf_iorequest+0x59/0xd0 [xfs]
[267127.811554]  [<ffffffffa0250e05>] xfs_bdstrat_cb+0x65/0xf0 [xfs]
[267127.811579]  [<ffffffffa0251013>] __xfs_buf_delwri_submit+0x183/0x210 [xfs]
[267127.811604]  [<ffffffffa0251ac0>] ? xfs_buf_delwri_submit_nowait+0x20/0x30 [xfs]
[267127.811630]  [<ffffffffa0251ac0>] xfs_buf_delwri_submit_nowait+0x20/0x30 [xfs]
[267127.811673]  [<ffffffffa02b4f66>] xfsaild_push+0x126/0x5b0 [xfs]
[267127.811716]  [<ffffffffa02b548f>] xfsaild+0x9f/0x140 [xfs]
[267127.811757]  [<ffffffffa02b53f0>] ? xfsaild_push+0x5b0/0x5b0 [xfs]
[267127.811766]  [<ffffffff8108fcd9>] kthread+0xc9/0xe0
[267127.811772]  [<ffffffff8108fc10>] ? flush_kthread_worker+0xb0/0xb0
[267127.811781]  [<ffffffff8176f928>] ret_from_fork+0x58/0x90
[267127.811787]  [<ffffffff8108fc10>] ? flush_kthread_worker+0xb0/0xb0
[267127.811791] XFS (sda1): Corruption detected. Unmount and run xfs_repair
[267127.818679] XFS (sda1): xfs_do_force_shutdown(0x8) called from line 1320 of file /build/buildd/linux-lts-trusty-3.13.0/fs/xfs/xfs_buf.c.  Return address = 0xffffffffa0250bad
[267127.818927] XFS (sda1): xfs_imap_to_bp: xfs_trans_read_buf() returned error 5.
[267127.820914] XFS (sda1): Corruption of in-memory data detected.  Shutting down filesystem
[267127.829131] XFS (sda1): Please umount the filesystem and rectify the problem(s)

I've not been able to reproduce this on demand outside our production
workload. Downgrading the machines back to 3.8.0-44 has returned us to
stability.

** Affects: linux-lts-trusty (Ubuntu)
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux-lts-trusty in Ubuntu.
https://bugs.launchpad.net/bugs/1468039

Title:
  XFS: Corruption of in-memory data detected

Status in linux-lts-trusty package in Ubuntu:
  New

Bug description:
  I've got 2 NFS servers that have been running Ubuntu 12.04 for a
  number of years. For the last 9 months or so they've been happily
  running kernel 3.8.0-44 with no problem. As soon as I upgraded them to
  3.13.0-53 the both experienced the same problem on an XFS filesystem
  when it started to have a heavy write load:

  [267127.795322] XFS (sda1): Internal error xfs_attr3_leaf_write_verify at line 216 of file /build/buildd/linux-lts-trusty-3.13.0/fs/xfs/xfs_attr_leaf.c.  Caller 0xffffffffa0250ada
  [267127.811202] CPU: 10 PID: 28744 Comm: xfsaild/sda1 Tainted: G          I   3.13.0-53-generic #89~precise1-Ubuntu
  [267127.811206] Hardware name: IBM System x3690 X5 -[7148AC1]-/Node 1, System Card, BIOS -[MLE170CUS-1.70]- 09/23/2011
  [267127.811211]  0000000000000001 ffff881016b7bb38 ffffffff8175a3e4 ffff88103f34f030
  [267127.811221]  ffff881023d9c000 ffff881016b7bb58 ffffffffa025317f ffffffffa0250ada
  [267127.811228]  ffffffff00000001 ffff881016b7bb98 ffffffffa02531ee 00000fe000200000
  [267127.811236] Call Trace:
  [267127.811251]  [<ffffffff8175a3e4>] dump_stack+0x46/0x58
  [267127.811311]  [<ffffffffa025317f>] xfs_error_report+0x3f/0x50 [xfs]
  [267127.811337]  [<ffffffffa0250ada>] ? _xfs_buf_ioapply+0x6a/0x140 [xfs]
  [267127.811363]  [<ffffffffa02531ee>] xfs_corruption_error+0x5e/0x90 [xfs]
  [267127.811396]  [<ffffffffa02711b0>] xfs_attr3_leaf_write_verify+0x100/0x120 [xfs]
  [267127.811421]  [<ffffffffa0250ada>] ? _xfs_buf_ioapply+0x6a/0x140 [xfs]
  [267127.811445]  [<ffffffffa0250e05>] ? xfs_bdstrat_cb+0x65/0xf0 [xfs]
  [267127.811470]  [<ffffffffa0250e05>] ? xfs_bdstrat_cb+0x65/0xf0 [xfs]
  [267127.811494]  [<ffffffffa0250ada>] _xfs_buf_ioapply+0x6a/0x140 [xfs]
  [267127.811505]  [<ffffffff8109fba0>] ? try_to_wake_up+0x210/0x210
  [267127.811530]  [<ffffffffa0250d29>] xfs_buf_iorequest+0x59/0xd0 [xfs]
  [267127.811554]  [<ffffffffa0250e05>] xfs_bdstrat_cb+0x65/0xf0 [xfs]
  [267127.811579]  [<ffffffffa0251013>] __xfs_buf_delwri_submit+0x183/0x210 [xfs]
  [267127.811604]  [<ffffffffa0251ac0>] ? xfs_buf_delwri_submit_nowait+0x20/0x30 [xfs]
  [267127.811630]  [<ffffffffa0251ac0>] xfs_buf_delwri_submit_nowait+0x20/0x30 [xfs]
  [267127.811673]  [<ffffffffa02b4f66>] xfsaild_push+0x126/0x5b0 [xfs]
  [267127.811716]  [<ffffffffa02b548f>] xfsaild+0x9f/0x140 [xfs]
  [267127.811757]  [<ffffffffa02b53f0>] ? xfsaild_push+0x5b0/0x5b0 [xfs]
  [267127.811766]  [<ffffffff8108fcd9>] kthread+0xc9/0xe0
  [267127.811772]  [<ffffffff8108fc10>] ? flush_kthread_worker+0xb0/0xb0
  [267127.811781]  [<ffffffff8176f928>] ret_from_fork+0x58/0x90
  [267127.811787]  [<ffffffff8108fc10>] ? flush_kthread_worker+0xb0/0xb0
  [267127.811791] XFS (sda1): Corruption detected. Unmount and run xfs_repair
  [267127.818679] XFS (sda1): xfs_do_force_shutdown(0x8) called from line 1320 of file /build/buildd/linux-lts-trusty-3.13.0/fs/xfs/xfs_buf.c.  Return address = 0xffffffffa0250bad
  [267127.818927] XFS (sda1): xfs_imap_to_bp: xfs_trans_read_buf() returned error 5.
  [267127.820914] XFS (sda1): Corruption of in-memory data detected.  Shutting down filesystem
  [267127.829131] XFS (sda1): Please umount the filesystem and rectify the problem(s)

  I've not been able to reproduce this on demand outside our production
  workload. Downgrading the machines back to 3.8.0-44 has returned us to
  stability.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux-lts-trusty/+bug/1468039/+subscriptions