kernel-packages team mailing list archive
-
kernel-packages team
-
Mailing list archive
-
Message #175293
[Bug 1576599] Re: XFS Oopsing 4.2.0-35-generic Kernel - Block out of range
On one of my servers I've reverted to the 3.19.0-58-generic kernel. I
still can't mount these broken filesystems.
root@ceph-store4:~# mount /var/lib/ceph/osd/ceph-2
Killed
root@ceph-store4:~# dmesg -T
...
[Fri Apr 29 11:09:27 2016] XFS (bcache4): Mounting V5 Filesystem
[Fri Apr 29 11:09:27 2016] XFS (bcache4): Starting recovery (logdev: internal)
[Fri Apr 29 11:09:29 2016] XFS (bcache4): _xfs_buf_find: Block out of range: block 0x8e8e05f48, EOFS 0x1d1c0bea0
[Fri Apr 29 11:09:29 2016] ------------[ cut here ]------------
[Fri Apr 29 11:09:29 2016] WARNING: CPU: 2 PID: 5427 at /build/linux-lts-vivid-GISjUd/linux-lts-vivid-3.19.0/fs/xfs/xfs_buf.c:473 _xfs_buf_find+0x289/0x2d0 [xfs]()
[Fri Apr 29 11:09:29 2016] Modules linked in: bridge openvswitch ipmi_devintf xfs dcdbas intel_rapl iosf_mbi x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel ipmi_ssif kvm crct10dif_pclmul crc32_pclmul aesni_intel aes_x86_64 lrw gf128mul glue_helper ablk_helper cryptd joydev sb_edac dm_multipath edac_core scsi_dh mei_me lpc_ich mei ipmi_si 8250_fintek ipmi_msghandler wmi acpi_pad mac_hid acpi_power_meter shpchp 8021q garp mrp stp llc lp parport btrfs bcache hid_generic usbhid hid raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq raid1 bnx2x raid0 mpt2sas ptp multipath raid_class pps_core mdio scsi_transport_sas nvme linear libcrc32c
[Fri Apr 29 11:09:29 2016] CPU: 2 PID: 5427 Comm: mount Not tainted 3.19.0-58-generic #64~14.04.1-Ubuntu
[Fri Apr 29 11:09:29 2016] Hardware name: Dell Inc. PowerEdge R720xd/0HJK12, BIOS 2.4.3 07/09/2014
[Fri Apr 29 11:09:29 2016] 0000000000000000 ffff88080aaeb928 ffffffff817b6c3d 0000000000000000
[Fri Apr 29 11:09:29 2016] ffffffffc0931e58 ffff88080aaeb968 ffffffff81075f3a ffff88080aaeb988
[Fri Apr 29 11:09:29 2016] ffff881008135980 0000000000000008 0000000000000000 ffff88080aaeba78
[Fri Apr 29 11:09:29 2016] Call Trace:
[Fri Apr 29 11:09:29 2016] [<ffffffff817b6c3d>] dump_stack+0x63/0x81
[Fri Apr 29 11:09:29 2016] [<ffffffff81075f3a>] warn_slowpath_common+0x8a/0xc0
[Fri Apr 29 11:09:29 2016] [<ffffffff8107602a>] warn_slowpath_null+0x1a/0x20
[Fri Apr 29 11:09:29 2016] [<ffffffffc08e9ec9>] _xfs_buf_find+0x289/0x2d0 [xfs]
[Fri Apr 29 11:09:29 2016] [<ffffffffc08e9f3a>] xfs_buf_get_map+0x2a/0x280 [xfs]
[Fri Apr 29 11:09:29 2016] [<ffffffffc08abbf3>] ? xfs_free_ag_extent+0x453/0x750 [xfs]
[Fri Apr 29 11:09:29 2016] [<ffffffffc09176d1>] xfs_trans_get_buf_map+0x111/0x1a0 [xfs]
[Fri Apr 29 11:09:29 2016] [<ffffffffc08c2690>] xfs_btree_get_bufs+0x50/0x60 [xfs]
[Fri Apr 29 11:09:29 2016] [<ffffffffc08ac5d7>] xfs_alloc_fix_freelist+0x1a7/0x4c0 [xfs]
[Fri Apr 29 11:09:29 2016] [<ffffffffc0907ee7>] ? kmem_zone_alloc+0x77/0x100 [xfs]
[Fri Apr 29 11:09:29 2016] [<ffffffff813b01bd>] ? radix_tree_lookup+0xd/0x10
[Fri Apr 29 11:09:29 2016] [<ffffffffc08de52a>] ? xfs_perag_get+0x2a/0xd0 [xfs]
[Fri Apr 29 11:09:29 2016] [<ffffffffc08ad01a>] xfs_free_extent+0x8a/0x110 [xfs]
[Fri Apr 29 11:09:29 2016] [<ffffffffc0910ea0>] xlog_recover_process_efi+0x170/0x1b0 [xfs]
[Fri Apr 29 11:09:29 2016] [<ffffffffc0913024>] xlog_recover_process_efis.isra.27+0x64/0xb0 [xfs]
[Fri Apr 29 11:09:29 2016] [<ffffffffc0916281>] xlog_recover_finish+0x21/0xb0 [xfs]
[Fri Apr 29 11:09:29 2016] [<ffffffffc090a394>] xfs_log_mount_finish+0x34/0x50 [xfs]
[Fri Apr 29 11:09:29 2016] [<ffffffffc0901509>] xfs_mountfs+0x509/0x760 [xfs]
[Fri Apr 29 11:09:29 2016] [<ffffffffc0904626>] xfs_fs_fill_super+0x2c6/0x340 [xfs]
[Fri Apr 29 11:09:29 2016] [<ffffffff811f23f0>] mount_bdev+0x1b0/0x1f0
[Fri Apr 29 11:09:29 2016] [<ffffffffc0904360>] ? xfs_parseargs+0xbe0/0xbe0 [xfs]
[Fri Apr 29 11:09:29 2016] [<ffffffffc0902895>] xfs_fs_mount+0x15/0x20 [xfs]
[Fri Apr 29 11:09:29 2016] [<ffffffff811f2d49>] mount_fs+0x39/0x1b0
[Fri Apr 29 11:09:29 2016] [<ffffffff8120e57b>] vfs_kern_mount+0x6b/0x120
[Fri Apr 29 11:09:29 2016] [<ffffffff81211324>] do_mount+0x224/0xce0
[Fri Apr 29 11:09:29 2016] [<ffffffff8117c8fe>] ? __get_free_pages+0xe/0x50
[Fri Apr 29 11:09:29 2016] [<ffffffff81210faa>] ? copy_mount_options+0x3a/0x160
[Fri Apr 29 11:09:29 2016] [<ffffffff812120db>] SyS_mount+0x8b/0xe0
[Fri Apr 29 11:09:29 2016] [<ffffffff817be78d>] system_call_fastpath+0x16/0x1b
[Fri Apr 29 11:09:29 2016] ---[ end trace fe0de8e0241625ce ]---
[Fri Apr 29 11:09:29 2016] XFS (bcache4): _xfs_buf_find: Block out of range: block 0x8e8e05f48, EOFS 0x1d1c0bea0
[Fri Apr 29 11:09:29 2016] ------------[ cut here ]------------
[Fri Apr 29 11:09:29 2016] WARNING: CPU: 2 PID: 5427 at /build/linux-lts-vivid-GISjUd/linux-lts-vivid-3.19.0/fs/xfs/xfs_buf.c:473 _xfs_buf_find+0x289/0x2d0 [xfs]()
[Fri Apr 29 11:09:29 2016] Modules linked in: bridge openvswitch ipmi_devintf xfs dcdbas intel_rapl iosf_mbi x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel ipmi_ssif kvm crct10dif_pclmul crc32_pclmul aesni_intel aes_x86_64 lrw gf128mul glue_helper ablk_helper cryptd joydev sb_edac dm_multipath edac_core scsi_dh mei_me lpc_ich mei ipmi_si 8250_fintek ipmi_msghandler wmi acpi_pad mac_hid acpi_power_meter shpchp 8021q garp mrp stp llc lp parport btrfs bcache hid_generic usbhid hid raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq raid1 bnx2x raid0 mpt2sas ptp multipath raid_class pps_core mdio scsi_transport_sas nvme linear libcrc32c
[Fri Apr 29 11:09:29 2016] CPU: 2 PID: 5427 Comm: mount Tainted: G W 3.19.0-58-generic #64~14.04.1-Ubuntu
[Fri Apr 29 11:09:29 2016] Hardware name: Dell Inc. PowerEdge R720xd/0HJK12, BIOS 2.4.3 07/09/2014
[Fri Apr 29 11:09:29 2016] 0000000000000000 ffff88080aaeb928 ffffffff817b6c3d 0000000000000000
[Fri Apr 29 11:09:29 2016] ffffffffc0931e58 ffff88080aaeb968 ffffffff81075f3a ffff88080aaeb988
[Fri Apr 29 11:09:29 2016] ffff881008135980 0000000000000008 ffff8808075be700 ffff88080aaeba78
[Fri Apr 29 11:09:29 2016] Call Trace:
[Fri Apr 29 11:09:29 2016] [<ffffffff817b6c3d>] dump_stack+0x63/0x81
[Fri Apr 29 11:09:29 2016] [<ffffffff81075f3a>] warn_slowpath_common+0x8a/0xc0
[Fri Apr 29 11:09:29 2016] [<ffffffff8107602a>] warn_slowpath_null+0x1a/0x20
[Fri Apr 29 11:09:29 2016] [<ffffffffc08e9ec9>] _xfs_buf_find+0x289/0x2d0 [xfs]
[Fri Apr 29 11:09:29 2016] [<ffffffffc08ea103>] xfs_buf_get_map+0x1f3/0x280 [xfs]
[Fri Apr 29 11:09:29 2016] [<ffffffffc09176d1>] xfs_trans_get_buf_map+0x111/0x1a0 [xfs]
[Fri Apr 29 11:09:29 2016] [<ffffffffc08c2690>] xfs_btree_get_bufs+0x50/0x60 [xfs]
[Fri Apr 29 11:09:29 2016] [<ffffffffc08ac5d7>] xfs_alloc_fix_freelist+0x1a7/0x4c0 [xfs]
[Fri Apr 29 11:09:29 2016] [<ffffffffc0907ee7>] ? kmem_zone_alloc+0x77/0x100 [xfs]
[Fri Apr 29 11:09:29 2016] [<ffffffff813b01bd>] ? radix_tree_lookup+0xd/0x10
[Fri Apr 29 11:09:29 2016] [<ffffffffc08de52a>] ? xfs_perag_get+0x2a/0xd0 [xfs]
[Fri Apr 29 11:09:29 2016] [<ffffffffc08ad01a>] xfs_free_extent+0x8a/0x110 [xfs]
[Fri Apr 29 11:09:29 2016] [<ffffffffc0910ea0>] xlog_recover_process_efi+0x170/0x1b0 [xfs]
[Fri Apr 29 11:09:29 2016] [<ffffffffc0913024>] xlog_recover_process_efis.isra.27+0x64/0xb0 [xfs]
[Fri Apr 29 11:09:29 2016] [<ffffffffc0916281>] xlog_recover_finish+0x21/0xb0 [xfs]
[Fri Apr 29 11:09:29 2016] [<ffffffffc090a394>] xfs_log_mount_finish+0x34/0x50 [xfs]
[Fri Apr 29 11:09:29 2016] [<ffffffffc0901509>] xfs_mountfs+0x509/0x760 [xfs]
[Fri Apr 29 11:09:29 2016] [<ffffffffc0904626>] xfs_fs_fill_super+0x2c6/0x340 [xfs]
[Fri Apr 29 11:09:29 2016] [<ffffffff811f23f0>] mount_bdev+0x1b0/0x1f0
[Fri Apr 29 11:09:29 2016] [<ffffffffc0904360>] ? xfs_parseargs+0xbe0/0xbe0 [xfs]
[Fri Apr 29 11:09:29 2016] [<ffffffffc0902895>] xfs_fs_mount+0x15/0x20 [xfs]
[Fri Apr 29 11:09:29 2016] [<ffffffff811f2d49>] mount_fs+0x39/0x1b0
[Fri Apr 29 11:09:29 2016] [<ffffffff8120e57b>] vfs_kern_mount+0x6b/0x120
[Fri Apr 29 11:09:29 2016] [<ffffffff81211324>] do_mount+0x224/0xce0
[Fri Apr 29 11:09:29 2016] [<ffffffff8117c8fe>] ? __get_free_pages+0xe/0x50
[Fri Apr 29 11:09:29 2016] [<ffffffff81210faa>] ? copy_mount_options+0x3a/0x160
[Fri Apr 29 11:09:29 2016] [<ffffffff812120db>] SyS_mount+0x8b/0xe0
[Fri Apr 29 11:09:29 2016] [<ffffffff817be78d>] system_call_fastpath+0x16/0x1b
[Fri Apr 29 11:09:29 2016] ---[ end trace fe0de8e0241625cf ]---
[Fri Apr 29 11:09:29 2016] BUG: unable to handle kernel NULL pointer dereference at 00000000000000f8
[Fri Apr 29 11:09:29 2016] IP: [<ffffffffc0917f36>] xfs_trans_binval+0x16/0x120 [xfs]
[Fri Apr 29 11:09:29 2016] PGD 8083d5067 PUD 80ad6a067 PMD 0
[Fri Apr 29 11:09:29 2016] Oops: 0000 [#1] SMP
[Fri Apr 29 11:09:29 2016] Modules linked in: bridge openvswitch ipmi_devintf xfs dcdbas intel_rapl iosf_mbi x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel ipmi_ssif kvm crct10dif_pclmul crc32_pclmul aesni_intel aes_x86_64 lrw gf128mul glue_helper ablk_helper cryptd joydev sb_edac dm_multipath edac_core scsi_dh mei_me lpc_ich mei ipmi_si 8250_fintek ipmi_msghandler wmi acpi_pad mac_hid acpi_power_meter shpchp 8021q garp mrp stp llc lp parport btrfs bcache hid_generic usbhid hid raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq raid1 bnx2x raid0 mpt2sas ptp multipath raid_class pps_core mdio scsi_transport_sas nvme linear libcrc32c
[Fri Apr 29 11:09:29 2016] CPU: 2 PID: 5427 Comm: mount Tainted: G W 3.19.0-58-generic #64~14.04.1-Ubuntu
[Fri Apr 29 11:09:29 2016] Hardware name: Dell Inc. PowerEdge R720xd/0HJK12, BIOS 2.4.3 07/09/2014
[Fri Apr 29 11:09:29 2016] task: ffff880809f23110 ti: ffff88080aae8000 task.ti: ffff88080aae8000
[Fri Apr 29 11:09:29 2016] RIP: 0010:[<ffffffffc0917f36>] [<ffffffffc0917f36>] xfs_trans_binval+0x16/0x120 [xfs]
[Fri Apr 29 11:09:29 2016] RSP: 0018:ffff88080aaeba68 EFLAGS: 00010292
[Fri Apr 29 11:09:29 2016] RAX: 0000000000000000 RBX: ffff8807f6c96000 RCX: 0000000000000866
[Fri Apr 29 11:09:29 2016] RDX: 0000000000000865 RSI: 0000000000000000 RDI: ffff8807f6c96000
[Fri Apr 29 11:09:29 2016] RBP: ffff88080aaeba88 R08: 000000000001efe0 R09: ffff88080fa3efe0
[Fri Apr 29 11:09:29 2016] R10: ffffffffc08e934c R11: ffffea00201d6f00 R12: ffff88080aaebb70
[Fri Apr 29 11:09:29 2016] R13: ffff8807f6c96000 R14: 0000000000000000 R15: ffff881007e6cfc0
[Fri Apr 29 11:09:29 2016] FS: 00007f378fedd880(0000) GS:ffff88080fa20000(0000) knlGS:0000000000000000
[Fri Apr 29 11:09:29 2016] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[Fri Apr 29 11:09:29 2016] CR2: 00000000000000f8 CR3: 0000000808e0d000 CR4: 00000000000407e0
[Fri Apr 29 11:09:29 2016] Stack:
[Fri Apr 29 11:09:29 2016] ffff8807f6c96000 ffff88080aaebb70 ffff88080bb2c800 ffff881007f59400
[Fri Apr 29 11:09:29 2016] ffff88080aaebb58 ffffffffc08ac5e2 ffffffff00000000 0000000800000002
[Fri Apr 29 11:09:29 2016] ffffffff00000002 ffff8808075d7a80 0000000000000000 0000000000000000
[Fri Apr 29 11:09:29 2016] Call Trace:
[Fri Apr 29 11:09:29 2016] [<ffffffffc08ac5e2>] xfs_alloc_fix_freelist+0x1b2/0x4c0 [xfs]
[Fri Apr 29 11:09:29 2016] [<ffffffffc0907ee7>] ? kmem_zone_alloc+0x77/0x100 [xfs]
[Fri Apr 29 11:09:29 2016] [<ffffffff813b01bd>] ? radix_tree_lookup+0xd/0x10
[Fri Apr 29 11:09:29 2016] [<ffffffffc08de52a>] ? xfs_perag_get+0x2a/0xd0 [xfs]
[Fri Apr 29 11:09:29 2016] [<ffffffffc08ad01a>] xfs_free_extent+0x8a/0x110 [xfs]
[Fri Apr 29 11:09:29 2016] [<ffffffffc0910ea0>] xlog_recover_process_efi+0x170/0x1b0 [xfs]
[Fri Apr 29 11:09:29 2016] [<ffffffffc0913024>] xlog_recover_process_efis.isra.27+0x64/0xb0 [xfs]
[Fri Apr 29 11:09:29 2016] [<ffffffffc0916281>] xlog_recover_finish+0x21/0xb0 [xfs]
[Fri Apr 29 11:09:29 2016] [<ffffffffc090a394>] xfs_log_mount_finish+0x34/0x50 [xfs]
[Fri Apr 29 11:09:29 2016] [<ffffffffc0901509>] xfs_mountfs+0x509/0x760 [xfs]
[Fri Apr 29 11:09:29 2016] [<ffffffffc0904626>] xfs_fs_fill_super+0x2c6/0x340 [xfs]
[Fri Apr 29 11:09:29 2016] [<ffffffff811f23f0>] mount_bdev+0x1b0/0x1f0
[Fri Apr 29 11:09:29 2016] [<ffffffffc0904360>] ? xfs_parseargs+0xbe0/0xbe0 [xfs]
[Fri Apr 29 11:09:29 2016] [<ffffffffc0902895>] xfs_fs_mount+0x15/0x20 [xfs]
[Fri Apr 29 11:09:29 2016] [<ffffffff811f2d49>] mount_fs+0x39/0x1b0
[Fri Apr 29 11:09:29 2016] [<ffffffff8120e57b>] vfs_kern_mount+0x6b/0x120
[Fri Apr 29 11:09:29 2016] [<ffffffff81211324>] do_mount+0x224/0xce0
[Fri Apr 29 11:09:29 2016] [<ffffffff8117c8fe>] ? __get_free_pages+0xe/0x50
[Fri Apr 29 11:09:29 2016] [<ffffffff81210faa>] ? copy_mount_options+0x3a/0x160
[Fri Apr 29 11:09:29 2016] [<ffffffff812120db>] SyS_mount+0x8b/0xe0
[Fri Apr 29 11:09:29 2016] [<ffffffff817be78d>] system_call_fastpath+0x16/0x1b
[Fri Apr 29 11:09:29 2016] Code: 75 eb e9 53 ff ff ff 66 66 66 66 66 2e 0f 1f 84 00 00 00 00 00 66 66 66 66 90 55 48 89 e5 41 56 49 89 f6 41 55 49 89 fd 41 54 53 <4c> 8b a6 f8 00 00 00 66 66 66 66 90 41 f6 44 24 78 04 74 5b 5b
[Fri Apr 29 11:09:29 2016] RIP [<ffffffffc0917f36>] xfs_trans_binval+0x16/0x120 [xfs]
[Fri Apr 29 11:09:29 2016] RSP <ffff88080aaeba68>
[Fri Apr 29 11:09:29 2016] CR2: 00000000000000f8
[Fri Apr 29 11:09:29 2016] ---[ end trace fe0de8e0241625d0 ]---
--
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1576599
Title:
XFS Oopsing 4.2.0-35-generic Kernel - Block out of range
Status in linux package in Ubuntu:
Confirmed
Bug description:
After updating the Kernel on two of our Trusty Ceph servers from
3.19.0-33-generic (vivid) to 4.2.0-35-generic (wily), both servers
have now Oopsed multiple times accessing some of their XFS
filesystems.
Once this error occurs, the XFS filesystem becomes permanently
inaccessible. Even after a reboot, any attempt to access the
filesystem in question either by mounting it, or attempting an
xfs_repair will trigger the same Oops again.
These filesystems did not immediately fail until several hours after
the upgrade. I suspect that this is related to some housekeeping being
triggered within the filesystem.
This is the call trace from the moment a filesystem first failed:
Apr 28 22:47:25 ceph-store5 kernel: [26692.804773] ------------[ cut here ]------------
Apr 28 22:47:25 ceph-store5 kernel: [26692.810046] WARNING: CPU: 8 PID: 5195 at /build/linux-lts-wily-H6pWeh/linux-lts-wily-4.2.0/fs/xfs/xfs_buf.c:473 _xfs_buf_find+0x27e/0x2d0 [xfs]()
Apr 28 22:47:25 ceph-store5 kernel: [26692.824715] Modules linked in: bridge openvswitch xfs dcdbas ipmi_devintf intel_rapl x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel ipmi_ssif kvm crct10dif_pclmul crc32_pclmul aesni_intel aes_x86_64 lrw gf128mul glue_helper ablk_helper cryptd joydev dm_multipath sb_edac scsi_dh edac_core input_leds mei_me lpc_ich mei ipmi_si ipmi_msghandler 8250_fintek wmi acpi_power_meter 8021q acpi_pad mac_hid garp mrp stp llc shpchp lp parport btrfs bcache raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx hid_generic xor usbhid hid raid6_pq bnx2x raid1 ptp mpt2sas raid0 pps_core multipath raid_class mdio scsi_transport_sas nvme libcrc32c linear
Apr 28 22:47:25 ceph-store5 kernel: [26692.891939] CPU: 8 PID: 5195 Comm: ceph-osd Tainted: G D W 4.2.0-35-generic #40~14.04.1-Ubuntu
Apr 28 22:47:25 ceph-store5 kernel: [26692.902689] Hardware name: Dell Inc. PowerEdge R720xd/0HJK12, BIOS 2.4.3 07/09/2014
Apr 28 22:47:25 ceph-store5 kernel: [26692.911303] 0000000000000000 ffff8800c7bf74a8 ffffffff817bcbf8 0000000000000000
Apr 28 22:47:25 ceph-store5 kernel: [26692.919677] ffffffffc0a7f300 ffff8800c7bf74e8 ffffffff81079b5a ffff8800c7bf7508
Apr 28 22:47:25 ceph-store5 kernel: [26692.928033] ffff88100a28ed80 0000000000000008 0000000000000000 ffff8800c7bf75f8
Apr 28 22:47:25 ceph-store5 kernel: [26692.936441] Call Trace:
Apr 28 22:47:25 ceph-store5 kernel: [26692.939208] [<ffffffff817bcbf8>] dump_stack+0x63/0x81
Apr 28 22:47:25 ceph-store5 kernel: [26692.944992] [<ffffffff81079b5a>] warn_slowpath_common+0x8a/0xc0
Apr 28 22:47:25 ceph-store5 kernel: [26692.951752] [<ffffffff81079c4a>] warn_slowpath_null+0x1a/0x20
Apr 28 22:47:25 ceph-store5 kernel: [26692.958346] [<ffffffffc0a3e12e>] _xfs_buf_find+0x27e/0x2d0 [xfs]
Apr 28 22:47:25 ceph-store5 kernel: [26692.965221] [<ffffffffc0a3e1aa>] xfs_buf_get_map+0x2a/0x260 [xfs]
Apr 28 22:47:25 ceph-store5 kernel: [26692.972194] [<ffffffffc0a0130a>] ? xfs_free_ag_extent+0x43a/0x730 [xfs]
Apr 28 22:47:25 ceph-store5 kernel: [26692.979763] [<ffffffffc0a69f26>] xfs_trans_get_buf_map+0x106/0x190 [xfs]
Apr 28 22:47:25 ceph-store5 kernel: [26692.987412] [<ffffffffc0a17380>] xfs_btree_get_bufs+0x50/0x60 [xfs]
Apr 28 22:47:25 ceph-store5 kernel: [26692.994564] [<ffffffffc0a01d1b>] xfs_alloc_fix_freelist+0x20b/0x3c0 [xfs]
Apr 28 22:47:25 ceph-store5 kernel: [26693.002302] [<ffffffffc0a3dd3f>] ? xfs_buf_trylock+0x1f/0x90 [xfs]
Apr 28 22:47:25 ceph-store5 kernel: [26693.009379] [<ffffffffc0a3281a>] ? xfs_perag_get+0x2a/0xc0 [xfs]
Apr 28 22:47:25 ceph-store5 kernel: [26693.016229] [<ffffffff813b0eed>] ? radix_tree_lookup+0xd/0x10
Apr 28 22:47:25 ceph-store5 kernel: [26693.022814] [<ffffffffc0a3281a>] ? xfs_perag_get+0x2a/0xc0 [xfs]
Apr 28 22:47:25 ceph-store5 kernel: [26693.029669] [<ffffffffc0a020a2>] xfs_alloc_vextent+0x1d2/0x660 [xfs]
Apr 28 22:47:25 ceph-store5 kernel: [26693.036915] [<ffffffffc0a12746>] xfs_bmap_btalloc+0x376/0x6b0 [xfs]
Apr 28 22:47:25 ceph-store5 kernel: [26693.044074] [<ffffffffc0a12aa4>] xfs_bmap_alloc+0x24/0x40 [xfs]
Apr 28 22:47:25 ceph-store5 kernel: [26693.050833] [<ffffffffc0a13456>] xfs_bmapi_write+0x486/0x9a0 [xfs]
Apr 28 22:47:25 ceph-store5 kernel: [26693.057882] [<ffffffffc0a1d1ce>] xfs_da_grow_inode_int+0xae/0x2d0 [xfs]
Apr 28 22:47:25 ceph-store5 kernel: [26693.065403] [<ffffffff811d1bc2>] ? __kmalloc+0x1d2/0x280
Apr 28 22:47:25 ceph-store5 kernel: [26693.071509] [<ffffffffc0a5b080>] ? kmem_alloc+0x70/0xf0 [xfs]
Apr 28 22:47:25 ceph-store5 kernel: [26693.078078] [<ffffffffc0a1d427>] xfs_da_grow_inode+0x37/0x90 [xfs]
Apr 28 22:47:25 ceph-store5 kernel: [26693.085129] [<ffffffffc0a08562>] xfs_attr_shortform_to_leaf+0xa2/0x2b0 [xfs]
Apr 28 22:47:25 ceph-store5 kernel: [26693.093151] [<ffffffffc0a5b2f0>] ? kmem_zone_alloc+0x70/0xf0 [xfs]
Apr 28 22:47:25 ceph-store5 kernel: [26693.100199] [<ffffffffc0a5b2f0>] ? kmem_zone_alloc+0x70/0xf0 [xfs]
Apr 28 22:47:25 ceph-store5 kernel: [26693.107251] [<ffffffffc0a047eb>] xfs_attr_set+0x31b/0x3e0 [xfs]
Apr 28 22:47:25 ceph-store5 kernel: [26693.114009] [<ffffffffc0a5acc2>] xfs_xattr_set+0x42/0x60 [xfs]
Apr 28 22:47:25 ceph-store5 kernel: [26693.120647] [<ffffffff81212a42>] generic_setxattr+0x62/0x80
Apr 28 22:47:25 ceph-store5 kernel: [26693.127107] [<ffffffff812134e1>] __vfs_setxattr_noperm+0x61/0x1a0
Apr 28 22:47:25 ceph-store5 kernel: [26693.134045] [<ffffffff812136c7>] vfs_setxattr+0xa7/0xb0
Apr 28 22:47:25 ceph-store5 kernel: [26693.145904] [<ffffffff8121376d>] ? setxattr+0x9d/0x1b0
Apr 28 22:47:25 ceph-store5 kernel: [26693.151781] [<ffffffff812137fa>] setxattr+0x12a/0x1b0
Apr 28 22:47:25 ceph-store5 kernel: [26693.157626] [<ffffffffc0a73fcb>] ? xfs_acl_exists+0x4b/0x60 [xfs]
Apr 28 22:47:25 ceph-store5 kernel: [26693.164599] [<ffffffffc0a74505>] ? posix_acl_access_exists+0x15/0x20 [xfs]
Apr 28 22:47:25 ceph-store5 kernel: [26693.172451] [<ffffffffc0a5af52>] ? xfs_vn_listxattr+0xf2/0x160 [xfs]
Apr 28 22:47:25 ceph-store5 kernel: [26693.179694] [<ffffffff811f1089>] ? __sb_start_write+0x49/0xf0
Apr 28 22:47:25 ceph-store5 kernel: [26693.186278] [<ffffffffc0a5ad30>] ? xfs_xattr_get+0x50/0x50 [xfs]
Apr 28 22:47:25 ceph-store5 kernel: [26693.193122] [<ffffffff81213a90>] SyS_fsetxattr+0xa0/0xd0
Apr 28 22:47:25 ceph-store5 kernel: [26693.199193] [<ffffffff817c40b2>] entry_SYSCALL_64_fastpath+0x16/0x75
Apr 28 22:47:25 ceph-store5 kernel: [26693.206471] ---[ end trace c0a9568b8830fc8b ]---
Apr 28 22:47:25 ceph-store5 kernel: [26693.211904] XFS (bcache7): _xfs_buf_find: Block out of range: block 0x874702fa0, EOFS 0x1d1c0bea0
Apr 28 22:47:25 ceph-store5 kernel: [26693.221992] ------------[ cut here ]------------
I notice that xfs_da_grow_inode. Am I right in thinking that this was
a new feature add in the 4.2 kernel?
I expect that the history of these filesystems is important. These
would more have been formatted on a 3.16 kernel. Although I can't
inspect these failed filesystems, I can give information about the
other filesystems on the same servers, which I an 99% sure are
identical:
root@ceph-store5:~# xfs_info /var/lib/ceph/osd/ceph-35
meta-data=/dev/bcache6 isize=512 agcount=4, agsize=244188661 blks
= sectsz=512 attr=2, projid32bit=1
= crc=1 finobt=1
data = bsize=4096 blocks=976754644, imaxpct=5
= sunit=0 swidth=0 blks
naming =version 2 bsize=4096 ascii-ci=0 ftype=1
log =internal bsize=4096 blocks=476930, version=2
= sectsz=512 sunit=0 blks, lazy-count=1
realtime =none extsz=4096 blocks=0, rtextents=0
==== Attempt to repair a filesystem (after a reboot) ====
root@ceph-store5:~# xfs_repair /dev/disk/by-uuid/398f78ec-c20b-468f-a2d6-1eb14804834c
Phase 1 - find and verify superblock...
Phase 2 - using internal log
- zero log...
ERROR: The filesystem has valuable metadata changes in a log which needs to
be replayed. Mount the filesystem to replay the log, and unmount it before
re-running xfs_repair. If you are unable to mount the filesystem, then use
the -L option to destroy the log and attempt a repair.
Note that destroying the log may cause corruption -- please attempt a mount
of the filesystem before doing this.
root@ceph-store5:~# mount -o noatime,inode64 /dev/disk/by-uuid/398f78ec-c20b-468f-a2d6-1eb14804834c /var/lib/ceph/osd/ceph-39/
Killed
Dmesg output (dmesg -T):
[Fri Apr 29 10:18:45 2016] XFS (bcache2): Mounting V5 Filesystem
[Fri Apr 29 10:18:45 2016] XFS (bcache2): Starting recovery (logdev: internal)
[Fri Apr 29 10:18:49 2016] XFS (bcache2): _xfs_buf_find: Block out of range: block 0x874702fa0, EOFS 0x1d1c0bea0
[Fri Apr 29 10:18:49 2016] ------------[ cut here ]------------
[Fri Apr 29 10:18:49 2016] WARNING: CPU: 6 PID: 211620 at /build/linux-lts-wily-H6pWeh/linux-lts-wily-4.2.0/fs/xfs/xfs_buf.c:473 _xfs_buf_find+0x27e/0x2d0 [xfs]()
[Fri Apr 29 10:18:49 2016] Modules linked in: bridge openvswitch xfs ipmi_devintf dcdbas intel_rapl x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel kvm ipmi_ssif crct10dif_pclmul crc32_pclmul aesni_intel aes_x86_64 lrw gf128mul glue_helper ablk_helper cryptd joydev input_leds dm_multipath sb_edac scsi_dh edac_core mei_me ipmi_si lpc_ich mei ipmi_msghandler 8250_fintek wmi acpi_pad acpi_power_meter 8021q mac_hid garp mrp stp llc shpchp lp parport btrfs bcache hid_generic usbhid hid raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq raid1 bnx2x raid0 ptp mpt2sas multipath pps_core raid_class mdio scsi_transport_sas nvme libcrc32c linear
[Fri Apr 29 10:18:49 2016] CPU: 6 PID: 211620 Comm: mount Not tainted 4.2.0-35-generic #40~14.04.1-Ubuntu
[Fri Apr 29 10:18:49 2016] Hardware name: Dell Inc. PowerEdge R720xd/0HJK12, BIOS 2.4.3 07/09/2014
[Fri Apr 29 10:18:49 2016] 0000000000000000 ffff88018e11b8c8 ffffffff817bcbf8 0000000000000000
[Fri Apr 29 10:18:49 2016] ffffffffc0653300 ffff88018e11b908 ffffffff81079b5a ffff88018e11b928
[Fri Apr 29 10:18:49 2016] ffff880806ae8840 0000000000000008 0000000000000000 ffff88018e11ba18
[Fri Apr 29 10:18:49 2016] Call Trace:
[Fri Apr 29 10:18:49 2016] [<ffffffff817bcbf8>] dump_stack+0x63/0x81
[Fri Apr 29 10:18:49 2016] [<ffffffff81079b5a>] warn_slowpath_common+0x8a/0xc0
[Fri Apr 29 10:18:49 2016] [<ffffffff81079c4a>] warn_slowpath_null+0x1a/0x20
[Fri Apr 29 10:18:49 2016] [<ffffffffc061212e>] _xfs_buf_find+0x27e/0x2d0 [xfs]
[Fri Apr 29 10:18:49 2016] [<ffffffffc06121aa>] xfs_buf_get_map+0x2a/0x260 [xfs]
[Fri Apr 29 10:18:49 2016] [<ffffffffc05d530a>] ? xfs_free_ag_extent+0x43a/0x730 [xfs]
[Fri Apr 29 10:18:49 2016] [<ffffffffc063df26>] xfs_trans_get_buf_map+0x106/0x190 [xfs]
[Fri Apr 29 10:18:49 2016] [<ffffffffc05eb380>] xfs_btree_get_bufs+0x50/0x60 [xfs]
[Fri Apr 29 10:18:49 2016] [<ffffffffc05d5d1b>] xfs_alloc_fix_freelist+0x20b/0x3c0 [xfs]
[Fri Apr 29 10:18:49 2016] [<ffffffff811d1075>] ? kmem_cache_alloc+0x1e5/0x210
[Fri Apr 29 10:18:49 2016] [<ffffffffc062f2f0>] ? kmem_zone_alloc+0x70/0xf0 [xfs]
[Fri Apr 29 10:18:49 2016] [<ffffffff813b0eed>] ? radix_tree_lookup+0xd/0x10
[Fri Apr 29 10:18:49 2016] [<ffffffffc060681a>] ? xfs_perag_get+0x2a/0xc0 [xfs]
[Fri Apr 29 10:18:49 2016] [<ffffffffc05d65bd>] xfs_free_extent+0x8d/0x110 [xfs]
[Fri Apr 29 10:18:49 2016] [<ffffffffc0637ca5>] xlog_recover_process_efi+0x165/0x1a0 [xfs]
[Fri Apr 29 10:18:49 2016] [<ffffffffc0639c0a>] xlog_recover_process_efis.isra.27+0x5a/0xb0 [xfs]
[Fri Apr 29 10:18:49 2016] [<ffffffffc063cc01>] xlog_recover_finish+0x21/0xb0 [xfs]
[Fri Apr 29 10:18:49 2016] [<ffffffffc06315df>] xfs_log_mount_finish+0x2f/0x50 [xfs]
[Fri Apr 29 10:18:49 2016] [<ffffffffc0628dbb>] xfs_mountfs+0x4db/0x740 [xfs]
[Fri Apr 29 10:18:49 2016] [<ffffffffc062b96d>] xfs_fs_fill_super+0x36d/0x460 [xfs]
[Fri Apr 29 10:18:49 2016] [<ffffffff811f1f82>] mount_bdev+0x1a2/0x1e0
[Fri Apr 29 10:18:49 2016] [<ffffffffc062b600>] ? xfs_parseargs+0xac0/0xac0 [xfs]
[Fri Apr 29 10:18:49 2016] [<ffffffffc0629e55>] xfs_fs_mount+0x15/0x20 [xfs]
[Fri Apr 29 10:18:49 2016] [<ffffffff811f2879>] mount_fs+0x39/0x160
[Fri Apr 29 10:18:49 2016] [<ffffffff8120d93b>] vfs_kern_mount+0x6b/0x120
[Fri Apr 29 10:18:49 2016] [<ffffffff8120fe8e>] do_mount+0x21e/0xcf0
[Fri Apr 29 10:18:49 2016] [<ffffffff8117db6e>] ? __get_free_pages+0xe/0x40
[Fri Apr 29 10:18:49 2016] [<ffffffff8120fb3a>] ? copy_mount_options+0x3a/0x150
[Fri Apr 29 10:18:49 2016] [<ffffffff81210c4b>] SyS_mount+0x8b/0xe0
[Fri Apr 29 10:18:49 2016] [<ffffffff817c40b2>] entry_SYSCALL_64_fastpath+0x16/0x75
[Fri Apr 29 10:18:49 2016] ---[ end trace ce3e7a80324237e0 ]---
[Fri Apr 29 10:18:49 2016] XFS (bcache2): _xfs_buf_find: Block out of range: block 0x874702fa0, EOFS 0x1d1c0bea0
[Fri Apr 29 10:18:49 2016] ------------[ cut here ]------------
[Fri Apr 29 10:18:49 2016] WARNING: CPU: 6 PID: 211620 at /build/linux-lts-wily-H6pWeh/linux-lts-wily-4.2.0/fs/xfs/xfs_buf.c:473 _xfs_buf_find+0x27e/0x2d0 [xfs]()
[Fri Apr 29 10:18:49 2016] Modules linked in: bridge openvswitch xfs ipmi_devintf dcdbas intel_rapl x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel kvm ipmi_ssif crct10dif_pclmul crc32_pclmul aesni_intel aes_x86_64 lrw gf128mul glue_helper ablk_helper cryptd joydev input_leds dm_multipath sb_edac scsi_dh edac_core mei_me ipmi_si lpc_ich mei ipmi_msghandler 8250_fintek wmi acpi_pad acpi_power_meter 8021q mac_hid garp mrp stp llc shpchp lp parport btrfs bcache hid_generic usbhid hid raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq raid1 bnx2x raid0 ptp mpt2sas multipath pps_core raid_class mdio scsi_transport_sas nvme libcrc32c linear
[Fri Apr 29 10:18:49 2016] CPU: 6 PID: 211620 Comm: mount Tainted: G W 4.2.0-35-generic #40~14.04.1-Ubuntu
[Fri Apr 29 10:18:49 2016] Hardware name: Dell Inc. PowerEdge R720xd/0HJK12, BIOS 2.4.3 07/09/2014
[Fri Apr 29 10:18:49 2016] 0000000000000000 ffff88018e11b8c8 ffffffff817bcbf8 0000000000000000
[Fri Apr 29 10:18:49 2016] ffffffffc0653300 ffff88018e11b908 ffffffff81079b5a ffff88018e11b928
[Fri Apr 29 10:18:49 2016] ffff880806ae8840 0000000000000008 0000000000000000 ffff88018e11ba18
[Fri Apr 29 10:18:49 2016] Call Trace:
[Fri Apr 29 10:18:49 2016] [<ffffffff817bcbf8>] dump_stack+0x63/0x81
[Fri Apr 29 10:18:49 2016] [<ffffffff81079b5a>] warn_slowpath_common+0x8a/0xc0
[Fri Apr 29 10:18:49 2016] [<ffffffff81079c4a>] warn_slowpath_null+0x1a/0x20
[Fri Apr 29 10:18:49 2016] [<ffffffffc061212e>] _xfs_buf_find+0x27e/0x2d0 [xfs]
[Fri Apr 29 10:18:49 2016] [<ffffffffc061235d>] xfs_buf_get_map+0x1dd/0x260 [xfs]
[Fri Apr 29 10:18:49 2016] [<ffffffffc063df26>] xfs_trans_get_buf_map+0x106/0x190 [xfs]
[Fri Apr 29 10:18:49 2016] [<ffffffffc05eb380>] xfs_btree_get_bufs+0x50/0x60 [xfs]
[Fri Apr 29 10:18:49 2016] [<ffffffffc05d5d1b>] xfs_alloc_fix_freelist+0x20b/0x3c0 [xfs]
[Fri Apr 29 10:18:49 2016] [<ffffffff811d1075>] ? kmem_cache_alloc+0x1e5/0x210
[Fri Apr 29 10:18:49 2016] [<ffffffffc062f2f0>] ? kmem_zone_alloc+0x70/0xf0 [xfs]
[Fri Apr 29 10:18:49 2016] [<ffffffff813b0eed>] ? radix_tree_lookup+0xd/0x10
[Fri Apr 29 10:18:49 2016] [<ffffffffc060681a>] ? xfs_perag_get+0x2a/0xc0 [xfs]
[Fri Apr 29 10:18:49 2016] [<ffffffffc05d65bd>] xfs_free_extent+0x8d/0x110 [xfs]
[Fri Apr 29 10:18:49 2016] [<ffffffffc0637ca5>] xlog_recover_process_efi+0x165/0x1a0 [xfs]
[Fri Apr 29 10:18:49 2016] [<ffffffffc0639c0a>] xlog_recover_process_efis.isra.27+0x5a/0xb0 [xfs]
[Fri Apr 29 10:18:49 2016] [<ffffffffc063cc01>] xlog_recover_finish+0x21/0xb0 [xfs]
[Fri Apr 29 10:18:49 2016] [<ffffffffc06315df>] xfs_log_mount_finish+0x2f/0x50 [xfs]
[Fri Apr 29 10:18:49 2016] [<ffffffffc0628dbb>] xfs_mountfs+0x4db/0x740 [xfs]
[Fri Apr 29 10:18:49 2016] [<ffffffffc062b96d>] xfs_fs_fill_super+0x36d/0x460 [xfs]
[Fri Apr 29 10:18:49 2016] [<ffffffff811f1f82>] mount_bdev+0x1a2/0x1e0
[Fri Apr 29 10:18:49 2016] [<ffffffffc062b600>] ? xfs_parseargs+0xac0/0xac0 [xfs]
[Fri Apr 29 10:18:49 2016] [<ffffffffc0629e55>] xfs_fs_mount+0x15/0x20 [xfs]
[Fri Apr 29 10:18:49 2016] [<ffffffff811f2879>] mount_fs+0x39/0x160
[Fri Apr 29 10:18:49 2016] [<ffffffff8120d93b>] vfs_kern_mount+0x6b/0x120
[Fri Apr 29 10:18:49 2016] [<ffffffff8120fe8e>] do_mount+0x21e/0xcf0
[Fri Apr 29 10:18:49 2016] [<ffffffff8117db6e>] ? __get_free_pages+0xe/0x40
[Fri Apr 29 10:18:49 2016] [<ffffffff8120fb3a>] ? copy_mount_options+0x3a/0x150
[Fri Apr 29 10:18:49 2016] [<ffffffff81210c4b>] SyS_mount+0x8b/0xe0
[Fri Apr 29 10:18:49 2016] [<ffffffff817c40b2>] entry_SYSCALL_64_fastpath+0x16/0x75
[Fri Apr 29 10:18:49 2016] ---[ end trace ce3e7a80324237e1 ]---
[Fri Apr 29 10:18:49 2016] BUG: unable to handle kernel NULL pointer dereference at 00000000000000f8
[Fri Apr 29 10:18:49 2016] IP: [<ffffffffc063e6f6>] xfs_trans_binval+0x16/0x110 [xfs]
[Fri Apr 29 10:18:49 2016] PGD 564eec067 PUD 11408b067 PMD 0
[Fri Apr 29 10:18:49 2016] Oops: 0000 [#1] SMP
[Fri Apr 29 10:18:49 2016] Modules linked in: bridge openvswitch xfs ipmi_devintf dcdbas intel_rapl x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel kvm ipmi_ssif crct10dif_pclmul crc32_pclmul aesni_intel aes_x86_64 lrw gf128mul glue_helper ablk_helper cryptd joydev input_leds dm_multipath sb_edac scsi_dh edac_core mei_me ipmi_si lpc_ich mei ipmi_msghandler 8250_fintek wmi acpi_pad acpi_power_meter 8021q mac_hid garp mrp stp llc shpchp lp parport btrfs bcache hid_generic usbhid hid raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq raid1 bnx2x raid0 ptp mpt2sas multipath pps_core raid_class mdio scsi_transport_sas nvme libcrc32c linear
[Fri Apr 29 10:18:49 2016] CPU: 6 PID: 211620 Comm: mount Tainted: G W 4.2.0-35-generic #40~14.04.1-Ubuntu
[Fri Apr 29 10:18:49 2016] Hardware name: Dell Inc. PowerEdge R720xd/0HJK12, BIOS 2.4.3 07/09/2014
[Fri Apr 29 10:18:49 2016] task: ffff88080967ee00 ti: ffff88018e118000 task.ti: ffff88018e118000
[Fri Apr 29 10:18:49 2016] RIP: 0010:[<ffffffffc063e6f6>] [<ffffffffc063e6f6>] xfs_trans_binval+0x16/0x110 [xfs]
[Fri Apr 29 10:18:49 2016] RSP: 0018:ffff88018e11ba08 EFLAGS: 00010292
[Fri Apr 29 10:18:49 2016] RAX: 0000000000000000 RBX: ffff88018e11bb18 RCX: 00000000003d2d52
[Fri Apr 29 10:18:49 2016] RDX: 00000000003d2d51 RSI: 0000000000000000 RDI: ffff880afa4e4828
[Fri Apr 29 10:18:49 2016] RBP: ffff88018e11ba28 R08: 000000000001a4f0 R09: ffff88080f8da4f0
[Fri Apr 29 10:18:49 2016] R10: ffffffffc061164a R11: ffffea001281d600 R12: ffff880afa4e4828
[Fri Apr 29 10:18:49 2016] R13: ffff880afa4e4828 R14: 0000000000000000 R15: 0000000000000008
[Fri Apr 29 10:18:49 2016] FS: 00007f42d515f880(0000) GS:ffff88080f8c0000(0000) knlGS:0000000000000000
[Fri Apr 29 10:18:49 2016] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[Fri Apr 29 10:18:49 2016] CR2: 00000000000000f8 CR3: 0000000139a06000 CR4: 00000000000406e0
[Fri Apr 29 10:18:49 2016] Stack:
[Fri Apr 29 10:18:49 2016] ffff88018e11bb18 ffff880afa4e4828 ffff88075d649ec0 ffff880164bc8800
[Fri Apr 29 10:18:49 2016] ffff88018e11bb08 ffffffffc05d5d26 ffff88015e77eb80 0000000200000000
[Fri Apr 29 10:18:49 2016] ffff88018e11ba98 ffffffff811d1075 ffff88015e77eb80 0000000000000000
[Fri Apr 29 10:18:49 2016] Call Trace:
[Fri Apr 29 10:18:49 2016] [<ffffffffc05d5d26>] xfs_alloc_fix_freelist+0x216/0x3c0 [xfs]
[Fri Apr 29 10:18:49 2016] [<ffffffff811d1075>] ? kmem_cache_alloc+0x1e5/0x210
[Fri Apr 29 10:18:49 2016] [<ffffffffc062f2f0>] ? kmem_zone_alloc+0x70/0xf0 [xfs]
[Fri Apr 29 10:18:49 2016] [<ffffffff813b0eed>] ? radix_tree_lookup+0xd/0x10
[Fri Apr 29 10:18:49 2016] [<ffffffffc060681a>] ? xfs_perag_get+0x2a/0xc0 [xfs]
[Fri Apr 29 10:18:49 2016] [<ffffffffc05d65bd>] xfs_free_extent+0x8d/0x110 [xfs]
[Fri Apr 29 10:18:49 2016] [<ffffffffc0637ca5>] xlog_recover_process_efi+0x165/0x1a0 [xfs]
[Fri Apr 29 10:18:49 2016] [<ffffffffc0639c0a>] xlog_recover_process_efis.isra.27+0x5a/0xb0 [xfs]
[Fri Apr 29 10:18:49 2016] [<ffffffffc063cc01>] xlog_recover_finish+0x21/0xb0 [xfs]
[Fri Apr 29 10:18:49 2016] [<ffffffffc06315df>] xfs_log_mount_finish+0x2f/0x50 [xfs]
[Fri Apr 29 10:18:49 2016] [<ffffffffc0628dbb>] xfs_mountfs+0x4db/0x740 [xfs]
[Fri Apr 29 10:18:49 2016] [<ffffffffc062b96d>] xfs_fs_fill_super+0x36d/0x460 [xfs]
[Fri Apr 29 10:18:49 2016] [<ffffffff811f1f82>] mount_bdev+0x1a2/0x1e0
[Fri Apr 29 10:18:49 2016] [<ffffffffc062b600>] ? xfs_parseargs+0xac0/0xac0 [xfs]
[Fri Apr 29 10:18:49 2016] [<ffffffffc0629e55>] xfs_fs_mount+0x15/0x20 [xfs]
[Fri Apr 29 10:18:49 2016] [<ffffffff811f2879>] mount_fs+0x39/0x160
[Fri Apr 29 10:18:49 2016] [<ffffffff8120d93b>] vfs_kern_mount+0x6b/0x120
[Fri Apr 29 10:18:49 2016] [<ffffffff8120fe8e>] do_mount+0x21e/0xcf0
[Fri Apr 29 10:18:49 2016] [<ffffffff8117db6e>] ? __get_free_pages+0xe/0x40
[Fri Apr 29 10:18:50 2016] [<ffffffff8120fb3a>] ? copy_mount_options+0x3a/0x150
[Fri Apr 29 10:18:50 2016] [<ffffffff81210c4b>] SyS_mount+0x8b/0xe0
[Fri Apr 29 10:18:50 2016] [<ffffffff817c40b2>] entry_SYSCALL_64_fastpath+0x16/0x75
[Fri Apr 29 10:18:50 2016] Code: 13 48 85 d2 75 eb e9 5f ff ff ff 66 2e 0f 1f 84 00 00 00 00 00 66 66 66 66 90 55 48 89 e5 41 56 49 89 f6 41 55 49 89 fd 41 54 53 <4c> 8b a6 f8 00 00 00 66 66 66 66 90 41 f6 44 24 78 04 74 4f 5b
[Fri Apr 29 10:18:50 2016] RIP [<ffffffffc063e6f6>] xfs_trans_binval+0x16/0x110 [xfs]
[Fri Apr 29 10:18:50 2016] RSP <ffff88018e11ba08>
[Fri Apr 29 10:18:50 2016] CR2: 00000000000000f8
[Fri Apr 29 10:18:50 2016] ---[ end trace ce3e7a80324237e2 ]---
root@ceph-store5:~#
Thanks to the magic of Ceph, I've so far managed to avoid data-loss
and serious downtime, but I suspect there's a timebomb here waiting
for anyone who upgrades from an older kernel to 4.2.0+.
Please let me know is there's any further information I might be able to provide to track this down.
---
AlsaDevices:
total 0
crw-rw---- 1 root audio 116, 1 Apr 29 01:29 seq
crw-rw---- 1 root audio 116, 33 Apr 29 01:29 timer
AplayDevices: Error: [Errno 2] No such file or directory
ApportVersion: 2.14.1-0ubuntu3.19
Architecture: amd64
ArecordDevices: Error: [Errno 2] No such file or directory
AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1:
CRDA: Error: [Errno 2] No such file or directory
DistroRelease: Ubuntu 14.04
MachineType: Dell Inc. PowerEdge R720xd
Package: linux (not installed)
PciMultimedia:
ProcEnviron:
TERM=xterm-256color
PATH=(custom, no user)
XDG_RUNTIME_DIR=<set>
LANG=en_US.UTF-8
SHELL=/bin/bash
ProcFB:
ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.2.0-35-generic root=UUID=f022f2e6-eeba-4e27-b286-6acd2c4bd1ea ro net.ifnames=1 biosdevname=0 console=tty1 console=ttyS1,115200 initrd=ubuntu/amd64/generic/trusty/release/di-initrd BOOTIF=01-b0-83-fe-e6-17-cc nomdmonddf nomdmonisw crashkernel=384M-:128M
ProcVersionSignature: Ubuntu 4.2.0-35.40~14.04.1-generic 4.2.8-ckt5
RelatedPackageVersions:
linux-restricted-modules-4.2.0-35-generic N/A
linux-backports-modules-4.2.0-35-generic N/A
linux-firmware 1.127.22
RfKill: Error: [Errno 2] No such file or directory
Tags: trusty uec-images
Uname: Linux 4.2.0-35-generic x86_64
UpgradeStatus: No upgrade log present (probably fresh install)
UserGroups:
_MarkForUpload: True
dmi.bios.date: 07/09/2014
dmi.bios.vendor: Dell Inc.
dmi.bios.version: 2.4.3
dmi.board.name: 0HJK12
dmi.board.vendor: Dell Inc.
dmi.board.version: A03
dmi.chassis.asset.tag: 707
dmi.chassis.type: 23
dmi.chassis.vendor: Dell Inc.
dmi.modalias: dmi:bvnDellInc.:bvr2.4.3:bd07/09/2014:svnDellInc.:pnPowerEdgeR720xd:pvr:rvnDellInc.:rn0HJK12:rvrA03:cvnDellInc.:ct23:cvr:
dmi.product.name: PowerEdge R720xd
dmi.sys.vendor: Dell Inc.
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1576599/+subscriptions
References
