kernel-packages team mailing list archive

Thread
Date
[Bug 1564950] Re: Kernel panic when using CEPHFS (Hammer) /w kernel vivid-lts on Ubuntu 14.04

To: kernel-packages@xxxxxxxxxxxxxxxxxxx
From: Kamal Mostafa <kamal@xxxxxxxxxxxxx>
Date: Fri, 01 Apr 2016 21:21:53 -0000
Reply-to: Bug 1564950 <1564950@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
On Fri, Apr 1, 2016 at 2:20 PM,  <redmine@xxxxxxxxxxxxxxxx> wrote:
> Issue #15302 has been updated by Randy Orr.
>
> I can confirm that mounting a cephfs filesystem is successful using the
> provided test kernel.
>

-- 
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux-lts-utopic in Ubuntu.
https://bugs.launchpad.net/bugs/1564950

Title:
  Kernel panic when using CEPHFS (Hammer) /w kernel vivid-lts on Ubuntu
  14.04

Status in linux package in Ubuntu:
  In Progress
Status in linux-lts-utopic package in Ubuntu:
  New
Status in linux-lts-vivid package in Ubuntu:
  New
Status in linux-lts-utopic source package in Trusty:
  New
Status in linux-lts-vivid source package in Trusty:
  New
Status in linux source package in Vivid:
  New

Bug description:
  Reported by Dennis Kramer at http://tracker.ceph.com/issues/15302

  ----------

  Nowadays i'm getting a kernel panic when i use the vivid-lts (3.19.x)
  kernel on ubuntu 14.04 after I try to mount cephfs. Log indicates:

  kernel BUG at /build/linux-lts-vivid-ZTSmDy/linux-lts-vivid-3.19.0/fs/ceph/mds_client.c:1928[ 27.305173] kernel BUG at /build/linux-lts-vivid-ZTSmDy/linux-lts-vivid-3.19.0/fs/ceph/mds_client.c:1928!
  [ 27.307226] invalid opcode: 0000 [#1] SMP 
  [ 27.307599] Modules linked in: ceph libceph libcrc32c fscache ib_iser rdma_cm iw_cm ib_cm ib_sa ib_mad ib_core ib_addr iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi dm_crypt ppdev parport_pc 8250_fintek pvpanic serio_raw joydev parport i2c_piix4 mac_hid hid_generic usbhid hid syscopyarea sysfillrect sysimgblt ttm drm_kms_helper psmouse drm pata_acpi floppy
  [ 27.307599] CPU: 0 PID: 118 Comm: kworker/0:2 Not tainted 3.19.0-56-generic #62~14.04.1-Ubuntu
  [ 27.307599] Hardware name: OpenStack Foundation OpenStack Nova, BIOS 1.7.5-20150310_111955-batsu 04/01/2014
  [ 27.307599] Workqueue: ceph-msgr con_work [libceph]
  [ 27.307599] task: ffff88007ca0ce80 ti: ffff88003657c000 task.ti: ffff88003657c000
  [ 27.307599] RIP: 0010:[<ffffffffc02e3ed0>] [<ffffffffc02e3ed0>] __prepare_send_request+0x7d0/0x800 [ceph]
  [ 27.307599] RSP: 0018:ffff88003657fb48 EFLAGS: 00010283
  [ 27.307599] RAX: ffff880077e1bb02 RBX: ffff8800364af000 RCX: 0000000000000000
  [ 27.307599] RDX: 0000000038a516d3 RSI: 0000000000000000 RDI: ffff880077e1baf2
  [ 27.307599] RBP: ffff88003657fbe8 R08: 0000000000000000 R09: 0000000000000000
  [ 27.307599] R10: ffffffffc0275fe6 R11: 0000000000000000 R12: ffff88007873c700
  [ 27.307599] R13: ffff880078458000 R14: 0000000000000000 R15: ffff880077e1ba80
  [ 27.307599] FS: 0000000000000000(0000) GS:ffff88007fc00000(0000) knlGS:0000000000000000
  [ 27.307599] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
  [ 27.307599] CR2: 00007ff6e2973000 CR3: 0000000036387000 CR4: 00000000000006f0
  [ 27.307599] Stack:
  [ 27.307599] ffff88003657fb8c ffffffffc02e1109 0100000000000015 0000000000000001
  [ 27.307599] ffff8800362e0118 ffff880077e1bafa ffff88003657fbe8 0000000000000000
  [ 27.307599] 0000000000000000 ffff8800362e0118 0000000000000000 0000000000000001
  [ 27.307599] Call Trace:
  [ 27.307599] [<ffffffffc02e1109>] ? __choose_mds+0x119/0x470 [ceph]
  [ 27.307599] [<ffffffffc02e4130>] __do_request+0x230/0x310 [ceph]
  [ 27.307599] [<ffffffffc02e4288>] __wake_requests+0x78/0xb0 [ceph]
  [ 27.307599] [<ffffffffc02e7430>] dispatch+0x550/0xad0 [ceph]
  [ 27.307599] [<ffffffffc027abbb>] try_read+0x4cb/0x10f0 [libceph]
  [ 27.307599] [<ffffffff810ab41e>] ? dequeue_task_fair+0x44e/0x660
  [ 27.307599] [<ffffffff810ac171>] ? put_prev_entity+0x31/0x3f0
  [ 27.307599] [<ffffffff810a41c8>] ? sched_clock_cpu+0x98/0xc0
  [ 27.307599] [<ffffffffc027b899>] con_work+0xb9/0x620 [libceph]
  [ 27.307599] [<ffffffff8108dc8f>] process_one_work+0x14f/0x400
  [ 27.307599] [<ffffffff8108e428>] worker_thread+0x118/0x510
  [ 27.307599] [<ffffffff8108e310>] ? rescuer_thread+0x3d0/0x3d0
  [ 27.307599] [<ffffffff81093902>] kthread+0xd2/0xf0
  [ 27.307599] [<ffffffff81093830>] ? kthread_create_on_node+0x1c0/0x1c0
  [ 27.307599] [<ffffffff817b8b98>] ret_from_fork+0x58/0x90
  [ 27.307599] [<ffffffff81093830>] ? kthread_create_on_node+0x1c0/0x1c0
  [ 27.307599] Code: e9 27 f9 ff ff 44 89 a3 70 02 00 00 48 89 de 4c 89 ef 44 89 65 88 e8 a0 c3 ff ff 8b 45 88 e9 09 f9 ff ff 4c 63 e0 e9 d9 f9 ff ff <0f> 0b 49 8b 8f 98 fc ff ff 4d 8b 87 a0 fc ff ff 4c 89 fa 48 c7 
  [ 27.307599] RIP [<ffffffffc02e3ed0>] __prepare_send_request+0x7d0/0x800 [ceph]
  [ 27.307599] RSP <ffff88003657fb48>
  [ 27.379039] ---[ end trace 1928182fad693b4d ]---
  [ 27.379983] BUG: unable to handle kernel paging request at ffffffffffffffd8
  [ 27.381275] IP: [<ffffffff81094000>] kthread_data+0x10/0x20
  [ 27.382382] PGD 1c19067 PUD 1c1b067 PMD 0 
  [ 27.383004] Oops: 0000 [#2] SMP 
  [ 27.383004] Modules linked in: ceph libceph libcrc32c fscache ib_iser rdma_cm iw_cm ib_cm ib_sa ib_mad ib_core ib_addr iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi dm_crypt ppdev parport_pc 8250_fintek pvpanic serio_raw joydev parport i2c_piix4 mac_hid hid_generic usbhid hid syscopyarea sysfillrect sysimgblt ttm drm_kms_helper psmouse drm pata_acpi floppy
  [ 27.383004] CPU: 0 PID: 118 Comm: kworker/0:2 Tainted: G D 3.19.0-56-generic #62~14.04.1-Ubuntu
  [ 27.383004] Hardware name: OpenStack Foundation OpenStack Nova, BIOS 1.7.5-20150310_111955-batsu 04/01/2014
  [ 27.383004] task: ffff88007ca0ce80 ti: ffff88003657c000 task.ti: ffff88003657c000
  [ 27.383004] RIP: 0010:[<ffffffff81094000>] [<ffffffff81094000>] kthread_data+0x10/0x20
  [ 27.383004] RSP: 0018:ffff88003657f7c8 EFLAGS: 00010096
  [ 27.383004] RAX: 0000000000000000 RBX: 0000000000000000 RCX: 000000000000000d
  [ 27.383004] RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffff88007ca0ce80
  [ 27.383004] RBP: ffff88003657f7c8 R08: 0000000000000000 R09: 0000000000000246
  [ 27.383004] R10: ffffffff810721a4 R11: ffffea0001dfba00 R12: ffff88007ca0d3a0
  [ 27.383004] R13: 0000000000000000 R14: 0000000000000000 R15: ffff88007ca0ce80
  [ 27.383004] FS: 0000000000000000(0000) GS:ffff88007fc00000(0000) knlGS:0000000000000000
  [ 27.383004] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
  [ 27.383004] CR2: 0000000000000028 CR3: 0000000036387000 CR4: 00000000000006f0
  [ 27.383004] Stack:
  [ 27.383004] ffff88003657f7e8 ffffffff8108ed05 ffff88003657f7e8 ffff88007fc13e80
  [ 27.383004] ffff88003657f858 ffffffff817b45fb ffff88007ca0ce80 0000000000013e80
  [ 27.383004] ffff88003657ffd8 0000000000013e80 0000000000000004 ffff88007ca0ce80
  [ 27.383004] Call Trace:
  [ 27.383004] [<ffffffff8108ed05>] wq_worker_sleeping+0x15/0xa0
  [ 27.383004] [<ffffffff817b45fb>] __schedule+0x5bb/0x820
  [ 27.383004] [<ffffffff817b4889>] schedule+0x29/0x70
  [ 27.383004] [<ffffffff810777ff>] do_exit+0x69f/0xb00
  [ 27.383004] [<ffffffff810187f8>] oops_end+0xa8/0x120
  [ 27.383004] [<ffffffff81018deb>] die+0x4b/0x70
  [ 27.383004] [<ffffffff810153a0>] do_trap+0xb0/0x150
  [ 27.383004] [<ffffffff81015a37>] do_error_trap+0x97/0x150
  [ 27.383004] [<ffffffffc02e3ed0>] ? __prepare_send_request+0x7d0/0x800 [ceph]
  [ 27.383004] [<ffffffff810160c0>] do_invalid_op+0x20/0x30
  [ 27.383004] [<ffffffff817ba7be>] invalid_op+0x1e/0x30
  [ 27.383004] [<ffffffffc0275fe6>] ? ceph_kvmalloc+0x26/0x50 [libceph]
  [ 27.383004] [<ffffffffc02e3ed0>] ? __prepare_send_request+0x7d0/0x800 [ceph]
  [ 27.383004] [<ffffffffc02e3a44>] ? __prepare_send_request+0x344/0x800 [ceph]
  [ 27.383004] [<ffffffffc02e1109>] ? __choose_mds+0x119/0x470 [ceph]
  [ 27.383004] [<ffffffffc02e4130>] __do_request+0x230/0x310 [ceph]
  [ 27.383004] [<ffffffffc02e4288>] __wake_requests+0x78/0xb0 [ceph]
  [ 27.383004] [<ffffffffc02e7430>] dispatch+0x550/0xad0 [ceph]
  [ 27.383004] [<ffffffffc027abbb>] try_read+0x4cb/0x10f0 [libceph]
  [ 27.383004] [<ffffffff810ab41e>] ? dequeue_task_fair+0x44e/0x660
  [ 27.383004] [<ffffffff810ac171>] ? put_prev_entity+0x31/0x3f0
  [ 27.383004] [<ffffffff810a41c8>] ? sched_clock_cpu+0x98/0xc0
  [ 27.383004] [<ffffffffc027b899>] con_work+0xb9/0x620 [libceph]
  [ 27.383004] [<ffffffff8108dc8f>] process_one_work+0x14f/0x400
  [ 27.383004] [<ffffffff8108e428>] worker_thread+0x118/0x510
  [ 27.383004] [<ffffffff8108e310>] ? rescuer_thread+0x3d0/0x3d0
  [ 27.383004] [<ffffffff81093902>] kthread+0xd2/0xf0
  [ 27.383004] [<ffffffff81093830>] ? kthread_create_on_node+0x1c0/0x1c0
  [ 27.383004] [<ffffffff817b8b98>] ret_from_fork+0x58/0x90
  [ 27.383004] [<ffffffff81093830>] ? kthread_create_on_node+0x1c0/0x1c0
  [ 27.383004] Code: 00 48 89 e5 5d 48 8b 40 c8 48 c1 e8 02 83 e0 01 c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 48 8b 87 c8 04 00 00 55 48 89 e5 <48> 8b 40 d8 5d c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 
  [ 27.383004] RIP [<ffffffff81094000>] kthread_data+0x10/0x20
  [ 27.383004] RSP <ffff88003657f7c8>
  [ 27.383004] CR2: ffffffffffffffd8
  [ 27.383004] ---[ end trace 1928182fad693b4e ]---
  [ 27.383004] Fixing recursive fault but reboot is needed!

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1564950/+subscriptions
References

[Bug 1564950] [NEW] Kernel panic when using CEPHFS (Hammer) /w kernel vivid-lts on Ubuntu 14.04
From: Kamal Mostafa, 2016-04-01