← Back to team overview

kernel-packages team mailing list archive

[Bug 1449910] Re: System hangs apparently randomly when creating LVM snapshots

 

So, all the systems have been up and running for 35 nights as of now. The upstream kernel is doing good. What can we do from here?
Thank you for your help.

-- 
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1449910

Title:
  System hangs apparently randomly when creating LVM snapshots

Status in linux package in Ubuntu:
  Confirmed

Bug description:
  We have several servers here which mount Ubuntu LTS Trusty Tahr 14.04. We use LVM snapshots to do backups during the night. Randomly (witha rate of 1 event every 10-12 days) the LVM snapshot creation runs in some problem. The server hangs and it must be rebooted hardly. No shell, no even any screen output, just a black screen, no ping, nothing. And Magic SysRq key doesn't help.
  No logs are written either (I also tried redirect logs to another machine, just in case). This happens with different hardware, and different backup software. The only constant is LVM snapshots. Kernel is 3.13.0-49. We tried the 14.10 kernel (3.16.0-34), we had the same hangs, but this time we had something logged:

  Apr 21 23:02:56 server-name kernel: [654840.108023] INFO: task kswapd0:50 blocked for more than 120 seconds.
  Apr 21 23:02:56 server-name kernel: [654840.108145]       Not tainted 3.16.0-34-generic #47~14.04.1-Ubuntu
  Apr 21 23:02:56 server-name kernel: [654840.108245] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
  Apr 21 23:02:56 server-name kernel: [654840.108361] kswapd0         D ffff88007fc130c0     0    50      2 0x00000000
  Apr 21 23:02:56 server-name kernel: [654840.108367]  ffff880077bcf998 0000000000000046 ffff880077bd0000 ffff880077bcffd8
  Apr 21 23:02:56 server-name kernel: [654840.108372]  00000000000130c0 00000000000130c0 ffff88007bdef010 ffffc90010d3c000
  Apr 21 23:02:56 server-name kernel: [654840.108377]  ffffc90010d3c0d8 0000000000000000 0000000000000000 ffffc90010d3c000
  Apr 21 23:02:56 server-name kernel: [654840.108382] Call Trace:
  /0x70
  Apr 21 23:02:56 server-name kernel: [654840.108419]  [<ffffffffc0707aad>] reiserfs_wait_on_write_block+0x4d/0x80 [reiserfs]
  Apr 21 23:02:56 server-name kernel: [654840.108426]  [<ffffffff810b4d70>] ? prepare_to_wait_event+0x100/0x100
  Apr 21 23:02:56 server-name kernel: [654840.108437]  [<ffffffffc0709271>] do_journal_begin_r+0xe1/0x3e0 [reiserfs]
  Apr 21 23:02:56 server-name kernel: [654840.108443]  [<ffffffff810a5fa8>] ? __enqueue_entity+0x78/0x80
  Apr 21 23:02:56 server-name kernel: [654840.108454]  [<ffffffffc070967a>] journal_begin+0x8a/0x160 [reiserfs]
  Apr 21 23:02:56 server-name kernel: [654840.108464]  [<ffffffffc06f67fc>] reiserfs_release_dquot+0x4c/0xd0 [reiserfs]
  Apr 21 23:02:56 server-name kernel: [654840.108470]  [<ffffffff813ab4f1>] ? __percpu_counter_add+0x51/0x70
  Apr 21 23:02:56 server-name kernel: [654840.108476]  [<ffffffff8123003d>] dqput+0x9d/0x200
  Apr 21 23:02:56 server-name kernel: [654840.108480]  [<ffffffff8123166d>] __dquot_drop+0x5d/0x70
  Apr 21 23:02:56 server-name kernel: [654840.108485]  [<ffffffff812316ad>] dquot_drop+0x2d/0x40
  Apr 21 23:02:56 server-name kernel: [654840.108494]  [<ffffffffc06ec7a0>] reiserfs_evic
  t_inode+0xa0/0x180 [reiserfs]
  Apr 21 23:02:56 server-name kernel: [654840.108501]  [<ffffffff811ff7de>] ? inode_wait_for_writeback+0x2e/0x40
  Apr 21 23:02:56 server-name kernel: [654840.108507]  [<ffffffff811eef34>] evict+0xb4/0x180
  Apr 21 23:02:56 server-name kernel: [654840.108511]  [<ffffffff811ef039>] dispose_list+0x39/0x50
  Apr 21 23:02:56 server-name kernel: [654840.108515]  [<ffffffff811eff47>] prune_icache_sb+0x47/0x60
  Apr 21 23:02:56 server-name kernel: [654840.108520]  [<ffffffff811d78f5>] super_cache_scan+0x105/0x170
  Apr 21 23:02:56 server-name kernel: [654840.108526]  [<ffffffff81171b78>] shrink_slab_node+0x138/0x290
  Apr 21 23:02:56 server-name kernel: [654840.108532]  [<ffffffff810f8ccb>] ? css_next_descendant_pre+0x3b/0x40
  Apr 21 23:02:56 server-name kernel: [654840.108536]  [<ffffffff8117367b>] shrink_slab+0x8b/0x160
  Apr 21 23:02:56 server-name kernel: [654840.108540]  [<ffffffff811770e2>] balance_pgdat+0x3f2/0x620
  Apr 21 23:02:56 server-name kernel: [654840.108544]  [<ffffffff8117746b>] kswapd+0x15b/0x3f0
  Apr 21 23:02:56 server-name kernel: [654840.108549]  [<ffffffff810b4d70>] ? prepare_to_wait_event+0x100/0x100
  Apr 21 23:02:56 server-name kernel: [654840.108552]  [<ffffffff81177310>] ? balance_pgdat+0x620/0x620
  Apr 21 23:02:56 server-name kernel: [654840.108558]  [<ffffffff81091332>] kthread+0xd2/0xf0
  Apr 21 23:02:56 server-name kernel: [654840.108562]  [<ffffffff81091260>] ? kthread_create_on_node+0x1c0/0x1c0
  Apr 21 23:02:56 server-name kernel: [654840.108567]  [<ffffffff8176c9bc>] ret_from_fork+0x7c/0xb0
  Apr 21 23:02:56 server-name kernel: [654840.108571]  [<ffffffff81091260>] ? kthread_create_on_node+0x1c0/0x1c0
  Apr 21 23:02:56 server-name kernel: [654840.108595] INFO: task kworker/1:1:25606 blocked for more than 120 seconds.
  Apr 21 23:02:56 server-name kernel: [654840.108709]       Not tainted 3.16.0-34-generic #47~14.04.1-Ubuntu
  Apr 21 23:02:56 server-name kernel: [654840.108809] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
  Apr 21 23:02:56 server-name kernel: [654840.108923] kworker/1:1     D ffff88007fc530c0     0 25606      2 0x00000000
  Apr 21 23:02:56 server-name kernel: [654840.108936] Workqueue: events_long flush_old_commits [reiserfs]
  Apr 21 23:02:56 server-name kernel: [654840.108939]  ffff88000311fc90 0000000000000046 ffff880079db65e0 ffff88000311ffd8
  Apr 21 23:02:56 server-name kernel: [654840.108943]  00000000000130c0 00000000000130c0 ffff880036b065e0 ffffc90010d3c000
  Apr 21 23:02:56 server-name kernel: [654840.108948]  ffffc90010d3c0d8 0000000000000000 0000000000000000 ffffc90010d3c000
  Apr 21 23:02:56 server-name kernel: [654840.108952] Call Trace:
  Apr 21 23:02:56 server-name kernel: [654840.108956]  [<ffffffff817689d9>] schedule+0x29/0x70
  Apr 21 23:02:56 server-name kernel: [654840.108967]  [<ffffffffc0707aad>] reiserfs_wait_on_write_block+0x4d/0x80 [reiserfs]
  Apr 21 23:02:56 server-name kernel: [654840.108972]  [<ffffffff810b4d70>] ? prepare_to_wait_event+0x100/0x100
  Apr 21 23:02:56 server-name kernel: [654840.108983]  [<ffffffffc0709271>] do_journal_begin_r+0xe1/0x3e0 [reiserfs]
  Apr 21 23:02:56 server-name kernel: [654840.108989]  [<ffffffff8101c355>] ? native_sched_clock+0x35/0x90
  Apr 21 23:02:56 server-name kernel: [654840.109000]  [<ffffffffc070967a>] journal_begin+0x8a/0x160 [reiserfs]
  Apr 21 23:02:56 server-name kernel: [654840.109010]  [<ffffffffc06f5a44>] reiserfs_sync_fs+0x34/0x70 [reiserfs]
  Apr 21 23:02:56 server-name kernel: [654840.109019]  [<ffffffffc06f5aca>] flush_old_commits+0x4a/0x60 [reiserfs]
  Apr 21 23:02:56 server-name kernel: [654840.109024]  [<ffffffff8108a322>] process_one_work+0x182/0x450
  Apr 21 23:02:56 server-name kernel: [654840.109028]  [<ffffffff8108aa91>] worker_thread+0x121/0x570
  Apr 21 23:02:56 server-name kernel: [654840.109032]  [<ffffffff8108a970>] ? rescuer_thread+0x380/0x380
  Apr 21 23:02:56 server-name kernel: [654840.109036]  [<ffffffff81091332>] kthread+0xd2/0xf0
  Apr 21 23:02:56 server-name kernel: [654840.109040]  [<ffffffff81091260>] ? kthread_create_on_node+0x1c0/0x1c0
  Apr 21 23:02:56 server-name kernel: [654840.109044]  [<ffffffff8176c9bc>] ret_from_fork+0x7c/0xb0
  Apr 21 23:02:56 server-name kernel: [654840.109048]  [<ffffffff81091260>] ? kthread_create_on_node+0x1c0/0x1c0
  Apr 21 23:02:56 server-name kernel: [654840.109053] INFO: task lvcreate:25648 blocked for more than 120 seconds.
  Apr 21 23:02:56 server-name kernel: [654840.109159]       Not tainted 3.16.0-34-generic #47~14.04.1-Ubuntu
  Apr 21 23:02:56 server-name kernel: [654840.109258] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
  Apr 21 23:02:56 server-name kernel: [654840.109372] lvcreate        D ffff88007fc530c0     0 25648  25623 0x00000000
  Apr 21 23:02:56 server-name kernel: [654840.109377]  ffff880000053b60 0000000000000086 ffff8800773432f0 ffff880000053fd8
  Apr 21 23:02:56 server-name kernel: [654840.109381]  00000000000130c0 00000000000130c0 ffff88007c0c9460 ffff8800773432f0
  Apr 21 23:02:56 server-name kernel: [654840.109385]  ffff880037a23080 ffff880037a23068 ffffffff00000000 ffff880037a23070
  Apr 21 23:02:56 server-name kernel: [654840.109390] Call Trace:
  Apr 21 23:02:56 server-name kernel: [654840.109394]  [<ffffffff817689d9>] schedule+0x29/0x70
  Apr 21 23:02:56 server-name kernel: [654840.109398]  [<ffffffff8176b77d>] rwsem_down_write_failed+0x1ed/0x390
  Apr 21 23:02:56 server-name kernel: [654840.109404]  [<ffffffff81011627>] ? __switch_to+0x167/0x590
  Apr 21 23:02:56 server-name kernel: [654840.109409]  [<ffffffff81394013>] call_rwsem_down_write_failed+0x13/0x20
  Apr 21 23:02:56 server-name kernel: [654840.109416]  [<ffffffff815e6c05>] ? dm_free_md_mempools+0x35/0x40
  Apr 21 23:02:56 server-name kernel: [654840.109419]  [<ffffffff8176b0ad>] ? down_write+0x2d/0x40
  Apr 21 23:02:56 server-name kernel: [654840.109423]  [<ffffffff811d6a6d>] thaw_super+0x1d/0xb0
  Apr 21 23:02:56 server-name kernel: [654840.109428]  [<ffffffff81209935>] thaw_bdev+0x65/0x80
  Apr 21 23:02:56 server-name kernel: [654840.109432]  [<ffffffff815e4a80>] unlock_fs.part.23+0x20/0x40
  Apr 21 23:02:56 server-name kernel: [654840.109436]  [<ffffffff815e6898>] dm_resume+0xb8/0xd0
  Apr 21 23:02:56 server-name kernel: [654840.109441]  [<ffffffff815eba5b>] dev_suspend+0x12b/0x220
  Apr 21 23:02:56 server-name kernel: [654840.109445]  [<ffffffff815eb930>] ? table_load+0x350/0x350
  Apr 21 23:02:56 server-name kernel: [654840.109449]  [<ffffffff815ec2f5>] ctl_ioctl+0x255/0x500
  Apr 21 23:02:56 server-name kernel: [654840.109454]  [<ffffffff812d75ef>] ? SYSC_semtimedop+0x23f/0xcf0
  Apr 21 23:02:56 server-name kernel: [654840.109459]  [<ffffffff815ec5b3>] dm_ctl_ioctl+0x13/0x20
  Apr 21 23:02:56 server-name kernel: [654840.109463]  [<ffffffff811e7090>] do_vfs_ioctl+0x2e0/0x4c0
  Apr 21 23:02:56 server-name kernel: [654840.109468]  [<ffffffff812ebff6>] ? security_sem_associate+0x16/0x20
  Apr 21 23:02:56 server-name kernel: [654840.109472]  [<ffffffff812d65a9>] ? sem_security+0x9/0x10
  Apr 21 23:02:56 server-name kernel: [654840.109476]  [<ffffffff812d4a09>] ? ipcget+0x149/0x1c0
  Apr 21 23:02:56 server-name kernel: [654840.109480]  [<ffffffff811e72f1>] SyS_ioctl+0x81/0xa0
  Apr 21 23:02:56 server-name kernel: [654840.109484]  [<ffffffff8176ca6d>] system_call_fastpath+0x1a/0x1f
  Apr 21 23:04:56 server-name kernel: [654960.108025] INFO: task kswapd0:50 blocked for more than 120 seconds.
  Apr 21 23:04:56 server-name kernel: [654960.108124]       Not tainted 3.16.0-34-generic #47~14.04.1-Ubuntu
  Apr 21 23:04:56 server-name kernel: [654960.108224] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
  Apr 21 23:04:56 server-name kernel: [654960.108339] kswapd0         D ffff88007fc130c0     0    50      2 0x00000000
  Apr 21 23:04:56 server-name kernel: [654960.108346]  ffff880077bcf998 0000000000000046 ffff880077bd0000 ffff880077bcffd8
  Apr 21 23:04:56 server-name kernel: [654960.108351]  00000000000130c0 00000000000130c0 ffff88007bdef010 ffffc90010d3c000
  Apr 21 23:04:56 server-name kernel: [654960.108355]  ffffc90010d3c0d8 0000000000000000 0000000000000000 ffffc90010d3c000
  Apr 21 23:04:56 server-name kernel: [654960.108360] Call Trace:
  Apr 21 23:04:56 server-name kernel: [654960.108372]  [<ffffffff817689d9>] schedule+0x29/0x70
  Apr 21 23:04:56 server-name kernel: [654960.108397]  [<ffffffffc0707aad>] reiserfs_wait_on_write_block+0x4d/0x80 [reiserfs]
  Apr 21 23:04:56 server-name kernel: [654960.108404]  [<ffffffff810b4d70>] ? prepare_to_wait_event+0x100/0x100
  Apr 21 23:04:56 server-name kernel: [654960.108416]  [<ffffffffc0709271>] do_journal_begin_r+0xe1/0x3e0 [reiserfs]
  Apr 21 23:04:56 server-name kernel: [654960.108422]  [<ffffffff810a5fa8>] ? __enqueue_entity+0x78/0x80
  Apr 21 23:04:56 server-name kernel: [654960.108433]  [<ffffffffc070967a>] journal_begin+0x8a/0x160 [reiserfs]
  Apr 21 23:04:56 server-name kernel: [654960.108443]  [<ffffffffc06f67fc>] reiserfs_release_dquot+0x4c/0xd0 [reiserfs]
  Apr 21 23:04:56 server-name kernel: [654960.108449]  [<ffffffff813ab4f1>] ? __percpu_counter_add+0x51/0x70
  Apr 21 23:04:56 server-name kernel: [654960.108455]  [<ffffffff8123003d>] dqput+0x9d/0x200
  Apr 21 23:04:56 server-name kernel: [654960.108459]  [<ffffffff8123166d>] __dquot_drop+0x5d/0x70
  Apr 21 23:04:56 server-name kernel: [654960.108464]  [<ffffffff812316ad>] dquot_drop+0x2d/0x40
  Apr 21 23:04:56 server-name kernel: [654960.108473]  [<ffffffffc06ec7a0>] reiserfs_evict_inode+0xa0/0x180 [reiserfs]
  Apr 21 23:04:56 server-name kernel: [654960.108480]  [<ffffffff811ff7de>] ? inode_wait_for_writeback+0x2e/0x40
  Apr 21 23:04:56 server-name kernel: [654960.108485]  [<ffffffff811eef34>] evict+0xb4/0x180
  Apr 21 23:04:56 server-name kernel: [654960.108490]  [<ffffffff811ef039>] dispose_list+0x39/0x50
  Apr 21 23:04:56 server-name kernel: [654960.108494]  [<ffffffff811eff47>] prune_icache_sb+0x47/0x60
  Apr 21 23:04:56 server-name kernel: [654960.108499]  [<ffffffff811d78f5>] super_cache_scan+0x105/0x170
  Apr 21 23:04:56 server-name kernel: [654960.108504]  [<ffffffff81171b78>] shrink_slab_node+0x138/0x290
  Apr 21 23:04:56 server-name kernel: [654960.108510]  [<ffffffff810f8ccb>] ? css_next_descendant_pre+0x3b/0x40
  Apr 21 23:04:56 server-name kernel: [654960.108515]  [<ffffffff8117367b>] shrink_slab+0x8b/0x160
  Apr 21 23:04:56 server-name kernel: [654960.108519]  [<ffffffff811770e2>] balance_pgdat+0x3f2/0x620
  Apr 21 23:04:56 server-name kernel: [654960.108523]  [<ffffffff8117746b>] kswapd+0x15b/0x3f0
  Apr 21 23:04:56 server-name kernel: [654960.108528]  [<ffffffff810b4d70>] ? prepare_to_wait_event+0x100/0x100
  Apr 21 23:04:56 server-name kernel: [654960.108531]  [<ffffffff81177310>] ? balance_pgdat+0x620/0x620
  Apr 21 23:04:56 server-name kernel: [654960.108536]  [<ffffffff81091332>] kthread+0xd2/0xf0
  Apr 21 23:04:56 server-name kernel: [654960.108541]  [<ffffffff81091260>] ? kthread_create_on_node+0x1c0/0x1c0
  Apr 21 23:04:56 server-name kernel: [654960.108546]  [<ffffffff8176c9bc>] ret_from_fork+0x7c/0xb0
  Apr 21 23:04:56 server-name kernel: [654960.108550]  [<ffffffff81091260>] ? kthread_create_on_node+0x1c0/0x1c0
  Apr 21 23:04:56 server-name kernel: [654960.108573] INFO: task kworker/1:1:25606 blocked for more than 120 seconds.
  Apr 21 23:04:56 server-name kernel: [654960.108688]       Not tainted 3.16.0-34-generic #47~14.04.1-Ubuntu
  Apr 21 23:04:56 server-name kernel: [654960.108788] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
  Apr 21 23:04:56 server-name kernel: [654960.108905] kworker/1:1     D ffff88007fc530c0     0 25606      2 0x00000000
  Apr 21 23:04:56 server-name kernel: [654960.108918] Workqueue: events_long flush_old_commits [reiserfs]
  Apr 21 23:04:56 server-name kernel: [654960.108920]  ffff88000311fc90 0000000000000046 ffff880079db65e0 ffff88000311ffd8
  Apr 21 23:04:56 server-name kernel: [654960.108925]  00000000000130c0 00000000000130c0 ffff880036b065e0 ffffc90010d3c000
  Apr 21 23:04:56 server-name kernel: [654960.108929]  ffffc90010d3c0d8 0000000000000000 0000000000000000 ffffc90010d3c000
  Apr 21 23:04:56 server-name kernel: [654960.108934] Call Trace:
  Apr 21 23:04:56 server-name kernel: [654960.108938]  [<ffffffff817689d9>] schedule+0x29/0x70
  Apr 21 23:04:56 server-name kernel: [654960.108949]  [<ffffffffc0707aad>] reiserfs_wait_on_write_block+0x4d/0x80 [reiserfs]
  Apr 21 23:04:56 server-name kernel: [654960.108953]  [<ffffffff810b4d70>] ? prepare_to_wait_event+0x100/0x100
  Apr 21 23:04:56 server-name kernel: [654960.108964]  [<ffffffffc0709271>] do_journal_begin_r+0xe1/0x3e0 [reiserfs]
  Apr 21 23:04:56 server-name kernel: [654960.108971]  [<ffffffff8101c355>] ? native_sched_clock+0x35/0x90
  Apr 21 23:04:56 server-name kernel: [654960.108982]  [<ffffffffc070967a>] journal_begin+0x8a/0x160 [reiserfs]
  Apr 21 23:04:56 server-name kernel: [654960.108991]  [<ffffffffc06f5a44>] reiserfs_sync_fs+0x34/0x70 [reiserfs]
  Apr 21 23:04:56 server-name kernel: [654960.109001]  [<ffffffffc06f5aca>] flush_old_commits+0x4a/0x60 [reiserfs]
  Apr 21 23:04:56 server-name kernel: [654960.109006]  [<ffffffff8108a322>] process_one_work+0x182/0x450
  Apr 21 23:04:56 server-name kernel: [654960.109010]  [<ffffffff8108aa91>] worker_thread+0x121/0x570
  Apr 21 23:04:56 server-name kernel: [654960.109014]  [<ffffffff8108a970>] ? rescuer_thread+0x380/0x380
  Apr 21 23:04:56 server-name kernel: [654960.109018]  [<ffffffff81091332>] kthread+0xd2/0xf0
  Apr 21 23:04:56 server-name kernel: [654960.109022]  [<ffffffff81091260>] ? kthread_create_on_node+0x1c0/0x1c0
  Apr 21 23:04:56 server-name kernel: [654960.109026]  [<ffffffff8176c9bc>] ret_from_fork+0x7c/0xb0
  Apr 21 23:04:56 server-name kernel: [654960.109030]  [<ffffffff81091260>] ? kthread_create_on_node+0x1c0/0x1c0
  Apr 21 23:04:56 server-name kernel: [654960.109035] INFO: task lvcreate:25648 blocked for more than 120 seconds.
  Apr 21 23:04:56 server-name kernel: [654960.109141]       Not tainted 3.16.0-34-generic #47~14.04.1-Ubuntu
  Apr 21 23:04:56 server-name kernel: [654960.109240] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
  Apr 21 23:04:56 server-name kernel: [654960.109354] lvcreate        D ffff88007fc530c0     0 25648  25623 0x00000000
  Apr 21 23:04:56 server-name kernel: [654960.109359]  ffff880000053b60 0000000000000086 ffff8800773432f0 ffff880000053fd8
  Apr 21 23:04:56 server-name kernel: [654960.109363]  00000000000130c0 00000000000130c0 ffff88007c0c9460 ffff8800773432f0
  Apr 21 23:04:56 server-name kernel: [654960.109367]  ffff880037a23080 ffff880037a23068 ffffffff00000000 ffff880037a23070
  Apr 21 23:04:56 server-name kernel: [654960.109372] Call Trace:
  Apr 21 23:04:56 server-name kernel: [654960.109376]  [<ffffffff817689d9>] schedule+0x29/0x70
  Apr 21 23:04:56 server-name kernel: [654960.109380]  [<ffffffff8176b77d>] rwsem_down_write_failed+0x1ed/0x390
  Apr 21 23:04:56 server-name kernel: [654960.109386]  [<ffffffff81011627>] ? __switch_to+0x167/0x590
  Apr 21 23:04:56 server-name kernel: [654960.109391]  [<ffffffff81394013>] call_rwsem_down_write_failed+0x13/0x20
  Apr 21 23:04:56 server-name kernel: [654960.109398]  [<ffffffff815e6c05>] ? dm_free_md_mempools+0x35/0x40
  Apr 21 23:04:56 server-name kernel: [654960.109401]  [<ffffffff8176b0ad>] ? down_write+0x2d/0x40
  Apr 21 23:04:56 server-name kernel: [654960.109405]  [<ffffffff811d6a6d>] thaw_super+0x1d/0xb0
  Apr 21 23:04:56 server-name kernel: [654960.109410]  [<ffffffff81209935>] thaw_bdev+0x65/0x80
  Apr 21 23:04:56 server-name kernel: [654960.109414]  [<ffffffff815e4a80>] unlock_fs.part.23+0x20/0x40
  Apr 21 23:04:56 server-name kernel: [654960.109419]  [<ffffffff815e6898>] dm_resume+0xb8/0xd0
  Apr 21 23:04:56 server-name kernel: [654960.109424]  [<ffffffff815eba5b>] dev_suspend+0x12b/0x220
  Apr 21 23:04:56 server-name kernel: [654960.109428]  [<ffffffff815eb930>] ? table_load+0x350/0x350
  Apr 21 23:04:56 server-name kernel: [654960.109431]  [<ffffffff815ec2f5>] ctl_ioctl+0x255/0x500
  Apr 21 23:04:56 server-name kernel: [654960.109437]  [<ffffffff812d75ef>] ? SYSC_semtimedop+0x23f/0xcf0
  Apr 21 23:04:56 server-name kernel: [654960.109441]  [<ffffffff815ec5b3>] dm_ctl_ioctl+0x13/0x20
  Apr 21 23:04:56 server-name kernel: [654960.109446]  [<ffffffff811e7090>] do_vfs_ioctl+0x2e0/0x4c0
  Apr 21 23:04:56 server-name kernel: [654960.109451]  [<ffffffff812ebff6>] ? security_sem_associate+0x16/0x20
  Apr 21 23:04:56 server-name kernel: [654960.109455]  [<ffffffff812d65a9>] ? sem_security+0x9/0x10
  Apr 21 23:04:56 server-name kernel: [654960.109459]  [<ffffffff812d4a09>] ? ipcget+0x149/0x1c0
  Apr 21 23:04:56 server-name kernel: [654960.109462]  [<ffffffff811e72f1>] SyS_ioctl+0x81/0xa0
  Apr 21 23:04:56 server-name kernel: [654960.109467]  [<ffffffff8176ca6d>] system_call_fastpath+0x1a/0x1f
  Apr 21 23:06:10 server-name nmbd[962]: [2015/04/21 23:06:10.798735,  0] ../source3/nmbd/nmbd_packets.c:759(queue_query_name)
  Apr 21 23:06:10 server-name nmbd[962]:   queue_query_name: interface 0 has NULL IP address !
  Apr 21 23:06:56 server-name kernel: [655080.108025] INFO: task kswapd0:50 blocked for more than 120 seconds.
  Apr 21 23:06:56 server-name kernel: [655080.108123]       Not tainted 3.16.0-34-generic #47~14.04.1-Ubuntu
  Apr 21 23:06:56 server-name kernel: [655080.108223] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
  Apr 21 23:06:56 server-name kernel: [655080.108347] kswapd0         D ffff88007fc130c0     0    50      2 0x00000000
  Apr 21 23:06:56 server-name kernel: [655080.108354]  ffff880077bcf998 0000000000000046 ffff880077bd0000 ffff880077bcffd8
  Apr 21 23:06:56 server-name kernel: [655080.108359]  00000000000130c0 00000000000130c0 ffff88007bdef010 ffffc90010d3c000
  Apr 21 23:06:56 server-name kernel: [655080.108363]  ffffc90010d3c0d8 0000000000000000 0000000000000000 ffffc90010d3c000
  Apr 21 23:06:56 server-name kernel: [655080.108369] Call Trace:
  Apr 21 23:06:56 server-name kernel: [655080.108379]  [<ffffffff817689d9>] schedule+0x29/0x70
  _on_write_block+0x4d/0x80 [reiserfs]
  Apr 21 23:06:56 server-name kernel: [655080.108413]  [<ffffffff810b4d70>] ? prepare_to_wait_event+0x100/0x100
  Apr 21 23:06:56 server-name kernel: [655080.108424]  [<ffffffffc0709271>] do_journal_begin_r+0xe1/0x3e0 [reiserfs]
  Apr 21 23:06:56 server-name kernel: [655080.108430]  [<ffffffff810a5fa8>] ? __enqueue_entity+0x78/0x80
  Apr 21 23:06:56 server-name kernel: [655080.108441]  [<ffffffffc070967a>] journal_begin+0x8a/0x160 [reiserfs]
  Apr 21 23:06:56 server-name kernel: [655080.108451]  [<ffffffffc06f67fc>] reiserfs_release_dquot+0x4c/0xd0 [reiserfs]
  Apr 21 23:06:56 server-name kernel: [655080.108457]  [<ffffffff813ab4f1>] ? __percpu_counter_add+0x51/0x70
  Apr 21 23:06:56 server-name kernel: [655080.108463]  [<ffffffff8123003d>] dqput+0x9d/0x200
  Apr 21 23:06:56 server-name kernel: [655080.108467]  [<ffffffff8123166d>] __dquot_drop+0x5d/0x70
  Apr 21 23:06:56 server-name kernel: [655080.108472]  [<ffffffff812316ad>] dquot_drop+0x2d/0x40
  Apr 21 23:06:56 server-name kernel: [655080.108481]  [<ffffffffc06ec7a0>] reiserfs_evict_inode+0xa0/0x180 [reiserfs]
  Apr 21 23:06:56 server-name kernel: [655080.108488]  [<ffffffff811ff7de>] ? inode_wait_Apr 21 23:06:56 server-name kernel: [655080.108493]  [<ffffffff811eef34>] evict+0xb4/0x180
  Apr 21 23:06:56 server-name kernel: [655080.108498]  [<ffffffff811ef039>] dispose_list+0x39/0x50
  Apr 21 23:06:56 server-name kernel: [655080.108502]  [<ffffffff811eff47>] prune_icache_sb+0x47/0x60
  Apr 21 23:06:56 server-name kernel: [655080.108507]  [<ffffffff811d78f5>] super_cache_scan+0x105/0x170
  Apr 21 23:06:56 server-name kernel: [655080.108512]  [<ffffffff81171b78>] shrink_slab_node+0x138/0x290
  Apr 21 23:06:56 server-name kernel: [655080.108519]  [<ffffffff810f8ccb>] ? css_next_descendant_pre+0x3b/0x40
  Apr 21 23:06:56 server-name kernel: [655080.108523]  [<ffffffff8117367b>] shrink_slab+0x8b/0x160
  Apr 21 23:06:56 server-name kernel: [655080.108527]  [<ffffffff811770e2>] balance_pgdat+0x3f2/0x620
  Apr 21 23:06:56 server-name kernel: [655080.108531]  [<ffffffff8117746b>] kswapd+0x15b/0x3f0
  Apr 21 23:06:56 server-name kernel: [655080.108536]  [<ffffffff810b4d70>] ? prepare_to_wait_event+0x100/0x100
  Apr 21 23:06:56 server-name kernel: [655080.108539]  [<ffffffff81177310>] ? balance_pgdat+0x620/0x620
  Apr 21 23:06:56 server-name kernel: [655080.108545]  [<ffffffff81091332>] kthread+0xd2/0xf0
  Apr 21 23:06:56 server-name kernel: [655080.108549]  [<ffffffff81091260>] ? kthread_create_on_node+0x1c0/0x1c0
  Apr 21 23:06:56 server-name kernel: [655080.108554]  [<ffffffff8176c9bc>] ret_from_fork+0x7c/0xb0
  Apr 21 23:06:56 server-name kernel: [655080.108558]  [<ffffffff81091260>] ? kthread_create_on_node+0x1c0/0x1c0
  Apr 21 23:06:56 server-name kernel: [655080.108582] INFO: task kworker/1:1:25606 blocked for more than 120 seconds.
  Apr 21 23:06:56 server-name kernel: [655080.108697]       Not tainted 3.16.0-34-generic #47~14.04.1-Ubuntu
  Apr 21 23:06:56 server-name kernel: [655080.108796] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
  Apr 21 23:06:56 server-name kernel: [655080.108910] kworker/1:1     D ffff88007fc530c0     0 25606      2 0x00000000
  Apr 21 23:06:56 server-name kernel: [655080.108923] Workqueue: events_long flush_old_commits [reiserfs]
  Apr 21 23:06:56 server-name kernel: [655080.108925]  ffff88000311fc90 0000000000000046 ffff880079db65e0 ffff88000311ffd8

  and this keeps going for some minutes, then the next morning the
  server is hung.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1449910/+subscriptions


References