kernel-packages team mailing list archive

[Rafael David Tinoco <rafael.tinoco@xxxxxxxxxxxxx>]

Attached script as an example on how to make kmem_alloc to fail (using
/var/lib/ceph as filesystem, it can be changed).

** Description changed:

+ === SRU Justification ===
+ 
+ Impact: xfs can hang on lack of contiguous memory page to be allocated.
+ Fix: upstream patch (b3f03bac8132207a20286d5602eda64500c19724).
+ Testcase:
+  - buddyinfo showing lack of contiguous blocks to be allocated (fragmented memory)
+  - to create 1 million files in 1 directory (attached script as an example)
+  - to observe message: XFS: possible memory allocation deadlock in kmem_alloc (mode:0x250)
+ 
+ === Original Description ===
+ 
  It was brought to my attention the following situation:
  
  http://tracker.ceph.com/issues/6301
  
  Precise kernel does not have XFS fix for kmem_alloc deadlock and users
  are facing this problem.
  
  Output example:
  
  """
  NFO: task ceph-osd:17047 blocked for more than 120 seconds.
  [153972.073476] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
  [153972.076322] ceph-osd        D ffff880869a28a60     0 17047   5423 0x00000000
  [153972.076324]  ffff880869a28750 0000000000000002 ffff880867788ee8 ffff8807e4e47500
  [153972.079259]  ffff880517addfd8 ffff880517addfd8 ffff880517addfd8 ffff880869a28750
  [153972.082244]  0000000000000004 ffff880517addd48 ffff880517addd50 7fffffffffffffff
  [153972.085278] Call Trace:
  [153972.088310]  [<ffffffff81410f4a>] ? schedule_timeout+0x1ca/0x270
  [153972.091371]  [<ffffffff8100abf1>] ? native_sched_clock+0x11/0x70
  [153972.094386]  [<ffffffff81070cda>] ? try_to_wake_up+0x1ea/0x270
  [153972.097358]  [<ffffffff81412623>] ? wait_for_completion+0xa3/0x120
  [153972.100322]  [<ffffffff81070d60>] ? try_to_wake_up+0x270/0x270
  [153972.103292]  [<ffffffff811a3702>] ? do_coredump+0x1b2/0xee0
  [153972.106212]  [<ffffffff811a3899>] ? do_coredump+0x349/0xee0
  [153972.109085]  [<ffffffff8134e0c4>] ? skb_queue_tail+0x24/0x60
  [153972.111954]  [<ffffffff813eb73a>] ? unix_dgram_sendmsg+0x5aa/0x640
  [153972.114839]  [<ffffffff81053049>] ? get_signal_to_deliver+0x199/0x5a0
  [153972.117743]  [<ffffffff81002353>] ? do_signal+0x63/0x8c0
  [153972.120632]  [<ffffffff81052030>] ? do_send_sig_info+0x60/0x90
  [153972.123521]  [<ffffffff81002c38>] ? do_notify_resume+0x88/0xa0
  [153972.126400]  [<ffffffff81414d6a>] ? int_signal+0x12/0x17
  [153972.299643] XFS: possible memory allocation deadlock in kmem_alloc (mode:0x250)
  [153972.868782] XFS: possible memory allocation deadlock in kmem_alloc (mode:0x250)
  [153973.038189] XFS: possible memory allocation deadlock in kmem_alloc (mode:0x250)
  [153974.309978] XFS: possible memory allocation deadlock in kmem_alloc (mode:0x250)
  """
  
  Fix, already included in Utopic, is upstream commit:
  b3f03bac8132207a20286d5602eda64500c19724
  
  Author: Dave Chinner <dchinner@xxxxxxxxxx>
  Date:   Tue Dec 3 23:50:57 2013 +1100
  
-     xfs: xfs_dir2_block_to_sf temp buffer allocation fails
-     
-     If we are using a large directory block size, and memory becomes
-     fragmented, we can get memory allocation failures trying to
-     kmem_alloc(64k) for a temporary buffer. However, there is not need
-     for a directory buffer sized allocation, as the end result ends up
-     in the inode literal area. This is, at most, slightly less than 2k
-     of space, and hence we don't need an allocation larger than that
-     fora temporary buffer.
-     
-     Signed-off-by: Dave Chinner <dchinner@xxxxxxxxxx>
-     Reviewed-by: Ben Myers <bpm@xxxxxxx>
-     Signed-off-by: Ben Myers <bpm@xxxxxxx>
+     xfs: xfs_dir2_block_to_sf temp buffer allocation fails
+ 
+     If we are using a large directory block size, and memory becomes
+     fragmented, we can get memory allocation failures trying to
+     kmem_alloc(64k) for a temporary buffer. However, there is not need
+     for a directory buffer sized allocation, as the end result ends up
+     in the inode literal area. This is, at most, slightly less than 2k
+     of space, and hence we don't need an allocation larger than that
+     fora temporary buffer.
+ 
+     Signed-off-by: Dave Chinner <dchinner@xxxxxxxxxx>
+     Reviewed-by: Ben Myers <bpm@xxxxxxx>
+     Signed-off-by: Ben Myers <bpm@xxxxxxx>

-- 
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1382333

Title:
  XFS: memory allocation deadlock in kmem_alloc (mode:0x8250)

Status in “linux” package in Ubuntu:
  Fix Released
Status in “linux” source package in Trusty:
  Confirmed
Status in “linux” source package in Utopic:
  Fix Released

Bug description:
  === SRU Justification ===

  Impact: xfs can hang on lack of contiguous memory page to be allocated.
  Fix: upstream patch (b3f03bac8132207a20286d5602eda64500c19724).
  Testcase:
   - buddyinfo showing lack of contiguous blocks to be allocated (fragmented memory)
   - to create 1 million files in 1 directory (attached script as an example)
   - to observe message: XFS: possible memory allocation deadlock in kmem_alloc (mode:0x250)

  === Original Description ===

  It was brought to my attention the following situation:

  http://tracker.ceph.com/issues/6301

  Precise kernel does not have XFS fix for kmem_alloc deadlock and users
  are facing this problem.

  Output example:

  """
  NFO: task ceph-osd:17047 blocked for more than 120 seconds.
  [153972.073476] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
  [153972.076322] ceph-osd        D ffff880869a28a60     0 17047   5423 0x00000000
  [153972.076324]  ffff880869a28750 0000000000000002 ffff880867788ee8 ffff8807e4e47500
  [153972.079259]  ffff880517addfd8 ffff880517addfd8 ffff880517addfd8 ffff880869a28750
  [153972.082244]  0000000000000004 ffff880517addd48 ffff880517addd50 7fffffffffffffff
  [153972.085278] Call Trace:
  [153972.088310]  [<ffffffff81410f4a>] ? schedule_timeout+0x1ca/0x270
  [153972.091371]  [<ffffffff8100abf1>] ? native_sched_clock+0x11/0x70
  [153972.094386]  [<ffffffff81070cda>] ? try_to_wake_up+0x1ea/0x270
  [153972.097358]  [<ffffffff81412623>] ? wait_for_completion+0xa3/0x120
  [153972.100322]  [<ffffffff81070d60>] ? try_to_wake_up+0x270/0x270
  [153972.103292]  [<ffffffff811a3702>] ? do_coredump+0x1b2/0xee0
  [153972.106212]  [<ffffffff811a3899>] ? do_coredump+0x349/0xee0
  [153972.109085]  [<ffffffff8134e0c4>] ? skb_queue_tail+0x24/0x60
  [153972.111954]  [<ffffffff813eb73a>] ? unix_dgram_sendmsg+0x5aa/0x640
  [153972.114839]  [<ffffffff81053049>] ? get_signal_to_deliver+0x199/0x5a0
  [153972.117743]  [<ffffffff81002353>] ? do_signal+0x63/0x8c0
  [153972.120632]  [<ffffffff81052030>] ? do_send_sig_info+0x60/0x90
  [153972.123521]  [<ffffffff81002c38>] ? do_notify_resume+0x88/0xa0
  [153972.126400]  [<ffffffff81414d6a>] ? int_signal+0x12/0x17
  [153972.299643] XFS: possible memory allocation deadlock in kmem_alloc (mode:0x250)
  [153972.868782] XFS: possible memory allocation deadlock in kmem_alloc (mode:0x250)
  [153973.038189] XFS: possible memory allocation deadlock in kmem_alloc (mode:0x250)
  [153974.309978] XFS: possible memory allocation deadlock in kmem_alloc (mode:0x250)
  """

  Fix, already included in Utopic, is upstream commit:
  b3f03bac8132207a20286d5602eda64500c19724

  Author: Dave Chinner <dchinner@xxxxxxxxxx>
  Date:   Tue Dec 3 23:50:57 2013 +1100

      xfs: xfs_dir2_block_to_sf temp buffer allocation fails

      If we are using a large directory block size, and memory becomes
      fragmented, we can get memory allocation failures trying to
      kmem_alloc(64k) for a temporary buffer. However, there is not need
      for a directory buffer sized allocation, as the end result ends up
      in the inode literal area. This is, at most, slightly less than 2k
      of space, and hence we don't need an allocation larger than that
      fora temporary buffer.

      Signed-off-by: Dave Chinner <dchinner@xxxxxxxxxx>
      Reviewed-by: Ben Myers <bpm@xxxxxxx>
      Signed-off-by: Ben Myers <bpm@xxxxxxx>

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1382333/+subscriptions