kernel-packages team mailing list archive

Thread
Date
[Bug 1495863] [NEW] ISST-KVM: R3-0: Firestone: PowerNV : Call traces w.r.t filesystem while running stress test

To: kernel-packages@xxxxxxxxxxxxxxxxxxx
From: Launchpad Bug Tracker <1495863@xxxxxxxxxxxxxxxxxx>
Date: Mon, 28 Sep 2015 20:38:03 -0000
Reply-to: Bug 1495863 <1495863@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
You have been subscribed to a public bug:

== Comment: #0 - Krishnaja Balachandran <kribalac@xxxxxxxxxx> - 2015-09-02 05:01:44 ==
---Problem Description---
While running stress tests( IO BASE TCP NFS) on Firestone system "amp" I see the following call traces in "dmesg". Also, few commands are hanging in amp.


 
Contact Information = kribalac@xxxxxxxxxx 
 
---uname output---
Linux amp 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:10:52 UTC 2015 ppc64le ppc64le ppc64le GNU/Linux
 

Machine Type = PowerNV 8335-GTA

 ---Debugger---
"xmon" was configured, however the system did not enter into the debugger
 
 
Stack trace output:
----------------------------
 [69633.095738] INFO: rcu_sched detected stalls on CPUs/tasks: { 11 59} (detected by 171, t=10356287 jiffies, g=2131264, c=2131263, q=51507717)
[69633.096031] Task dump for CPU 11:
[69633.096058] kswapd8         R  running task        0   989      2 0x00000804
[69633.096110] Call Trace:
[69633.096138] [c000003c9f93f070] [c000003c9f93f0b0] 0xc000003c9f93f0b0 (unreliable)
[69633.096347] [c000003c9f93f240] [c000003c9f93f370] 0xc000003c9f93f370
[69633.096399] Task dump for CPU 59:
[69633.096426] kworker/u386:14 D 0000000000000000     0 77488      2 0x00000804
[69633.096484] Workqueue: writeback bdi_writeback_workfn (flush-8:32)
[69633.096536] Call Trace:
[69633.096556] [c000003c9f936f00] [c000001ff4ee76f0] 0xc000001ff4ee76f0 (unreliable)
[69633.096618] [c000003c9f9370d0] [c000003c9f937130] 0xc000003c9f937130
[69633.096671] [c000003c9f937130] [c000000000a11370] __schedule+0x370/0x8d0
[69633.096726] [c000003c9f937350] [c000000000a153e8] rwsem_down_write_failed+0x288/0x400
[69633.096788] [c000003c9f9373e0] [c000000000a147f8] down_write+0x88/0x90
[69633.096864] [c000003c9f937410] [d000000029bc5254] xfs_ilock+0xf4/0x160 [xfs]
[69633.096933] [c000003c9f937450] [d000000029bc1bc8] xfs_iomap_write_allocate+0x238/0x3f0 [xfs]
[69633.097011] [c000003c9f937580] [d000000029ba60bc] xfs_map_blocks+0x1cc/0x2f0 [xfs]
[69633.097087] [c000003c9f9375f0] [d000000029ba7b24] xfs_vm_writepage+0x194/0x630 [xfs]
[69633.097148] [c000003c9f9376d0] [c00000000021c66c] __writepage+0x4c/0xb0
[69633.097373] [c000003c9f937710] [c00000000021cda4] write_cache_pages+0x1e4/0x4c0
[69633.097437] [c000003c9f937850] [c00000000021d0e4] generic_writepages+0x64/0x90
[69633.097513] [c000003c9f9378b0] [d000000029ba5e70] xfs_vm_writepages+0x70/0xa0 [xfs]
[69633.097575] [c000003c9f9378f0] [c00000000021e720] do_writepages+0x60/0xc0
[69633.097627] [c000003c9f937920] [c0000000002f1db8] __writeback_single_inode+0x68/0x370
[69633.097687] [c000003c9f937970] [c0000000002f2478] writeback_sb_inodes+0x2c8/0x4f0
[69633.097747] [c000003c9f937a40] [c0000000002f2784] __writeback_inodes_wb+0xe4/0x150
[69633.097807] [c000003c9f937aa0] [c0000000002f354c] wb_writeback+0x30c/0x3e0
[69633.097859] [c000003c9f937b40] [c0000000002f405c] bdi_writeback_workfn+0x14c/0x550
[69633.097919] [c000003c9f937c60] [c0000000000d28bc] process_one_work+0x19c/0x480
[69633.097980] [c000003c9f937cf0] [c0000000000d3160] worker_thread+0x190/0x5b0
[69633.098032] [c000003c9f937d80] [c0000000000da494] kthread+0x114/0x140
[69633.098085] [c000003c9f937e30] [c00000000000956c] ret_from_kernel_thread+0x5c/0x70

 
 NOTE:

System is on a private network.  Access the private network via SSH to "banner.isst.aus.stglabs.ibm.com" using your GSA ID and password. 
(Banner itself is behind a BSO, so must authenticate through that first.)

Login details :  
ssh banner.isst.aus.stglabs.ibm.com [debug/don2rry ]

Host login:-

amp.isst.aus.stglabs.ibm.com [10.33.31.106 ]
[root/don2rry]

login via GUI:
bmc-amp.isst.aus.stglabs.ibm.com [10.33.31.106 ]


IPMI Login to host amp console :- 
-------------------------------------------------
>From banner machine run the following :
ssh banner.isst.aus.stglabs.ibm.com [debug/don2rry ]

ipmitool -I lanplus -H bmc-amp -U ADMIN -P admin sol deactivate
ipmitool -I lanplus -H bmc-amp -U ADMIN -P admin sol activate


-----------------------------------------------------------------------------
                             TESTING INFORMATION
-----------------------------------------------------------------------------

SYSTEM INFORMATION
---------------------------------

   HOST NAME or NETWORK ADDRESS:   amp.isst.aus.stglabs.ibm.com [ 10.33.31.106 ]
   
   BMC NAME and BMC ip: 	bmc-amp.isst.aus.stglabs.ibm.com [ 10.33.31.108 ]

   Firmware Revision: 		2.02.82263
   Firmware Build Time: 	Aug 24 2015 10:44:30 CDT


RECENT SYSTEM CHANGES  : none
-----------------------------------------
  none

** Affects: linux (Ubuntu)
     Importance: Undecided
     Assignee: Taco Screen team (taco-screen-team)
         Status: New


** Tags: architecture-ppc64le bot-comment bugnameltc-129789 severity-high targetmilestone-inin---
-- 
ISST-KVM: R3-0: Firestone: PowerNV : Call traces w.r.t filesystem while running stress test
https://bugs.launchpad.net/bugs/1495863
You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu.