kernel-packages team mailing list archive

[Ming Lei <1574814@xxxxxxxxxxxxxxxxxx>]

On Tue, May 17, 2016 at 12:12 PM, Ming Lei <ming.lei@xxxxxxxxxxxxx> wrote:
> On Mon, May 16, 2016 at 5:25 PM, Ming Lei <ming.lei@xxxxxxxxxxxxx> wrote:
>> On Fri, May 13, 2016 at 7:22 AM, dann frazier
>> <dann.frazier@xxxxxxxxxxxxx> wrote:
>>> I used ftrace to do some duration measuring of the timer function
>>> fb_flashcursor(). I noticed several places where this timer takes around
>>> 98 ms to complete. This time seems to be due to multiple calls to
>>> __memcpy_toio() in ast_dirty_update():
>>>
>>>         for (i = y; i <= y2; i++) {
>>>                 /* assume equal stride for now */
>>>                 src_offset = dst_offset = i * afbdev->afb.base.pitches[0] + (x * bpp);
>>>                 memcpy_toio(bo->kmap.virtual + src_offset, afbdev->sysram + src_offset, (x2 - x + 1) * bpp);
>>>
>>>
>>> My theory is that this is causing mod_timer() to block on the other CPU, resulting in the soft lockup.
>>>
>>> Also - I built a custom d-i using pristine 4.6-rc7, and I am able to
>>> easily reproduce this. I think the next step here is to report this to
>>> upstream.
>>
>> Hi Dann,
>>
>> Andrew asked me to take a look at the issue, and from my tracing,
>> most of times, the cpu 'hangs' in the following line of code:
>>
>> __mod_timer():
>> ....
>> out_unlock:
>>      spin_unlock_irqrestore(&base->lock, flags);
>>
>> If I added two trace points around the above line, most of times only
>> the trace point before the line is dumped, and the one after the line
>> can't dumped.
>
> Looks the above issue is caused by passing 'jiffies' to mod_timer,
> and 'ops->cur_blink_jiffies' is observed as zero in cursor_timer_handler()
> when the issue happened.
>
> The following patch(workaround) can make the issue disappeared:
>
> diff --git a/drivers/video/console/fbcon.c b/drivers/video/console/fbcon.c
> index 6e92917..5e880ee 100644
> --- a/drivers/video/console/fbcon.c
> +++ b/drivers/video/console/fbcon.c
> @@ -1095,6 +1095,8 @@ static void fbcon_init(struct vc_data *vc, int init)
>                 con_copy_unimap(vc, svc);
>
>         ops = info->fbcon_par;
> +       if (vc->vc_cur_blink_ms)

oops, it should be 'if (!vc->vc_cur_blink_ms)'

> +               vc->vc_cur_blink_ms = 125;
>         ops->cur_blink_jiffies = msecs_to_jiffies(vc->vc_cur_blink_ms);
>         p->con_rotate = initial_rotation;
>         set_blitting_type(vc, info);
>
> Thanks,
>
>>
>> Thanks,
>>
>>>
>>> --
>>> You received this bug notification because you are subscribed to linux
>>> in Ubuntu.
>>> https://bugs.launchpad.net/bugs/1574814
>>>
>>> Title:
>>>   ThunderX: soft lockup in cursor_timer_handler() Edit
>>>
>>> Status in linux package in Ubuntu:
>>>   Confirmed
>>>
>>> Bug description:
>>>   I booted a Cavium ThunderX crb1s 2.0 system using the netboot mini iso via virtual media:
>>>     http://ports.ubuntu.com/ubuntu-ports/dists/xenial/main/installer-arm64/20101020ubuntu451/images/netboot/mini.iso
>>>
>>>   During boot I observed the following lockup on the serial console:
>>>
>>>   [ 28.128327] usb 1-1.1: reset high-speed USB device number 3 using xhci_hcd
>>>   [ 84.912299] NMI watchdog: BUG: soft lockup - CPU#14 stuck for 23s! [swapper/14:0]
>>>   [ 84.922718] Modules linked in: hid_generic(E) usbhid(E) hid(E) usb_storage(E) mdio_thunder(E) nicvf(E) ast(E) i2c_algo_bit(E) drm_kms_helper(E) syscopyarea(E) sysfillrect(E) sysimgblt(E) fb_sys_fops(E) ttm(E) drm(E) nicpf(E) thunder_bgx(E) mdio_cavium(E)
>>>   [ 84.922749]
>>>   [ 84.922754] CPU: 14 PID: 0 Comm: swapper/14 Tainted: G E 4.4.0-21-generic #37-Ubuntu
>>>   [ 84.922757] Hardware name: Cavium ThunderX CN88XX board (DT)
>>>   [ 84.922761] task: ffff801f6c9d4100 ti: ffff801f6c9e8000 task.ti: ffff801f6c9e8000
>>>   [ 84.922771] PC is at cursor_timer_handler+0x30/0x58
>>>   [ 84.922775] LR is at cursor_timer_handler+0x30/0x58
>>>   [ 84.922778] pc : [<ffff8000004ec4f0>] lr : [<ffff8000004ec4f0>] pstate: 00400145
>>>   [ 84.922781] sp : ffff801f6c9ebc20
>>>   [ 84.922784] x29: ffff801f6c9ebc20 x28: ffff8000f94398d8
>>>   [ 84.922789] x27: ffff801f6c9ebd00 x26: ffff801f7b3bebb8
>>>   [ 84.922793] x25: ffff801f6c9e8000 x24: ffff800000e5ec00
>>>   [ 84.922798] x23: ffff801f667d9800 x22: ffff8000004ec4c0
>>>   [ 84.922802] x21: 0000000000000100 x20: ffff8000f94398d8
>>>   [ 84.922807] x19: ffff8000f9439800 x18: 0000ffffc76a5358
>>>   [ 84.922811] x17: 0000ffff97bbd2a8 x16: ffff8000002a5040
>>>   [ 84.922816] x15: 000000003e4cf1e0 x14: 0000000000000008
>>>   [ 84.922820] x13: 0000000000000000 x12: 003d090000000000
>>>   [ 84.922824] x11: 00000000003d0900 x10: ffff80000090f200
>>>   [ 84.922829] x9 : 00003d0900000000 x8 : 000000000000000e
>>>   [ 84.922833] x7 : ffff801f7b3c5008 x6 : 00000000ffffffff
>>>   [ 84.922837] x5 : 0000000000000000 x4 : 0000000000000001
>>>   [ 84.922842] x3 : 0000000000000000 x2 : ffff801f6c899e05
>>>   [ 84.922846] x1 : ffff801f667d99e0 x0 : 0000000000000000
>>>   [ 84.922850]
>>>   [ 101.008387] usb 1-1.1: reset high-speed USB device number 3 using xhci_hcd
>>>   [ 101.180375] usb 1-1.1: reset high-speed USB device number 3 using xhci_hcd
>>>   [ 101.342677] random: nonblocking pool is initialized
>>>
>>> To manage notifications about this bug go to:
>>> https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1574814/+subscriptions

-- 
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1574814

Title:
  ThunderX: soft lockup in cursor_timer_handler()

Status in linux package in Ubuntu:
  Confirmed

Bug description:
  I booted a Cavium ThunderX crb1s 2.0 system using the netboot mini iso via virtual media:
    http://ports.ubuntu.com/ubuntu-ports/dists/xenial/main/installer-arm64/20101020ubuntu451/images/netboot/mini.iso

  During boot I observed the following lockup on the serial console:

  [ 28.128327] usb 1-1.1: reset high-speed USB device number 3 using xhci_hcd
  [ 84.912299] NMI watchdog: BUG: soft lockup - CPU#14 stuck for 23s! [swapper/14:0]
  [ 84.922718] Modules linked in: hid_generic(E) usbhid(E) hid(E) usb_storage(E) mdio_thunder(E) nicvf(E) ast(E) i2c_algo_bit(E) drm_kms_helper(E) syscopyarea(E) sysfillrect(E) sysimgblt(E) fb_sys_fops(E) ttm(E) drm(E) nicpf(E) thunder_bgx(E) mdio_cavium(E)
  [ 84.922749]
  [ 84.922754] CPU: 14 PID: 0 Comm: swapper/14 Tainted: G E 4.4.0-21-generic #37-Ubuntu
  [ 84.922757] Hardware name: Cavium ThunderX CN88XX board (DT)
  [ 84.922761] task: ffff801f6c9d4100 ti: ffff801f6c9e8000 task.ti: ffff801f6c9e8000
  [ 84.922771] PC is at cursor_timer_handler+0x30/0x58
  [ 84.922775] LR is at cursor_timer_handler+0x30/0x58
  [ 84.922778] pc : [<ffff8000004ec4f0>] lr : [<ffff8000004ec4f0>] pstate: 00400145
  [ 84.922781] sp : ffff801f6c9ebc20
  [ 84.922784] x29: ffff801f6c9ebc20 x28: ffff8000f94398d8
  [ 84.922789] x27: ffff801f6c9ebd00 x26: ffff801f7b3bebb8
  [ 84.922793] x25: ffff801f6c9e8000 x24: ffff800000e5ec00
  [ 84.922798] x23: ffff801f667d9800 x22: ffff8000004ec4c0
  [ 84.922802] x21: 0000000000000100 x20: ffff8000f94398d8
  [ 84.922807] x19: ffff8000f9439800 x18: 0000ffffc76a5358
  [ 84.922811] x17: 0000ffff97bbd2a8 x16: ffff8000002a5040
  [ 84.922816] x15: 000000003e4cf1e0 x14: 0000000000000008
  [ 84.922820] x13: 0000000000000000 x12: 003d090000000000
  [ 84.922824] x11: 00000000003d0900 x10: ffff80000090f200
  [ 84.922829] x9 : 00003d0900000000 x8 : 000000000000000e
  [ 84.922833] x7 : ffff801f7b3c5008 x6 : 00000000ffffffff
  [ 84.922837] x5 : 0000000000000000 x4 : 0000000000000001
  [ 84.922842] x3 : 0000000000000000 x2 : ffff801f6c899e05
  [ 84.922846] x1 : ffff801f667d99e0 x0 : 0000000000000000
  [ 84.922850]
  [ 101.008387] usb 1-1.1: reset high-speed USB device number 3 using xhci_hcd
  [ 101.180375] usb 1-1.1: reset high-speed USB device number 3 using xhci_hcd
  [ 101.342677] random: nonblocking pool is initialized

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1574814/+subscriptions