debcrafters-packages team mailing list archive

Thread
Date
[Bug 2111213] Re: System hang due refcount_t underflow issue and VFS: Close: file count is 0

To: debcrafters-packages@xxxxxxxxxxxxxxxxxxx
From: Paul White <2111213@xxxxxxxxxxxxxxxxxx>
Date: Sun, 18 May 2025 20:00:32 -0000
Reply-to: Bug 2111213 <2111213@xxxxxxxxxxxxxxxxxx>
Sender: noreply@xxxxxxxxxxxxx
** Package changed: ubuntu => nfs-utils (Ubuntu)

** Tags added: jammy

-- 
You received this bug notification because you are a member of
Debcrafters packages, which is subscribed to nfs-utils in Ubuntu.
https://bugs.launchpad.net/bugs/2111213

Title:
  System hang due refcount_t underflow issue and VFS: Close: file count
  is 0

Status in nfs-utils package in Ubuntu:
  New

Bug description:
  ================
  Environment details:
  ================

  - Linux Kernel Version: 5.15.0-117
  - Ubuntu 22.04
  - nfs-utils package: 1:1.3.4-2.5ubuntu3.3 and 1:2.6.1-1ubuntu1.2
  - Using NFSv3

  ================
  Issue observed:
  ================

  - I see `refcount_t underflow` and `use-after-free` warnings associated specifically with `nfsd_write` operations in the syslogs.
  - After this occurs, the system eventually becomes non-functional, requiring a manual reboot.
  - The last NFS-related operation seems to be an `unexport` of a filesystem: exportfs -u 10.20.30.40:/fs/4d148fdf-6b18-48c8-8215-7c5dd58cac7d/c0
  - After the problem occurs, syslogs flood continuously with these repetitive and only entries: VFS: Close: file count is 0
  - On the system console, I consistently notice one `exportfs` process stuck in a CPU soft lockup—this persists until the system gets rebooted.


  ================
  Stack trace
  ================

  2025-05-15T10:22:24.988837+00:00 <4> kernel - [7144513.654935] ------------[ cut here ]------------
  2025-05-15T10:22:24.989181+00:00 <4> kernel - [7144513.654941] refcount_t: underflow; use-after-free.
  2025-05-15T10:22:24.989186+00:00 <4> kernel - [7144513.654963] WARNING: CPU: 71 PID: 2010514 at lib/refcount.c:28 refcount_warn_saturate+0xf7/0x150
  ...

  ...
  2025-05-15T10:22:24.989221+00:00 <4> kernel - [7144513.655146] RIP: 0010:refcount_warn_saturate+0xf7/0x150
  2025-05-15T10:22:24.989229+00:00 <4> kernel - [7144513.655149] Code: eb 9e 0f b6 1d 12 2b b9 01 80 fb 01 0f 87 10 92 6f 00 83 e3 01 75 89 48 c7 c7 30 10 e4 8e c6 05 f6 2a b9 01 01 e8 b3 0b 6c 00 <0f> 0b e9 6f ff ff ff 0f b6 1d e1 2a b9 01 80 fb 01 0f 87 cd 91 6f
  2025-05-15T10:22:24.989231+00:00 <4> kernel - [7144513.655153] RSP: 0018:ff297e72f41e7d48 EFLAGS: 00010286
  2025-05-15T10:22:24.989232+00:00 <4> kernel - [7144513.655156] RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000000027
  2025-05-15T10:22:24.989233+00:00 <4> kernel - [7144513.655158] RDX: ff249fc37fae0588 RSI: 0000000000000001 RDI: ff249fc37fae0580
  2025-05-15T10:22:24.989234+00:00 <4> kernel - [7144513.655160] RBP: ff297e72f41e7d50 R08: 0000000000000003 R09: fffffffffff6bb28
  2025-05-15T10:22:24.989235+00:00 <4> kernel - [7144513.655162] R10: ff249fc37f16bb30 R11: 0000000000000001 R12: ff249fb2124ba380
  2025-05-15T10:22:24.989239+00:00 <4> kernel - [7144513.655164] R13: ff249fb2124ba3b8 R14: 00000000000000e5 R15: ff249f4589a40000
  2025-05-15T10:22:24.989240+00:00 <4> kernel - [7144513.655166] FS:  0000000000000000(0000) GS:ff249fc37fac0000(0000) knlGS:0000000000000000
  2025-05-15T10:22:24.989241+00:00 <4> kernel - [7144513.655169] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
  2025-05-15T10:22:24.989242+00:00 <4> kernel - [7144513.655171] CR2: 00007f02a7291000 CR3: 000000413b00e006 CR4: 0000000000771ee0
  2025-05-15T10:22:24.989243+00:00 <4> kernel - [7144513.655173] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
  2025-05-15T10:22:24.989244+00:00 <4> kernel - [7144513.655175] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
  2025-05-15T10:22:24.989247+00:00 <4> kernel - [7144513.655177] PKRU: 55555554
  2025-05-15T10:22:24.989248+00:00 <4> kernel - [7144513.655178] Call Trace:
  2025-05-15T10:22:24.989248+00:00 <4> kernel - [7144513.655181]  <TASK>
  2025-05-15T10:22:24.989249+00:00 <4> kernel - [7144513.655185]  ? show_trace_log_lvl+0x1d6/0x2ea
  2025-05-15T10:22:24.989250+00:00 <4> kernel - [7144513.655193]  ? show_trace_log_lvl+0x1d6/0x2ea
  2025-05-15T10:22:24.989251+00:00 <4> kernel - [7144513.655197]  ? nfsd_file_put+0x10f/0x170 [nfsd]
  2025-05-15T10:22:24.989251+00:00 <4> kernel - [7144513.655255]  ? show_regs.part.0+0x23/0x29
  2025-05-15T10:22:24.989255+00:00 <4> kernel - [7144513.655258]  ? show_regs.cold+0x8/0xd
  2025-05-15T10:22:24.989255+00:00 <4> kernel - [7144513.655260]  ? refcount_warn_saturate+0xf7/0x150
  2025-05-15T10:22:24.989256+00:00 <4> kernel - [7144513.655263]  ? __warn+0x8c/0x100
  2025-05-15T10:22:24.989257+00:00 <4> kernel - [7144513.655268]  ? refcount_warn_saturate+0xf7/0x150
  2025-05-15T10:22:24.989257+00:00 <4> kernel - [7144513.655270]  ? report_bug+0xa4/0xd0
  2025-05-15T10:22:24.989258+00:00 <4> kernel - [7144513.655275]  ? handle_bug+0x39/0x90
  2025-05-15T10:22:24.989261+00:00 <4> kernel - [7144513.655279]  ? exc_invalid_op+0x19/0x70
  2025-05-15T10:22:24.989262+00:00 <4> kernel - [7144513.655281]  ? asm_exc_invalid_op+0x1b/0x20
  2025-05-15T10:22:24.989262+00:00 <4> kernel - [7144513.655287]  ? refcount_warn_saturate+0xf7/0x150
  2025-05-15T10:22:24.989263+00:00 <4> kernel - [7144513.655290]  ? refcount_warn_saturate+0xf7/0x150
  2025-05-15T10:22:24.989264+00:00 <4> kernel - [7144513.655292]  nfsd_file_put+0x10f/0x170 [nfsd]
  2025-05-15T10:22:24.989265+00:00 <4> kernel - [7144513.655326]  nfsd_write+0x8a/0x140 [nfsd]
  2025-05-15T10:22:24.989267+00:00 <4> kernel - [7144513.655361]  nfsd3_proc_write+0xe0/0x170 [nfsd]
  2025-05-15T10:22:24.989268+00:00 <4> kernel - [7144513.655395]  nfsd_dispatch+0x173/0x270 [nfsd]
  2025-05-15T10:22:24.989268+00:00 <4> kernel - [7144513.655427]  svc_process_common+0x3d7/0x720 [sunrpc]
  2025-05-15T10:22:24.989269+00:00 <4> kernel - [7144513.655484]  ? nfsd_svc+0x200/0x200 [nfsd]
  2025-05-15T10:22:24.989270+00:00 <4> kernel - [7144513.655518]  svc_process+0xbc/0x100 [sunrpc]
  2025-05-15T10:22:24.989270+00:00 <4> kernel - [7144513.655561]  nfsd+0xd3/0x100 [nfsd]
  2025-05-15T10:22:24.989271+00:00 <4> kernel - [7144513.655590]  ? trace_event_raw_event_nfsd_export_update+0x190/0x190 [nfsd]
  2025-05-15T10:22:24.989272+00:00 <4> kernel - [7144513.655619]  kthread+0x127/0x150
  2025-05-15T10:22:24.989273+00:00 <4> kernel - [7144513.655624]  ? set_kthread_struct+0x50/0x50
  2025-05-15T10:22:24.989274+00:00 <4> kernel - [7144513.655628]  ret_from_fork+0x1f/0x30
  2025-05-15T10:22:24.989275+00:00 <4> kernel - [7144513.655636]  </TASK>
  2025-05-15T10:22:24.989276+00:00 <4> kernel - [7144513.655637] ---[ end trace 2c2d196f39b3f573 ]---
  2025-05-15T10:22:25.000964+00:00 <3> kernel - [7144513.668313] VFS: Close: file count is 0
  2025-05-15T10:22:25.000984+00:00 <3> kernel - [7144513.668317] VFS: Close: file count is 0
  2025-05-15T10:22:25.000986+00:00 <3> kernel - [7144513.668318] VFS: Close: file count is 0

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/nfs-utils/+bug/2111213/+subscriptions