kernel-packages team mailing list archive

Thread
Date
[Bug 1353730] Re: nfs4 daemon crashes, eventually PC locks up

To: kernel-packages@xxxxxxxxxxxxxxxxxxx
From: Kim Tyler <1353730@xxxxxxxxxxxxxxxxxx>
Date: Fri, 24 Oct 2014 00:51:46 -0000
Reply-to: Bug 1353730 <1353730@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
yes - it still happens (happened) with trusty.  I use a backup program
called backintime which uses rsync, and I backup the laptop to an nfs4
mount.  During the backup, the nfs4 server crashes just as specified
above in trusty (kernel 3.13.0-32-generic).  The backup program then
stops, leaving the rsync running.

If I restart the server, without restarting the client, the backup
program continues and corrupts the disk that is being backed up to,
requiring an fsck.

>From the logs of backintime, the command is:- (where pio is the laptop
being backed up, mike is the nfs4 server, and the disk /media/seagate on
mike is mounted nfs4 at pio as /media/mike/seagate

rsync -rtDH --links --no-p --no-g --no-o  --delete --delete-excluded  -v
--chmod=Du+wx  --exclude="/media/mike/seagate/backintime"
--exclude="/home/kit284/.local/share/backintime"
--include="/home/kit284/" --include="/home/" --include="/home/mythtv/"
--exclude="*.backup*" --exclude="*~" --exclude="/home/kit284/root"
--exclude="/home/kit284/tmp" --exclude="/home/kit284/Ubuntu One"
--exclude="*.thumbnails" --exclude="*.iso"
--exclude="/home/kit284/.gvfs" --exclude="[Cc]ache*"
--include="/home/kit284/**" --include="/home/mythtv/**" --exclude="*" /
"/media/mike/seagate/backintime/backintime/pio/kit284/1/new_snapshot/backup/"


I am currently running backintime via a FUSE mounted ssh connection built in to backintime, and this is working OK.  However, is is very disquieting to know that NFS4 can crash, and take out the disk, requiring a reboot and fsck.

I note that some research on the internet show that rsync can have
problems with change attributes on nfs4 mounts, and the suggestion is
that this is a bug with nfs4.

-- 
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1353730

Title:
  nfs4 daemon crashes, eventually PC locks up

Status in “linux” package in Ubuntu:
  Incomplete

Bug description:
  Aug  7 01:05:46 mike kernel: [60329.252525] PGD 5b4ac067 PUD 5b4ad067 PMD 0 
  Aug  7 01:05:46 mike kernel: [60329.252532] Oops: 0000 [#1] SMP 
  Aug  7 01:05:46 mike kernel: [60329.252538] Modules linked in: rfcomm bnep bluetooth nfsd auth_rpcgss nfs_acl nfs lockd sunrpc fscache ir_lirc_codec lirc_dev ir_mce_kbd_decoder ir_sanyo_decoder ir_sony_decoder ir_jvc_decoder ir_rc6_decoder ir_nec_decoder ir_rc5_decoder fc0012 rtl2832 dvb_usb_rtl28xxu rtl2830 dvb_usb_v2 dvb_core rc_core nvidia(POF) snd_intel8x0 snd_ac97_codec ac97_bus snd_pcm snd_page_alloc snd_seq_midi snd_seq_midi_event serio_raw snd_rawmidi k8temp edac_core edac_mce_amd snd_seq snd_seq_device usblp snd_timer snd soundcore i2c_nforce2 mac_hid parport_pc ppdev lp parport zfs(POF) zunicode(POF) zavl(POF) zcommon(POF) znvpair(POF) spl(OF) hid_generic usbhid pata_acpi hid firewire_ohci firewire_core psmouse e100 forcedeth mii crc_itu_t sata_nv pata_amd
  Aug  7 01:05:46 mike kernel: [60329.252626] CPU: 0 PID: 2099 Comm: nfsd Tainted: PF          O 3.13.0-32-generic #57-Ubuntu
  Aug  7 01:05:46 mike kernel: [60329.252632] Hardware name:    /GA-K8N Pro-SLI, BIOS F4 07/16/2005
  Aug  7 01:05:46 mike kernel: [60329.252638] task: ffff880060f5c7d0 ti: ffff88005b42e000 task.ti: ffff88005b42e000
  Aug  7 01:05:46 mike kernel: [60329.252642] RIP: 0010:[<ffffffffa0fbb06b>]  [<ffffffffa0fbb06b>] set_nfsv4_acl_one+0x1b/0x90 [nfsd]
  Aug  7 01:05:46 mike kernel: [60329.252660] RSP: 0018:ffff88005b42fcf8  EFLAGS: 00010246
  Aug  7 01:05:46 mike kernel: [60329.252664] RAX: 0000000000000000 RBX: ffff88002e9f3d80 RCX: 00000000008036d0
  Aug  7 01:05:46 mike kernel: [60329.252668] RDX: ffffffffa0fe86e7 RSI: 0000000000000000 RDI: ffff88002e9f3d80
  Aug  7 01:05:46 mike kernel: [60329.252672] RBP: ffff88005b42fd20 R08: 00000000000171e0 R09: ffff88007fc171e0
  Aug  7 01:05:46 mike kernel: [60329.252676] R10: ffffea0000282500 R11: ffffffffa0fbb0c7 R12: 0000000000000000
  Aug  7 01:05:46 mike kernel: [60329.252680] R13: ffff88002e9f3d80 R14: ffffffffa0fe86e7 R15: 0000000000000660
  Aug  7 01:05:46 mike kernel: [60329.252685] FS:  00007f62c5fb5780(0000) GS:ffff88007fc00000(0000) knlGS:0000000000000000
  Aug  7 01:05:46 mike kernel: [60329.252689] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
  Aug  7 01:05:46 mike kernel: [60329.252693] CR2: 0000000000000010 CR3: 000000005b4ab000 CR4: 00000000000007f0
  Aug  7 01:05:46 mike kernel: [60329.252697] Stack:
  Aug  7 01:05:46 mike kernel: [60329.252700]  ffff88002e9f3d80 ffff88000a094940 ffff88005c2540b0 ffff880060fe0000
  Aug  7 01:05:46 mike kernel: [60329.252707]  0000000000000660 ffff88005b42fd60 ffffffffa0fbb492 ffff880060fe0000
  Aug  7 01:05:46 mike kernel: [60329.252714]  ffff88000a094900 0000000000000000 ffff880060fe8040 ffff8800667ee240
  Aug  7 01:05:46 mike kernel: [60329.252720] Call Trace:
  Aug  7 01:05:46 mike kernel: [60329.252740]  [<ffffffffa0fbb492>] nfsd4_set_nfs4_acl+0x122/0x130 [nfsd]
  Aug  7 01:05:46 mike kernel: [60329.252759]  [<ffffffffa0fc8f5d>] nfsd4_setattr+0x10d/0x1a0 [nfsd]
  Aug  7 01:05:46 mike kernel: [60329.252777]  [<ffffffffa0fca5ca>] nfsd4_proc_compound+0x56a/0x7c0 [nfsd]
  Aug  7 01:05:46 mike kernel: [60329.252792]  [<ffffffffa0fb6d2b>] nfsd_dispatch+0xbb/0x200 [nfsd]
  Aug  7 01:05:46 mike kernel: [60329.252838]  [<ffffffffa0f1162d>] svc_process_common+0x46d/0x6d0 [sunrpc]
  Aug  7 01:05:46 mike kernel: [60329.252864]  [<ffffffffa0f11997>] svc_process+0x107/0x170 [sunrpc]
  Aug  7 01:05:46 mike kernel: [60329.252879]  [<ffffffffa0fb671f>] nfsd+0xbf/0x130 [nfsd]
  Aug  7 01:05:46 mike kernel: [60329.252894]  [<ffffffffa0fb6660>] ? nfsd_destroy+0x80/0x80 [nfsd]
  Aug  7 01:05:46 mike kernel: [60329.252904]  [<ffffffff8108b3d2>] kthread+0xd2/0xf0
  Aug  7 01:05:46 mike kernel: [60329.252911]  [<ffffffff8108b300>] ? kthread_create_on_node+0x1d0/0x1d0
  Aug  7 01:05:46 mike kernel: [60329.252922]  [<ffffffff8172c5bc>] ret_from_fork+0x7c/0xb0
  Aug  7 01:05:46 mike kernel: [60329.252928]  [<ffffffff8108b300>] ? kthread_create_on_node+0x1d0/0x1d0
  Aug  7 01:05:46 mike kernel: [60329.252932] Code: eb e6 66 66 66 66 66 2e 0f 1f 84 00 00 00 00 00 66 66 66 66 90 55 48 89 e5 41 57 41 56 49 89 d6 41 55 49 89 fd 41 54 49 89 f4 53 <48> 63 46 10 be d0 00 00 00 4c 8d 3c c5 04 00 00 00 4c 89 ff e8 
  Aug  7 01:05:46 mike kernel: [60329.253003]  RSP <ffff88005b42fcf8>
  Aug  7 01:05:46 mike kernel: [60329.253006] CR2: 0000000000000010
  Aug  7 01:05:46 mike kernel: [60329.253012] ---[ end trace 737bd44e2270bead ]---




  and




  Aug  7 03:00:10 mike kernel: [67193.680580] PGD 36391067 PUD 36392067 PMD 0 
  Aug  7 03:00:10 mike kernel: [67193.680588] Oops: 0000 [#2] SMP 
  Aug  7 03:00:10 mike kernel: [67193.680594] Modules linked in: rfcomm bnep bluetooth nfsd auth_rpcgss nfs_acl nfs lockd sunrpc fscache ir_lirc_codec lirc_dev ir_mce_kbd_decoder ir_sanyo_decoder ir_sony_decoder ir_jvc_decoder ir_rc6_decoder ir_nec_decoder ir_rc5_decoder fc0012 rtl2832 dvb_usb_rtl28xxu rtl2830 dvb_usb_v2 dvb_core rc_core nvidia(POF) snd_intel8x0 snd_ac97_codec ac97_bus snd_pcm snd_page_alloc snd_seq_midi snd_seq_midi_event serio_raw snd_rawmidi k8temp edac_core edac_mce_amd snd_seq snd_seq_device usblp snd_timer snd soundcore i2c_nforce2 mac_hid parport_pc ppdev lp parport zfs(POF) zunicode(POF) zavl(POF) zcommon(POF) znvpair(POF) spl(OF) hid_generic usbhid pata_acpi hid firewire_ohci firewire_core psmouse e100 forcedeth mii crc_itu_t sata_nv pata_amd
  Aug  7 03:00:10 mike kernel: [67193.680681] CPU: 0 PID: 2098 Comm: nfsd Tainted: PF     D    O 3.13.0-32-generic #57-Ubuntu
  Aug  7 03:00:10 mike kernel: [67193.680688] Hardware name:    /GA-K8N Pro-SLI, BIOS F4 07/16/2005
  Aug  7 03:00:10 mike kernel: [67193.680693] task: ffff880060f5afe0 ti: ffff880060fde000 task.ti: ffff880060fde000
  Aug  7 03:00:10 mike kernel: [67193.680697] RIP: 0010:[<ffffffffa0fbb06b>]  [<ffffffffa0fbb06b>] set_nfsv4_acl_one+0x1b/0x90 [nfsd]
  Aug  7 03:00:10 mike kernel: [67193.680715] RSP: 0018:ffff880060fdfcf8  EFLAGS: 00010246
  Aug  7 03:00:10 mike kernel: [67193.680719] RAX: 0000000000000000 RBX: ffff88004a9e0840 RCX: 00000000008c923e
  Aug  7 03:00:10 mike kernel: [67193.680723] RDX: ffffffffa0fe86e7 RSI: 0000000000000000 RDI: ffff88004a9e0840
  Aug  7 03:00:10 mike kernel: [67193.680727] RBP: ffff880060fdfd20 R08: 00000000000171e0 R09: ffff88007fc171e0
  Aug  7 03:00:10 mike kernel: [67193.680731] R10: ffffea0001eb8040 R11: ffffffffa0fbb0c7 R12: 0000000000000000
  Aug  7 03:00:10 mike kernel: [67193.680735] R13: ffff88004a9e0840 R14: ffffffffa0fe86e7 R15: 0000000000000660
  Aug  7 03:00:10 mike kernel: [67193.680740] FS:  00007f08bffff700(0000) GS:ffff88007fc00000(0000) knlGS:0000000000000000
  Aug  7 03:00:10 mike kernel: [67193.680744] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
  Aug  7 03:00:10 mike kernel: [67193.680748] CR2: 0000000000000010 CR3: 0000000036390000 CR4: 00000000000007f0
  Aug  7 03:00:10 mike kernel: [67193.680752] Stack:
  Aug  7 03:00:10 mike kernel: [67193.680755]  ffff88004a9e0840 ffff88007ae019c0 ffff880006bf72a8 ffff880060eb6000
  Aug  7 03:00:10 mike kernel: [67193.680762]  0000000000000660 ffff880060fdfd60 ffffffffa0fbb492 ffff880060eb6000
  Aug  7 03:00:10 mike kernel: [67193.680768]  ffff88007ae01980 0000000000000000 ffff8800667e9040 ffff8800667eb240
  Aug  7 03:00:10 mike kernel: [67193.680775] Call Trace:
  Aug  7 03:00:10 mike kernel: [67193.680795]  [<ffffffffa0fbb492>] nfsd4_set_nfs4_acl+0x122/0x130 [nfsd]
  Aug  7 03:00:10 mike kernel: [67193.680814]  [<ffffffffa0fc8f5d>] nfsd4_setattr+0x10d/0x1a0 [nfsd]
  Aug  7 03:00:10 mike kernel: [67193.680832]  [<ffffffffa0fca5ca>] nfsd4_proc_compound+0x56a/0x7c0 [nfsd]
  Aug  7 03:00:10 mike kernel: [67193.680847]  [<ffffffffa0fb6d2b>] nfsd_dispatch+0xbb/0x200 [nfsd]
  Aug  7 03:00:10 mike kernel: [67193.680894]  [<ffffffffa0f1162d>] svc_process_common+0x46d/0x6d0 [sunrpc]
  Aug  7 03:00:10 mike kernel: [67193.680920]  [<ffffffffa0f11997>] svc_process+0x107/0x170 [sunrpc]
  Aug  7 03:00:10 mike kernel: [67193.680934]  [<ffffffffa0fb671f>] nfsd+0xbf/0x130 [nfsd]
  Aug  7 03:00:10 mike kernel: [67193.680949]  [<ffffffffa0fb6660>] ? nfsd_destroy+0x80/0x80 [nfsd]
  Aug  7 03:00:10 mike kernel: [67193.680959]  [<ffffffff8108b3d2>] kthread+0xd2/0xf0
  Aug  7 03:00:10 mike kernel: [67193.680966]  [<ffffffff8108b300>] ? kthread_create_on_node+0x1d0/0x1d0
  Aug  7 03:00:10 mike kernel: [67193.680977]  [<ffffffff8172c5bc>] ret_from_fork+0x7c/0xb0
  Aug  7 03:00:10 mike kernel: [67193.680983]  [<ffffffff8108b300>] ? kthread_create_on_node+0x1d0/0x1d0
  Aug  7 03:00:10 mike kernel: [67193.680987] Code: eb e6 66 66 66 66 66 2e 0f 1f 84 00 00 00 00 00 66 66 66 66 90 55 48 89 e5 41 57 41 56 49 89 d6 41 55 49 89 fd 41 54 49 89 f4 53 <48> 63 46 10 be d0 00 00 00 4c 8d 3c c5 04 00 00 00 4c 89 ff e8 
  Aug  7 03:00:10 mike kernel: [67193.681058]  RSP <ffff880060fdfcf8>
  Aug  7 03:00:10 mike kernel: [67193.681061] CR2: 0000000000000010
  Aug  7 03:00:10 mike kernel: [67193.681067] ---[ end trace 737bd44e2270beae ]---


  After this sequence, the server will lockup solid, and even REISUB
  will not reset it, it requires either hardware reset or power cycle.

  It happens about 3 times a week (it has a function as file server for
  other linux machines), and only started happening after upgrade from
  saucy to trusty.

  Hardware has been tested, with no obvious errors.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1353730/+subscriptions