kernel-packages team mailing list archive

Thread
Date
[Bug 1513157] Re: Kernel bug caused by AUFS

To: kernel-packages@xxxxxxxxxxxxxxxxxxx
From: Joseph Salisbury <joseph.salisbury@xxxxxxxxxxxxx>
Date: Wed, 04 Nov 2015 18:02:02 -0000
Reply-to: Bug 1513157 <1513157@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
Did this issue start happening after an update/upgrade?  Was there a
prior kernel version where you were not having this particular problem?

Would it be possible for you to test the latest upstream kernel? Refer
to https://wiki.ubuntu.com/KernelMainlineBuilds . Please test the latest
v4.3 kernel[0].

If this bug is fixed in the mainline kernel, please add the following
tag 'kernel-fixed-upstream'.

If the mainline kernel does not fix this bug, please add the tag:
'kernel-bug-exists-upstream'.

Once testing of the upstream kernel is complete, please mark this bug as
"Confirmed".


Thanks in advance.

[0] http://kernel.ubuntu.com/~kernel-ppa/mainline/v4.3-unstable/


** Changed in: linux (Ubuntu)
   Importance: Undecided => Medium

** Changed in: linux (Ubuntu)
       Status: Confirmed => Incomplete

-- 
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1513157

Title:
  Kernel bug caused by AUFS

Status in linux package in Ubuntu:
  Incomplete

Bug description:
  Basically, when I write "a lot" of data while in a docker container I
  can consistently crash my machine.  Here are some logs that I gathered
  when trying to figure this out:

  Nov 04 08:58:54 zfp kernel: BUG: unable to handle kernel NULL pointer dereference at 0000000000000098
  Nov 04 08:58:54 zfp kernel: IP: [<ffffffffc0b5a93b>] au_write_pre+0x8b/0x110 [aufs]
  Nov 04 08:58:54 zfp kernel: PGD 3b64ba067 PUD 3b5e32067 PMD 0 
  Nov 04 08:58:54 zfp kernel: Oops: 0000 [#1] SMP 
  Nov 04 08:58:54 zfp kernel: Modules linked in: rfcomm veth xt_conntrack ipt_MASQUERADE nf_nat_masquerade_ipv4 xt_multiport iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 xt_addrtype iptable_filter ip_tables x_tables nf_nat nf_conntrack br_netfilter bridge stp llc aufs bnep arc4 snd_hda_codec_hdmi intel_rapl iosf_mbi x86_pkg_temp_thermal intel_powerclamp coretemp snd_hda_codec_realtek snd_hda_codec_generic kvm_intel kvm snd_hda_intel iwlmvm snd_hda_codec mac80211 snd_hda_core snd_hwdep snd_pcm uvcvideo iwlwifi serio_raw snd_seq_midi lpc_ich snd_seq_midi_event videobuf2_vmalloc videobuf2_memops thinkpad_acpi snd_rawmidi videobuf2_core v4l2_common nvram videodev cfg80211 media snd_seq joydev input_leds snd_seq_device snd_timer btusb btrtl btbcm btintel bluetooth rtsx_pci_ms snd memstick mei_me mei soundcore
  Nov 04 08:58:54 zfp kernel:  ie31200_edac edac_core shpchp mac_hid sch_fq_codel nfsd auth_rpcgss nfs_acl lockd grace sunrpc parport_pc ppdev lp parport autofs4 drbg ansi_cprng algif_skcipher af_alg dm_crypt hid_generic hid_thingm usbhid hid rtsx_pci_sdmmc crct10dif_pclmul crc32_pclmul ghash_clmulni_intel aesni_intel aes_x86_64 lrw gf128mul glue_helper ablk_helper cryptd psmouse nouveau i915 ahci libahci rtsx_pci mxm_wmi e1000e ttm i2c_algo_bit drm_kms_helper ptp drm pps_core wmi video
  Nov 04 08:58:54 zfp kernel: CPU: 3 PID: 23904 Comm: mysqld Not tainted 4.2.0-16-generic #19-Ubuntu
  Nov 04 08:58:54 zfp kernel: Hardware name: LENOVO 20AN006LUS/20AN006LUS, BIOS GLET77WW (2.31 ) 01/27/2015
  Nov 04 08:58:54 zfp kernel: task: ffff8803b5aae040 ti: ffff8803b71e4000 task.ti: ffff8803b71e4000
  Nov 04 08:58:54 zfp kernel: RIP: 0010:[<ffffffffc0b5a93b>]  [<ffffffffc0b5a93b>] au_write_pre+0x8b/0x110 [aufs]
  Nov 04 08:58:54 zfp kernel: RSP: 0018:ffff8803b71e7c58  EFLAGS: 00010286
  Nov 04 08:58:54 zfp kernel: RAX: 0000000000000000 RBX: ffff8803b72e7500 RCX: 0000000000000000
  Nov 04 08:58:54 zfp kernel: RDX: 0000000100000000 RSI: 0000000000000000 RDI: ffff8803becff208
  Nov 04 08:58:54 zfp kernel: RBP: ffff8803b71e7cd8 R08: ffff88042b2d3200 R09: ffff8803a71ea458
  Nov 04 08:58:54 zfp kernel: R10: 00007ff42b18c000 R11: ffff88042acfcb00 R12: ffff8803b71e7cf0
  Nov 04 08:58:54 zfp kernel: R13: 0000000000000001 R14: ffff8803a7344300 R15: ffff8803b72e6400
  Nov 04 08:58:54 zfp kernel: FS:  00007ff3fa5a6700(0000) GS:ffff88043e2c0000(0000) knlGS:0000000000000000
  Nov 04 08:58:54 zfp kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
  Nov 04 08:58:54 zfp kernel: CR2: 0000000000000098 CR3: 00000003b64b9000 CR4: 00000000001406e0
  Nov 04 08:58:54 zfp kernel: Stack:
  Nov 04 08:58:54 zfp kernel:  ffff8803a7344300 0000070300000004 ffff880397a5a600 ffff8803b7980e00
  Nov 04 08:58:54 zfp kernel:  0000000000000000 ffff8803a7344480 ffff880397a5a3c0 ffff88042b2d3200
  Nov 04 08:58:54 zfp kernel:  ffff8803b5aae040 00000000ca877142 ffff8803b71e7cd8 ffff8803a71ea458
  Nov 04 08:58:54 zfp kernel: Call Trace:
  Nov 04 08:58:54 zfp kernel:  [<ffffffffc0b5b1ac>] aufs_write_iter+0x4c/0x100 [aufs]
  Nov 04 08:58:54 zfp kernel:  [<ffffffff81325abd>] ? security_file_permission+0x3d/0xc0
  Nov 04 08:58:54 zfp kernel:  [<ffffffffc0b5b160>] ? aufs_splice_write+0x110/0x110 [aufs]
  Nov 04 08:58:54 zfp kernel:  [<ffffffff8124a44f>] aio_run_iocb+0x27f/0x2e0
  Nov 04 08:58:54 zfp kernel:  [<ffffffff810e080d>] ? call_rcu_sched+0x1d/0x20
  Nov 04 08:58:54 zfp kernel:  [<ffffffff812bfe4e>] ? ext4_discard_preallocations+0x38e/0x4b0
  Nov 04 08:58:54 zfp kernel:  [<ffffffff8124b278>] ? do_io_submit+0x198/0x4f0
  Nov 04 08:58:54 zfp kernel:  [<ffffffff811de4ba>] ? kmem_cache_alloc+0x1ca/0x200
  Nov 04 08:58:54 zfp kernel:  [<ffffffff8124b32b>] do_io_submit+0x24b/0x4f0
  Nov 04 08:58:54 zfp kernel:  [<ffffffff8124b5e0>] SyS_io_submit+0x10/0x20
  Nov 04 08:58:54 zfp kernel:  [<ffffffff817ef9f2>] entry_SYSCALL_64_fastpath+0x16/0x75
  Nov 04 08:58:54 zfp kernel: Code: 74 10 49 8b 87 d0 00 00 00 0f b6 40 30 41 88 44 24 08 49 8b 87 d0 00 00 00 48 8b 58 38 f0 48 ff 43 38 4d 85 e4 74 0f 48 8b 43 20 <48> 8b 80 98 00 00 00 49 89 04 24 45 85 ed 75 63 31 f6 4c 89 f7 
  Nov 04 08:58:54 zfp kernel: RIP  [<ffffffffc0b5a93b>] au_write_pre+0x8b/0x110 [aufs]
  Nov 04 08:58:54 zfp kernel:  RSP <ffff8803b71e7c58>
  Nov 04 08:58:54 zfp kernel: CR2: 0000000000000098
  Nov 04 08:58:54 zfp kernel: ---[ end trace 8adeb327cb22564d ]---
  Nov 04 08:58:57 zfp kernel: BUG: unable to handle kernel paging request at 0000000000003078
  Nov 04 08:58:57 zfp kernel: IP: [<ffffffff8122fc59>] vfs_fsync_range+0x9/0xb0
  Nov 04 08:58:57 zfp kernel: PGD 3b64ba067 PUD 3b5e32067 PMD 0 
  Nov 04 08:58:57 zfp kernel: Oops: 0000 [#2] SMP 
  Nov 04 08:58:57 zfp kernel: Modules linked in: rfcomm veth xt_conntrack ipt_MASQUERADE nf_nat_masquerade_ipv4 xt_multiport iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 xt_addrtype iptable_filter ip_tables x_tables nf_nat nf_conntrack br_netfilter bridge stp llc aufs bnep arc4 snd_hda_codec_hdmi intel_rapl iosf_mbi x86_pkg_temp_thermal intel_powerclamp coretemp snd_hda_codec_realtek snd_hda_codec_generic kvm_intel kvm snd_hda_intel iwlmvm snd_hda_codec mac80211 snd_hda_core snd_hwdep snd_pcm uvcvideo iwlwifi serio_raw snd_seq_midi lpc_ich snd_seq_midi_event videobuf2_vmalloc videobuf2_memops thinkpad_acpi snd_rawmidi videobuf2_core v4l2_common nvram videodev cfg80211 media snd_seq joydev input_leds snd_seq_device snd_timer btusb btrtl btbcm btintel bluetooth rtsx_pci_ms snd memstick mei_me mei soundcore
  Nov 04 08:58:57 zfp kernel:  ie31200_edac edac_core shpchp mac_hid sch_fq_codel nfsd auth_rpcgss nfs_acl lockd grace sunrpc parport_pc ppdev lp parport autofs4 drbg ansi_cprng algif_skcipher af_alg dm_crypt hid_generic hid_thingm usbhid hid rtsx_pci_sdmmc crct10dif_pclmul crc32_pclmul ghash_clmulni_intel aesni_intel aes_x86_64 lrw gf128mul glue_helper ablk_helper cryptd psmouse nouveau i915 ahci libahci rtsx_pci mxm_wmi e1000e ttm i2c_algo_bit drm_kms_helper ptp drm pps_core wmi video
  Nov 04 08:58:57 zfp kernel: CPU: 1 PID: 23902 Comm: mysqld Tainted: G      D         4.2.0-16-generic #19-Ubuntu
  Nov 04 08:58:57 zfp kernel: Hardware name: LENOVO 20AN006LUS/20AN006LUS, BIOS GLET77WW (2.31 ) 01/27/2015
  Nov 04 08:58:57 zfp kernel: task: ffff8803b5aac4c0 ti: ffff8803b64d4000 task.ti: ffff8803b64d4000
  Nov 04 08:58:57 zfp kernel: RIP: 0010:[<ffffffff8122fc59>]  [<ffffffff8122fc59>] vfs_fsync_range+0x9/0xb0
  Nov 04 08:58:57 zfp kernel: RSP: 0018:ffff8803b64d7f00  EFLAGS: 00010246
  Nov 04 08:58:57 zfp kernel: RAX: 0000000000003000 RBX: ffff8803b72e6401 RCX: 0000000000000000
  Nov 04 08:58:57 zfp kernel: RDX: 7fffffffffffffff RSI: 0000000000000000 RDI: ffff8803b72e6400
  Nov 04 08:58:57 zfp kernel: RBP: ffff8803b64d7f38 R08: 0000100000000004 R09: 0000000000000b90
  Nov 04 08:58:57 zfp kernel: R10: ffff8803b72e6438 R11: ffff88042acfcb00 R12: ffff8803b72e6400
  Nov 04 08:58:57 zfp kernel: R13: 0000000000000000 R14: 00007ff3fcff3cf0 R15: 0000000000000172
  Nov 04 08:58:57 zfp kernel: FS:  00007ff3fcff4700(0000) GS:ffff88043e240000(0000) knlGS:0000000000000000
  Nov 04 08:58:57 zfp kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
  Nov 04 08:58:57 zfp kernel: CR2: 000000000040b410 CR3: 00000003b64b9000 CR4: 00000000001406e0
  Nov 04 08:58:57 zfp kernel: Stack:
  Nov 04 08:58:57 zfp kernel:  ffffffff8122fd5d 0000000000000000 000000007c494ce8 0000000000000032
  Nov 04 08:58:57 zfp kernel:  0000000000000172 0000000000000000 0000000051eb851f ffff8803b64d7f48
  Nov 04 08:58:57 zfp kernel:  ffffffff81230000 00007ff3fcff3cd0 ffffffff817ef9f2 00000000563a394e
  Nov 04 08:58:57 zfp kernel: Call Trace:
  Nov 04 08:58:57 zfp kernel:  [<ffffffff8122fd5d>] ? do_fsync+0x3d/0x70
  Nov 04 08:58:57 zfp kernel:  [<ffffffff81230000>] SyS_fsync+0x10/0x20
  Nov 04 08:58:57 zfp kernel:  [<ffffffff817ef9f2>] entry_SYSCALL_64_fastpath+0x16/0x75
  Nov 04 08:58:57 zfp kernel: Code: 04 25 28 00 00 00 75 07 48 83 c4 18 5b 5d c3 e8 5e b5 e4 ff 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 48 8b 47 28 <48> 8b 40 78 48 85 c0 0f 84 8b 00 00 00 55 48 89 e5 41 56 41 55 
  Nov 04 08:58:57 zfp kernel: RIP  [<ffffffff8122fc59>] vfs_fsync_range+0x9/0xb0
  Nov 04 08:58:57 zfp kernel:  RSP <ffff8803b64d7f00>
  Nov 04 08:58:57 zfp kernel: CR2: 0000000000003078
  Nov 04 08:58:57 zfp kernel: ---[ end trace 8adeb327cb22564e ]---

  I am pretty sure that if you were to just create a container and write
  /dev/urandom to some file at a reasonable rate you could repro this.
  I haven't tried because this is my work machine and I'm having trouble
  justifying crashing my machine a 5th time today :)

  Let me know if there are more details you need.

  Thanks.

  ProblemType: Bug
  DistroRelease: Ubuntu 15.10
  Package: linux-image-4.2.0-16-generic 4.2.0-16.19
  ProcVersionSignature: Ubuntu 4.2.0-16.19-generic 4.2.3
  Uname: Linux 4.2.0-16-generic x86_64
  ApportVersion: 2.19.1-0ubuntu4
  Architecture: amd64
  Date: Wed Nov  4 09:27:07 2015
  EcryptfsInUse: Yes
  HibernationDevice: RESUME=UUID=729dd129-ad72-4d34-87d2-b698d6f581ae
  InstallationDate: Installed on 2015-06-16 (140 days ago)
  InstallationMedia: Ubuntu 15.04 "Vivid Vervet" - Release amd64 (20150422)
  MachineType: LENOVO 20AN006LUS
  ProcFB: 0 inteldrmfb
  ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-4.2.0-16-generic root=/dev/mapper/ubuntu--vg-root ro quiet splash vt.handoff=7
  RelatedPackageVersions:
   linux-restricted-modules-4.2.0-16-generic N/A
   linux-backports-modules-4.2.0-16-generic  N/A
   linux-firmware                            1.149
  SourcePackage: linux
  UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev'
  UpgradeStatus: Upgraded to wily on 2015-11-02 (1 days ago)
  dmi.bios.date: 01/27/2015
  dmi.bios.vendor: LENOVO
  dmi.bios.version: GLET77WW (2.31 )
  dmi.board.asset.tag: Not Available
  dmi.board.name: 20AN006LUS
  dmi.board.vendor: LENOVO
  dmi.board.version: 0B98401 WIN
  dmi.chassis.asset.tag: No Asset Information
  dmi.chassis.type: 10
  dmi.chassis.vendor: LENOVO
  dmi.chassis.version: Not Available
  dmi.modalias: dmi:bvnLENOVO:bvrGLET77WW(2.31):bd01/27/2015:svnLENOVO:pn20AN006LUS:pvrThinkPadT440p:rvnLENOVO:rn20AN006LUS:rvr0B98401WIN:cvnLENOVO:ct10:cvrNotAvailable:
  dmi.product.name: 20AN006LUS
  dmi.product.version: ThinkPad T440p
  dmi.sys.vendor: LENOVO

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1513157/+subscriptions
References

[Bug 1513157] [NEW] Kernel bug caused by AUFS
From: frew, 2015-11-04