kernel-packages team mailing list archive
-
kernel-packages team
-
Mailing list archive
-
Message #77611
[Bug 1350522] Re: EC2 kernel crash due to vmalloc
Ok, finally it seems I found the problem. It seems Xen setup code is accidentally setting up the kernel page tables in a way that causes the last 2G of memory to be identically mapped (kernel mappings). This would just work normally because the page table that covers the first 1G would correctly be clean for the second 512M (which started the module space before). If modules ever reached more than 512M of memory this just would have happened with the old layout, too.
But now that the kernel image is increased to 1G, we start to use the bad page table immediately.
Sent this patch upstream (right now only tested with the new layout)
** Patch added: "0001-x86-xen-Fix-setup-of-64bit-kernel-pagetables.patch"
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1350522/+attachment/4189340/+files/0001-x86-xen-Fix-setup-of-64bit-kernel-pagetables.patch
--
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1350522
Title:
EC2 kernel crash due to vmalloc
Status in “linux” package in Ubuntu:
Confirmed
Status in “linux” source package in Utopic:
Confirmed
Bug description:
During Alpha-2 automated testing, I saw the following in a log:
19:35:02 [ 2.475810] systemd-udevd[95]: starting version 204
19:35:02 [ 2.547049] ------------[ cut here ]------------
19:35:02 [ 2.547065] WARNING: CPU: 0 PID: 97 at /build/buildd/linux-3.16.0/mm/vmalloc.c:128 vmap_page_range_noflush+0x2d1/0x370()
19:35:02 [ 2.547069] Modules linked in:
19:35:02 [ 2.547073] CPU: 0 PID: 97 Comm: systemd-udevd Not tainted 3.16.0-6-generic #11-Ubuntu
19:35:02 [ 2.547077] 0000000000000009 ffff880002defb98 ffffffff81755538 0000000000000000
19:35:02 [ 2.547082] ffff880002defbd0 ffffffff8106bb0d ffff88000400ec88 0000000000000001
19:35:02 [ 2.547086] ffff880002fcfb00 ffffffffc0391000 0000000000000000 ffff880002defbe0
19:35:02 [ 2.547090] Call Trace:
19:35:02 [ 2.547096] [<ffffffff81755538>] dump_stack+0x45/0x56
19:35:02 [ 2.547101] [<ffffffff8106bb0d>] warn_slowpath_common+0x7d/0xa0
19:35:02 [ 2.547104] [<ffffffff8106bbea>] warn_slowpath_null+0x1a/0x20
19:35:02 [ 2.547108] [<ffffffff81197c31>] vmap_page_range_noflush+0x2d1/0x370
19:35:02 [ 2.547112] [<ffffffff81197cfe>] map_vm_area+0x2e/0x40
19:35:02 [ 2.547115] [<ffffffff8119a058>] __vmalloc_node_range+0x188/0x280
19:35:02 [ 2.547120] [<ffffffff810e92b4>] ? module_alloc_update_bounds+0x14/0x70
19:35:02 [ 2.547124] [<ffffffff810e92b4>] ? module_alloc_update_bounds+0x14/0x70
19:35:02 [ 2.547129] [<ffffffff8104f294>] module_alloc+0x74/0xd0
19:35:02 [ 2.547132] [<ffffffff810e92b4>] ? module_alloc_update_bounds+0x14/0x70
19:35:02 [ 2.547135] [<ffffffff810e92b4>] module_alloc_update_bounds+0x14/0x70
19:35:02 [ 2.547146] [<ffffffff810e9a6c>] layout_and_allocate+0x74c/0xc70
19:35:02 [ 2.547149] [<ffffffff810ea063>] load_module+0xd3/0x1b70
19:35:02 [ 2.547154] [<ffffffff811cfeb1>] ? vfs_read+0xf1/0x170
19:35:02 [ 2.547157] [<ffffffff810e7aa1>] ? copy_module_from_fd.isra.46+0x121/0x180
19:35:02 [ 2.547161] [<ffffffff810ebc76>] SyS_finit_module+0x86/0xb0
19:35:02 [ 2.547167] [<ffffffff8175de7f>] tracesys+0xe1/0xe6
19:35:02 [ 2.547169] ---[ end trace 8a5de7fc66e75fe4 ]---
19:35:02 [ 2.547172] vmalloc: allocation failure, allocated 20480 of 24576 bytes
19:35:02 [ 2.547175] systemd-udevd: page allocation failure: order:0, mode:0xd2
19:35:02 [ 2.547180] CPU: 0 PID: 97 Comm: systemd-udevd Tainted: G W 3.16.0-6-generic #11-Ubuntu
19:35:02 [ 2.547183] ffffffff81a88bc0 ffff880002defc08 ffffffff81755538 00000000000000d2
19:35:02 [ 2.547187] ffff880002defc90 ffffffff811642bf ffffffff81a88bc0 ffff880002defc28
19:35:02 [ 2.547191] 00003fff00000018 ffff880002defca0 ffff880002defc40 0000000000000163
19:35:02 [ 2.547195] Call Trace:
19:35:02 [ 2.547197] [<ffffffff81755538>] dump_stack+0x45/0x56
19:35:02 [ 2.547202] [<ffffffff811642bf>] warn_alloc_failed+0xdf/0x130
19:35:02 [ 2.547207] [<ffffffff8119a118>] __vmalloc_node_range+0x248/0x280
19:35:02 [ 2.547210] [<ffffffff810e92b4>] ? module_alloc_update_bounds+0x14/0x70
19:35:02 [ 2.547214] [<ffffffff810e92b4>] ? module_alloc_update_bounds+0x14/0x70
19:35:02 [ 2.547217] [<ffffffff8104f294>] module_alloc+0x74/0xd0
19:35:02 [ 2.547220] [<ffffffff810e92b4>] ? module_alloc_update_bounds+0x14/0x70
19:35:02 [ 2.547224] [<ffffffff810e92b4>] module_alloc_update_bounds+0x14/0x70
19:35:02 [ 2.547229] [<ffffffff810e9a6c>] layout_and_allocate+0x74c/0xc70
19:35:02 [ 2.547232] [<ffffffff810ea063>] load_module+0xd3/0x1b70
19:35:02 [ 2.547235] [<ffffffff811cfeb1>] ? vfs_read+0xf1/0x170
19:35:02 [ 2.547238] [<ffffffff810e7aa1>] ? copy_module_from_fd.isra.46+0x121/0x180
19:35:02 [ 2.547242] [<ffffffff810ebc76>] SyS_finit_module+0x86/0xb0
19:35:02 [ 2.547246] [<ffffffff8175de7f>] tracesys+0xe1/0xe6
19:35:02 [ 2.547248] Mem-Info:
---
AlsaDevices:
total 0
crw-rw---- 1 root audio 116, 1 Jul 30 20:19 seq
crw-rw---- 1 root audio 116, 33 Jul 30 20:19 timer
AplayDevices: Error: [Errno 2] No such file or directory
ApportVersion: 2.14.4-0ubuntu2
Architecture: amd64
ArecordDevices: Error: [Errno 2] No such file or directory
AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1:
CRDA: Error: [Errno 2] No such file or directory
CurrentDmesg:
[ 42.901579] systemd-logind[1378]: New seat seat0.
[ 42.932887] systemd-logind[1378]: New session 1 of user ubuntu.
DistroRelease: Ubuntu 14.10
Ec2AMI: ami-c571d8d8
Ec2AMIManifest: (unknown)
Ec2AvailabilityZone: sa-east-1b
Ec2InstanceType: m3.medium
Ec2Kernel: aki-5553f448
Ec2Ramdisk: unavailable
IwConfig: Error: [Errno 2] No such file or directory
Lspci:
Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99
Package: linux-image-virtual 3.16.0.5.6
PackageArchitecture: amd64
PciMultimedia:
ProcFB:
ProcKernelCmdLine: root=LABEL=cloudimg-rootfs ro console=hvc0
ProcModules:
ProcVersionSignature: User Name 3.16.0-5.10-generic 3.16.0-rc6
RelatedPackageVersions:
linux-restricted-modules-3.16.0-5-generic N/A
linux-backports-modules-3.16.0-5-generic N/A
linux-firmware N/A
RfKill: Error: [Errno 2] No such file or directory
Tags: utopic ec2-images third-party-packages
Uname: Linux 3.16.0-5-generic x86_64
UpgradeStatus: No upgrade log present (probably fresh install)
UserGroups: adm audio cdrom dialout dip floppy netdev plugdev sudo video
_MarkForUpload: True
---
AlsaDevices:
total 0
crw-rw---- 1 root audio 116, 1 Jul 31 20:13 seq
crw-rw---- 1 root audio 116, 33 Jul 31 20:13 timer
AplayDevices: Error: [Errno 2] No such file or directory
ApportVersion: 2.14.5-0ubuntu1
Architecture: amd64
ArecordDevices: Error: [Errno 2] No such file or directory
AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1:
CRDA: Error: [Errno 2] No such file or directory
CurrentDmesg:
[ 17.975584] systemd-logind[1248]: New seat seat0.
[ 17.993902] systemd-logind[1248]: New session 1 of user ubuntu.
DistroRelease: Ubuntu 14.10
Ec2AMI: ami-b7ce1dc0
Ec2AMIManifest: (unknown)
Ec2AvailabilityZone: eu-west-1c
Ec2InstanceType: m3.medium
Ec2Kernel: aki-52a34525
Ec2Ramdisk: unavailable
IwConfig: Error: [Errno 2] No such file or directory
Lspci:
Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99
Package: linux-image-virtual 3.16.0.6.7
PackageArchitecture: amd64
PciMultimedia:
ProcFB:
ProcKernelCmdLine: root=LABEL=cloudimg-rootfs ro console=hvc0 initcall_debug debug ignore_loglevel LOGLEVEL=8
ProcModules:
ProcVersionSignature: Ubuntu 3.16.0-6.11-generic 3.16.0-rc7
RelatedPackageVersions:
linux-restricted-modules-3.16.0-6-generic N/A
linux-backports-modules-3.16.0-6-generic N/A
linux-firmware N/A
RfKill: Error: [Errno 2] No such file or directory
Tags: utopic ec2-images
Uname: Linux 3.16.0-6-generic x86_64
UpgradeStatus: No upgrade log present (probably fresh install)
UserGroups:
_MarkForUpload: True
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1350522/+subscriptions
References
