kernel-packages team mailing list archive

Thread
Date
[Bug 1448269] Re: qemu guest hangs on nested kvm startup with host kernel oops

To: kernel-packages@xxxxxxxxxxxxxxxxxxx
From: Chris J Arges <1448269@xxxxxxxxxxxxxxxxxx>
Date: Thu, 25 Jun 2015 15:44:26 -0000
Reply-to: Bug 1448269 <1448269@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
** Description changed:

+ [Impact]
+ Users of nested KVM may experience the L1 VM hanging when booting an L2 VM. Overall this seems to be due to issues with external interrupts not reaching L1 when L2 gets booted.
+ 
+ [Test Case]
+ Run a nested KVM instance:
+ https://gist.github.com/arges/9d21c6da03a8c10d3980
+ 
+ [Fix]
+ commit 4fa7734c62cdd8c07edd54fa5a5e91482273071a
+ commit f3380ca5d7edb5e31932998ab2e29dfdce39c5ed
+ 
+ --
+ 
  I'm creating a vivid qemu guest on a trusty host with 3.13.0-48-generic
  kernel.  When I start a guest inside that guest, I get the oops below on
  the host while the first guest hangs and must be (virsh) destroyed.
  
  Apr 24 20:40:08 sergeh2 kernel: [1575627.844208] ------------[ cut here ]------------
  Apr 24 20:40:08 sergeh2 kernel: [1575627.844227] WARNING: CPU: 2 PID: 17176 at /build/buildd/linux-3.13.0/arch/x86/kvm/vmx.c:8414 nested_vmx_vmexit+0x11c/0x150 [kvm_intel]()
  Apr 24 20:40:08 sergeh2 kernel: [1575627.844229] Modules linked in: vhost_net vhost macvtap macvlan xts gf128mul xt_conntrack ipt_REJECT ip6table_filter ip6_tables ebtable_nat ebtables veth xt_nat xt_CHECKSUM iptable_mangle ipt_MASQUERADE
  iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack xt_tcpudp bridge stp llc iptable_filter ip_tables x_tables dm_crypt gpio_ich coretemp kvm_intel kvm i7core_edac edac_core lpc_ich shpchp mac_hid serio_raw lp parp
  ort btrfs libcrc32c raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq raid0 multipath linear dm_snapshot raid1 nouveau mxm_wmi video i2c_algo_bit ttm drm_kms_helper drm ahci r8169 libahci mii wmi
  Apr 24 20:40:08 sergeh2 kernel: [1575627.844281] CPU: 2 PID: 17176 Comm: qemu-system-x86 Not tainted 3.13.0-48-generic #80-Ubuntu
  Apr 24 20:40:08 sergeh2 kernel: [1575627.844283] Hardware name: MSI MS-7522/MSI X58 Pro (MS-7522)  , BIOS V8.14B8 11/09/2012
  Apr 24 20:40:08 sergeh2 kernel: [1575627.844286]  0000000000000009 ffff880907561c98 ffffffff81721506 0000000000000000
  Apr 24 20:40:08 sergeh2 kernel: [1575627.844290]  ffff880907561cd0 ffffffff810677dd ffff880bfa808000 0000000000000014
  Apr 24 20:40:08 sergeh2 kernel: [1575627.844293]  ffff8806da7a7000 ffff880bfca9c800 0000000000000000 ffff880907561ce0
  Apr 24 20:40:08 sergeh2 kernel: [1575627.844297] Call Trace:
  Apr 24 20:40:08 sergeh2 kernel: [1575627.844305]  [<ffffffff81721506>] dump_stack+0x45/0x56
  Apr 24 20:40:08 sergeh2 kernel: [1575627.844310]  [<ffffffff810677dd>] warn_slowpath_common+0x7d/0xa0
  Apr 24 20:40:08 sergeh2 kernel: [1575627.844314]  [<ffffffff810678ba>] warn_slowpath_null+0x1a/0x20
  Apr 24 20:40:08 sergeh2 kernel: [1575627.844321]  [<ffffffffa081f8ec>] nested_vmx_vmexit+0x11c/0x150 [kvm_intel]
  Apr 24 20:40:08 sergeh2 kernel: [1575627.844327]  [<ffffffffa081fafd>] vmx_queue_exception+0xfd/0x140 [kvm_intel]
  Apr 24 20:40:08 sergeh2 kernel: [1575627.844347]  [<ffffffffa03b7020>] vcpu_enter_guest+0x9f0/0xce0 [kvm]
  Apr 24 20:40:08 sergeh2 kernel: [1575627.844364]  [<ffffffffa03bb2d8>] kvm_arch_vcpu_ioctl_run+0x1e8/0x460 [kvm]
  Apr 24 20:40:08 sergeh2 kernel: [1575627.844376]  [<ffffffffa03a5042>] kvm_vcpu_ioctl+0x2a2/0x5e0 [kvm]
  Apr 24 20:40:08 sergeh2 kernel: [1575627.844381]  [<ffffffff810aaa38>] ? __wake_up_common+0x58/0x90
  Apr 24 20:40:08 sergeh2 kernel: [1575627.844387]  [<ffffffff811ffc91>] ? fsnotify+0x241/0x320
  Apr 24 20:40:08 sergeh2 kernel: [1575627.844391]  [<ffffffff811d11c0>] do_vfs_ioctl+0x2e0/0x4c0
  Apr 24 20:40:08 sergeh2 kernel: [1575627.844406]  [<ffffffffa03b0504>] ? kvm_on_user_return+0x74/0x80 [kvm]
  Apr 24 20:40:08 sergeh2 kernel: [1575627.844409]  [<ffffffff811d1421>] SyS_ioctl+0x81/0xa0
  Apr 24 20:40:08 sergeh2 kernel: [1575627.844414]  [<ffffffff81731fbd>] system_call_fastpath+0x1a/0x1f
  Apr 24 20:40:08 sergeh2 kernel: [1575627.844416] ---[ end trace 351396e62b6ef224 ]---
  Apr 24 20:48:29 sergeh2 dnsmasq-dhcp[1409]: DHCPREQUEST(lxcbr0) 10.0.3.104 00:16:3e:72:73:32
  
  ProblemType: Bug
  DistroRelease: Ubuntu 14.04
  Package: linux-image-3.13.0-48-generic 3.13.0-48.80
  ProcVersionSignature: Ubuntu 3.13.0-48.80-generic 3.13.11-ckt16
  Uname: Linux 3.13.0-48-generic x86_64
  AlsaDevices:
-  total 0
-  crw-rw---- 1 root audio 116,  1 Apr 10 14:22 seq
-  crw-rw---- 1 root audio 116, 33 Apr 10 14:22 timer
+  total 0
+  crw-rw---- 1 root audio 116,  1 Apr 10 14:22 seq
+  crw-rw---- 1 root audio 116, 33 Apr 10 14:22 timer
  AplayDevices: Error: [Errno 2] No such file or directory: 'aplay'
  ApportVersion: 2.14.1-0ubuntu3.10
  Architecture: amd64
  ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord'
  AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1:
  CRDA: Error: [Errno 2] No such file or directory: 'iw'
  CurrentDmesg: Error: command ['sh', '-c', 'dmesg | comm -13 --nocheck-order /var/log/dmesg -'] failed with exit code 1: comm: /var/log/dmesg: Permission denied
  Date: Fri Apr 24 20:59:31 2015
  IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig'
  MachineType: MSI MS-7522
  PciMultimedia:
-  
+ 
  ProcFB:
-  
+ 
  ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-3.13.0-48-generic root=UUID=d1920c3b-419d-484b-b1f2-5cbc69ef62f5 ro nomodeset intel_pstate=enable nomdmonddf nomdmonisw
  RfKill: Error: [Errno 2] No such file or directory: 'rfkill'
  SourcePackage: linux
  UpgradeStatus: No upgrade log present (probably fresh install)
  WifiSyslog:
-  
+ 
  dmi.bios.date: 11/09/2012
  dmi.bios.vendor: American Megatrends Inc.
  dmi.bios.version: V8.14B8
  dmi.board.asset.tag: To Be Filled By O.E.M.
  dmi.board.name: MSI X58 Pro (MS-7522)
  dmi.board.vendor: MSI
  dmi.board.version: 3.0
  dmi.chassis.asset.tag: To Be Filled By O.E.M.
  dmi.chassis.type: 3
  dmi.chassis.vendor: MICRO-STAR INTERNATIONAL CO.,LTD
  dmi.chassis.version: 3.0
  dmi.modalias: dmi:bvnAmericanMegatrendsInc.:bvrV8.14B8:bd11/09/2012:svnMSI:pnMS-7522:pvr3.0:rvnMSI:rnMSIX58Pro(MS-7522):rvr3.0:cvnMICRO-STARINTERNATIONALCO.,LTD:ct3:cvr3.0:
  dmi.product.name: MS-7522
  dmi.product.version: 3.0
  dmi.sys.vendor: MSI

-- 
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1448269

Title:
  qemu guest hangs on nested kvm startup with host kernel oops

Status in linux package in Ubuntu:
  Fix Released
Status in linux source package in Utopic:
  In Progress

Bug description:
  [Impact]
  Users of nested KVM may experience the L1 VM hanging when booting an L2 VM. Overall this seems to be due to issues with external interrupts not reaching L1 when L2 gets booted.

  [Test Case]
  Run a nested KVM instance:
  https://gist.github.com/arges/9d21c6da03a8c10d3980

  [Fix]
  commit 4fa7734c62cdd8c07edd54fa5a5e91482273071a
  commit f3380ca5d7edb5e31932998ab2e29dfdce39c5ed

  --

  I'm creating a vivid qemu guest on a trusty host with
  3.13.0-48-generic kernel.  When I start a guest inside that guest, I
  get the oops below on the host while the first guest hangs and must be
  (virsh) destroyed.

  Apr 24 20:40:08 sergeh2 kernel: [1575627.844208] ------------[ cut here ]------------
  Apr 24 20:40:08 sergeh2 kernel: [1575627.844227] WARNING: CPU: 2 PID: 17176 at /build/buildd/linux-3.13.0/arch/x86/kvm/vmx.c:8414 nested_vmx_vmexit+0x11c/0x150 [kvm_intel]()
  Apr 24 20:40:08 sergeh2 kernel: [1575627.844229] Modules linked in: vhost_net vhost macvtap macvlan xts gf128mul xt_conntrack ipt_REJECT ip6table_filter ip6_tables ebtable_nat ebtables veth xt_nat xt_CHECKSUM iptable_mangle ipt_MASQUERADE
  iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack xt_tcpudp bridge stp llc iptable_filter ip_tables x_tables dm_crypt gpio_ich coretemp kvm_intel kvm i7core_edac edac_core lpc_ich shpchp mac_hid serio_raw lp parp
  ort btrfs libcrc32c raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq raid0 multipath linear dm_snapshot raid1 nouveau mxm_wmi video i2c_algo_bit ttm drm_kms_helper drm ahci r8169 libahci mii wmi
  Apr 24 20:40:08 sergeh2 kernel: [1575627.844281] CPU: 2 PID: 17176 Comm: qemu-system-x86 Not tainted 3.13.0-48-generic #80-Ubuntu
  Apr 24 20:40:08 sergeh2 kernel: [1575627.844283] Hardware name: MSI MS-7522/MSI X58 Pro (MS-7522)  , BIOS V8.14B8 11/09/2012
  Apr 24 20:40:08 sergeh2 kernel: [1575627.844286]  0000000000000009 ffff880907561c98 ffffffff81721506 0000000000000000
  Apr 24 20:40:08 sergeh2 kernel: [1575627.844290]  ffff880907561cd0 ffffffff810677dd ffff880bfa808000 0000000000000014
  Apr 24 20:40:08 sergeh2 kernel: [1575627.844293]  ffff8806da7a7000 ffff880bfca9c800 0000000000000000 ffff880907561ce0
  Apr 24 20:40:08 sergeh2 kernel: [1575627.844297] Call Trace:
  Apr 24 20:40:08 sergeh2 kernel: [1575627.844305]  [<ffffffff81721506>] dump_stack+0x45/0x56
  Apr 24 20:40:08 sergeh2 kernel: [1575627.844310]  [<ffffffff810677dd>] warn_slowpath_common+0x7d/0xa0
  Apr 24 20:40:08 sergeh2 kernel: [1575627.844314]  [<ffffffff810678ba>] warn_slowpath_null+0x1a/0x20
  Apr 24 20:40:08 sergeh2 kernel: [1575627.844321]  [<ffffffffa081f8ec>] nested_vmx_vmexit+0x11c/0x150 [kvm_intel]
  Apr 24 20:40:08 sergeh2 kernel: [1575627.844327]  [<ffffffffa081fafd>] vmx_queue_exception+0xfd/0x140 [kvm_intel]
  Apr 24 20:40:08 sergeh2 kernel: [1575627.844347]  [<ffffffffa03b7020>] vcpu_enter_guest+0x9f0/0xce0 [kvm]
  Apr 24 20:40:08 sergeh2 kernel: [1575627.844364]  [<ffffffffa03bb2d8>] kvm_arch_vcpu_ioctl_run+0x1e8/0x460 [kvm]
  Apr 24 20:40:08 sergeh2 kernel: [1575627.844376]  [<ffffffffa03a5042>] kvm_vcpu_ioctl+0x2a2/0x5e0 [kvm]
  Apr 24 20:40:08 sergeh2 kernel: [1575627.844381]  [<ffffffff810aaa38>] ? __wake_up_common+0x58/0x90
  Apr 24 20:40:08 sergeh2 kernel: [1575627.844387]  [<ffffffff811ffc91>] ? fsnotify+0x241/0x320
  Apr 24 20:40:08 sergeh2 kernel: [1575627.844391]  [<ffffffff811d11c0>] do_vfs_ioctl+0x2e0/0x4c0
  Apr 24 20:40:08 sergeh2 kernel: [1575627.844406]  [<ffffffffa03b0504>] ? kvm_on_user_return+0x74/0x80 [kvm]
  Apr 24 20:40:08 sergeh2 kernel: [1575627.844409]  [<ffffffff811d1421>] SyS_ioctl+0x81/0xa0
  Apr 24 20:40:08 sergeh2 kernel: [1575627.844414]  [<ffffffff81731fbd>] system_call_fastpath+0x1a/0x1f
  Apr 24 20:40:08 sergeh2 kernel: [1575627.844416] ---[ end trace 351396e62b6ef224 ]---
  Apr 24 20:48:29 sergeh2 dnsmasq-dhcp[1409]: DHCPREQUEST(lxcbr0) 10.0.3.104 00:16:3e:72:73:32

  ProblemType: Bug
  DistroRelease: Ubuntu 14.04
  Package: linux-image-3.13.0-48-generic 3.13.0-48.80
  ProcVersionSignature: Ubuntu 3.13.0-48.80-generic 3.13.11-ckt16
  Uname: Linux 3.13.0-48-generic x86_64
  AlsaDevices:
   total 0
   crw-rw---- 1 root audio 116,  1 Apr 10 14:22 seq
   crw-rw---- 1 root audio 116, 33 Apr 10 14:22 timer
  AplayDevices: Error: [Errno 2] No such file or directory: 'aplay'
  ApportVersion: 2.14.1-0ubuntu3.10
  Architecture: amd64
  ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord'
  AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1:
  CRDA: Error: [Errno 2] No such file or directory: 'iw'
  CurrentDmesg: Error: command ['sh', '-c', 'dmesg | comm -13 --nocheck-order /var/log/dmesg -'] failed with exit code 1: comm: /var/log/dmesg: Permission denied
  Date: Fri Apr 24 20:59:31 2015
  IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig'
  MachineType: MSI MS-7522
  PciMultimedia:

  ProcFB:

  ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-3.13.0-48-generic root=UUID=d1920c3b-419d-484b-b1f2-5cbc69ef62f5 ro nomodeset intel_pstate=enable nomdmonddf nomdmonisw
  RfKill: Error: [Errno 2] No such file or directory: 'rfkill'
  SourcePackage: linux
  UpgradeStatus: No upgrade log present (probably fresh install)
  WifiSyslog:

  dmi.bios.date: 11/09/2012
  dmi.bios.vendor: American Megatrends Inc.
  dmi.bios.version: V8.14B8
  dmi.board.asset.tag: To Be Filled By O.E.M.
  dmi.board.name: MSI X58 Pro (MS-7522)
  dmi.board.vendor: MSI
  dmi.board.version: 3.0
  dmi.chassis.asset.tag: To Be Filled By O.E.M.
  dmi.chassis.type: 3
  dmi.chassis.vendor: MICRO-STAR INTERNATIONAL CO.,LTD
  dmi.chassis.version: 3.0
  dmi.modalias: dmi:bvnAmericanMegatrendsInc.:bvrV8.14B8:bd11/09/2012:svnMSI:pnMS-7522:pvr3.0:rvnMSI:rnMSIX58Pro(MS-7522):rvr3.0:cvnMICRO-STARINTERNATIONALCO.,LTD:ct3:cvr3.0:
  dmi.product.name: MS-7522
  dmi.product.version: 3.0
  dmi.sys.vendor: MSI

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1448269/+subscriptions
References

[Bug 1448269] [NEW] qemu guest hangs on nested kvm startup with host kernel oops
From: Serge Hallyn, 2015-04-24