kernel-packages team mailing list archive

[Salvatore Orlando <1273386@xxxxxxxxxxxxxxxxxx>]

The kernel bug manifests when mounting of nbd devices is combined with ip namespace operations.
Using openstack it can be reproduced only with the following configuration:
- compute service must run on the same node as the dhcp-agent and/or the l3-agent
- file injection should be turned on: libvirt.inject_partition != -2
- key injection should be enabled: libvirt.inject_key = True
- config drive should be disabled: force_config_drive = 'False' or empty string or None

If these conditions are met, nbd mount will be used to inject the key
into the instance. This will trigger something in the kernel which
subsequently will cause the crash in a process running in a network
namespace. After this crash nbd mount won't work anymore.

Using openstack the crash can be reproduced within a few minutes with
the scripts available here: https://gist.github.com/salv-orlando/8715991

- keep_booting_stuff.sh creates and destroys vms continuously, ensuring a key is always injected
- stress_me_to_death creates network namespaces, launches a process in them, kills the process and then the namespace

The two scripts require openstack and should be executed concurrently.
Crashdump data will posted as soon as possible.

It should not be too hard to provide a script that reproduces the issue
and is indepedent from openstack.

** Changed in: neutron
       Status: New => Triaged

-- 
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1273386

Title:
  Neutron namespace metadata proxy triggers kernel crash on Ubuntu
  12.04/3.2 kernel

Status in OpenStack Neutron (virtual network service):
  Triaged
Status in OpenStack Compute (Nova):
  New
Status in “linux” package in Ubuntu:
  Incomplete

Bug description:
  In the past 9 days we have been seeing very frequent occurences of
  this kernel crash: http://paste.openstack.org/show/61869/

  Even if the particular crash pasted here is triggered by dnsmasq, in
  almost all cases the crash is actually triggered by the neutron metada
  proxy.

  This also affects nova badly since this issue, which appears namespace
  related, results in a hang while mounting the ndb device for key
  injection.

  logstash query:
  http://logstash.openstack.org/#eyJzZWFyY2giOiJcImtlcm5lbCBCVUcgYXQgL2J1aWxkL2J1aWxkZC9saW51eC0zLjIuMC9mcy9idWZmZXIuYzoyOTE3XCIgYW5kIGZpbGVuYW1lOnN5c2xvZy50eHQiLCJmaWVsZHMiOltdLCJvZmZzZXQiOjAsInRpbWVmcmFtZSI6ImN1c3RvbSIsImdyYXBobW9kZSI6ImNvdW50IiwidGltZSI6eyJmcm9tIjoiMjAxNC0wMS0xNlQxODo1MDo0OCswMDowMCIsInRvIjoiMjAxNC0wMS0yN1QxOToxNjoxMSswMDowMCIsInVzZXJfaW50ZXJ2YWwiOiIwIn0sInN0YW1wIjoxMzkwODUwMzI2ODY0fQ==

  We have seen about 398 hits since the bug started to manifest.
  Decreased hit rate in the past few days is due to less neutron patches being pushed.

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1273386/+subscriptions