yahoo-eng-team team mailing list archive

[Thierry Carrez <thierry.carrez+lp@xxxxxxxxx>]

** Changed in: neutron
       Status: Fix Committed => Fix Released

** Changed in: neutron
    Milestone: None => juno-1

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1294254

Title:
  dhcp cannot find tap device exceptions

Status in OpenStack Neutron (virtual network service):
  Fix Released

Bug description:
  This happens in tempest tests a lot:

  http://logstash.openstack.org/#eyJzZWFyY2giOiJtZXNzYWdlOiBcImNhbm5vdCBmaW5kIGRldmljZVwiIEFORCBmaWxlbmFtZTpcImxvZ3Mvc2NyZWVuLXEtZGhjcC50eHRcIiBBTkQgYnVpbGRfYnJhbmNoOlwibWFzdGVyXCIgQU5EIHByb2plY3Q6XCJvcGVuc3RhY2svbmV1dHJvblwiIiwiZmllbGRzIjpbXSwib2Zmc2V0IjowLCJ0aW1lZnJhbWUiOiI0MzIwMCIsImdyYXBobW9kZSI6ImNvdW50IiwidGltZSI6eyJ1c2VyX2ludGVydmFsIjowfSwic3RhbXAiOjEzOTUwNTgyMTI3OTF9

  The _set_default_route function is called when the driver is called
  with 'enable' or 'reload_allocations'. It calls the get_dhcp_port rpc
  which tries to get the previously created dhcp port for the network
  and host. But sometimes that port has been deleted by this time, and
  then get_dhcp_port actually creates a new one. This will have a
  different uuid than the original, so the tap name will be different
  and the call to list the routes on it gives an exception as that tap
  was never created.

  It is not clear from the code why _set_default_route needs to do this.

  
  Example from call_driver reload_allocations
  http://logs.openstack.org/12/59212/12/check/check-tempest-dsvm-neutron-full/e3c77d0/logs/screen-q-dhcp.txt.gz#_2014-03-17_10_16_03_166

  [req-024b81f2-eac3-490f-920d-b9fa1f70f51e None] Unable to reload_allocations dhcp for 9edab258-c056-40e3-a340-1c58622380bf.
  Traceback (most recent call last):
    File "/opt/stack/new/neutron/neutron/agent/dhcp_agent.py", line 127, in call_driver
      getattr(driver, action)(**action_kwargs)
    File "/opt/stack/new/neutron/neutron/agent/linux/dhcp.py", line 400, in reload_allocations
      self.device_manager.update(self.network)
    File "/opt/stack/new/neutron/neutron/agent/linux/dhcp.py", line 810, in update
      self._set_default_route(network)
    File "/opt/stack/new/neutron/neutron/agent/linux/dhcp.py", line 669, in _set_default_route
      gateway = device.route.get_gateway()
    File "/opt/stack/new/neutron/neutron/agent/linux/ip_lib.py", line 388, in get_gateway
      *filters).split('\n')
    File "/opt/stack/new/neutron/neutron/agent/linux/ip_lib.py", line 211, in _run
      return self._parent._run(kwargs.get('options', []), self.COMMAND, args)
    File "/opt/stack/new/neutron/neutron/agent/linux/ip_lib.py", line 52, in _run
      return self._as_root(options, command, args)
    File "/opt/stack/new/neutron/neutron/agent/linux/ip_lib.py", line 70, in _as_root
      namespace)
    File "/opt/stack/new/neutron/neutron/agent/linux/ip_lib.py", line 81, in _execute
      root_helper=root_helper)
    File "/opt/stack/new/neutron/neutron/agent/linux/utils.py", line 76, in execute
      raise RuntimeError(m)
  RuntimeError: 
  Command: ['sudo', '/usr/local/bin/neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip', 'netns', 'exec', 'qdhcp-9edab258-c056-40e3-a340-1c58622380bf', 'ip', 'route', 'list', 'dev', 'tapa01643ce-47']
  Exit code: 1
  Stdout: ''
  Stderr: 'Cannot find device "tapa01643ce-47"\n'

  
  Example from call_driver 'enable'
  http://logs.openstack.org/12/59212/12/check/check-tempest-dsvm-neutron-full/e3c77d0/logs/screen-q-dhcp.txt.gz#_2014-03-17_10_16_03_166

  [req-e5cdbe16-076d-4012-80af-7873ff14cdeb None] Unable to enable dhcp for 3236628d-4ccd-4c29-a2b6-ca4752725874.
  Traceback (most recent call last):
    File "/opt/stack/new/neutron/neutron/agent/dhcp_agent.py", line 127, in call_driver
      getattr(driver, action)(**action_kwargs)
    File "/opt/stack/new/neutron/neutron/agent/linux/dhcp.py", line 166, in enable
      reuse_existing=True)
    File "/opt/stack/new/neutron/neutron/agent/linux/dhcp.py", line 803, in setup
      self._set_default_route(network)
    File "/opt/stack/new/neutron/neutron/agent/linux/dhcp.py", line 669, in _set_default_route
      gateway = device.route.get_gateway()
    File "/opt/stack/new/neutron/neutron/agent/linux/ip_lib.py", line 388, in get_gateway
      *filters).split('\n')
    File "/opt/stack/new/neutron/neutron/agent/linux/ip_lib.py", line 211, in _run
      return self._parent._run(kwargs.get('options', []), self.COMMAND, args)
    File "/opt/stack/new/neutron/neutron/agent/linux/ip_lib.py", line 52, in _run
      return self._as_root(options, command, args)
    File "/opt/stack/new/neutron/neutron/agent/linux/ip_lib.py", line 70, in _as_root
      namespace)
    File "/opt/stack/new/neutron/neutron/agent/linux/ip_lib.py", line 81, in _execute
      root_helper=root_helper)
    File "/opt/stack/new/neutron/neutron/agent/linux/utils.py", line 76, in execute
      raise RuntimeError(m)
  RuntimeError: 
  Command: ['sudo', '/usr/local/bin/neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip', 'netns', 'exec', 'qdhcp-3236628d-4ccd-4c29-a2b6-ca4752725874', 'ip', 'route', 'list', 'dev', 'tap88655de5-5f']
  Exit code: 1
  Stdout: ''
  Stderr: 'Cannot find device "tap88655de5-5f"\n'

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1294254/+subscriptions