yahoo-eng-team team mailing list archive

[Swaminathan Vasudevan <swaminathan.vasudevan@xxxxxx>]

** Summary changed:

- DVR qrouter created without OVS qr device
+ Cannot find device "qr-" error message found in logtrace with DVR routers while trying to update arp entry

** Changed in: neutron
       Status: Expired => Confirmed

** Changed in: neutron
     Assignee: (unassigned) => Swaminathan Vasudevan (swaminathan-vasudevan)

** Tags added: l3-dvr-backlog

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1419175

Title:
  Cannot find device "qr-" error message found in logtrace with DVR
  routers while trying to update arp entry

Status in neutron:
  In Progress

Bug description:
  We have running stable/juno with DVR enabled.
  During tests, we created router, gateway and instance.

  There is one qrouter on one compute node was created with
  RuntimeError:

  Command: ['sudo', '/usr/bin/neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip', 'netns', 'exec', 'qrouter-086cf9e6-4c43-4b65-b623-fbd5d593f687', 'ip', '-4', 'neigh', 'replace', '10.100.100.13', 'lladdr', 'fa:16:3e:84:fe:e4', 'nud', 'permanent', 'dev', 'qr-00d7d90b-01']
  Exit code: 1
  Stdout: ''
  Stderr: 'Cannot find device "qr-00d7d90b-01"\n'
  2015-02-05 20:48:11.834 27031 ERROR neutron.agent.l3_agent [req-2c71f61b-c036-4d90-bcfd-75ffdd5340ff None] DVR: Failed updating arp entry
  2015-02-05 20:48:11.834 27031 TRACE neutron.agent.l3_agent Traceback (most recent call last):
  2015-02-05 20:48:11.834 27031 TRACE neutron.agent.l3_agent   File "/opt/stack/venvs/openstack/local/lib/python2.7/site-packages/neutron/agent/l3_agent.py", line 1719, in _update_arp_entry
  2015-02-05 20:48:11.834 27031 TRACE neutron.agent.l3_agent     device.neigh.add(net.version, ip, mac)
  2015-02-05 20:48:11.834 27031 TRACE neutron.agent.l3_agent   File "/opt/stack/venvs/openstack/local/lib/python2.7/site-packages/neutron/agent/linux/ip_lib.py", line 515, in add
  2015-02-05 20:48:11.834 27031 TRACE neutron.agent.l3_agent     options=[ip_version])
  2015-02-05 20:48:11.834 27031 TRACE neutron.agent.l3_agent   File "/opt/stack/venvs/openstack/local/lib/python2.7/site-packages/neutron/agent/linux/ip_lib.py", line 247, in _as_root
  2015-02-05 20:48:11.834 27031 TRACE neutron.agent.l3_agent     kwargs.get('use_root_namespace', False))
  2015-02-05 20:48:11.834 27031 TRACE neutron.agent.l3_agent   File "/opt/stack/venvs/openstack/local/lib/python2.7/site-packages/neutron/agent/linux/ip_lib.py", line 79, in _as_root
  2015-02-05 20:48:11.834 27031 TRACE neutron.agent.l3_agent     log_fail_as_error=self.log_fail_as_error)
  2015-02-05 20:48:11.834 27031 TRACE neutron.agent.l3_agent   File "/opt/stack/venvs/openstack/local/lib/python2.7/site-packages/neutron/agent/linux/ip_lib.py", line 91, in _execute
  2015-02-05 20:48:11.834 27031 TRACE neutron.agent.l3_agent     log_fail_as_error=log_fail_as_error)
  2015-02-05 20:48:11.834 27031 TRACE neutron.agent.l3_agent   File "/opt/stack/venvs/openstack/local/lib/python2.7/site-packages/neutron/agent/linux/utils.py", line 82, in execute
  2015-02-05 20:48:11.834 27031 TRACE neutron.agent.l3_agent     raise RuntimeError(m)
  2015-02-05 20:48:11.834 27031 TRACE neutron.agent.l3_agent RuntimeError:

  As the result, all future router update failed as well. When the router was removed, the qrouter namespace was left on the compute node as well because of error:
  2015-02-05 20:48:11.834 27031 TRACE neutron.agent.l3_agent Stderr: 'Cannot find device "qr-00d7d90b-01"\n'

  Logs also can be read at: http://paste.openstack.org/show/168348/

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1419175/+subscriptions