yahoo-eng-team team mailing list archive

[Kailun Qin <1775496@xxxxxxxxxxxxxxxxxx>]

Public bug reported:

When a network is removed from a dhcp agent, in some scenarios if the
agent releases its port concurrently, there is chance that the removal
of network from agent will fail due to that the target port is not
found.

The issue can be reproduced on the latest devstack.
Steps to reproduce:
1. neutron port-list, identify the port to delete
2. Remove one network from a dhcp agent: neutron dhcp-agent-network-remove --dhcp_agent xxx --network xxx
   AND at the same time delete the port associated: neutron port-delete xxx

Failed CLI:
vagrant@control:~/devstack$ neutron dhcp-agent-network-remove --dhcp_agent 73721261-41c6-4f82-b0f4-ef9a750c7f70 --network net
neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead.
Port 6089a77e-1975-40a5-9d4d-819e0d9e8fd5 could not be found.
Neutron server returns request_ids: ['req-dfecf6a3-8d61-435b-a6a2-919ac6ca972f']
Failed Log:
DEBUG oslo.privsep.daemon [^[[00;36m-] ^[[01;35mprivsep: Exception during request[140677005388208]: Network interface tap83924265-3e not found in namespace qdhcp-d57b2982-69e8-4e62-8dd0-6241f204e132.^[[00m ^[[00;33m{{(pid=10686) loop /usr/local/lib/python2.7/dist-packages/oslo_privsep/daemon.py:449}}^[[00m
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/oslo_privsep/daemon.py", line 445, in loop
    reply = self._process_cmd(*msg)
  File "/usr/local/lib/python2.7/dist-packages/oslo_privsep/daemon.py", line 428, in _process_cmd
    ret = func(*f_args, **f_kwargs)
  File "/usr/local/lib/python2.7/dist-packages/oslo_privsep/priv_context.py", line 209, in _wrap
    return func(*args, **kwargs)
  File "/opt/stack/neutron/neutron/privileged/agent/linux/ip_lib.py", line 272, in get_link_attributes
    link = _run_iproute_link("get", device, namespace)[0]
  File "/opt/stack/neutron/neutron/privileged/agent/linux/ip_lib.py", line 130, in _run_iproute_link
    idx = _get_link_id(device, namespace)
  File "/opt/stack/neutron/neutron/privileged/agent/linux/ip_lib.py", line 124, in _get_link_id
    raise NetworkInterfaceNotFound(device=device, namespace=namespace)
NetworkInterfaceNotFound: Network interface tap83924265-3e not found in namespace qdhcp-d57b2982-69e8-4e62-8dd0-6241f204e132.

** Affects: neutron
     Importance: Undecided
     Assignee: Kailun Qin (kailun.qin)
         Status: New

** Changed in: neutron
     Assignee: (unassigned) => Kailun Qin (kailun.qin)

** Summary changed:

- agentschedulers: concurrent port delete on unscheduling may cause port not found
+ agentschedulers: concurrent port delete on unscheduling may cause unscheduling to fail

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1775496

Title:
  agentschedulers: concurrent port delete on unscheduling may cause
  unscheduling to fail

Status in neutron:
  New

Bug description:
  When a network is removed from a dhcp agent, in some scenarios if the
  agent releases its port concurrently, there is chance that the removal
  of network from agent will fail due to that the target port is not
  found.

  The issue can be reproduced on the latest devstack.
  Steps to reproduce:
  1. neutron port-list, identify the port to delete
  2. Remove one network from a dhcp agent: neutron dhcp-agent-network-remove --dhcp_agent xxx --network xxx
     AND at the same time delete the port associated: neutron port-delete xxx

  Failed CLI:
  vagrant@control:~/devstack$ neutron dhcp-agent-network-remove --dhcp_agent 73721261-41c6-4f82-b0f4-ef9a750c7f70 --network net
  neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead.
  Port 6089a77e-1975-40a5-9d4d-819e0d9e8fd5 could not be found.
  Neutron server returns request_ids: ['req-dfecf6a3-8d61-435b-a6a2-919ac6ca972f']
  Failed Log:
  DEBUG oslo.privsep.daemon [^[[00;36m-] ^[[01;35mprivsep: Exception during request[140677005388208]: Network interface tap83924265-3e not found in namespace qdhcp-d57b2982-69e8-4e62-8dd0-6241f204e132.^[[00m ^[[00;33m{{(pid=10686) loop /usr/local/lib/python2.7/dist-packages/oslo_privsep/daemon.py:449}}^[[00m
  Traceback (most recent call last):
    File "/usr/local/lib/python2.7/dist-packages/oslo_privsep/daemon.py", line 445, in loop
      reply = self._process_cmd(*msg)
    File "/usr/local/lib/python2.7/dist-packages/oslo_privsep/daemon.py", line 428, in _process_cmd
      ret = func(*f_args, **f_kwargs)
    File "/usr/local/lib/python2.7/dist-packages/oslo_privsep/priv_context.py", line 209, in _wrap
      return func(*args, **kwargs)
    File "/opt/stack/neutron/neutron/privileged/agent/linux/ip_lib.py", line 272, in get_link_attributes
      link = _run_iproute_link("get", device, namespace)[0]
    File "/opt/stack/neutron/neutron/privileged/agent/linux/ip_lib.py", line 130, in _run_iproute_link
      idx = _get_link_id(device, namespace)
    File "/opt/stack/neutron/neutron/privileged/agent/linux/ip_lib.py", line 124, in _get_link_id
      raise NetworkInterfaceNotFound(device=device, namespace=namespace)
  NetworkInterfaceNotFound: Network interface tap83924265-3e not found in namespace qdhcp-d57b2982-69e8-4e62-8dd0-6241f204e132.

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1775496/+subscriptions