yahoo-eng-team team mailing list archive

[Brian Haley <1627902@xxxxxxxxxxxxxxxxxx>]

I actually broke this with my update, I'll explain.

Although we do correctly disable accept_ra in the dhcp namespace now, on
the grenade upgrade job we are restarting a new dhcp agent on an
existing installation.  This means that the old dhcp namespace could
have auto-configured an IPv6 address already, in which case we fall on
our face:

'ip', 'netns', 'exec', 'qdhcp-38b38c7e-d336-4955-a140-ef83a4410c2a', 'ip', '-6', 'addr', 'add', 'fdb9:d744:18da:0:f816:3eff:fe61:c32b/64', 'scope', 'global', 'dev', 'tap62476185-f1'
ProcessExecutionError: Exit code: 2; Stdin: ; Stdout: ; Stderr: RTNETLINK answers: File exists

http://logs.openstack.org/43/406243/1/check/gate-grenade-dsvm-neutron-
dvr-multinode-ubuntu-
xenial/6cee38c/logs/new/screen-q-dhcp.txt.gz#_2016-12-02_17_40_30_861

Looking a few lines up we can see:

    Reusing existing device: tap8fb1bc28-fa

That's the clue that we might have a conflict.

In init_l3() we filter the IP(v6) addresses with the "permanent" flag,
but that will filter any previous IPv6 addresses created via SLAAC on
receipt of an RA.  Just dropping the "permanent" filter seems like a
quick fix, but that will leave the SLAAC-based address alone, which will
eventually be removed as it's lifetime expires, leading to other
problems.

The best thing to do would be to detect the old address (it has a
"dynamic" flag) and remove it, allowing us to correctly add the new
"permanent" one.  I'll work on a fix.

** Changed in: neutron
       Status: Fix Released => Confirmed

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1627902

Title:
  DHCP agent conflicting with dynamic IPv6 addresses

Status in neutron:
  Confirmed

Bug description:
  Below in gate logs. Completely breaks the DHCP for that network
  because it's trying to add an address that conflicts with one given to
  it via RA. Cause is the merge of
  d86f1b87f01c53c3e0b085086133b311e5bf3ab5 which allowed the agent to be
  configured with stateless v6 addresses to serve metadata correctly.

  http://logs.openstack.org/12/343312/5/gate/gate-tempest-dsvm-neutron-
  full-ubuntu-
  xenial/c11b933/logs/screen-q-dhcp.txt.gz?level=TRACE#_2016-09-26_21_45_40_604

  2016-09-26 21:45:40.604 13605 ERROR neutron.agent.linux.utils [-] Exit
  code: 2; Stdin: ; Stdout: ; Stderr: RTNETLINK answers: File exists

  2016-09-26 21:45:40.604 13605 ERROR neutron.agent.dhcp.agent [-] Unable to enable dhcp for 81d252a2-8207-4e8c-a286-07fb3494a3ec.
  2016-09-26 21:45:40.604 13605 ERROR neutron.agent.dhcp.agent Traceback (most recent call last):
  2016-09-26 21:45:40.604 13605 ERROR neutron.agent.dhcp.agent   File "/opt/stack/new/neutron/neutron/agent/dhcp/agent.py", line 114, in call_driver
  2016-09-26 21:45:40.604 13605 ERROR neutron.agent.dhcp.agent     getattr(driver, action)(**action_kwargs)
  2016-09-26 21:45:40.604 13605 ERROR neutron.agent.dhcp.agent   File "/opt/stack/new/neutron/neutron/agent/linux/dhcp.py", line 212, in enable
  2016-09-26 21:45:40.604 13605 ERROR neutron.agent.dhcp.agent     interface_name = self.device_manager.setup(self.network)
  2016-09-26 21:45:40.604 13605 ERROR neutron.agent.dhcp.agent   File "/opt/stack/new/neutron/neutron/agent/linux/dhcp.py", line 1396, in setup
  2016-09-26 21:45:40.604 13605 ERROR neutron.agent.dhcp.agent     namespace=network.namespace)
  2016-09-26 21:45:40.604 13605 ERROR neutron.agent.dhcp.agent   File "/opt/stack/new/neutron/neutron/agent/linux/interface.py", line 129, in init_l3
  2016-09-26 21:45:40.604 13605 ERROR neutron.agent.dhcp.agent     device.addr.add(ip_cidr)
  2016-09-26 21:45:40.604 13605 ERROR neutron.agent.dhcp.agent   File "/opt/stack/new/neutron/neutron/agent/linux/ip_lib.py", line 577, in add
  2016-09-26 21:45:40.604 13605 ERROR neutron.agent.dhcp.agent     self._as_root([net.version], tuple(args))
  2016-09-26 21:45:40.604 13605 ERROR neutron.agent.dhcp.agent   File "/opt/stack/new/neutron/neutron/agent/linux/ip_lib.py", line 364, in _as_root
  2016-09-26 21:45:40.604 13605 ERROR neutron.agent.dhcp.agent     use_root_namespace=use_root_namespace)
  2016-09-26 21:45:40.604 13605 ERROR neutron.agent.dhcp.agent   File "/opt/stack/new/neutron/neutron/agent/linux/ip_lib.py", line 95, in _as_root
  2016-09-26 21:45:40.604 13605 ERROR neutron.agent.dhcp.agent     log_fail_as_error=self.log_fail_as_error)
  2016-09-26 21:45:40.604 13605 ERROR neutron.agent.dhcp.agent   File "/opt/stack/new/neutron/neutron/agent/linux/ip_lib.py", line 104, in _execute
  2016-09-26 21:45:40.604 13605 ERROR neutron.agent.dhcp.agent     log_fail_as_error=log_fail_as_error)
  2016-09-26 21:45:40.604 13605 ERROR neutron.agent.dhcp.agent   File "/opt/stack/new/neutron/neutron/agent/linux/utils.py", line 138, in execute
  2016-09-26 21:45:40.604 13605 ERROR neutron.agent.dhcp.agent     raise RuntimeError(msg)
  2016-09-26 21:45:40.604 13605 ERROR neutron.agent.dhcp.agent RuntimeError: Exit code: 2; Stdin: ; Stdout: ; Stderr: RTNETLINK answers: File exists
  2016-09-26 21:45:40.604 13605 ERROR neutron.agent.dhcp.agent 
  2016-09-26 21:45:40.604 13605 ERROR neutron.agent.dhcp.agent

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1627902/+subscriptions