← Back to team overview

yahoo-eng-team team mailing list archive

  1. yahoo-eng-team team
  2. Mailing list archive
  3. Message #58348

[Bug 1639042] [NEW] Prefix delegation doesn't work in the described scenario

  Thread Previous • Date Previous • • Thread Next 
Public bug reported:

When creating the router and enable pd subnet with the below commands:
    neutron router-create ha-router
    neutron router-gateway-set ha-router public
    neutron router-interface-add ha-router ipv6-pd-subnet

lla timeout on creating with the following trace in l3 agent log:

raceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/eventlet/hubs/hub.py", line 457, in fire_timers
    timer()
  File "/usr/local/lib/python2.7/dist-packages/eventlet/hubs/timer.py", line 58, in __call__
    cb(*args, **kw)
  File "/opt/stack/neutron/neutron/agent/linux/pd.py", line 228, in _ensure_lla_task
    sleep=2)
  File "/opt/stack/neutron/neutron/common/utils.py", line 750, in wait_until_true
    eventlet.sleep(sleep)
  File "/usr/local/lib/python2.7/dist-packages/eventlet/greenthread.py", line 34, in sleep
    hub.switch()
  File "/usr/local/lib/python2.7/dist-packages/eventlet/hubs/hub.py", line 294, in switch
    return self.greenlet.switch()
Timeout: 40 seconds

Note that the order of neutron commands are significant.

Checking the l3 agent log further, gateways were added twice in a row.
The later one removed the lla added in the earlier, which caused the
timeout.

2016-11-03 11:03:26.517 ^[[00;32mDEBUG neutron.agent.l3.router_info [^[[00;36m-^[[00;32m] ^[[01;35m^[[00;32mExternal gateway added: port({u'allowed_address_pairs': [], u'extra_dhcp_opts': [], u'updated_at': u'2016-11-03T15:03:22Z', u'device_owner': u'network:router_gateway', u'revision_number': 5, u'port_security_enabled': False, u'binding:profile': {}, u'binding:vnic_type': u'normal', u'fixed_ips': [{u'subnet_id': u'e8cbec96-2ea4-43d5-a717-93b03231d393', u'prefixlen': 24, u'ip_address': u'172.24.4.7'}, {u'subnet_id': u'68fc0073-619a-43ca-be62-569842366e48', u'prefixlen': 64, u'ip_address': u'2001:db8::8'}], u'id': u'95b57ead-f411-4f7b-8eda-e2a7b4623ac6', u'security_groups': [], u'binding:vif_details': {}, u'address_scopes': {u'4': None, u'6': None}, u'binding:vif_type': u'unbound', u'mac_address': u'fa:16:3e:2e:2f:58', u'project_id': u'', u'status': u'DOWN', u'subnets': [{u'dns_nameservers': [], u'ipv6_ra_mode': None, u'gateway_ip': u'172.24.4.1', u'cidr': u'172.24.4.0/24', u'id': u'e8cbec96-2ea4-43d5-a717-93b03231d393', u'subnetpool_id': None}, {u'dns_nameservers': [], u'ipv6_ra_mode': None, u'gateway_ip': u'2001:db8::2', u'cidr': u'2001:db8::/64', u'id': u'68fc0073-619a-43ca-be62-569842366e48', u'subnetpool_id': None}], u'binding:host_id': u'ip6-devstack-43', u'description': u'', u'device_id': u'0269742e-56e5-468c-974a-859bb1966a62', u'name': u'', u'admin_state_up': True, u'network_id': u'8da945b9-8ac2-4d1b-840f-094dbd794500', u'tenant_id': u'', u'created_at': u'2016-11-03T15:03:22Z', u'mtu': 1500, u'extra_subnets': []}), interface(qg-95b57ead-f4), ns(qrouter-0269742e-56e5-468c-974a-859bb1966a62)^[[00m ^[[00;33mfrom (pid=28145) _external_gateway_added /opt/stack/neutron/neutron/agent/l3/router_info.py:632^[[00m
.....

2016-11-03 11:03:34.338 DEBUG neutron.agent.linux.utils [-] Running
command (rootwrap daemon): ['ip', 'netns', 'exec', 'qrouter-0269742e-
56e5-468c-974a-859bb1966a62', 'ip', '-6', 'addr', 'add',
'fe80::f816:3eff:fecc:7a22/64', 'scope', 'link', 'dev', 'qg-
95b57ead-f4'] from (pid=28145) execute_rootwrap_daemon
/opt/stack/neutron/neutron/agent/linux/utils.py:100

The above log took place when the gw_port was added the first time .

2016-11-03 11:03:34.413 ^[[00;32mDEBUG neutron.agent.l3.router_info [^[[00;36m-^[[00;32m] ^[[01;35m^[[00;32mExternal gateway added: port({u'allowed_address_pairs': [], u'extra_dhcp_opts': [], u'updated_at': u'2016-11-03T15:03:30Z', u'device_owner': u'network:router_gateway', u'revision_number': 8, u'port_security_enabled': False, u'binding:profile': {}, u'binding:vnic_type': u'normal', u'fixed_ips': [{u'subnet_id': u'e8cbec96-2ea4-43d5-a717-93b03231d393', u'prefixlen': 24, u'ip_address': u'172.24.4.7'}, {u'subnet_id': u'68fc0073-619a-43ca-be62-569842366e48', u'prefixlen': 64, u'ip_address': u'2001:db8::8'}], u'id': u'95b57ead-f411-4f7b-8eda-e2a7b4623ac6', u'security_groups': [], u'binding:vif_details': {u'port_filter': True, u'ovs_hybrid_plug': True}, u'address_scopes': {u'4': None, u'6': None}, u'binding:vif_type': u'ovs', u'mac_address': u'fa:16:3e:2e:2f:58', u'project_id': u'', u'status': u'ACTIVE', u'subnets': [{u'dns_nameservers': [], u'ipv6_ra_mode': None, u'gateway_ip': u'172.24.4.1', u'cidr': u'172.24.4.0/24', u'id': u'e8cbec96-2ea4-43d5-a717-93b03231d393', u'subnetpool_id': None}, {u'dns_nameservers': [], u'ipv6_ra_mode': None, u'gateway_ip': u'2001:db8::2', u'cidr': u'2001:db8::/64', u'id': u'68fc0073-619a-43ca-be62-569842366e48', u'subnetpool_id': None}], u'binding:host_id': u'ip6-devstack-43', u'description': u'', u'device_id': u'0269742e-56e5-468c-974a-859bb1966a62', u'name': u'', u'admin_state_up': True, u'network_id': u'8da945b9-8ac2-4d1b-840f-094dbd794500', u'tenant_id': u'', u'created_at': u'2016-11-03T15:03:22Z', u'mtu': 1500, u'extra_subnets': []}), interface(qg-95b57ead-f4), ns(qrouter-0269742e-56e5-468c-974a-859bb1966a62)^[[00m ^[[00;33mfrom (pid=28145) _external_gateway_added /opt/stack/neutron/neutron/agent/l3/router_info.py:632^[[00m
......
2016-11-03 11:03:38.513 DEBUG neutron.agent.linux.utils [-] Running command (rootwrap daemon): ['ip', 'netns', 'exec', 'qrouter-0269742e-56e5-468c-974a-859bb1966a62', 'ip', '-6', 'addr', 'del', 'fe80::f816:3eff:fecc:7a22/64', 'dev', 'qg-95b57ead-f4'] from (pid=28145) execute_rootwrap_daemon /opt/stack/neutron/neutron/agent/linux/utils.py:100

the above log took place when the gw_port was added the second time

** Affects: neutron
     Importance: Undecided
     Assignee: Baodong (Robert) Li (baoli)
         Status: New

** Changed in: neutron
     Assignee: (unassigned) => Baodong (Robert) Li (baoli)

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1639042

Title:
  Prefix delegation doesn't work in the described scenario

Status in neutron:
  New

Bug description:
  When creating the router and enable pd subnet with the below commands:
      neutron router-create ha-router
      neutron router-gateway-set ha-router public
      neutron router-interface-add ha-router ipv6-pd-subnet

  lla timeout on creating with the following trace in l3 agent log:

  raceback (most recent call last):
    File "/usr/local/lib/python2.7/dist-packages/eventlet/hubs/hub.py", line 457, in fire_timers
      timer()
    File "/usr/local/lib/python2.7/dist-packages/eventlet/hubs/timer.py", line 58, in __call__
      cb(*args, **kw)
    File "/opt/stack/neutron/neutron/agent/linux/pd.py", line 228, in _ensure_lla_task
      sleep=2)
    File "/opt/stack/neutron/neutron/common/utils.py", line 750, in wait_until_true
      eventlet.sleep(sleep)
    File "/usr/local/lib/python2.7/dist-packages/eventlet/greenthread.py", line 34, in sleep
      hub.switch()
    File "/usr/local/lib/python2.7/dist-packages/eventlet/hubs/hub.py", line 294, in switch
      return self.greenlet.switch()
  Timeout: 40 seconds

  Note that the order of neutron commands are significant.

  Checking the l3 agent log further, gateways were added twice in a row.
  The later one removed the lla added in the earlier, which caused the
  timeout.

  2016-11-03 11:03:26.517 ^[[00;32mDEBUG neutron.agent.l3.router_info [^[[00;36m-^[[00;32m] ^[[01;35m^[[00;32mExternal gateway added: port({u'allowed_address_pairs': [], u'extra_dhcp_opts': [], u'updated_at': u'2016-11-03T15:03:22Z', u'device_owner': u'network:router_gateway', u'revision_number': 5, u'port_security_enabled': False, u'binding:profile': {}, u'binding:vnic_type': u'normal', u'fixed_ips': [{u'subnet_id': u'e8cbec96-2ea4-43d5-a717-93b03231d393', u'prefixlen': 24, u'ip_address': u'172.24.4.7'}, {u'subnet_id': u'68fc0073-619a-43ca-be62-569842366e48', u'prefixlen': 64, u'ip_address': u'2001:db8::8'}], u'id': u'95b57ead-f411-4f7b-8eda-e2a7b4623ac6', u'security_groups': [], u'binding:vif_details': {}, u'address_scopes': {u'4': None, u'6': None}, u'binding:vif_type': u'unbound', u'mac_address': u'fa:16:3e:2e:2f:58', u'project_id': u'', u'status': u'DOWN', u'subnets': [{u'dns_nameservers': [], u'ipv6_ra_mode': None, u'gateway_ip': u'172.24.4.1', u'cidr': u'172.24.4.0/24', u'id': u'e8cbec96-2ea4-43d5-a717-93b03231d393', u'subnetpool_id': None}, {u'dns_nameservers': [], u'ipv6_ra_mode': None, u'gateway_ip': u'2001:db8::2', u'cidr': u'2001:db8::/64', u'id': u'68fc0073-619a-43ca-be62-569842366e48', u'subnetpool_id': None}], u'binding:host_id': u'ip6-devstack-43', u'description': u'', u'device_id': u'0269742e-56e5-468c-974a-859bb1966a62', u'name': u'', u'admin_state_up': True, u'network_id': u'8da945b9-8ac2-4d1b-840f-094dbd794500', u'tenant_id': u'', u'created_at': u'2016-11-03T15:03:22Z', u'mtu': 1500, u'extra_subnets': []}), interface(qg-95b57ead-f4), ns(qrouter-0269742e-56e5-468c-974a-859bb1966a62)^[[00m ^[[00;33mfrom (pid=28145) _external_gateway_added /opt/stack/neutron/neutron/agent/l3/router_info.py:632^[[00m
  .....

  2016-11-03 11:03:34.338 DEBUG neutron.agent.linux.utils [-] Running
  command (rootwrap daemon): ['ip', 'netns', 'exec', 'qrouter-0269742e-
  56e5-468c-974a-859bb1966a62', 'ip', '-6', 'addr', 'add',
  'fe80::f816:3eff:fecc:7a22/64', 'scope', 'link', 'dev', 'qg-
  95b57ead-f4'] from (pid=28145) execute_rootwrap_daemon
  /opt/stack/neutron/neutron/agent/linux/utils.py:100

  The above log took place when the gw_port was added the first time .

  2016-11-03 11:03:34.413 ^[[00;32mDEBUG neutron.agent.l3.router_info [^[[00;36m-^[[00;32m] ^[[01;35m^[[00;32mExternal gateway added: port({u'allowed_address_pairs': [], u'extra_dhcp_opts': [], u'updated_at': u'2016-11-03T15:03:30Z', u'device_owner': u'network:router_gateway', u'revision_number': 8, u'port_security_enabled': False, u'binding:profile': {}, u'binding:vnic_type': u'normal', u'fixed_ips': [{u'subnet_id': u'e8cbec96-2ea4-43d5-a717-93b03231d393', u'prefixlen': 24, u'ip_address': u'172.24.4.7'}, {u'subnet_id': u'68fc0073-619a-43ca-be62-569842366e48', u'prefixlen': 64, u'ip_address': u'2001:db8::8'}], u'id': u'95b57ead-f411-4f7b-8eda-e2a7b4623ac6', u'security_groups': [], u'binding:vif_details': {u'port_filter': True, u'ovs_hybrid_plug': True}, u'address_scopes': {u'4': None, u'6': None}, u'binding:vif_type': u'ovs', u'mac_address': u'fa:16:3e:2e:2f:58', u'project_id': u'', u'status': u'ACTIVE', u'subnets': [{u'dns_nameservers': [], u'ipv6_ra_mode': None, u'gateway_ip': u'172.24.4.1', u'cidr': u'172.24.4.0/24', u'id': u'e8cbec96-2ea4-43d5-a717-93b03231d393', u'subnetpool_id': None}, {u'dns_nameservers': [], u'ipv6_ra_mode': None, u'gateway_ip': u'2001:db8::2', u'cidr': u'2001:db8::/64', u'id': u'68fc0073-619a-43ca-be62-569842366e48', u'subnetpool_id': None}], u'binding:host_id': u'ip6-devstack-43', u'description': u'', u'device_id': u'0269742e-56e5-468c-974a-859bb1966a62', u'name': u'', u'admin_state_up': True, u'network_id': u'8da945b9-8ac2-4d1b-840f-094dbd794500', u'tenant_id': u'', u'created_at': u'2016-11-03T15:03:22Z', u'mtu': 1500, u'extra_subnets': []}), interface(qg-95b57ead-f4), ns(qrouter-0269742e-56e5-468c-974a-859bb1966a62)^[[00m ^[[00;33mfrom (pid=28145) _external_gateway_added /opt/stack/neutron/neutron/agent/l3/router_info.py:632^[[00m
  ......
  2016-11-03 11:03:38.513 DEBUG neutron.agent.linux.utils [-] Running command (rootwrap daemon): ['ip', 'netns', 'exec', 'qrouter-0269742e-56e5-468c-974a-859bb1966a62', 'ip', '-6', 'addr', 'del', 'fe80::f816:3eff:fecc:7a22/64', 'dev', 'qg-95b57ead-f4'] from (pid=28145) execute_rootwrap_daemon /opt/stack/neutron/neutron/agent/linux/utils.py:100

  the above log took place when the gw_port was added the second time

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1639042/+subscriptions

Follow ups

  Thread Previous • Date Previous • • Thread Next