yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1624515] [NEW] DVR: SNAT port not found in the list error in check jobs

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Brian Haley <1624515@xxxxxxxxxxxxxxxxxx>
Date: Fri, 16 Sep 2016 19:41:31 -0000
Reply-to: Bug 1624515 <1624515@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
Public bug reported:

While looking through logs I came across this ERROR:

http://logs.openstack.org/30/370430/6/check/gate-tempest-dsvm-neutron-
dvr-ubuntu-
xenial/c853e94/logs/screen-q-l3.txt.gz#_2016-09-15_16_27_41_804

Pasting here since that log could go away (even if the paste is horribly
wrapped, sorry):

ERROR neutron.agent.l3.dvr_router_base [-] DVR: SNAT port not found in
the list [{u'allowed_address_pairs': [], u'extra_dhcp_opts': [],
u'updated_at': u'2016-09-15T16:26:02', u'device_owner':
u'network:router_centralized_snat', u'revision_number': 6,
u'port_security_enabled': False, u'binding:profile': {},
u'binding:vnic_type': u'normal', u'fixed_ips': [{u'subnet_id':
u'af427fca-9194-440f-87d6-e74e4d1c8a27', u'prefixlen': 28,
u'ip_address': u'10.1.0.4'}], u'id': u'54df3773-e6cf-4d9c-
b1b9-0e43f77ba21a', u'security_groups': [], u'binding:vif_details':
{u'port_filter': True, u'ovs_hybrid_plug': True}, u'address_scopes':
{u'4': None, u'6': None}, u'binding:vif_type': u'ovs', u'mac_address':
u'fa:16:3e:ed:f7:77', u'project_id': u'', u'status': u'ACTIVE',
u'subnets': [{u'dns_nameservers': [], u'ipv6_ra_mode': None,
u'gateway_ip': u'10.1.0.1', u'cidr': u'10.1.0.0/28', u'id':
u'af427fca-9194-440f-87d6-e74e4d1c8a27', u'subnetpool_id': None}],
u'binding:host_id': u'ubuntu-xenial-osic-cloud1-4314226',
u'description': u'', u'device_id': u'6e1fd121-cf73-4e72-b6fa-
431761591de6', u'name': u'', u'admin_state_up': True, u'network_id':
u'86a63ce4-50b6-48ee-b192-c77fe2481db9', u'tenant_id': u'',
u'created_at': u'2016-09-15T16:25:51', u'mtu': 1450, u'extra_subnets':
[]}, {u'allowed_address_pairs': [], u'extra_dhcp_opts': [],
u'updated_at': u'2016-09-15T16:26:06', u'device_owner':
u'network:router_centralized_snat', u'revision_number': 7,
u'port_security_enabled': False, u'binding:profile': {},
u'binding:vnic_type': u'normal', u'fixed_ips': [{u'subnet_id':
u'b433a3c3-eb7c-404b-aea3-5e454b4dd0bd', u'prefixlen': 64,
u'ip_address': u'2003:0:0:1::2'}, {u'subnet_id':
u'3dd250b4-eade-4239-9011-9860ccf31364', u'prefixlen': 64,
u'ip_address': u'2003::4'}], u'id': u'7454b08a-3431-4099-a3bc-
7e46c56f310b', u'security_groups': [], u'binding:vif_details':
{u'port_filter': True, u'ovs_hybrid_plug': True}, u'address_scopes':
{u'4': None, u'6': None}, u'binding:vif_type': u'ovs', u'mac_address':
u'fa:16:3e:ce:db:80', u'project_id': u'', u'status': u'ACTIVE',
u'subnets': [{u'dns_nameservers': [], u'ipv6_ra_mode':
u'dhcpv6-stateless', u'gateway_ip': u'2003:0:0:1::1', u'cidr':
u'2003:0:0:1::/64', u'id': u'b433a3c3-eb7c-404b-aea3-5e454b4dd0bd',
u'subnetpool_id': None}, {u'dns_nameservers': [], u'ipv6_ra_mode':
u'dhcpv6-stateless', u'gateway_ip': u'2003::1', u'cidr': u'2003::/64',
u'id': u'3dd250b4-eade-4239-9011-9860ccf31364', u'subnetpool_id':
None}], u'binding:host_id': u'ubuntu-xenial-osic-cloud1-4314226',
u'description': u'', u'device_id': u'6e1fd121-cf73-4e72-b6fa-
431761591de6', u'name': u'', u'admin_state_up': True, u'network_id':
u'7862083c-a928-4170-b99e-3cfd0fb3ae77', u'tenant_id': u'',
u'created_at': u'2016-09-15T16:25:55', u'mtu': 1450, u'extra_subnets':
[]}] for the given router  internal port {u'allowed_address_pairs': [],
u'extra_dhcp_opts': [], u'updated_at': u'2016-09-15T16:27:37',
u'device_owner': u'network:router_interface_distributed',
u'revision_number': 8, u'port_security_enabled': False,
u'binding:profile': {}, u'binding:vnic_type': u'normal', u'fixed_ips':
[{u'subnet_id': u'3dd250b4-eade-4239-9011-9860ccf31364', u'prefixlen':
64, u'ip_address': u'2003::1'}], u'id':
u'6092e3b0-f833-4611-82e4-7fe5d1d31021', u'security_groups': [],
u'binding:vif_details': {}, u'address_scopes': {u'4': None, u'6': None},
u'binding:vif_type': u'distributed', u'mac_address':
u'fa:16:3e:ea:5f:f0', u'project_id':
u'b557a0938e3748d6a34b0ca2efdee658', u'status': u'ACTIVE', u'subnets':
[{u'dns_nameservers': [], u'ipv6_ra_mode': u'dhcpv6-stateless',
u'gateway_ip': u'2003::1', u'cidr': u'2003::/64', u'id':
u'3dd250b4-eade-4239-9011-9860ccf31364', u'subnetpool_id': None}],
u'binding:host_id': u'', u'description': u'', u'device_id':
u'6e1fd121-cf73-4e72-b6fa-431761591de6', u'name': u'',
u'admin_state_up': True, u'network_id': u'7862083c-a928-4170-b99e-
3cfd0fb3ae77', u'tenant_id': u'b557a0938e3748d6a34b0ca2efdee658',
u'created_at': u'2016-09-15T16:25:54', u'mtu': 1450, u'extra_subnets':
[{u'dns_nameservers': [], u'ipv6_ra_mode': u'dhcpv6-stateless',
u'gateway_ip': u'2003:0:0:1::1', u'cidr': u'2003:0:0:1::/64', u'id':
u'b433a3c3-eb7c-404b-aea3-5e454b4dd0bd', u'subnetpool_id': None}]}

>From looking at the trace, I do see the subnet in question in the list,
but since the code is only checking it's in the first port it doesn't
find it.  I'm curious why it doesn't look in them all, so will propose a
patch to verify it works and get some feedback.

** Affects: neutron
     Importance: Medium
     Assignee: Brian Haley (brian-haley)
         Status: In Progress


** Tags: l3-dvr-backlog

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1624515

Title:
  DVR: SNAT port not found in the list error in check jobs

Status in neutron:
  In Progress

Bug description:
  While looking through logs I came across this ERROR:

  http://logs.openstack.org/30/370430/6/check/gate-tempest-dsvm-neutron-
  dvr-ubuntu-
  xenial/c853e94/logs/screen-q-l3.txt.gz#_2016-09-15_16_27_41_804

  Pasting here since that log could go away (even if the paste is
  horribly wrapped, sorry):

  ERROR neutron.agent.l3.dvr_router_base [-] DVR: SNAT port not found in
  the list [{u'allowed_address_pairs': [], u'extra_dhcp_opts': [],
  u'updated_at': u'2016-09-15T16:26:02', u'device_owner':
  u'network:router_centralized_snat', u'revision_number': 6,
  u'port_security_enabled': False, u'binding:profile': {},
  u'binding:vnic_type': u'normal', u'fixed_ips': [{u'subnet_id':
  u'af427fca-9194-440f-87d6-e74e4d1c8a27', u'prefixlen': 28,
  u'ip_address': u'10.1.0.4'}], u'id': u'54df3773-e6cf-4d9c-
  b1b9-0e43f77ba21a', u'security_groups': [], u'binding:vif_details':
  {u'port_filter': True, u'ovs_hybrid_plug': True}, u'address_scopes':
  {u'4': None, u'6': None}, u'binding:vif_type': u'ovs', u'mac_address':
  u'fa:16:3e:ed:f7:77', u'project_id': u'', u'status': u'ACTIVE',
  u'subnets': [{u'dns_nameservers': [], u'ipv6_ra_mode': None,
  u'gateway_ip': u'10.1.0.1', u'cidr': u'10.1.0.0/28', u'id':
  u'af427fca-9194-440f-87d6-e74e4d1c8a27', u'subnetpool_id': None}],
  u'binding:host_id': u'ubuntu-xenial-osic-cloud1-4314226',
  u'description': u'', u'device_id': u'6e1fd121-cf73-4e72-b6fa-
  431761591de6', u'name': u'', u'admin_state_up': True, u'network_id':
  u'86a63ce4-50b6-48ee-b192-c77fe2481db9', u'tenant_id': u'',
  u'created_at': u'2016-09-15T16:25:51', u'mtu': 1450, u'extra_subnets':
  []}, {u'allowed_address_pairs': [], u'extra_dhcp_opts': [],
  u'updated_at': u'2016-09-15T16:26:06', u'device_owner':
  u'network:router_centralized_snat', u'revision_number': 7,
  u'port_security_enabled': False, u'binding:profile': {},
  u'binding:vnic_type': u'normal', u'fixed_ips': [{u'subnet_id':
  u'b433a3c3-eb7c-404b-aea3-5e454b4dd0bd', u'prefixlen': 64,
  u'ip_address': u'2003:0:0:1::2'}, {u'subnet_id':
  u'3dd250b4-eade-4239-9011-9860ccf31364', u'prefixlen': 64,
  u'ip_address': u'2003::4'}], u'id': u'7454b08a-3431-4099-a3bc-
  7e46c56f310b', u'security_groups': [], u'binding:vif_details':
  {u'port_filter': True, u'ovs_hybrid_plug': True}, u'address_scopes':
  {u'4': None, u'6': None}, u'binding:vif_type': u'ovs', u'mac_address':
  u'fa:16:3e:ce:db:80', u'project_id': u'', u'status': u'ACTIVE',
  u'subnets': [{u'dns_nameservers': [], u'ipv6_ra_mode':
  u'dhcpv6-stateless', u'gateway_ip': u'2003:0:0:1::1', u'cidr':
  u'2003:0:0:1::/64', u'id': u'b433a3c3-eb7c-404b-aea3-5e454b4dd0bd',
  u'subnetpool_id': None}, {u'dns_nameservers': [], u'ipv6_ra_mode':
  u'dhcpv6-stateless', u'gateway_ip': u'2003::1', u'cidr': u'2003::/64',
  u'id': u'3dd250b4-eade-4239-9011-9860ccf31364', u'subnetpool_id':
  None}], u'binding:host_id': u'ubuntu-xenial-osic-cloud1-4314226',
  u'description': u'', u'device_id': u'6e1fd121-cf73-4e72-b6fa-
  431761591de6', u'name': u'', u'admin_state_up': True, u'network_id':
  u'7862083c-a928-4170-b99e-3cfd0fb3ae77', u'tenant_id': u'',
  u'created_at': u'2016-09-15T16:25:55', u'mtu': 1450, u'extra_subnets':
  []}] for the given router  internal port {u'allowed_address_pairs':
  [], u'extra_dhcp_opts': [], u'updated_at': u'2016-09-15T16:27:37',
  u'device_owner': u'network:router_interface_distributed',
  u'revision_number': 8, u'port_security_enabled': False,
  u'binding:profile': {}, u'binding:vnic_type': u'normal', u'fixed_ips':
  [{u'subnet_id': u'3dd250b4-eade-4239-9011-9860ccf31364', u'prefixlen':
  64, u'ip_address': u'2003::1'}], u'id':
  u'6092e3b0-f833-4611-82e4-7fe5d1d31021', u'security_groups': [],
  u'binding:vif_details': {}, u'address_scopes': {u'4': None, u'6':
  None}, u'binding:vif_type': u'distributed', u'mac_address':
  u'fa:16:3e:ea:5f:f0', u'project_id':
  u'b557a0938e3748d6a34b0ca2efdee658', u'status': u'ACTIVE', u'subnets':
  [{u'dns_nameservers': [], u'ipv6_ra_mode': u'dhcpv6-stateless',
  u'gateway_ip': u'2003::1', u'cidr': u'2003::/64', u'id':
  u'3dd250b4-eade-4239-9011-9860ccf31364', u'subnetpool_id': None}],
  u'binding:host_id': u'', u'description': u'', u'device_id':
  u'6e1fd121-cf73-4e72-b6fa-431761591de6', u'name': u'',
  u'admin_state_up': True, u'network_id': u'7862083c-a928-4170-b99e-
  3cfd0fb3ae77', u'tenant_id': u'b557a0938e3748d6a34b0ca2efdee658',
  u'created_at': u'2016-09-15T16:25:54', u'mtu': 1450, u'extra_subnets':
  [{u'dns_nameservers': [], u'ipv6_ra_mode': u'dhcpv6-stateless',
  u'gateway_ip': u'2003:0:0:1::1', u'cidr': u'2003:0:0:1::/64', u'id':
  u'b433a3c3-eb7c-404b-aea3-5e454b4dd0bd', u'subnetpool_id': None}]}

  From looking at the trace, I do see the subnet in question in the
  list, but since the code is only checking it's in the first port it
  doesn't find it.  I'm curious why it doesn't look in them all, so will
  propose a patch to verify it works and get some feedback.

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1624515/+subscriptions
Follow ups

[Bug 1624515] Re: DVR: SNAT port not found in the list error in check jobs
From: OpenStack Infra, 2017-02-10