yahoo-eng-team team mailing list archive
-
yahoo-eng-team team
-
Mailing list archive
-
Message #91120
[Bug 2003842] [NEW] [OVN] A route inferred from a subnet's default gateway is not added to ovn-nb if segment_id is not None for a subnet
Public bug reported:
Context:
* Neutron is configured to use OVN
* An external provider network with one segment is created
* A subnet with a default gateway IP set is associated with this segment explicitly (segment_id != None)
* A router's gateway port is set to use the provider network (external_gateway_info is set with a network_id passed)
Result: OVN NB does not contain a default route and instance traffic is
blackholed.
--
Detailed description:
The first time a external gateway info is set as follows
$ openstack router set --external-gateway pubnet r1
does not result in OVN getting a default route with the next-hop set to
the subnet's gateway IP:
$ sudo ovn-nbctl list logical_router_static_route ; echo $?
0
Doing it twice in a row does (the default route appears in the table
after the second command):
$ openstack router set --external-gateway pubnet r1 && openstack router
set --external-gateway pubnet r1
$ sudo ovn-nbctl list logical_router_static_route
_uuid : df7c6020-83e7-446c-8f5c-31db96eb2dd3
bfd : []
external_ids : {"neutron:is_ext_gw"="true", "neutron:subnet_id"="abdae752-034c-4845-b6b3-92bf40cf24a6"}
ip_prefix : "0.0.0.0/0"
nexthop : "10.1.1.1"
options : {}
output_port : []
policy : []
route_table : ""
The inferred route is normally installed by this portion of code:
https://github.com/openstack/neutron/blob/21927e79075ce0f3e521e56fca0bed8f1de61066/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py#L1264-L1279
Based on the result from _get_gw_info:
https://github.com/openstack/neutron/blob/21927e79075ce0f3e521e56fca0bed8f1de61066/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py#L1197-L1204
`_get_gw_info` returns an empty list since `external_fixed_ips` is an
empty list:
self._l3_plugin.get_router(context, 'd51ec4b0-c847-41e0-b43d-5dbf8ddcca32')
{'id': 'd51ec4b0-c847-41e0-b43d-5dbf8ddcca32', 'name': 'r1', 'tenant_id': 'dbfcc6c6a50f481685fda546abd00cd3', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': 'eef0120b-d01f-4cf7-9d1a-65f1da1eb67c', 'external_fixed_ips': [], 'enable_snat': True}, 'gw_port_id': '2da99728-b04e-4a4f-ac6f-d0930de8264a', 'description': '', 'availability_zones': [], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'tags': [], 'created_at': '2023-01-20T09:45:55Z', 'updated_at': '2023-01-24T12:44:14Z', 'revision_number': 35, 'project_id': 'dbfcc6c6a50f481685fda546abd00cd3'}
Meanwhile, the `external_fixed_ips` field is empty because of the
deferred IPAM logic triggered by the presence of `segment_id != None`
for the subnet on the external network. Based on this logic, the port is
unbound and does not get an IP allocation until a port update & port
binding:
https://github.com/openstack/neutron/blob/21927e79075ce0f3e521e56fca0bed8f1de61066/neutron/objects/subnet.py#L341-L343 (subnets attached to segments are excluded if a host isn't known)
https://github.com/openstack/neutron/blob/21927e79075ce0f3e521e56fca0bed8f1de61066/neutron/objects/subnet.py#L481-L486 (ipam_exceptions.DeferIpam is raised)
https://github.com/openstack/neutron/blob/21927e79075ce0f3e521e56fca0bed8f1de61066/neutron/db/db_base_plugin_v2.py#L1472-L1478 (DeferIpam is caught and the port gets IP_ALLOCATION_NONE for its IP allocation as it has no fixed ips.
Port state after it gets created in the unbound state (the code trying
to add a default route is trying to find fixed IPs at the same time the
gateway port is unbound and does not have any):
openstack port list --router r1
+--------------------------------------+------+-------------------+----------------------------------------------------------------------------+--------+
| ID | Name | MAC Address | Fixed IP Addresses | Status |
+--------------------------------------+------+-------------------+----------------------------------------------------------------------------+--------+
| 2da99728-b04e-4a4f-ac6f-d0930de8264a | | fa:16:3e:eb:cf:76 | | DOWN |
| 97d604f2-addb-46b8-9eaf-745257dddb2f | | fa:16:3e:c8:73:8b | ip_address='192.168.0.1', subnet_id='89227e7b-d2b0-4953-afe7-2b471736f85a' | ACTIVE |
+--------------------------------------+------+-------------------+----------------------------------------------------------------------------+--------+
openstack port show 2da99728-b04e-4a4f-ac6f-d0930de8264a
+-------------------------+--------------------------------------+
| Field | Value |
+-------------------------+--------------------------------------+
| admin_state_up | UP |
| allowed_address_pairs | |
| binding_host_id | |
| binding_profile | |
| binding_vif_details | |
| binding_vif_type | unbound |
| binding_vnic_type | normal |
| created_at | 2023-01-24T12:42:44Z |
| data_plane_status | None |
| description | |
| device_id | d51ec4b0-c847-41e0-b43d-5dbf8ddcca32 |
| device_owner | network:router_gateway |
| device_profile | None |
| dns_assignment | None |
| dns_domain | None |
| dns_name | None |
| extra_dhcp_opts | |
| fixed_ips | |
| id | 2da99728-b04e-4a4f-ac6f-d0930de8264a |
| ip_allocation | deferred |
| mac_address | fa:16:3e:eb:cf:76 |
| name | |
| network_id | eef0120b-d01f-4cf7-9d1a-65f1da1eb67c |
| numa_affinity_policy | None |
| port_security_enabled | False |
| project_id | |
| propagate_uplink_status | None |
| qos_network_policy_id | None |
| qos_policy_id | None |
| resource_request | None |
| revision_number | 1 |
| security_group_ids | |
| status | DOWN |
| tags | |
| trunk_details | None |
| updated_at | 2023-01-24T12:42:44Z |
+-------------------------+--------------------------------------+
Tested on Yoga, references are for the master branch.
** Affects: neutron
Importance: Undecided
Status: New
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/2003842
Title:
[OVN] A route inferred from a subnet's default gateway is not added to
ovn-nb if segment_id is not None for a subnet
Status in neutron:
New
Bug description:
Context:
* Neutron is configured to use OVN
* An external provider network with one segment is created
* A subnet with a default gateway IP set is associated with this segment explicitly (segment_id != None)
* A router's gateway port is set to use the provider network (external_gateway_info is set with a network_id passed)
Result: OVN NB does not contain a default route and instance traffic
is blackholed.
--
Detailed description:
The first time a external gateway info is set as follows
$ openstack router set --external-gateway pubnet r1
does not result in OVN getting a default route with the next-hop set
to the subnet's gateway IP:
$ sudo ovn-nbctl list logical_router_static_route ; echo $?
0
Doing it twice in a row does (the default route appears in the table
after the second command):
$ openstack router set --external-gateway pubnet r1 && openstack
router set --external-gateway pubnet r1
$ sudo ovn-nbctl list logical_router_static_route
_uuid : df7c6020-83e7-446c-8f5c-31db96eb2dd3
bfd : []
external_ids : {"neutron:is_ext_gw"="true", "neutron:subnet_id"="abdae752-034c-4845-b6b3-92bf40cf24a6"}
ip_prefix : "0.0.0.0/0"
nexthop : "10.1.1.1"
options : {}
output_port : []
policy : []
route_table : ""
The inferred route is normally installed by this portion of code:
https://github.com/openstack/neutron/blob/21927e79075ce0f3e521e56fca0bed8f1de61066/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py#L1264-L1279
Based on the result from _get_gw_info:
https://github.com/openstack/neutron/blob/21927e79075ce0f3e521e56fca0bed8f1de61066/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py#L1197-L1204
`_get_gw_info` returns an empty list since `external_fixed_ips` is an
empty list:
self._l3_plugin.get_router(context, 'd51ec4b0-c847-41e0-b43d-5dbf8ddcca32')
{'id': 'd51ec4b0-c847-41e0-b43d-5dbf8ddcca32', 'name': 'r1', 'tenant_id': 'dbfcc6c6a50f481685fda546abd00cd3', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': 'eef0120b-d01f-4cf7-9d1a-65f1da1eb67c', 'external_fixed_ips': [], 'enable_snat': True}, 'gw_port_id': '2da99728-b04e-4a4f-ac6f-d0930de8264a', 'description': '', 'availability_zones': [], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'tags': [], 'created_at': '2023-01-20T09:45:55Z', 'updated_at': '2023-01-24T12:44:14Z', 'revision_number': 35, 'project_id': 'dbfcc6c6a50f481685fda546abd00cd3'}
Meanwhile, the `external_fixed_ips` field is empty because of the
deferred IPAM logic triggered by the presence of `segment_id != None`
for the subnet on the external network. Based on this logic, the port
is unbound and does not get an IP allocation until a port update &
port binding:
https://github.com/openstack/neutron/blob/21927e79075ce0f3e521e56fca0bed8f1de61066/neutron/objects/subnet.py#L341-L343 (subnets attached to segments are excluded if a host isn't known)
https://github.com/openstack/neutron/blob/21927e79075ce0f3e521e56fca0bed8f1de61066/neutron/objects/subnet.py#L481-L486 (ipam_exceptions.DeferIpam is raised)
https://github.com/openstack/neutron/blob/21927e79075ce0f3e521e56fca0bed8f1de61066/neutron/db/db_base_plugin_v2.py#L1472-L1478 (DeferIpam is caught and the port gets IP_ALLOCATION_NONE for its IP allocation as it has no fixed ips.
Port state after it gets created in the unbound state (the code trying
to add a default route is trying to find fixed IPs at the same time
the gateway port is unbound and does not have any):
openstack port list --router r1
+--------------------------------------+------+-------------------+----------------------------------------------------------------------------+--------+
| ID | Name | MAC Address | Fixed IP Addresses | Status |
+--------------------------------------+------+-------------------+----------------------------------------------------------------------------+--------+
| 2da99728-b04e-4a4f-ac6f-d0930de8264a | | fa:16:3e:eb:cf:76 | | DOWN |
| 97d604f2-addb-46b8-9eaf-745257dddb2f | | fa:16:3e:c8:73:8b | ip_address='192.168.0.1', subnet_id='89227e7b-d2b0-4953-afe7-2b471736f85a' | ACTIVE |
+--------------------------------------+------+-------------------+----------------------------------------------------------------------------+--------+
openstack port show 2da99728-b04e-4a4f-ac6f-d0930de8264a
+-------------------------+--------------------------------------+
| Field | Value |
+-------------------------+--------------------------------------+
| admin_state_up | UP |
| allowed_address_pairs | |
| binding_host_id | |
| binding_profile | |
| binding_vif_details | |
| binding_vif_type | unbound |
| binding_vnic_type | normal |
| created_at | 2023-01-24T12:42:44Z |
| data_plane_status | None |
| description | |
| device_id | d51ec4b0-c847-41e0-b43d-5dbf8ddcca32 |
| device_owner | network:router_gateway |
| device_profile | None |
| dns_assignment | None |
| dns_domain | None |
| dns_name | None |
| extra_dhcp_opts | |
| fixed_ips | |
| id | 2da99728-b04e-4a4f-ac6f-d0930de8264a |
| ip_allocation | deferred |
| mac_address | fa:16:3e:eb:cf:76 |
| name | |
| network_id | eef0120b-d01f-4cf7-9d1a-65f1da1eb67c |
| numa_affinity_policy | None |
| port_security_enabled | False |
| project_id | |
| propagate_uplink_status | None |
| qos_network_policy_id | None |
| qos_policy_id | None |
| resource_request | None |
| revision_number | 1 |
| security_group_ids | |
| status | DOWN |
| tags | |
| trunk_details | None |
| updated_at | 2023-01-24T12:42:44Z |
+-------------------------+--------------------------------------+
Tested on Yoga, references are for the master branch.
To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/2003842/+subscriptions
Follow ups
