| Thread Previous • Date Previous • Date Next • Thread Next |
Public bug reported:
We install DVR to src mac flow only for fixed IPs but not for AAPs. As
result its not possible to communicate to IP address specified in AAP
via distributed router.
https://github.com/openstack/neutron/blob/f2c3e3e68c0fa77cff5cdd22d847e3f6ea7cdd84/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_dvr_neutron_agent.py#L569
Issue may be reproduced by creating the following topology.
ump-01 (instance) --- netA (10.10.11.0/24) --- router ---- net B
(192.168.0.0/24) ---- fw-01 (instance VIP: 192.168.0.10/32)
Port for instance B1
+-------------------------+---------------------------------------------------------------------------------------------------------------------------------------------+
| Field | Value |
+-------------------------+---------------------------------------------------------------------------------------------------------------------------------------------+
| admin_state_up | UP |
| allowed_address_pairs | ip_address='192.168.0.10/32', mac_address='00:00:5e:00:01:32' |
| | ip_address='0.0.0.0/0', mac_address='fa:16:3e:0c:7d:6c' |
| binding_host_id | vs-ps-y7eajqtcjwcm-2-v3xgrdrpb2ao-server-6rwxod6xp2yi |
| binding_profile | |
| binding_vif_details | bound_drivers.0='openvswitch', bridge_name='br-int', connectivity='l2', datapath_type='system', ovs_hybrid_plug='False', port_filter='True' |
| binding_vif_type | ovs |
| binding_vnic_type | normal |
| created_at | 2025-01-08T11:13:05Z |
| data_plane_status | None |
| description | |
| device_id | dd41d063-717f-4795-99b2-00783bce81f6 |
| device_owner | compute:nova |
| device_profile | None |
| dns_assignment | fqdn='oc-virtual-lab-server-fw-01.openstack.internal.', hostname='oc-virtual-lab-server-fw-01', ip_address='192.168.0.63' |
| dns_domain | |
| dns_name | oc-virtual-lab-server-fw-01 |
| extra_dhcp_opts | |
| fixed_ips | ip_address='192.168.0.188', subnet_id='7425f886-b2db-4474-af23-dc8e164be243' |
| hardware_offload_type | None |
| hints | |
| id | e71f0314-35a2-4a5c-8ac4-2216ded5addd |
| ip_allocation | None |
| mac_address | fa:16:3e:17:78:99 |
| name | oc-virtual-lab-port-fw-01 |
| network_id | fefa44ca-5b56-46f6-a31a-3e0a67b99ffe |
| numa_affinity_policy | None |
| port_security_enabled | True |
| project_id | 59592b24e56641afa2d7dc705da006e7 |
| propagate_uplink_status | None |
| resource_request | None |
| revision_number | 6 |
| qos_network_policy_id | None |
| qos_policy_id | None |
| security_group_ids | d926157a-91f0-491b-980a-3ab33d7e4a50 |
| status | ACTIVE |
| tags | |
| trunk_details | None |
| updated_at | 2025-01-08T11:21:56Z |
+-------------------------+---------------------------------------------------------------------------------------------------------------------------------------------+
port show for A1
+-------------------------+---------------------------------------------------------------------------------------------------------------------------------------------+
| Field | Value |
+-------------------------+---------------------------------------------------------------------------------------------------------------------------------------------+
| admin_state_up | UP |
| allowed_address_pairs | |
| binding_host_id | vs-ps-y7eajqtcjwcm-2-v3xgrdrpb2ao-server-6rwxod6xp2yi |
| binding_profile | |
| binding_vif_details | bound_drivers.0='openvswitch', bridge_name='br-int', connectivity='l2', datapath_type='system', ovs_hybrid_plug='False', port_filter='True' |
| binding_vif_type | ovs |
| binding_vnic_type | normal |
| created_at | 2025-01-08T11:13:05Z |
| data_plane_status | None |
| description | |
| device_id | c9844224-7154-4b91-a15f-92bb515964d7 |
| device_owner | compute:nova |
| device_profile | None |
| dns_assignment | fqdn='oc-virtual-lab-server-jump-01.openstack.internal.', hostname='oc-virtual-lab-server-jump-01', ip_address='10.10.11.203' |
| dns_domain | |
| dns_name | oc-virtual-lab-server-jump-01 |
| extra_dhcp_opts | |
| fixed_ips | ip_address='10.10.11.203', subnet_id='6cc27493-bafc-4c93-9290-1cefd6c74bd0' |
| hardware_offload_type | None |
| hints | |
| id | 006cca82-c38c-4311-a6db-0fc5acc6e977 |
| ip_allocation | None |
| mac_address | fa:16:3e:d1:4d:dc |
| name | oc-virtual-lab-jump-port-jump-01 |
| network_id | 8392d7ee-0c5f-46f6-805f-527afe68b5f8 |
| numa_affinity_policy | None |
| port_security_enabled | False |
| project_id | 59592b24e56641afa2d7dc705da006e7 |
| propagate_uplink_status | None |
| resource_request | None |
| revision_number | 4 |
| qos_network_policy_id | None |
| qos_policy_id | None |
| security_group_ids | |
| status | ACTIVE |
| tags | |
| trunk_details | None |
| updated_at | 2025-01-08T11:13:36Z |
+-------------------------+---------------------------------------------------------------------------------------------------------------------------------------------+
On instance fw-01 start VIP 192.168.0.10/32 with keepalived and
following configuration
ubuntu@oc-virtual-lab-server-fw-01:~$ cat /etc/keepalived/keepalived.conf
vrrp_instance VI_1 {
interface ens3
virtual_router_id 50
nopreempt
priority 100
advert_int 1
virtual_ipaddress {
192.168.0.10/32
}
notify_master "/etc/keepalived/notify_master.sh"
use_vmac
vmac_xmit_base
}
and sysctl settings
ubuntu@oc-virtual-lab-server-fw-01:~$ cat /etc/sysctl.d/vrrp.conf
net.ipv4.conf.all.arp_ignore=1
net.ipv4.conf.all.arp_announce=1
net.ipv4.conf.all.arp_filter=0
net.ipv4.conf.ens3.arp_filter = 1
When jump-01 and fw-01 are on same host connectivity works, but when
they are on different hypervisors its failed due to missing dvr flow
ovs-ofctl -O OpenFlow14 dump-flows br-int |grep table=1
cookie=0xf2a4abee1cb41511, duration=328.546s, table=1, n_packets=3179, n_bytes=231080, priority=20,dl_vlan=5,dl_dst=fa:16:3e:17:78:99 actions=set_field:fa:16:3e:59:84:ef->eth_src,goto_table:58
cookie=0xf2a4abee1cb41511, duration=336.209s, table=1, n_packets=116933, n_bytes=11456946, priority=1 actions=drop
If add this flow connectivity is restored
ovs-ofctl -O OpenFlow14 add-flow br-int 'table=1,priority=20,dl_vlan=5,dl_dst=00:00:5e:00:01:32,actions=set_field:fa:16:3e:59:84:ef->eth_src,goto_table:58'
** Affects: neutron
Importance: Undecided
Status: New
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/2093248
Title:
DVR flow is not installed for allowed address pairs mac in
DVR_TO_SRC_MAC table=1
Status in neutron:
New
Bug description:
We install DVR to src mac flow only for fixed IPs but not for AAPs. As
result its not possible to communicate to IP address specified in AAP
via distributed router.
https://github.com/openstack/neutron/blob/f2c3e3e68c0fa77cff5cdd22d847e3f6ea7cdd84/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_dvr_neutron_agent.py#L569
Issue may be reproduced by creating the following topology.
ump-01 (instance) --- netA (10.10.11.0/24) --- router ---- net B
(192.168.0.0/24) ---- fw-01 (instance VIP: 192.168.0.10/32)
Port for instance B1
+-------------------------+---------------------------------------------------------------------------------------------------------------------------------------------+
| Field | Value |
+-------------------------+---------------------------------------------------------------------------------------------------------------------------------------------+
| admin_state_up | UP |
| allowed_address_pairs | ip_address='192.168.0.10/32', mac_address='00:00:5e:00:01:32' |
| | ip_address='0.0.0.0/0', mac_address='fa:16:3e:0c:7d:6c' |
| binding_host_id | vs-ps-y7eajqtcjwcm-2-v3xgrdrpb2ao-server-6rwxod6xp2yi |
| binding_profile | |
| binding_vif_details | bound_drivers.0='openvswitch', bridge_name='br-int', connectivity='l2', datapath_type='system', ovs_hybrid_plug='False', port_filter='True' |
| binding_vif_type | ovs |
| binding_vnic_type | normal |
| created_at | 2025-01-08T11:13:05Z |
| data_plane_status | None |
| description | |
| device_id | dd41d063-717f-4795-99b2-00783bce81f6 |
| device_owner | compute:nova |
| device_profile | None |
| dns_assignment | fqdn='oc-virtual-lab-server-fw-01.openstack.internal.', hostname='oc-virtual-lab-server-fw-01', ip_address='192.168.0.63' |
| dns_domain | |
| dns_name | oc-virtual-lab-server-fw-01 |
| extra_dhcp_opts | |
| fixed_ips | ip_address='192.168.0.188', subnet_id='7425f886-b2db-4474-af23-dc8e164be243' |
| hardware_offload_type | None |
| hints | |
| id | e71f0314-35a2-4a5c-8ac4-2216ded5addd |
| ip_allocation | None |
| mac_address | fa:16:3e:17:78:99 |
| name | oc-virtual-lab-port-fw-01 |
| network_id | fefa44ca-5b56-46f6-a31a-3e0a67b99ffe |
| numa_affinity_policy | None |
| port_security_enabled | True |
| project_id | 59592b24e56641afa2d7dc705da006e7 |
| propagate_uplink_status | None |
| resource_request | None |
| revision_number | 6 |
| qos_network_policy_id | None |
| qos_policy_id | None |
| security_group_ids | d926157a-91f0-491b-980a-3ab33d7e4a50 |
| status | ACTIVE |
| tags | |
| trunk_details | None |
| updated_at | 2025-01-08T11:21:56Z |
+-------------------------+---------------------------------------------------------------------------------------------------------------------------------------------+
port show for A1
+-------------------------+---------------------------------------------------------------------------------------------------------------------------------------------+
| Field | Value |
+-------------------------+---------------------------------------------------------------------------------------------------------------------------------------------+
| admin_state_up | UP |
| allowed_address_pairs | |
| binding_host_id | vs-ps-y7eajqtcjwcm-2-v3xgrdrpb2ao-server-6rwxod6xp2yi |
| binding_profile | |
| binding_vif_details | bound_drivers.0='openvswitch', bridge_name='br-int', connectivity='l2', datapath_type='system', ovs_hybrid_plug='False', port_filter='True' |
| binding_vif_type | ovs |
| binding_vnic_type | normal |
| created_at | 2025-01-08T11:13:05Z |
| data_plane_status | None |
| description | |
| device_id | c9844224-7154-4b91-a15f-92bb515964d7 |
| device_owner | compute:nova |
| device_profile | None |
| dns_assignment | fqdn='oc-virtual-lab-server-jump-01.openstack.internal.', hostname='oc-virtual-lab-server-jump-01', ip_address='10.10.11.203' |
| dns_domain | |
| dns_name | oc-virtual-lab-server-jump-01 |
| extra_dhcp_opts | |
| fixed_ips | ip_address='10.10.11.203', subnet_id='6cc27493-bafc-4c93-9290-1cefd6c74bd0' |
| hardware_offload_type | None |
| hints | |
| id | 006cca82-c38c-4311-a6db-0fc5acc6e977 |
| ip_allocation | None |
| mac_address | fa:16:3e:d1:4d:dc |
| name | oc-virtual-lab-jump-port-jump-01 |
| network_id | 8392d7ee-0c5f-46f6-805f-527afe68b5f8 |
| numa_affinity_policy | None |
| port_security_enabled | False |
| project_id | 59592b24e56641afa2d7dc705da006e7 |
| propagate_uplink_status | None |
| resource_request | None |
| revision_number | 4 |
| qos_network_policy_id | None |
| qos_policy_id | None |
| security_group_ids | |
| status | ACTIVE |
| tags | |
| trunk_details | None |
| updated_at | 2025-01-08T11:13:36Z |
+-------------------------+---------------------------------------------------------------------------------------------------------------------------------------------+
On instance fw-01 start VIP 192.168.0.10/32 with keepalived and
following configuration
ubuntu@oc-virtual-lab-server-fw-01:~$ cat /etc/keepalived/keepalived.conf
vrrp_instance VI_1 {
interface ens3
virtual_router_id 50
nopreempt
priority 100
advert_int 1
virtual_ipaddress {
192.168.0.10/32
}
notify_master "/etc/keepalived/notify_master.sh"
use_vmac
vmac_xmit_base
}
and sysctl settings
ubuntu@oc-virtual-lab-server-fw-01:~$ cat /etc/sysctl.d/vrrp.conf
net.ipv4.conf.all.arp_ignore=1
net.ipv4.conf.all.arp_announce=1
net.ipv4.conf.all.arp_filter=0
net.ipv4.conf.ens3.arp_filter = 1
When jump-01 and fw-01 are on same host connectivity works, but when
they are on different hypervisors its failed due to missing dvr flow
ovs-ofctl -O OpenFlow14 dump-flows br-int |grep table=1
cookie=0xf2a4abee1cb41511, duration=328.546s, table=1, n_packets=3179, n_bytes=231080, priority=20,dl_vlan=5,dl_dst=fa:16:3e:17:78:99 actions=set_field:fa:16:3e:59:84:ef->eth_src,goto_table:58
cookie=0xf2a4abee1cb41511, duration=336.209s, table=1, n_packets=116933, n_bytes=11456946, priority=1 actions=drop
If add this flow connectivity is restored
ovs-ofctl -O OpenFlow14 add-flow br-int 'table=1,priority=20,dl_vlan=5,dl_dst=00:00:5e:00:01:32,actions=set_field:fa:16:3e:59:84:ef->eth_src,goto_table:58'
To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/2093248/+subscriptions
| Thread Previous • Date Previous • Date Next • Thread Next |
