yahoo-eng-team team mailing list archive
-
yahoo-eng-team team
-
Mailing list archive
-
Message #81872
[Bug 1866615] [NEW] Packets incorrectly marked as martian
Public bug reported:
Problem:
The following behaviour is observed:
The deployment has 2 provider networks. One of the them is the public one and another is
getting outside but through NAT. This second one is the one that they hypervisors use and this is what we have as "openstack public" (10.20.6.X). The VMs that are launched are attached to the fabric on the 10.10 public network. Therefore that network is not present on the hypervisor NICS. What we observe is that the switch is sending ARP requests (correctly) from the .250 active standby IP but the kernel is marking them as Martian despite the fact that neutron knows this network.
System:
triple-O Based Rocky Deployment . VXLAN tunneling, DVR enabled with Bond interfaces on 2 switches. (Open vSwitch) 2.11.0, Neutron 13.0.5
kernel: 3.10.0-957.21.3.el7.x86_64
Host OS: CentOS
Switches: Arista
------------------ --------------- ---------------
| SWITCH | | HYPERVISOR | | VM |
| 10.10.91.250 | --- | 10.20.6.X | --- | 10.10.X.Y |
------------------ --------------- ---------------
Subnet details :
+-------------------+--------------------------------------+
| Field | Value |
+-------------------+--------------------------------------+
| allocation_pools | 10.10.90.10-10.10.91.240 |
| cidr | 10.10.90.0/23 |
| created_at | 2019-09-24T08:43:54Z |
| description | |
| dns_nameservers | 10.10.D.Y |
| enable_dhcp | True |
| gateway_ip | 10.10.91.254 |
| host_routes | |
| id | f91d725a-89d1-4a32-97b5-95409177e8eb |
| ip_version | 4 |
| ipv6_address_mode | None |
| ipv6_ra_mode | None |
| name | public-subnet |
| network_id | a1a3280b-9c78-4e5f-883a-9b4bc4e72b1f |
| project_id | ec9851ba91854e10bb8d5e752260f5fd |
| revision_number | 14 |
| segment_id | None |
| service_types | |
| subnetpool_id | None |
| tags | |
| updated_at | 2020-03-03T14:34:26Z |
+-------------------+--------------------------------------+
cat openvswitch_agent.ini
[agent]
l2_population=True
arp_responder=True
enable_distributed_routing=True
drop_flows_on_start=False
extensions=qos
tunnel_csum=False
tunnel_types=vxlan
vxlan_udp_port=4789
[securitygroup]
firewall_driver=iptables_hybrid
Expected output:
No Martian Packets observed
Actual output:
Since The extra provider network is configured I would expect that the linux kernel would not mark the incoming packets as martian.
However,
Mar 9 10:45:41 compute0 kernel: IPv4: martian source 10.10.90.74 from 10.10.91.250 on dev qbrff08c591-e2
Mar 9 10:45:41 compute0 kernel: ll header: 00000000: ff ff ff ff ff ff 98 5d 82 a1 a6 cd 08 06 .......]......
Mar 9 10:45:42 compute0 kernel: IPv4: martian source 10.10.90.74 from 10.10.91.250 on dev qbrff08c591-e2
Mar 9 10:45:42 compute0 kernel: ll header: 00000000: ff ff ff ff ff ff 98 5d 82 a1 a6 cd 08 06 .......]......
Mar 9 10:45:43 compute0 kernel: IPv4: martian source 10.10.91.203 from 10.10.91.250 on dev qbrff08c591-e2
Mar 9 10:45:43 compute0 kernel: ll header: 00000000: ff ff ff ff ff ff 98 5d 82 a1 a6 cd 08 06 .......]......
Mar 9 10:45:44 compute0 kernel: IPv4: martian source 10.10.90.74 from 10.10.91.250 on dev qbrff08c591-e2
Mar 9 10:45:44 compute0 kernel: ll header: 00000000: ff ff ff ff ff ff 98 5d 82 a1 a6 cd 08 06 .......]......
Mar 9 10:45:44 compute0 kernel: IPv4: martian source 10.10.91.203 from 10.10.91.250 on dev qbrff08c591-e2
Mar 9 10:45:44 compute0 kernel: ll header: 00000000: ff ff ff ff ff ff 98 5d 82 a1 a6 cd 08 06 .......]......
Perceived severity:
Minor annoyance since /var/log/messages is flooded.
Minor security vulnerability since disabling martian packet logging will hide a real case of martian packet flooding
** Affects: neutron
Importance: Undecided
Status: New
** Tags: logging
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1866615
Title:
Packets incorrectly marked as martian
Status in neutron:
New
Bug description:
Problem:
The following behaviour is observed:
The deployment has 2 provider networks. One of the them is the public one and another is
getting outside but through NAT. This second one is the one that they hypervisors use and this is what we have as "openstack public" (10.20.6.X). The VMs that are launched are attached to the fabric on the 10.10 public network. Therefore that network is not present on the hypervisor NICS. What we observe is that the switch is sending ARP requests (correctly) from the .250 active standby IP but the kernel is marking them as Martian despite the fact that neutron knows this network.
System:
triple-O Based Rocky Deployment . VXLAN tunneling, DVR enabled with Bond interfaces on 2 switches. (Open vSwitch) 2.11.0, Neutron 13.0.5
kernel: 3.10.0-957.21.3.el7.x86_64
Host OS: CentOS
Switches: Arista
------------------ --------------- ---------------
| SWITCH | | HYPERVISOR | | VM |
| 10.10.91.250 | --- | 10.20.6.X | --- | 10.10.X.Y |
------------------ --------------- ---------------
Subnet details :
+-------------------+--------------------------------------+
| Field | Value |
+-------------------+--------------------------------------+
| allocation_pools | 10.10.90.10-10.10.91.240 |
| cidr | 10.10.90.0/23 |
| created_at | 2019-09-24T08:43:54Z |
| description | |
| dns_nameservers | 10.10.D.Y |
| enable_dhcp | True |
| gateway_ip | 10.10.91.254 |
| host_routes | |
| id | f91d725a-89d1-4a32-97b5-95409177e8eb |
| ip_version | 4 |
| ipv6_address_mode | None |
| ipv6_ra_mode | None |
| name | public-subnet |
| network_id | a1a3280b-9c78-4e5f-883a-9b4bc4e72b1f |
| project_id | ec9851ba91854e10bb8d5e752260f5fd |
| revision_number | 14 |
| segment_id | None |
| service_types | |
| subnetpool_id | None |
| tags | |
| updated_at | 2020-03-03T14:34:26Z |
+-------------------+--------------------------------------+
cat openvswitch_agent.ini
[agent]
l2_population=True
arp_responder=True
enable_distributed_routing=True
drop_flows_on_start=False
extensions=qos
tunnel_csum=False
tunnel_types=vxlan
vxlan_udp_port=4789
[securitygroup]
firewall_driver=iptables_hybrid
Expected output:
No Martian Packets observed
Actual output:
Since The extra provider network is configured I would expect that the linux kernel would not mark the incoming packets as martian.
However,
Mar 9 10:45:41 compute0 kernel: IPv4: martian source 10.10.90.74 from 10.10.91.250 on dev qbrff08c591-e2
Mar 9 10:45:41 compute0 kernel: ll header: 00000000: ff ff ff ff ff ff 98 5d 82 a1 a6 cd 08 06 .......]......
Mar 9 10:45:42 compute0 kernel: IPv4: martian source 10.10.90.74 from 10.10.91.250 on dev qbrff08c591-e2
Mar 9 10:45:42 compute0 kernel: ll header: 00000000: ff ff ff ff ff ff 98 5d 82 a1 a6 cd 08 06 .......]......
Mar 9 10:45:43 compute0 kernel: IPv4: martian source 10.10.91.203 from 10.10.91.250 on dev qbrff08c591-e2
Mar 9 10:45:43 compute0 kernel: ll header: 00000000: ff ff ff ff ff ff 98 5d 82 a1 a6 cd 08 06 .......]......
Mar 9 10:45:44 compute0 kernel: IPv4: martian source 10.10.90.74 from 10.10.91.250 on dev qbrff08c591-e2
Mar 9 10:45:44 compute0 kernel: ll header: 00000000: ff ff ff ff ff ff 98 5d 82 a1 a6 cd 08 06 .......]......
Mar 9 10:45:44 compute0 kernel: IPv4: martian source 10.10.91.203 from 10.10.91.250 on dev qbrff08c591-e2
Mar 9 10:45:44 compute0 kernel: ll header: 00000000: ff ff ff ff ff ff 98 5d 82 a1 a6 cd 08 06 .......]......
Perceived severity:
Minor annoyance since /var/log/messages is flooded.
Minor security vulnerability since disabling martian packet logging will hide a real case of martian packet flooding
To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1866615/+subscriptions
Follow ups
