← Back to team overview

yahoo-eng-team team mailing list archive

[Bug 1475786] [NEW] Cannot ping to a same subnet VM via floating IP

 

Public bug reported:

Suppose I have two VMs running and they are under the same subnet, they
are assigned with the floating IPs.  (See attached image). I am using
nova network model.

I cannot get ping working if I ping from one VM to another VM via the
floating IP. Ping to another vm which resides in another subnet via
floating IP seems fine.

I did some investigation, looks like the packet is being dropped after
the PREROUTING rules. Here is the modprobe iptable log:

Jul 17 10:15:40 localhost kernel: [ 1846.629048] TRACE: raw:PREROUTING:rule:2 IN=br100 OUT= PHYSIN=vlan100 MAC=fa:16:3e:c2:b9:7d:fa:16:3e:dd:e7:c9:08:00 SRC=10.0.0.3 DST=172.24.4.2 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=3620 DF PROTO=ICMP TYPE=8 CODE=0 ID=8705 SEQ=0 
Jul 17 10:15:40 localhost kernel: [ 1846.629055] TRACE: raw:PREROUTING:policy:3 IN=br100 OUT= PHYSIN=vlan100 MAC=fa:16:3e:c2:b9:7d:fa:16:3e:dd:e7:c9:08:00 SRC=10.0.0.3 DST=172.24.4.2 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=3620 DF PROTO=ICMP TYPE=8 CODE=0 ID=8705 SEQ=0 
Jul 17 10:15:40 localhost kernel: [ 1846.629063] TRACE: mangle:PREROUTING:policy:1 IN=br100 OUT= PHYSIN=vlan100 MAC=fa:16:3e:c2:b9:7d:fa:16:3e:dd:e7:c9:08:00 SRC=10.0.0.3 DST=172.24.4.2 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=3620 DF PROTO=ICMP TYPE=8 CODE=0 ID=8705 SEQ=0 
Jul 17 10:15:40 localhost kernel: [ 1846.629068] TRACE: nat:PREROUTING:rule:1 IN=br100 OUT= PHYSIN=vlan100 MAC=fa:16:3e:c2:b9:7d:fa:16:3e:dd:e7:c9:08:00 SRC=10.0.0.3 DST=172.24.4.2 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=3620 DF PROTO=ICMP TYPE=8 CODE=0 ID=8705 SEQ=0 
Jul 17 10:15:40 localhost kernel: [ 1846.629074] TRACE: nat:nova-network-PREROUTING:rule:3 IN=br100 OUT= PHYSIN=vlan100 MAC=fa:16:3e:c2:b9:7d:fa:16:3e:dd:e7:c9:08:00 SRC=10.0.0.3 DST=172.24.4.2 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=3620 DF PROTO=ICMP TYPE=8 CODE=0 ID=8705 SEQ=0 

And from the iptables counter, nothing got incremented after the
PREROUTING rule:

sudo iptables -t nat -L -v -n 
Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         
    1    84 nova-network-PREROUTING  all  --  *      *       0.0.0.0/0            0.0.0.0/0           
    0     0 nova-api-PREROUTING  all  --  *      *       0.0.0.0/0            0.0.0.0/0           

Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 nova-network-OUTPUT  all  --  *      *       0.0.0.0/0            0.0.0.0/0           
    0     0 nova-api-OUTPUT  all  --  *      *       0.0.0.0/0            0.0.0.0/0           

Chain POSTROUTING (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 nova-network-POSTROUTING  all  --  *      *       0.0.0.0/0            0.0.0.0/0           
    0     0 nova-api-POSTROUTING  all  --  *      *       0.0.0.0/0            0.0.0.0/0           
    0     0 nova-postrouting-bottom  all  --  *      *       0.0.0.0/0            0.0.0.0/0           

Chain nova-api-OUTPUT (1 references)
 pkts bytes target     prot opt in     out     source               destination         

Chain nova-api-POSTROUTING (1 references)
 pkts bytes target     prot opt in     out     source               destination         

Chain nova-api-PREROUTING (1 references)
 pkts bytes target     prot opt in     out     source               destination         

Chain nova-api-float-snat (1 references)
 pkts bytes target     prot opt in     out     source               destination         

Chain nova-api-snat (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 nova-api-float-snat  all  --  *      *       0.0.0.0/0            0.0.0.0/0           

Chain nova-network-OUTPUT (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 DNAT       all  --  *      *       0.0.0.0/0            172.24.4.1           to:11.0.0.3
    0     0 DNAT       all  --  *      *       0.0.0.0/0            172.24.4.2           to:10.0.0.4
    0     0 DNAT       all  --  *      *       0.0.0.0/0            172.24.4.3           to:10.0.0.3
    0     0 DNAT       all  --  *      *       0.0.0.0/0            172.24.4.4           to:11.0.0.4

Chain nova-network-POSTROUTING (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 ACCEPT     all  --  *      *       10.0.0.0/24          192.168.62.100      
    0     0 ACCEPT     all  --  *      *       10.0.0.0/24          10.0.0.0/24          ! ctstate DNAT
    0     0 ACCEPT     all  --  *      *       11.0.0.0/24          192.168.62.100      
    0     0 ACCEPT     all  --  *      *       11.0.0.0/24          11.0.0.0/24          ! ctstate DNAT
    0     0 SNAT       all  --  *      *       11.0.0.3             0.0.0.0/0            ctstate DNAT to:172.24.4.1
    0     0 SNAT       all  --  *      *       10.0.0.4             0.0.0.0/0            ctstate DNAT to:172.24.4.2
    0     0 SNAT       all  --  *      *       10.0.0.3             0.0.0.0/0            ctstate DNAT to:172.24.4.3
    0     0 SNAT       all  --  *      *       11.0.0.4             0.0.0.0/0            ctstate DNAT to:172.24.4.4

Chain nova-network-PREROUTING (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 DNAT       tcp  --  *      *       0.0.0.0/0            169.254.169.254      tcp dpt:80 to:192.168.62.100:8775
    0     0 DNAT       all  --  *      *       0.0.0.0/0            172.24.4.1           to:11.0.0.3
    1    84 DNAT       all  --  *      *       0.0.0.0/0            172.24.4.2           to:10.0.0.4
    0     0 DNAT       all  --  *      *       0.0.0.0/0            172.24.4.3           to:10.0.0.3
    0     0 DNAT       all  --  *      *       0.0.0.0/0            172.24.4.4           to:11.0.0.4

Chain nova-network-float-snat (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 SNAT       all  --  *      *       11.0.0.3             11.0.0.3             to:172.24.4.1
    0     0 SNAT       all  --  *      eth1    11.0.0.3             0.0.0.0/0            to:172.24.4.1
    0     0 SNAT       all  --  *      *       10.0.0.4             10.0.0.4             to:172.24.4.2
    0     0 SNAT       all  --  *      eth1    10.0.0.4             0.0.0.0/0            to:172.24.4.2
    0     0 SNAT       all  --  *      *       10.0.0.3             10.0.0.3             to:172.24.4.3
    0     0 SNAT       all  --  *      eth1    10.0.0.3             0.0.0.0/0            to:172.24.4.3
    0     0 SNAT       all  --  *      *       11.0.0.4             11.0.0.4             to:172.24.4.4
    0     0 SNAT       all  --  *      eth1    11.0.0.4             0.0.0.0/0            to:172.24.4.4

Chain nova-network-snat (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 nova-network-float-snat  all  --  *      *       0.0.0.0/0            0.0.0.0/0           
    0     0 SNAT       all  --  *      eth1    10.0.0.0/24          0.0.0.0/0            to:192.168.62.100
    0     0 SNAT       all  --  *      eth1    11.0.0.0/24          0.0.0.0/0            to:192.168.62.100

Chain nova-postrouting-bottom (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 nova-network-snat  all  --  *      *       0.0.0.0/0            0.0.0.0/0           
    0     0 nova-api-snat  all  --  *      *       0.0.0.0/0            0.0.0.0/0

** Affects: nova
     Importance: Undecided
         Status: New


** Tags: floating ip

** Attachment added: "Screen Shot 2015-07-17 at 12.06.09 PM.png"
   https://bugs.launchpad.net/bugs/1475786/+attachment/4430715/+files/Screen%20Shot%202015-07-17%20at%2012.06.09%20PM.png

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1475786

Title:
  Cannot ping to a same subnet VM via floating IP

Status in OpenStack Compute (nova):
  New

Bug description:
  Suppose I have two VMs running and they are under the same subnet,
  they are assigned with the floating IPs.  (See attached image). I am
  using nova network model.

  I cannot get ping working if I ping from one VM to another VM via the
  floating IP. Ping to another vm which resides in another subnet via
  floating IP seems fine.

  I did some investigation, looks like the packet is being dropped after
  the PREROUTING rules. Here is the modprobe iptable log:

  Jul 17 10:15:40 localhost kernel: [ 1846.629048] TRACE: raw:PREROUTING:rule:2 IN=br100 OUT= PHYSIN=vlan100 MAC=fa:16:3e:c2:b9:7d:fa:16:3e:dd:e7:c9:08:00 SRC=10.0.0.3 DST=172.24.4.2 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=3620 DF PROTO=ICMP TYPE=8 CODE=0 ID=8705 SEQ=0 
  Jul 17 10:15:40 localhost kernel: [ 1846.629055] TRACE: raw:PREROUTING:policy:3 IN=br100 OUT= PHYSIN=vlan100 MAC=fa:16:3e:c2:b9:7d:fa:16:3e:dd:e7:c9:08:00 SRC=10.0.0.3 DST=172.24.4.2 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=3620 DF PROTO=ICMP TYPE=8 CODE=0 ID=8705 SEQ=0 
  Jul 17 10:15:40 localhost kernel: [ 1846.629063] TRACE: mangle:PREROUTING:policy:1 IN=br100 OUT= PHYSIN=vlan100 MAC=fa:16:3e:c2:b9:7d:fa:16:3e:dd:e7:c9:08:00 SRC=10.0.0.3 DST=172.24.4.2 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=3620 DF PROTO=ICMP TYPE=8 CODE=0 ID=8705 SEQ=0 
  Jul 17 10:15:40 localhost kernel: [ 1846.629068] TRACE: nat:PREROUTING:rule:1 IN=br100 OUT= PHYSIN=vlan100 MAC=fa:16:3e:c2:b9:7d:fa:16:3e:dd:e7:c9:08:00 SRC=10.0.0.3 DST=172.24.4.2 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=3620 DF PROTO=ICMP TYPE=8 CODE=0 ID=8705 SEQ=0 
  Jul 17 10:15:40 localhost kernel: [ 1846.629074] TRACE: nat:nova-network-PREROUTING:rule:3 IN=br100 OUT= PHYSIN=vlan100 MAC=fa:16:3e:c2:b9:7d:fa:16:3e:dd:e7:c9:08:00 SRC=10.0.0.3 DST=172.24.4.2 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=3620 DF PROTO=ICMP TYPE=8 CODE=0 ID=8705 SEQ=0 

  And from the iptables counter, nothing got incremented after the
  PREROUTING rule:

  sudo iptables -t nat -L -v -n 
  Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes)
   pkts bytes target     prot opt in     out     source               destination         
      1    84 nova-network-PREROUTING  all  --  *      *       0.0.0.0/0            0.0.0.0/0           
      0     0 nova-api-PREROUTING  all  --  *      *       0.0.0.0/0            0.0.0.0/0           

  Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
   pkts bytes target     prot opt in     out     source               destination         

  Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
   pkts bytes target     prot opt in     out     source               destination         
      0     0 nova-network-OUTPUT  all  --  *      *       0.0.0.0/0            0.0.0.0/0           
      0     0 nova-api-OUTPUT  all  --  *      *       0.0.0.0/0            0.0.0.0/0           

  Chain POSTROUTING (policy ACCEPT 0 packets, 0 bytes)
   pkts bytes target     prot opt in     out     source               destination         
      0     0 nova-network-POSTROUTING  all  --  *      *       0.0.0.0/0            0.0.0.0/0           
      0     0 nova-api-POSTROUTING  all  --  *      *       0.0.0.0/0            0.0.0.0/0           
      0     0 nova-postrouting-bottom  all  --  *      *       0.0.0.0/0            0.0.0.0/0           

  Chain nova-api-OUTPUT (1 references)
   pkts bytes target     prot opt in     out     source               destination         

  Chain nova-api-POSTROUTING (1 references)
   pkts bytes target     prot opt in     out     source               destination         

  Chain nova-api-PREROUTING (1 references)
   pkts bytes target     prot opt in     out     source               destination         

  Chain nova-api-float-snat (1 references)
   pkts bytes target     prot opt in     out     source               destination         

  Chain nova-api-snat (1 references)
   pkts bytes target     prot opt in     out     source               destination         
      0     0 nova-api-float-snat  all  --  *      *       0.0.0.0/0            0.0.0.0/0           

  Chain nova-network-OUTPUT (1 references)
   pkts bytes target     prot opt in     out     source               destination         
      0     0 DNAT       all  --  *      *       0.0.0.0/0            172.24.4.1           to:11.0.0.3
      0     0 DNAT       all  --  *      *       0.0.0.0/0            172.24.4.2           to:10.0.0.4
      0     0 DNAT       all  --  *      *       0.0.0.0/0            172.24.4.3           to:10.0.0.3
      0     0 DNAT       all  --  *      *       0.0.0.0/0            172.24.4.4           to:11.0.0.4

  Chain nova-network-POSTROUTING (1 references)
   pkts bytes target     prot opt in     out     source               destination         
      0     0 ACCEPT     all  --  *      *       10.0.0.0/24          192.168.62.100      
      0     0 ACCEPT     all  --  *      *       10.0.0.0/24          10.0.0.0/24          ! ctstate DNAT
      0     0 ACCEPT     all  --  *      *       11.0.0.0/24          192.168.62.100      
      0     0 ACCEPT     all  --  *      *       11.0.0.0/24          11.0.0.0/24          ! ctstate DNAT
      0     0 SNAT       all  --  *      *       11.0.0.3             0.0.0.0/0            ctstate DNAT to:172.24.4.1
      0     0 SNAT       all  --  *      *       10.0.0.4             0.0.0.0/0            ctstate DNAT to:172.24.4.2
      0     0 SNAT       all  --  *      *       10.0.0.3             0.0.0.0/0            ctstate DNAT to:172.24.4.3
      0     0 SNAT       all  --  *      *       11.0.0.4             0.0.0.0/0            ctstate DNAT to:172.24.4.4

  Chain nova-network-PREROUTING (1 references)
   pkts bytes target     prot opt in     out     source               destination         
      0     0 DNAT       tcp  --  *      *       0.0.0.0/0            169.254.169.254      tcp dpt:80 to:192.168.62.100:8775
      0     0 DNAT       all  --  *      *       0.0.0.0/0            172.24.4.1           to:11.0.0.3
      1    84 DNAT       all  --  *      *       0.0.0.0/0            172.24.4.2           to:10.0.0.4
      0     0 DNAT       all  --  *      *       0.0.0.0/0            172.24.4.3           to:10.0.0.3
      0     0 DNAT       all  --  *      *       0.0.0.0/0            172.24.4.4           to:11.0.0.4

  Chain nova-network-float-snat (1 references)
   pkts bytes target     prot opt in     out     source               destination         
      0     0 SNAT       all  --  *      *       11.0.0.3             11.0.0.3             to:172.24.4.1
      0     0 SNAT       all  --  *      eth1    11.0.0.3             0.0.0.0/0            to:172.24.4.1
      0     0 SNAT       all  --  *      *       10.0.0.4             10.0.0.4             to:172.24.4.2
      0     0 SNAT       all  --  *      eth1    10.0.0.4             0.0.0.0/0            to:172.24.4.2
      0     0 SNAT       all  --  *      *       10.0.0.3             10.0.0.3             to:172.24.4.3
      0     0 SNAT       all  --  *      eth1    10.0.0.3             0.0.0.0/0            to:172.24.4.3
      0     0 SNAT       all  --  *      *       11.0.0.4             11.0.0.4             to:172.24.4.4
      0     0 SNAT       all  --  *      eth1    11.0.0.4             0.0.0.0/0            to:172.24.4.4

  Chain nova-network-snat (1 references)
   pkts bytes target     prot opt in     out     source               destination         
      0     0 nova-network-float-snat  all  --  *      *       0.0.0.0/0            0.0.0.0/0           
      0     0 SNAT       all  --  *      eth1    10.0.0.0/24          0.0.0.0/0            to:192.168.62.100
      0     0 SNAT       all  --  *      eth1    11.0.0.0/24          0.0.0.0/0            to:192.168.62.100

  Chain nova-postrouting-bottom (1 references)
   pkts bytes target     prot opt in     out     source               destination         
      0     0 nova-network-snat  all  --  *      *       0.0.0.0/0            0.0.0.0/0           
      0     0 nova-api-snat  all  --  *      *       0.0.0.0/0            0.0.0.0/0

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1475786/+subscriptions


Follow ups