openstack team mailing list archive

["Bruno Oliveira ~lychinus" <brunnop.oliveira@xxxxxxxxx>]

"(...)Do you have your vSwitch properly configured on your hyper-v host?(...)"

>> I can't say for sure, Peter, but I think so...

In troubleshooting we did (and are still doing) I can tell that
regardless of the network model that we're using (FLAT or VLAN
Network),
the instance that is provisioned on Hyper-V (for some reason) can't
reach the quantum-l3-agent "by default"
(I said "default" because, we just managed to do it after a hard, long
and boring troubleshoting,
 yet, we're not sure if that's how it should be done, indeed)

Since it's not something quick to explain, I'll present the scenario:
(I'm not sure if it might be a candidate for a fix in quantum-l3-agent,
 so quantum-devs might be interested too)


Here's how our network interfaces turns out, in our network controller:

==============================
External bridge network
==============================

Bridge "br-eth1"
        Port "br-eth1"
            Interface "br-eth1"
                type: internal
        Port "eth1.11"
            Interface "eth1.11"
        Port "phy-br-eth1"
            Interface "phy-br-eth1"

==============================
Internal network
==============================

   Bridge br-int
        Port "int-br-eth1"
            Interface "int-br-eth1"
        Port br-int
            Interface br-int
                type: internal
        Port "tapb610a695-46"
            tag: 1
            Interface "tapb610a695-46"
                type: internal
        Port "qr-ef10bef4-fa"
            tag: 1
            Interface "qr-ef10bef4-fa"
                type: internal

==============================

There's another iface named "br-ex" that we're using for floating_ips,
but it has nothing to do with what we're doing right now, so I'm skipping it...


************ So, for the hands-on ****************

I know it may be a little bit hard to understand, but I'll do my best
trying to explain:

1) the running instance in Hyper-V, which is linked to Hyper-V vSwitch
is actually
communicating to bridge: "br-eth1" (that is in the network controller).

NOTE: That's where the DHCP REQUEST (from the instance) lands


2) The interface MAC Address, of that running instance on Hyper-V, is:
fa:16:3e:95:95:e4. (we're gonna use it on later steps)
Since DHCP is not fully working yet, we had to manually set an IP for
that instance: "10.5.5.3"


3) From that instance interface, the dhcp_broadcast should be forward ->
   FROM interface "eth1.12" TO  "phy-br-eth1"
   And FROM interface "phy-br-eth1" TO the bridge "br-int"   *** THIS
IS WHERE THE PACKETS ARE DROPPED  ***.

Check it out for the "actions:drop"
---------------------------------------------------------------------------------------------
root@osnetwork:~# ovs-dpctl dump-flows br-int  |grep 10.5.5.3

in_port(4),eth(src=fa:16:3e:f0:ac:8e,dst=ff:ff:ff:ff:ff:ff),eth_type(0x0806),arp(sip=10.5.5.3,tip=10.5.5.1,op=1,sha=fa:16:3e:f0:ac:8e,tha=00:00:00:00:00:00),
packets:20, bytes:1120, used:0.412s, actions:drop
---------------------------------------------------------------------------------------------

4) Finally, when the packet reaches the bridge "br-int", the
DHCP_REQUEST should be forward to the
   dhcp_interface, that is: tapb610a695-46    *** WHICH IS NOT
HAPPENING EITHER ***


5) How to fix :: bridge br-eth1

-------------------------------------------
5.1. Getting to know the ifaces of 'br-eth1'
-------------------------------------------
root@osnetwork:~# ovs-ofctl show br-eth1

OFPT_FEATURES_REPLY (xid=0x1): ver:0x1, dpid:0000e0db554e164b
n_tables:255, n_buffers:256 features: capabilities:0xc7, actions:0xfff

1(eth1.11): addr:e0:db:55:4e:16:4b
     config:     0
     state:      0
     current:    10GB-FD AUTO_NEG
     advertised: 1GB-FD 10GB-FD FIBER AUTO_NEG
     supported:  1GB-FD 10GB-FD FIBER AUTO_NEG

3(phy-br-eth1): addr:26:9b:97:93:b9:70
     config:     0
     state:      0
     current:    10GB-FD COPPER

LOCAL(br-eth1): addr:e0:db:55:4e:16:4b
     config:     0
     state:      0

OFPT_GET_CONFIG_REPLY (xid=0x3): frags=normal miss_send_len=0


-------------------------------------------
5.2. Adding flow rules to enable passing (instead of dropping)
-------------------------------------------

# the source mac_address (dl_src) is the from the interface of the
# running instance on Hyper-V. This fix the DROP (only)

root@osnetwork:~# ovs-ofctl add-flow br-eth1
priority=10,in_port=3,dl_src=fa:16:3e:95:95:e4,actions=normal



6) How to fix :: bridge br-int

-------------------------------------------
6.1. Getting to know the ifaces of 'br-int'
-------------------------------------------

root@osnetwork:~# ovs-ofctl show br-int

OFPT_FEATURES_REPLY (xid=0x1): ver:0x1, dpid:000092976d64274d

n_tables:255, n_buffers:256  features: capabilities:0xc7, actions:0xfff

1(tapb610a695-46): addr:19:01:00:00:00:00
     config:     PORT_DOWN
     state:      LINK_DOWN

4(int-br-eth1): addr:5a:56:e1:53:e9:90
     config:     0
     state:      0
     current:    10GB-FD COPPER

5(qr-ef10bef4-fa): addr:19:01:00:00:00:00
     config:     PORT_DOWN
     state:      LINK_DOWN

LOCAL(br-int): addr:92:97:6d:64:27:4d
     config:     0
     state:      0

-------------------------------------------
6.2. Adding flow rules to enable FORWARD
     FROM: interface int-br-eth1  (4)
     TO:   interface tapb610a695-46 (1) -> dhcp_interface

     and the REVERSE_FORWARD: from (1) to (4)
-------------------------------------------
root@osnetwork:~# ovs-ofctl add-flow br-int
priority=12,in_port=4,dl_src=fa:16:3e:f0:ac:8e,action=1,normal
root@osnetwork:~# ovs-ofctl add-flow br-int
priority=12,in_port=1,dl_dst=fa:16:3e:f0:ac:8e,action=4,normal


==================
Conclusion
==================

That way, and only *that way*, the Hyper-V instance is able to
exchange ARP with the dhcp (network controller).

Even though, it is functional, we're not sure if that's how it HAS to
be done. May I have your thoughts on it?

Should we really have to create those rules/actions in openVSwitch to
make the instance (of hyper-v) to reach out
DHCP ? It seems either bug or something is wierd in my configurations...

May I have your opinions on it?


We'd greatly appreciate your feedback. Thank you very much.