openstack team mailing list archive

[Alessandro Pilotti <apilotti@xxxxxxxxxxxxxxxxxxxxxx>]

Hi Bruno,

I just hit the same (or a very similar) issue doing a multinode deployment with RDO on CentOS 6.4 (OVS 1.10) while we had no problem until now using Ubuntu 12.04 (OVS 1.4).
Can you please provide some more details about the Linux OS you are using and your multinode configuration?

I tested it with flat and VLAN networks, so far it doesn't look like an Hyper-V related issue.


Thanks,

Alessandro


On Jun 7, 2013, at 23:51 , Bruno Oliveira ~lychinus <brunnop.oliveira@xxxxxxxxx<mailto:brunnop.oliveira@xxxxxxxxx>> wrote:

"(...)Do you have your vSwitch properly configured on your hyper-v host?(...)"

I can't say for sure, Peter, but I think so...

In troubleshooting we did (and are still doing) I can tell that
regardless of the network model that we're using (FLAT or VLAN
Network),
the instance that is provisioned on Hyper-V (for some reason) can't
reach the quantum-l3-agent "by default"
(I said "default" because, we just managed to do it after a hard, long
and boring troubleshoting,
yet, we're not sure if that's how it should be done, indeed)

Since it's not something quick to explain, I'll present the scenario:
(I'm not sure if it might be a candidate for a fix in quantum-l3-agent,
so quantum-devs might be interested too)


Here's how our network interfaces turns out, in our network controller:

==============================
External bridge network
==============================

Bridge "br-eth1"
       Port "br-eth1"
           Interface "br-eth1"
               type: internal
       Port "eth1.11"
           Interface "eth1.11"
       Port "phy-br-eth1"
           Interface "phy-br-eth1"

==============================
Internal network
==============================

  Bridge br-int
       Port "int-br-eth1"
           Interface "int-br-eth1"
       Port br-int
           Interface br-int
               type: internal
       Port "tapb610a695-46"
           tag: 1
           Interface "tapb610a695-46"
               type: internal
       Port "qr-ef10bef4-fa"
           tag: 1
           Interface "qr-ef10bef4-fa"
               type: internal

==============================

There's another iface named "br-ex" that we're using for floating_ips,
but it has nothing to do with what we're doing right now, so I'm skipping it...


************ So, for the hands-on ****************

I know it may be a little bit hard to understand, but I'll do my best
trying to explain:

1) the running instance in Hyper-V, which is linked to Hyper-V vSwitch
is actually
communicating to bridge: "br-eth1" (that is in the network controller).

NOTE: That's where the DHCP REQUEST (from the instance) lands


2) The interface MAC Address, of that running instance on Hyper-V, is:
fa:16:3e:95:95:e4. (we're gonna use it on later steps)
Since DHCP is not fully working yet, we had to manually set an IP for
that instance: "10.5.5.3"


3) From that instance interface, the dhcp_broadcast should be forward ->
  FROM interface "eth1.12" TO  "phy-br-eth1"
  And FROM interface "phy-br-eth1" TO the bridge "br-int"   *** THIS
IS WHERE THE PACKETS ARE DROPPED  ***.

Check it out for the "actions:drop"
---------------------------------------------------------------------------------------------
root@osnetwork:~# ovs-dpctl dump-flows br-int  |grep 10.5.5.3

in_port(4),eth(src=fa:16:3e:f0:ac:8e,dst=ff:ff:ff:ff:ff:ff),eth_type(0x0806),arp(sip=10.5.5.3,tip=10.5.5.1,op=1,sha=fa:16:3e:f0:ac:8e,tha=00:00:00:00:00:00),
packets:20, bytes:1120, used:0.412s, actions:drop
---------------------------------------------------------------------------------------------

4) Finally, when the packet reaches the bridge "br-int", the
DHCP_REQUEST should be forward to the
  dhcp_interface, that is: tapb610a695-46    *** WHICH IS NOT
HAPPENING EITHER ***


5) How to fix :: bridge br-eth1

-------------------------------------------
5.1. Getting to know the ifaces of 'br-eth1'
-------------------------------------------
root@osnetwork:~# ovs-ofctl show br-eth1

OFPT_FEATURES_REPLY (xid=0x1): ver:0x1, dpid:0000e0db554e164b
n_tables:255, n_buffers:256 features: capabilities:0xc7, actions:0xfff

1(eth1.11): addr:e0:db:55:4e:16:4b
    config:     0
    state:      0
    current:    10GB-FD AUTO_NEG
    advertised: 1GB-FD 10GB-FD FIBER AUTO_NEG
    supported:  1GB-FD 10GB-FD FIBER AUTO_NEG

3(phy-br-eth1): addr:26:9b:97:93:b9:70
    config:     0
    state:      0
    current:    10GB-FD COPPER

LOCAL(br-eth1): addr:e0:db:55:4e:16:4b
    config:     0
    state:      0

OFPT_GET_CONFIG_REPLY (xid=0x3): frags=normal miss_send_len=0


-------------------------------------------
5.2. Adding flow rules to enable passing (instead of dropping)
-------------------------------------------

# the source mac_address (dl_src) is the from the interface of the
# running instance on Hyper-V. This fix the DROP (only)

root@osnetwork:~# ovs-ofctl add-flow br-eth1
priority=10,in_port=3,dl_src=fa:16:3e:95:95:e4,actions=normal



6) How to fix :: bridge br-int

-------------------------------------------
6.1. Getting to know the ifaces of 'br-int'
-------------------------------------------

root@osnetwork:~# ovs-ofctl show br-int

OFPT_FEATURES_REPLY (xid=0x1): ver:0x1, dpid:000092976d64274d

n_tables:255, n_buffers:256  features: capabilities:0xc7, actions:0xfff

1(tapb610a695-46): addr:19:01:00:00:00:00
    config:     PORT_DOWN
    state:      LINK_DOWN

4(int-br-eth1): addr:5a:56:e1:53:e9:90
    config:     0
    state:      0
    current:    10GB-FD COPPER

5(qr-ef10bef4-fa): addr:19:01:00:00:00:00
    config:     PORT_DOWN
    state:      LINK_DOWN

LOCAL(br-int): addr:92:97:6d:64:27:4d
    config:     0
    state:      0

-------------------------------------------
6.2. Adding flow rules to enable FORWARD
    FROM: interface int-br-eth1  (4)
    TO:   interface tapb610a695-46 (1) -> dhcp_interface

    and the REVERSE_FORWARD: from (1) to (4)
-------------------------------------------
root@osnetwork:~# ovs-ofctl add-flow br-int
priority=12,in_port=4,dl_src=fa:16:3e:f0:ac:8e,action=1,normal
root@osnetwork:~# ovs-ofctl add-flow br-int
priority=12,in_port=1,dl_dst=fa:16:3e:f0:ac:8e,action=4,normal


==================
Conclusion
==================

That way, and only *that way*, the Hyper-V instance is able to
exchange ARP with the dhcp (network controller).

Even though, it is functional, we're not sure if that's how it HAS to
be done. May I have your thoughts on it?

Should we really have to create those rules/actions in openVSwitch to
make the instance (of hyper-v) to reach out
DHCP ? It seems either bug or something is wierd in my configurations...

May I have your opinions on it?


We'd greatly appreciate your feedback. Thank you very much.

_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to     : openstack@xxxxxxxxxxxxxxxxxxx<mailto:openstack@xxxxxxxxxxxxxxxxxxx>
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp