openstack team mailing list archive

[Jānis Ģeņģeris <janis.gengeris@xxxxxxxxx>]

Hi,

I have managed to create the Quantum router so that all the VMs created on
provider network are going through it. I had to simply use
router-interface-add.

But I have now the problem with accessing metadata service. As L3 agent
adds DNAT rule in router, the packets are rewritten and arrives at metadata
host, but when coming back it doesn't go through the router but directly to
the source host resulting in TCP handshake failure. Could it be because all
the ports in br-int which are somehow related to this provider network are
withing the same VLAN(have the same tag in ovs-vsctl output) thus being in
the same broadcast domain?

So the above could be solved with SNAT rule to router address, but then
metadata service sees routers address instead of VMs and returns HTTP 404
error.

Is it possible to fix this with additional flow in openvswitch somehow? The
vswitch man pages say that it is possible to match against IPs.

What other solution is here to fix this?

Thanks,
--janis

On Mon, Oct 8, 2012 at 11:04 PM, Jānis Ģeņģeris <janis.gengeris@xxxxxxxxx>wrote:

> Here is the output, with few details.
>
> # quantum router-show router_vlan1501
> +-----------------------+--------------------------------------+
> | Field                 | Value                                |
> +-----------------------+--------------------------------------+
> | admin_state_up        | True                                 |
> | external_gateway_info |                                      |
> | id                    | 3c7383c7-7759-4db6-ba5b-19e754280cb8 |
> | name                  | router_vlan1501                      |
> | status                | ACTIVE                               |
> | tenant_id             | 7246a7e9d61f42b8a644bc1551a2a396     |
> +-----------------------+--------------------------------------+
>
> # quantum net-show vlan1501
> +---------------------------+--------------------------------------+
> | Field                     | Value                                |
> +---------------------------+--------------------------------------+
> | admin_state_up            | True                                 |
> | id                        | c2161824-a439-40e5-8809-5599f80df2fe |
> | name                      | vlan1501                             |
> | provider:network_type     | vlan                                 |
> | provider:physical_network | default                              |
> | provider:segmentation_id  | 1501                                 |
> | router:external           | False                                |
> | shared                    | False                                |
> | status                    | ACTIVE                               |
> | subnets                   | af6eac1e-dfec-49a0-bfc2-3fbf9a7063b3 |
> | tenant_id                 | 7246a7e9d61f42b8a644bc1551a2a396     |
> +---------------------------+--------------------------------------+
>
> # quantum router-gateway-set 3c7383c7-7759-4db6-ba5b-19e754280cb8
> c2161824-a439-40e5-8809-5599f80df2fe
> Bad router request: Network c2161824-a439-40e5-8809-5599f80df2fe is not a
> valid external network
>
> I assume this happens because 'router:external' if False, when switching
> it to True the above command succeeds. But then if I want to switch back
> later (with no ports attached, no routers, even no subnets), I get this:
>
> quantum net-update vlan1501 --router:external False
> External network e011d68b-6abd-43a4-b6c3-9dbadf5344ee cannot be updated to
> be made non-external, since it has existing gateway ports
>
> Anyway, I have set 'router:external' to True for 'vlan1501' network, set
> the gateway for my newly created quantum router(router_vlan1501) with
> 'router-gateway-set' as a result quantum have created router ns with single
> interface from my configured provider network 'vlan1501' with default
> gateway set to the one of the 'vlan1501' nets attached subnet.
>
> But now if I boot the VM with '--nic=<vlan1501_id>' option, I get VM with
> default gateway set to networks('vlan1501') gateway(not the freshly created
> router gateway, am I missing something in configs?), which is the same as
> the created routers gateway. Another thing if I switch to newly created
> 'router_vlan1501' namespace I can't actually ping the external gateway,
> that is there as a default gw for the 'vlan1501' net.
>
> So my thinking is I need to change the default gw in VM to the virtual
> router?
>
> On Mon, Oct 8, 2012 at 9:32 PM, Dan Wendlandt <dan@xxxxxxxxxx> wrote:
>
>> On Mon, Oct 8, 2012 at 12:27 PM, Jānis Ģeņģeris
>> <janis.gengeris@xxxxxxxxx> wrote:
>> > On Mon, Oct 8, 2012 at 6:24 PM, Dan Wendlandt <dan@xxxxxxxxxx> wrote:
>> >>
>> >> On Mon, Oct 8, 2012 at 7:52 AM, Jānis Ģeņģeris <
>> janis.gengeris@xxxxxxxxx>
>> >> wrote:
>> >> > Hello,
>> >> >
>> >> > When using provider networks in Quantum, where should the metadata
>> >> > service
>> >> > rule mapping (e.g. 169.254.169.254:80 ->
>> metadata_server:metadata_port)
>> >> > must
>> >> > be set?
>> >> >
>> >> > For example, for floating IPs l3-agent handles this, but for provider
>> >> > networks router is not used. I tried to set custom iptables rule for
>> >> > this,
>> >> > but have a hard time understanding where to set it, as there is
>> >> > openvswitch
>> >> > and namespaces.
>> >> >
>> >> > I'm using provider network configuration with VLANs.
>> >>
>> >> You actually could use the Quantum L3 router as your gateway even if
>> >> VMs are on a provider network, but I suspect your question is actually
>> >> more along the lines of: if I want my gateway to be a physical router
>> >> not managed by Quantum, how do I does the DNAT rule for metadata get
>> >> applied?  In this case, you need to apply the DNAT rule manually to
>> >> the physical router, which I believe is the same as if you were using
>> >> flat networking with Nova with a physical router.
>> >
>> > Adding the rule in physical router is not a good idea, because then the
>> > configuration of the OpenStack crosses the actual software/server border
>> > into network equipment, than can add to complexity later.
>>
>> Yes, its hard to have it both ways...  if you want everything done
>> automatically via software, I'd suggest using the quantum router as
>> the gateway, not an external physical router.
>>
>> >
>> > I tried to add provider network to quantum router, and the quantum CLI
>> was
>> > rejecting it.
>> > AFAIK router-interface-add is for internal networks, and
>> router-gateway-set
>> > is also failing.
>>
>> Can you post what you've run and what the resulting error was?  In
>> terms of the L3 API, quantum shouldn't care whether a network is
>> provider or not for router-interface-add.  Perhaps this is a
>> permissions issue?
>>
>> Dan
>>
>>
>> >
>> > Which CLI command to use for adding provider network to existing quantum
>> > router?
>> >>
>> >>
>> >>
>> >>
>> >> There may also be a more complex solution achievable via quantum in
>> >> which the provider creates a quantum router with an interface on the
>> >> provider network, VMs are each given a host route to route traffic
>> >> destined for 169.254.169.254/32 to this quantum router IP, rather than
>> >> the physical default gateway, and this quantum router performs the
>> >> DNAT.  However, its probably much easier to just apply this rule to
>> >> your physical router.
>> >
>> > No, this is no good.
>> >>
>> >>
>> >> Dan
>> >>
>> >>
>> >> >
>> >> > Regards,
>> >> > --janis
>> >> >
>> >> > _______________________________________________
>> >> > Mailing list: https://launchpad.net/~openstack
>> >> > Post to     : openstack@xxxxxxxxxxxxxxxxxxx
>> >> > Unsubscribe : https://launchpad.net/~openstack
>> >> > More help   : https://help.launchpad.net/ListHelp
>> >> >
>> >>
>> >>
>> >>
>> >> --
>> >> ~~~~~~~~~~~~~~~~~~~~~~~~~~~
>> >> Dan Wendlandt
>> >> Nicira, Inc: www.nicira.com
>> >> twitter: danwendlandt
>> >> ~~~~~~~~~~~~~~~~~~~~~~~~~~~
>> >
>> >
>> > Regards,
>> > --janis
>>
>>
>>
>> --
>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~
>> Dan Wendlandt
>> Nicira, Inc: www.nicira.com
>> twitter: danwendlandt
>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~
>>
>
>
>
> --
> --janis