cloud-init team mailing list archive

[Ryan Harper <ryan.harper@xxxxxxxxxxxxx>]

On Mon, Mar 23, 2020 at 4:11 PM Brian Haley <haleyb.dev@xxxxxxxxx> wrote:

> On 3/23/20 4:58 PM, Ryan Harper wrote:
> >
> >
> > On Mon, Mar 23, 2020 at 3:37 PM Brian Haley <haleyb.dev@xxxxxxxxx
> > <mailto:haleyb.dev@xxxxxxxxx>> wrote:
> >
>
> >     I think in our Openstack case, there are three possibilities.
> Assuming
> >     DHCP succeeds and assigns an IP to the interface, metadata should use
> >     the following address based on what is configured:
> >
> >     - IPv4-only: IPv4 metadata address
> >
> >     - IPv6-only: IPv6 metadata address
> >
> >     - dual-stack: either should work, prefer IPv4
> >
> >
> > I get your idea, let's reword this a bit to confirm since we'll always
> > have ipv6 link-local
> > on our nic; the only variable is whether DHCPv4 will respond with an IP.
> >
> >   - DHCP response with v4 address -> prefer v4 metadata address
> >   - NoDHCP response -> try IPv6 metadata url
>
> Right, that looks good to me.
>

Great!


> >     Does that make sense?  I'm assuming in the IPv6-only case there
> >     won't be
> >     an IPv4 address, or it will be something in the odd link-local
> subnet?
> >
> >
> > IIUC for v6 only, our DHCPv4 request will fail, so we'll then need to
> > try the v6 ip.
> >
> > A potential improvement in cloud-init would to DHCPv4 and crawl v6 in
> > parallel;
> > which ever returns first we'd take.  This would work on the assumption
> > that v4 and
> > v6 response won't diverge.
>
> I think that would work too, since both requests (v4/v6) should be
> hitting the same proxy and get the same response.
>

OK


> > Another variant is to have OpenStack have a
> > EphemeralLinkLocal setup instead of DHCP;  we'd need to fallback on
> > EphemeralDHCP as there are some Openstacks which include a
> > classless-static-route to point to the metadata service which would
> > would otherwise not be reachable via just link-local configuration.
>
> I think I understand what you're saying - in some cases the DHCP
> response has a route to 169.254.169.254/32 when there is no gateway IP
> address supplied.  I don't understand the Ephemeral part though, but I
>

Exactly

have not gone through the cloud-init code in a while.
>

I should have elaborated.  Inside cloud-init, we have a helper class
called EphemeralDHCP which runs during cloud-init "local" time which
is before the OS networking layer comes up.  On some platforms, initially
Ec2, DHCP is *required* to allow HTTP traffic to the metadata URL,
we call it ephemeral as we run dhclient in a way as to not execute any
scripts which could produce side-effects on the root filesystem as we're
only using DHCP to allow access to metadata service in an effort to obtain
the instance network-configuration.  We tear down the configuration after
crawling the metadata service and allow the OS networking layer to bring
up the complete network config.

This was not strictly required on OpenStack at the time but we re-used this
component for any platform which provides network configuration via
metadata service.
As recently as last year, we started seeing reports of errors on some
OpenStacks where
the the /32 route was provided via extended DHCP options which we now
support.

As Slawek pointed out, currently the v6 path is stuck behind a v4 only
policy.
I think the clear path would be to implement a EphemeralLinkLocal class
that will attempt to resolve metadata URLs via link-local.  And ideally we
run
these attempts in parallel and the first one to return will be utilized for
the instance.


> > Some food for thought on improvements over just trying DHCP4, and then
> > falling back on IPv6 link-local and the v5 metadata url.
>
> Yes, thanks for all the info and your help.
>

Sure.  Is there, or will there be, documentation on how to enable this v6
metadata
service?  Ideally cloud-init devs would like to test things out on a real
deployment
with the v6 metadata service in place.


> -Brian
>
> >      >      >    -
> >      >      > - cloud-init's EphemeralDHCP code currently relies upon
> >     dhclient
> >      >     and ipv4 ips/routes; this will need updating to work with v6;
> >     what
> >      >     is needed use fe80::a9fe:a9fe ?
> >      >
> >      >     As this is link local IP address it should be reachable
> >     without any
> >      >     configuration if IPv6 is enabled on the interface in the
> guest.
> >      >
> >      >
> >      > OK.   That makes the v6 support simpler.
> >      >
> >      >
> >      >      > - is there a chance that Metadata is available over either
> >     v4 or v6?
> >      >      >    - would the data ever diverge?
> >      >
> >      >     I don’t know if this data can diverge in the future.
> >      >
> >      >
> >      > Reading the spec, it appeared to me that the metadata service has
> no
> >      > changes, rather this was more about configuring which network
> ports
> >      > expose the service.
> >      > I would not expect there to be a different metadata service on v4
> >     from
> >      > v6 (the instance-id is the same either path);
> >      >
> >      >
> >      >      >
> >      >      >
> >      >      > Thanks,
> >      >      > Ryan Harper
> >      >      >
> >      >
> >      >     —
> >      >     Slawek Kaplonski
> >      >     Senior software engineer
> >      >     Red Hat
> >      >
> >
>
>