← Back to team overview

cloud-init team mailing list archive

Re: [OpenStack][Neutron] Metadata over IPv6

 

On Mon, Mar 23, 2020 at 3:37 PM Brian Haley <haleyb.dev@xxxxxxxxx> wrote:

> On 3/23/20 10:48 AM, Ryan Harper wrote:
> >
> > On Mon, Mar 16, 2020 at 8:44 AM Slawek Kaplonski <skaplons@xxxxxxxxxx
> > <mailto:skaplons@xxxxxxxxxx>> wrote:
> >
> >     Hi,
> >
> >      > On 11 Mar 2020, at 20:13, Ryan Harper <ryan.harper@xxxxxxxxxxxxx
> >     <mailto:ryan.harper@xxxxxxxxxxxxx>> wrote:
> >      >
> >      >
> >      >
> >      > On Wed, Mar 11, 2020 at 4:39 AM Slawek Kaplonski
> >     <skaplons@xxxxxxxxxx <mailto:skaplons@xxxxxxxxxx>> wrote:
> >      > Hi,
> >      >
> >      > Hi Slawek,
> >      >
> >      > Thanks for reaching out to the cloud-init community.
> >      >
> >      >
> >      > I’m OpenStack Neutron developer. Since some time we have proposed
> >     spec [1] about Metadata over IPv6 in IPv6 only networks.
> >      > This spec proposes to use fe80::a9fe:a9fe IP address as it is
> >     equivalent of 169.254.169.254 in IPv6 link-local subnet.
> >      >
> >      > According to [2] I know already that cloud-init can be configured
> >     to use any metadata urls with “metadata_urls” so we can use such
> >     IPv6 address (or any other) and can be configured to be used in
> >     cloud-init on guest vms.
> >      > But as cloud-init is one of the biggest users of Metadata
> >     service, I would like to ask cloud-init community what do You think
> >     about it and if this will work for You.
> >      > Any feedback here or in the spec's review will be appreciated :)
> >      >
> >      > Some initial thoughts and questions:
> >      >
> >      > - In what order should cloud-init try ipv6 and ipv4?
> >      >    - cloud-init would prefer to know which one it should use so
> >     we don't timeout on an endpoint that isn't there.
> >
> >     I would say that it it could be checked what IP addresses are
> >     configured in the guest OS already and use correct one if only one
> >     type of addresses is there. If there would be both IPv4 and IPv6, it
> >     could try IPv4 first as it is like that now. What do You think about
> >     such solution?
> >
> >
> > cloud-init runs before networking comes up.  In OpenStack, cloud-init
> > will to an Ephemeral DHCP to try to reach metadata service over IPV4.
> > Once metadata service is crawled, cloud-init will write out network
> > configuration obtained from metadata service.
> > Currently we will try a list of URLs/IPs as configured in the
> > datasource.   We can add the IPv6 metadata url to the list, my question
> > is where in the list should it go?  If we put it first, then on
> > deployments which lack IPv6 metadata service, all VMs will pay a
> > boot speed cost of trying to fetch metadata from a non-existent URL.  If
> > the v6 address is at the end, then IPv6-only deployments will waste time
> > trying to hit an IPv4 url which is not present.  This is why I'm
> > interested in being "told" in some way.
> > This knowledge would need to come from a non-network area.  Typically
> > this is done via DMI tables; however, DMI is not a complete solution as
> > this is not available on all architectures (s390x and ppc64le and some
> > arm64), which makes this
> > sub-optimal.
>
> I think in our Openstack case, there are three possibilities.  Assuming
> DHCP succeeds and assigns an IP to the interface, metadata should use
> the following address based on what is configured:
>
> - IPv4-only: IPv4 metadata address
>
- IPv6-only: IPv6 metadata address
>
- dual-stack: either should work, prefer IPv4
>

I get your idea, let's reword this a bit to confirm since we'll always have
ipv6 link-local
on our nic; the only variable is whether DHCPv4 will respond with an IP.

 - DHCP response with v4 address -> prefer v4 metadata address
 - NoDHCP response -> try IPv6 metadata url


> Does that make sense?  I'm assuming in the IPv6-only case there won't be
> an IPv4 address, or it will be something in the odd link-local subnet?
>

IIUC for v6 only, our DHCPv4 request will fail, so we'll then need to try
the v6 ip.

A potential improvement in cloud-init would to DHCPv4 and crawl v6 in
parallel;
which ever returns first we'd take.  This would work on the assumption that
v4 and
v6 response won't diverge.  Another variant is to have OpenStack have a
EphemeralLinkLocal setup instead of DHCP;  we'd need to fallback on
EphemeralDHCP as there are some Openstacks which include a
classless-static-route to point to the metadata service which would would
otherwise not be reachable via just link-local configuration.

Some food for thought on improvements over just trying DHCP4, and then
falling back on IPv6 link-local and the v5 metadata url.

Ryan Harper



> -Brian
>
>
> >      >    -
> >      > - cloud-init's EphemeralDHCP code currently relies upon dhclient
> >     and ipv4 ips/routes; this will need updating to work with v6; what
> >     is needed use fe80::a9fe:a9fe ?
> >
> >     As this is link local IP address it should be reachable without any
> >     configuration if IPv6 is enabled on the interface in the guest.
> >
> >
> > OK.   That makes the v6 support simpler.
> >
> >
> >      > - is there a chance that Metadata is available over either v4 or
> v6?
> >      >    - would the data ever diverge?
> >
> >     I don’t know if this data can diverge in the future.
> >
> >
> > Reading the spec, it appeared to me that the metadata service has no
> > changes, rather this was more about configuring which network ports
> > expose the service.
> > I would not expect there to be a different metadata service on v4 from
> > v6 (the instance-id is the same either path);
> >
> >
> >      >
> >      >
> >      > Thanks,
> >      > Ryan Harper
> >      >
> >
> >     —
> >     Slawek Kaplonski
> >     Senior software engineer
> >     Red Hat
> >
>
>

Follow ups

References