← Back to team overview

cloud-init team mailing list archive

Re: [OpenStack][Neutron] Metadata over IPv6


On 3/23/20 4:58 PM, Ryan Harper wrote:

On Mon, Mar 23, 2020 at 3:37 PM Brian Haley <haleyb.dev@xxxxxxxxx <mailto:haleyb.dev@xxxxxxxxx>> wrote:

    On 3/23/20 10:48 AM, Ryan Harper wrote:
     > On Mon, Mar 16, 2020 at 8:44 AM Slawek Kaplonski
    <skaplons@xxxxxxxxxx <mailto:skaplons@xxxxxxxxxx>
     > <mailto:skaplons@xxxxxxxxxx <mailto:skaplons@xxxxxxxxxx>>> wrote:
     >     Hi,
     >      > On 11 Mar 2020, at 20:13, Ryan Harper
    <ryan.harper@xxxxxxxxxxxxx <mailto:ryan.harper@xxxxxxxxxxxxx>
     >     <mailto:ryan.harper@xxxxxxxxxxxxx
    <mailto:ryan.harper@xxxxxxxxxxxxx>>> wrote:
     >      >
     >      >
     >      >
     >      > On Wed, Mar 11, 2020 at 4:39 AM Slawek Kaplonski
     >     <skaplons@xxxxxxxxxx <mailto:skaplons@xxxxxxxxxx>
    <mailto:skaplons@xxxxxxxxxx <mailto:skaplons@xxxxxxxxxx>>> wrote:
     >      > Hi,
     >      >
     >      > Hi Slawek,
     >      >
     >      > Thanks for reaching out to the cloud-init community.
     >      >
     >      >
     >      > I’m OpenStack Neutron developer. Since some time we have
     >     spec [1] about Metadata over IPv6 in IPv6 only networks.
     >      > This spec proposes to use fe80::a9fe:a9fe IP address as it is
     >     equivalent of in IPv6 link-local subnet.
     >      >
     >      > According to [2] I know already that cloud-init can be
     >     to use any metadata urls with “metadata_urls” so we can use such
     >     IPv6 address (or any other) and can be configured to be used in
     >     cloud-init on guest vms.
     >      > But as cloud-init is one of the biggest users of Metadata
     >     service, I would like to ask cloud-init community what do You
     >     about it and if this will work for You.
     >      > Any feedback here or in the spec's review will be
    appreciated :)
     >      >
     >      > Some initial thoughts and questions:
     >      >
     >      > - In what order should cloud-init try ipv6 and ipv4?
     >      >    - cloud-init would prefer to know which one it should
    use so
     >     we don't timeout on an endpoint that isn't there.
     >     I would say that it it could be checked what IP addresses are
     >     configured in the guest OS already and use correct one if
    only one
     >     type of addresses is there. If there would be both IPv4 and
    IPv6, it
     >     could try IPv4 first as it is like that now. What do You
    think about
     >     such solution?
     > cloud-init runs before networking comes up.  In OpenStack,
     > will to an Ephemeral DHCP to try to reach metadata service over
     > Once metadata service is crawled, cloud-init will write out network
     > configuration obtained from metadata service.
     > Currently we will try a list of URLs/IPs as configured in the
     > datasource.   We can add the IPv6 metadata url to the list, my
     > is where in the list should it go?  If we put it first, then on
     > deployments which lack IPv6 metadata service, all VMs will pay a
     > boot speed cost of trying to fetch metadata from a non-existent
    URL.  If
     > the v6 address is at the end, then IPv6-only deployments will
    waste time
     > trying to hit an IPv4 url which is not present.  This is why I'm
     > interested in being "told" in some way.
> This knowledge would need to come from a non-network area. Typically
     > this is done via DMI tables; however, DMI is not a complete
    solution as
     > this is not available on all architectures (s390x and ppc64le and
     > arm64), which makes this
     > sub-optimal.

    I think in our Openstack case, there are three possibilities.  Assuming
    DHCP succeeds and assigns an IP to the interface, metadata should use
    the following address based on what is configured:

    - IPv4-only: IPv4 metadata address

    - IPv6-only: IPv6 metadata address

    - dual-stack: either should work, prefer IPv4

I get your idea, let's reword this a bit to confirm since we'll always have ipv6 link-local
on our nic; the only variable is whether DHCPv4 will respond with an IP.

  - DHCP response with v4 address -> prefer v4 metadata address
  - NoDHCP response -> try IPv6 metadata url

Right, that looks good to me.

    Does that make sense?  I'm assuming in the IPv6-only case there
    won't be
    an IPv4 address, or it will be something in the odd link-local subnet?

IIUC for v6 only, our DHCPv4 request will fail, so we'll then need to try the v6 ip.

A potential improvement in cloud-init would to DHCPv4 and crawl v6 in parallel; which ever returns first we'd take.  This would work on the assumption that v4 and
v6 response won't diverge.

I think that would work too, since both requests (v4/v6) should be hitting the same proxy and get the same response.

Another variant is to have OpenStack have a EphemeralLinkLocal setup instead of DHCP;  we'd need to fallback on EphemeralDHCP as there are some Openstacks which include a classless-static-route to point to the metadata service which would would otherwise not be reachable via just link-local configuration.

I think I understand what you're saying - in some cases the DHCP response has a route to when there is no gateway IP address supplied. I don't understand the Ephemeral part though, but I have not gone through the cloud-init code in a while.

Some food for thought on improvements over just trying DHCP4, and then falling back on IPv6 link-local and the v5 metadata url.

Yes, thanks for all the info and your help.


     >      >    -
     >      > - cloud-init's EphemeralDHCP code currently relies upon
     >     and ipv4 ips/routes; this will need updating to work with v6;
     >     is needed use fe80::a9fe:a9fe ?
     >     As this is link local IP address it should be reachable
    without any
     >     configuration if IPv6 is enabled on the interface in the guest.
     > OK.   That makes the v6 support simpler.
     >      > - is there a chance that Metadata is available over either
    v4 or v6?
     >      >    - would the data ever diverge?
     >     I don’t know if this data can diverge in the future.
     > Reading the spec, it appeared to me that the metadata service has no
     > changes, rather this was more about configuring which network ports
     > expose the service.
     > I would not expect there to be a different metadata service on v4
     > v6 (the instance-id is the same either path);
     >      >
     >      >
     >      > Thanks,
     >      > Ryan Harper
     >      >
     >     —
     >     Slawek Kaplonski
     >     Senior software engineer
     >     Red Hat

Follow ups