cloud-init-dev team mailing list archive

[Scott Moser <smoser@xxxxxxxxxx>]

I have one fun comment in line.


Diff comments:

> diff --git a/cloudinit/sources/DataSourceEc2.py b/cloudinit/sources/DataSourceEc2.py
> index 4ec9592..dd9199e 100644
> --- a/cloudinit/sources/DataSourceEc2.py
> +++ b/cloudinit/sources/DataSourceEc2.py
> @@ -137,22 +157,27 @@ class DataSourceEc2(sources.DataSource):
>  
>          urls = []
>          url2base = {}
> +        url2apiver = {}
>          for url in mdurls:
> -            cur = "%s/%s/meta-data/instance-id" % (url, self.api_ver)
> -            urls.append(cur)
> -            url2base[cur] = url
> +            for api_ver in self.supported_metadata_versions:
> +                cur = "%s/%s/meta-data/instance-id" % (url, api_ver)
> +                urls.append(cur)
> +                url2base[cur] = url
> +                url2apiver[cur] = api_ver
>  
>          start_time = time.time()
> -        url = uhelp.wait_for_url(urls=urls, max_wait=max_wait,
> -                                 timeout=timeout, status_cb=LOG.warn)
> +        url = uhelp.wait_for_url(
> +            urls=urls, max_wait=max_wait, timeout=timeout, status_cb=LOG.warn,

the reason that we 'wait_for_url' here is to actually wait for the metadata service to come up.
we will wait up to timeout/max_wait seconds for it to appear.

I'm not sure what error code we would get if it wasnt "there yet".

In the early days of Ec2 (and I'm told it is still the case), the metadata service sometimes wasn't available when cloud-init would start looking.  So we had to wait for it.  This is the source of all sorts of problems in cloud-init.  (Users saying 'why is cloud-init keep trying to get data from 169.254.169.254' outside of Ec2).
Unfortunately, i'm not sure what response you would get if the data just hadn't materialized..., it may well have been 404.

That means that you can't tell the difference between "not there yet" and "wont ever be there.".

lets reach out to a AWS contact and see what sort of response they woudl think we'd get.

> +            exception_cb=_skip_metadata_error_codes)
>  
>          if url:
> -            LOG.debug("Using metadata source: '%s'", url2base[url])
> +            self.metadata_address = url2base[url]
> +            self.discovered_api_ver = url2apiver[url]
> +            LOG.debug("Using metadata source: '%s'", self.metadata_address)
>          else:
>              LOG.critical("Giving up on md from %s after %s seconds",
>                           urls, int(time.time() - start_time))
>  
> -        self.metadata_address = url2base.get(url)
>          return bool(url)
>  
>      def device_name_to_device(self, name):
> @@ -234,6 +259,40 @@ class DataSourceEc2(sources.DataSource):
>                  util.get_cfg_by_path(cfg, STRICT_ID_PATH, STRICT_ID_DEFAULT),
>                  cfg)
>  
> +    def _crawl_metadata(self):

if you just drop all the time info here, i think you can just use utl.log_time to do that for you.
change the caller to do:
  util.log_time(self._crawl_metadata, "Crawl of metadata service")

right?

> +        """Crawl metadata service when available.
> +
> +        @returns: True on success, False otherwise.
> +        """
> +        if not self.wait_for_metadata_service():
> +            return False
> +        try:
> +            start_time = time.time()
> +            self.userdata_raw = ec2.get_instance_userdata(
> +                self.discovered_api_ver, self.metadata_address)
> +            self.metadata = ec2.get_instance_metadata(
> +                self.discovered_api_ver, self.metadata_address)
> +        except Exception:
> +            util.logexc(
> +                LOG, "Failed reading from metadata address %s",
> +                self.metadata_address)
> +            return False
> +        LOG.debug(
> +            "Crawl of metadata service took %.3f seconds",
> +            time.time() - start_time)
> +        return True
> +
> +
> +class DataSourceEc2Local(DataSourceEc2):
> +    """Datasource run at init-local which sets up network to query metadata.
> +
> +    In init-local, no network is available. This subclass sets up minimal
> +    networking with dhclient on a viable nic so that it can talk to the
> +    metadata service. If the metadata service provides network configuration
> +    then render the network configuration for that instance based on metadata.
> +    """
> +    get_network_metadata = True  # Get metadata network config if present
> +
>  
>  def read_strict_mode(cfgval, default):
>      try:


-- 
https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/328241
Your team cloud-init commiters is requested to review the proposed merge of ~chad.smith/cloud-init:aws-local-dhcp into cloud-init:master.