yahoo-eng-team team mailing list archive

[Robert Van Voorhees <1888822@xxxxxxxxxxxxxxxxxx>]

Public bug reported:

In the upstream Kubernetes project Cluster API, specifically the Cluster
API AWS Provider, it will download a file securely from AWS Secrets
Manager in the cloud-init script, save that file to a well known
location, and then restart the cloud-init service through systemd.
After the cloud-init script is restarted, it will resolve the secrets
file (that had previously not been there) and execute its commands.

This worked fine on versions of cloud-init up until
19.4-33-gbb4131a2-0ubuntu1~18.04.1.  Once upgrading to
20.2-45-g5f7825e2-0ubuntu1~18.04.1 the secrets file is never resolved
again.

Some other information:

- cloud-init is definitely successfully running twice based on systemd and cloud-init-output.
- The /var/lib/cloud/instance/user-data.txt does show the reference to the well-known file at /etc/secret-userdata.txt
- The "resolved" version of user-data at /var/lib/cloud/instance/user-data.txt.i does not include the resolved file.  Deleting this file and then restarted cloud-init does not solve the problem, as the file resolves again without it.

Is there another command that is now required if you plan on restarting
cloud-init for another execution where files are now present that were
previously not?

1. Cloud Provider: AWS
2. Upstream issue: https://github.com/kubernetes-sigs/cluster-api-provider-aws/issues/1839 Instructions to recreate can be found in that issue including 2 public AMIs.

** Affects: cloud-init
     Importance: Undecided
         Status: New

** Attachment added: "results of cloud-init log collector"
   https://bugs.launchpad.net/bugs/1888822/+attachment/5395523/+files/cloud-init.tar.gz

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1888822

Title:
  cloud-init caches files and never checks again

Status in cloud-init:
  New

Bug description:
  In the upstream Kubernetes project Cluster API, specifically the
  Cluster API AWS Provider, it will download a file securely from AWS
  Secrets Manager in the cloud-init script, save that file to a well
  known location, and then restart the cloud-init service through
  systemd.  After the cloud-init script is restarted, it will resolve
  the secrets file (that had previously not been there) and execute its
  commands.

  This worked fine on versions of cloud-init up until
  19.4-33-gbb4131a2-0ubuntu1~18.04.1.  Once upgrading to
  20.2-45-g5f7825e2-0ubuntu1~18.04.1 the secrets file is never resolved
  again.

  Some other information:

  - cloud-init is definitely successfully running twice based on systemd and cloud-init-output.
  - The /var/lib/cloud/instance/user-data.txt does show the reference to the well-known file at /etc/secret-userdata.txt
  - The "resolved" version of user-data at /var/lib/cloud/instance/user-data.txt.i does not include the resolved file.  Deleting this file and then restarted cloud-init does not solve the problem, as the file resolves again without it.

  Is there another command that is now required if you plan on
  restarting cloud-init for another execution where files are now
  present that were previously not?

  1. Cloud Provider: AWS
  2. Upstream issue: https://github.com/kubernetes-sigs/cluster-api-provider-aws/issues/1839 Instructions to recreate can be found in that issue including 2 public AMIs.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1888822/+subscriptions