yahoo-eng-team team mailing list archive

[Brian Murray <brian@xxxxxxxxxx>]

Hello Wayne, or anyone else affected,

Accepted cloud-init into precise-proposed. The package will build now
and be available at https://launchpad.net/ubuntu/+source/cloud-
init/0.6.3-0ubuntu1.16 in a few hours, and then in the -proposed
repository.

Please help us by testing this new package.  See
https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to
enable and use -proposed.  Your feedback will aid us getting this update
out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug,
mentioning the version of the package you tested, and change the tag
from verification-needed to verification-done. If it does not fix the
bug for you, please add a comment stating that, and change the tag to
verification-failed.  In either case, details of your testing will help
us make a better decision.

Further information regarding the verification process can be found at
https://wiki.ubuntu.com/QATeam/PerformingSRUVerification .  Thank you in
advance!

** Also affects: cloud-init (Ubuntu Precise)
   Importance: Undecided
       Status: New

** Changed in: cloud-init (Ubuntu Precise)
       Status: New => Fix Committed

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1404311

Title:
  [SRU] gce metadata api doesn't properly stream binary data

Status in Init scripts for use on cloud images:
  Fix Committed
Status in cloud-init package in Ubuntu:
  Fix Released
Status in cloud-init source package in Precise:
  Fix Committed
Status in cloud-init source package in Trusty:
  Fix Committed
Status in cloud-init source package in Utopic:
  Fix Committed
Status in cloud-init source package in Vivid:
  Fix Released

Bug description:
  [IMPACT] Due to limitations in GCE, binary user-data is mangled when
  sent as user-data.

  [FIX] Allow user to declare binary encoding on user-data.

  [VERIFICATION]
  1. Create pristine image from -proposed
  2. For step 6
  3. Boot GCE instance w/ normal user-data, i.e.:
     $ cat user-data
     #cloud-config
     ssh_import_id: [utlemming]
     $ gcloud compute instances create <image from step 1> \
              --metadata-from-file user-data=user-data
  4. Confirm that user-data was parsed properly
  5. GZIP user-data, and encode using base64:
     gzip -c user-data.txt | base64 > user-data.b64
  6. Boot GCE instance w/ user-data.b64 and character encoding meta-data 
     set: 
     $ gcloud compute instances create <image from step 1> \
              --metadata-from-file user-data=user-data.b64 \
              --metadata user-data-encoding=base64
  7. Confirm that user-data was consumed; attach /var/log/cloud-init.log
     to report. 

  [RISK] If a user sets the user-data-encoding to base64, but does not
  provide base64 data the instance will fail to provision. However,
  since the user has to explicitly setup the circumstances, it is low
  risk.

  [ORIGINAL REPORT]
  The GCE datasource uses the long hostname. Hostnames longer than 64 characters can break several tools.
  While implementing the GCE provider for Juju we found that the metadata API breaks when trying to retrieve certain binary formats. In our case the gz of user-data. The API only streams out the first 5 bytes, encounters what it preceives as a EOF/nil character and truncates the rest of the request.

  We've opened an issue with Google directly, but in the meantime a work
  around is to allow an explicit encoding to be set for the user-data
  field of the GCE metadata. This will allow use to base64 encode the
  binary blob, which the API returns the entire contents of without
  issue.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1404311/+subscriptions