yahoo-eng-team team mailing list archive

[Scott Moser <smoser@xxxxxxxxxx>]

** Also affects: cloud-init (Ubuntu Yakkety)
   Importance: Undecided
       Status: New

** Changed in: cloud-init (Ubuntu Yakkety)
       Status: New => Confirmed

** Changed in: cloud-init (Ubuntu Yakkety)
   Importance: Undecided => Medium

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1582323

Title:
  Commissioning fails when competing cloud metadata resides on disk

Status in cloud-init:
  Fix Released
Status in MAAS:
  Fix Committed
Status in MAAS 2.1 series:
  Fix Released
Status in MAAS trunk series:
  Fix Committed
Status in cloud-init package in Ubuntu:
  Fix Released
Status in cloud-init source package in Xenial:
  Fix Committed
Status in cloud-init source package in Yakkety:
  Confirmed

Bug description:
  === Begin SRU Information ===
  [Impact]
  The issue originally reported was that when MAAS attempted to enlist a
  system by booting it with a remote iscsi disk with intent to have cloud-init
  utilize the MAAS metadata service, cloud-init found some metadata from a
  previous use of the system on the local disk.  cloud-init then went on
  to use that data and did not respond to maas.

  The impact in this case was that cloud-init failed to enlist.  The same problem
  could occur in any other case where there was data on the local disk that
  provided a datasource for cloud-init.

  The fix provided was for the scenario provided was for MAAS to provide
  configuration on the maas provided kernel command line that tells cloud-init
  it should read only attempt to use the MAAS datasource.

  Specifically, as mentioned in comment 7, this looked like:
     root=iscsi:.... cc:{'datasource_list': ['MAAS']}end_cc \
         cloud-config-url=http://maas/path/to/config ...

  cloud-init then reads that information on boot and it overrides the settings
  found inside the iscsi root device.

  [Test Case]
  A test case lives in unit tests now that ensures kernel config overrides
  system config.

  To further test this we could
  a.) cause this situation by
    1.) installing a node in maas
    2.) putting config drive or nocloud data onto one of the partitions
    3.) returning the system to maas
    4.) attempt re-deploy.

  b.) use a cloud image, kernel and initramfs and web server
    1.) download image, update cloud-init to -proposed.
    2.) set up a web service to serve files like MAAS described at
        https://maas.ubuntu.com/docs/development/metadata.html
    3.) boot image with kernel command line including the cc: and cloud-config-url  referencing that web service.
    4.) have provided a config drive or nocloud seed disk to the vm.

  The 'b' test above is easier to reproduce in that it does not rely on
  MAAS.

  [Regression Potential]
  Regression potential is low, in that this feature worked for some time
  in previous releases.  A bad reading of the code made me (smoser) change
  the code intending to fix the problem, but in fact regressed it.  So this
  change is actually reverting a previous change in behavior.

  This was first broken in 16.04 in 0.7.7~bzr1245-0ubuntu1~16.04.1 .

  [Other Info]
  The upstream commit that fixed this behavior (including the added tests)
  is 0b0f254a [1]

  --
  [1] https://git.launchpad.net/cloud-init/commit/?id=0b0f254a6935a1b1fff128fa177152dd519e1a3d

  === End SRU Information ===

  A customer reused hardware that had previously deployed a RHEL
  Overcloud-controller which places metadata on the disk as a legitimate
  source, that cloud-init looks at by default.  When the newly enlisted
  node appeared it had the name of "overcloud-controller-0" vs. maas-
  enlist, pulled from the disk metadata which had overridden MAAS'
  metadata.  Commissioning continually failed on all of the nodes until
  the disk metadata was manually removed (KVM boot Ubuntu ISO, rm -f
  data or dd zeros to disk).

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1582323/+subscriptions