yahoo-eng-team team mailing list archive

[Andrew Kutz <1992509@xxxxxxxxxxxxxxxxxx>]

Public bug reported:

At some point Canonical updated the OVF used to produce Jammy's OVA at
https://cloud-
images.ubuntu.com/releases/jammy/release-20220808/ubuntu-22.04-server-
cloudimg-amd64.ova from the OVF used to produce Impish's OVA at
https://cloud-
images.ubuntu.com/releases/impish/release-20220708/ubuntu-21.10-server-
cloudimg-amd64.ova. However, this update routinely causes failures in
the Cloud-Init Data Source for VMware at
https://cloudinit.readthedocs.io/en/latest/topics/datasources/vmware.html.

Technically speaking the VMware data source should have routinely failed
with Canonical's cloud OVA, but prior to Jammy there was a bug in the
OVF -- the CD-ROM device was parented to the wrong IDE controller. The
diff between the two OVFs is attached, but the important bit was here:

> -        <rasd:Parent>3</rasd:Parent>
> +        <rasd:Parent>5</rasd:Parent>

The CD-ROM device in Impish is parented to device ID 3, which is the
following device:

>      <Item>
>        <rasd:Address>0</rasd:Address>
>        <rasd:Description>SCSI Controller</rasd:Description>
>        <rasd:ElementName>SCSI Controller 0</rasd:ElementName>
>        <rasd:InstanceID>3</rasd:InstanceID>
>        <rasd:ResourceSubType>VirtualSCSI</rasd:ResourceSubType>
>        <rasd:ResourceType>6</rasd:ResourceType>
>      </Item>

Except when parenting a CD-ROM device to a SCSI controller like this
causes vSphere 7.x+ (and perhaps earlier versions -- I could not easily
check) to quietly drop the CD-ROM device. However, on Jammy the parent
is device ID 5, which is the following device:

>      <Item>
>        <rasd:Address>1</rasd:Address>
>        <rasd:Description>IDE Controller</rasd:Description>
>        <rasd:ElementName>VirtualIDEController 1</rasd:ElementName>
>        <rasd:InstanceID>5</rasd:InstanceID>
>        <rasd:ResourceType>5</rasd:ResourceType>
>      </Item>

Parenting a CD-ROM to an IDE controller is fine, and deploying the Jammy
OVA on vSphere 7.x+ results in a VM with a valid CD-ROM device. And that
is the root of the problem. Because now if the VM also has one or more
OVF properties defined, as the Ubuntu OVFs do, vSphere will send that
information into the guest using one of two transports: ISO or GuestInfo
(ESX --> VM RPC via VM Tools). Both Ubuntu OVFs indicate to use the ISO
transport:

>     <VirtualHardwareSection ovf:transport="iso">

The ISO transport logic in vSphere is as follows:

1. if there are one or more OVF properties
2. construct an ISO file with the OVF environment and its properties
3. upload the ISO to a datastore accessible by the ESXi host where the VM is scheduled
4. attach the ISO file to the VM's CD-ROM device

When the VM is powered on, VM Tools reads the contents of the ISO
mounted to the CD-ROM device. However, the presence of the CD-ROM with
valid OVF environment data *also* results in the Cloud-Init OVF data
source
(https://cloudinit.readthedocs.io/en/latest/topics/datasources/ovf.html)
detecting that it should be loaded (https://github.com/canonical/cloud-
init/blob/cd2cca35a1bf36b584422f431c3ddf55b820434c/tools/ds-
identify#L984-L1030). Once that occurs the VMware data source is not
loaded, even if there is valid data to do so
(https://github.com/canonical/cloud-
init/blob/cd2cca35a1bf36b584422f431c3ddf55b820434c/tools/ds-
identify#L1459-L1499).

The fix to the CD-ROM device is itself not an issue -- it *should* have
been fixed. In fact, if the CD-ROM was not fixed but the "ovf:transport"
value had been changed to "guestInfo" then the same problem would have
occurred as the OVF data source would have been triggered that way and
taken precedence over the VMware data source
(https://github.com/canonical/cloud-
init/blob/cd2cca35a1bf36b584422f431c3ddf55b820434c/tools/ds-
identify#L965-L982). However, while not strictly a problem in a vacuum,
the change *does* regress the prior behavior of the stock, Ubuntu cloud
images. All of a sudden the Jammy OVA, and presumably future images,
will trigger the OVF data source on vSphere instead of the VMware data
source when both were detected via ds_identify.

This is unfortunate, and I am not sure what can be done about it. The
fix to the CD-ROM was necessary, even if it did break the VMware data
source when OVF properties are defined in the OVF. What do y'all think
should occur?

** Affects: cloud-init
     Importance: Undecided
         Status: New

** Patch added: "A diff between Jammy's and Impish's OVF files"
   https://bugs.launchpad.net/bugs/1992509/+attachment/5623112/+files/jammy-v-impish.ovf.diff

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1992509

Title:
  Jammy's fix to CD-ROM device in OVF breaks VMware datasource

Status in cloud-init:
  New

Bug description:
  At some point Canonical updated the OVF used to produce Jammy's OVA at
  https://cloud-
  images.ubuntu.com/releases/jammy/release-20220808/ubuntu-22.04-server-
  cloudimg-amd64.ova from the OVF used to produce Impish's OVA at
  https://cloud-
  images.ubuntu.com/releases/impish/release-20220708/ubuntu-21.10-server-
  cloudimg-amd64.ova. However, this update routinely causes failures in
  the Cloud-Init Data Source for VMware at
  https://cloudinit.readthedocs.io/en/latest/topics/datasources/vmware.html.

  Technically speaking the VMware data source should have routinely
  failed with Canonical's cloud OVA, but prior to Jammy there was a bug
  in the OVF -- the CD-ROM device was parented to the wrong IDE
  controller. The diff between the two OVFs is attached, but the
  important bit was here:

  > -        <rasd:Parent>3</rasd:Parent>
  > +        <rasd:Parent>5</rasd:Parent>

  The CD-ROM device in Impish is parented to device ID 3, which is the
  following device:

  >      <Item>
  >        <rasd:Address>0</rasd:Address>
  >        <rasd:Description>SCSI Controller</rasd:Description>
  >        <rasd:ElementName>SCSI Controller 0</rasd:ElementName>
  >        <rasd:InstanceID>3</rasd:InstanceID>
  >        <rasd:ResourceSubType>VirtualSCSI</rasd:ResourceSubType>
  >        <rasd:ResourceType>6</rasd:ResourceType>
  >      </Item>

  Except when parenting a CD-ROM device to a SCSI controller like this
  causes vSphere 7.x+ (and perhaps earlier versions -- I could not
  easily check) to quietly drop the CD-ROM device. However, on Jammy the
  parent is device ID 5, which is the following device:

  >      <Item>
  >        <rasd:Address>1</rasd:Address>
  >        <rasd:Description>IDE Controller</rasd:Description>
  >        <rasd:ElementName>VirtualIDEController 1</rasd:ElementName>
  >        <rasd:InstanceID>5</rasd:InstanceID>
  >        <rasd:ResourceType>5</rasd:ResourceType>
  >      </Item>

  Parenting a CD-ROM to an IDE controller is fine, and deploying the
  Jammy OVA on vSphere 7.x+ results in a VM with a valid CD-ROM device.
  And that is the root of the problem. Because now if the VM also has
  one or more OVF properties defined, as the Ubuntu OVFs do, vSphere
  will send that information into the guest using one of two transports:
  ISO or GuestInfo (ESX --> VM RPC via VM Tools). Both Ubuntu OVFs
  indicate to use the ISO transport:

  >     <VirtualHardwareSection ovf:transport="iso">

  The ISO transport logic in vSphere is as follows:

  1. if there are one or more OVF properties
  2. construct an ISO file with the OVF environment and its properties
  3. upload the ISO to a datastore accessible by the ESXi host where the VM is scheduled
  4. attach the ISO file to the VM's CD-ROM device

  When the VM is powered on, VM Tools reads the contents of the ISO
  mounted to the CD-ROM device. However, the presence of the CD-ROM with
  valid OVF environment data *also* results in the Cloud-Init OVF data
  source
  (https://cloudinit.readthedocs.io/en/latest/topics/datasources/ovf.html)
  detecting that it should be loaded
  (https://github.com/canonical/cloud-
  init/blob/cd2cca35a1bf36b584422f431c3ddf55b820434c/tools/ds-
  identify#L984-L1030). Once that occurs the VMware data source is not
  loaded, even if there is valid data to do so
  (https://github.com/canonical/cloud-
  init/blob/cd2cca35a1bf36b584422f431c3ddf55b820434c/tools/ds-
  identify#L1459-L1499).

  The fix to the CD-ROM device is itself not an issue -- it *should*
  have been fixed. In fact, if the CD-ROM was not fixed but the
  "ovf:transport" value had been changed to "guestInfo" then the same
  problem would have occurred as the OVF data source would have been
  triggered that way and taken precedence over the VMware data source
  (https://github.com/canonical/cloud-
  init/blob/cd2cca35a1bf36b584422f431c3ddf55b820434c/tools/ds-
  identify#L965-L982). However, while not strictly a problem in a
  vacuum, the change *does* regress the prior behavior of the stock,
  Ubuntu cloud images. All of a sudden the Jammy OVA, and presumably
  future images, will trigger the OVF data source on vSphere instead of
  the VMware data source when both were detected via ds_identify.

  This is unfortunate, and I am not sure what can be done about it. The
  fix to the CD-ROM was necessary, even if it did break the VMware data
  source when OVF properties are defined in the OVF. What do y'all think
  should occur?

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1992509/+subscriptions