yahoo-eng-team team mailing list archive

[Sean McGinnis <sean_mcginnis@xxxxxxxx>]

Public bug reported:

When creating a new instance with the option to create a new volume from
an image and that image is large, instance creation can time out before
the image has had a chance to download.

With a larger image size (like Windows, but could be done with others) this process takes a long time. The instance creation times out with the default settings and the instance is cleaned up. But in the background the download is still happening.
                                                                                
So far, our Cinder driver is not involved in any way in this process.           
                                                                                
Once the image download finally completes it then finally calls in to the driver to create the volume. I would think this would be done first to make sure there is a volume to transfer the image to in the first place. It then attaches the volume to the compute host, then gets the failure rollback and deletes the volume.
                                                                                
If you watch the syslog (tail -f /var/log/syslog) while all this is happening you can see way after horizon errors out some messages from  the kernel that is discovered a new iSCSI device, then shortly afterwards a warning that it received an indication that the LUN            assignments have changed (from the device removal).

Horizon displays the message:

Error: Build of instance 84bba509-1727-4c32-83c4-925f91f12c6f aborted:
Block Device Mapping is Invalid

In the n-cpu.log there is a failure with the message:

VolumeNotCreated: Volume f5818ef3-c21d-44d4-b2e6-9996d4ac7bec did not
finish being created even after we waited 214 seconds or 61 attempts.
And its status is creating.

Someone from the nova team thought this could be the case if everything
is being passed to the novaclient rather than performing the operations
directly (with the volume likely being created first) like the tempest
tests do for boot images. It could be argued that the novaclient should
be updated to perform this correctly via that path, but this surfaces,
and could also be fixed, at the horizon level.

** Affects: horizon
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Dashboard (Horizon).
https://bugs.launchpad.net/bugs/1497358

Title:
  Creating instance with Boot from instance (creates new volume)
  timesout before image can be downloaded

Status in OpenStack Dashboard (Horizon):
  New

Bug description:
  When creating a new instance with the option to create a new volume
  from an image and that image is large, instance creation can time out
  before the image has had a chance to download.

  With a larger image size (like Windows, but could be done with others) this process takes a long time. The instance creation times out with the default settings and the instance is cleaned up. But in the background the download is still happening.
                                                                                  
  So far, our Cinder driver is not involved in any way in this process.           
                                                                                  
  Once the image download finally completes it then finally calls in to the driver to create the volume. I would think this would be done first to make sure there is a volume to transfer the image to in the first place. It then attaches the volume to the compute host, then gets the failure rollback and deletes the volume.
                                                                                  
  If you watch the syslog (tail -f /var/log/syslog) while all this is happening you can see way after horizon errors out some messages from  the kernel that is discovered a new iSCSI device, then shortly afterwards a warning that it received an indication that the LUN            assignments have changed (from the device removal).

  Horizon displays the message:

  Error: Build of instance 84bba509-1727-4c32-83c4-925f91f12c6f aborted:
  Block Device Mapping is Invalid

  In the n-cpu.log there is a failure with the message:

  VolumeNotCreated: Volume f5818ef3-c21d-44d4-b2e6-9996d4ac7bec did not
  finish being created even after we waited 214 seconds or 61 attempts.
  And its status is creating.

  Someone from the nova team thought this could be the case if
  everything is being passed to the novaclient rather than performing
  the operations directly (with the volume likely being created first)
  like the tempest tests do for boot images. It could be argued that the
  novaclient should be updated to perform this correctly via that path,
  but this surfaces, and could also be fixed, at the horizon level.

To manage notifications about this bug go to:
https://bugs.launchpad.net/horizon/+bug/1497358/+subscriptions