openstack team mailing list archive

["John Purrier" <john@xxxxxxxxxxxxx>]

A discussion in order to set a common understanding of supported image
formats in OpenStack and how the project can approach the issues surrounding
various hypervisors, cross-cloud interoperability, and potentially setting
some necessary development work items. Once the community has reached
consensus the Project Oversight Committee should consider and set a clear
project set of guidelines.

 

Image Formats and workload portability in OpenStack and public and/or
private cloud deployments.

 

The key point of the discussion: Virtual Image Formats matter. As the longer
term vision for OpenStack Compute and the cloud computing industry is
formulated it is becoming clear that over the next few years the issues of
portable workloads, federated cloud deployments, and seamless operations
across multi-hypervisor systems (whether they are inside a single DC or
federated across two or more deployments) will be a key element of success.
OpenStack Compute will form the technological basis for achieving this.

 

However, stating that Image Formats matter does not imply that there will be
a "winner" in the marketplace or that a singular format is pushed via the
OpenStack project. In order to achieve the goals stated above surrounding
workload portability the key element will be not the actual virtual disk
formats; but rather a standard VM interchange standard and the ability to
easily re-purpose the VM image to the target environment (whether it be a
public cloud or a private instance).

 

Today we are seeing most, if not all, of the popular virtualization systems
branching to support alternate VM image formats beyond what they consider
"native", either through conversion tools or the ability to natively mount
and run the alternative formats. This demonstrates that the market
understands the importance of being able to be somewhat image format
agnostic and that there is no value in trying to lock-in customers through
proprietary formats.

 

The Virtual Disk Format playing field today looks like:

 

.         VHD - Citrix XenServer and Microsoft Hyper-V

.         VMDK - VMWare ESX

.         VDI - Oracle VirtualBox

.         QCOW2 - KVM and QEMU

 

All the hypervisors also support Raw Disk Images. Additionally, deployments
can combine a disk partitioning and file-system (usually via LVM) to get
more direct control of Raw Images and to provide some additional features,
such as Snapshot with Coalesce (as Rackspace does on the current Linux Cloud
Servers product).

 

These disk formats are not too complicated, and there are free and cheap
tools that allow conversions between the various formats. As stated above,
there is a movement within all of these projects to reduce the friction in
moving VM images into their environment that were created by someone else.

 

Proposal for comment:

 

OpenStack has, at its core, a mission to provide widespread ubiquity of
Virtual/Cloud Computing and also has taken a hypervisor-agnostic approach.
In order to foster this and to move the project to a world where workloads
can be easily moved between installations that are running different
underlying virtualization systems the following should occur:

 

1.      A standardized VM Image exchange format should be described. The
DMTF already has a specification that is useful here, OVF. OVF is a
specification that provides for an exchange container that will include
standardized mechanisms to describe meta-data and also to encapsulate one or
more VM images or .ISO files. OVF does not dictate any particular VM Image
format, and hence is ideal for our purposes.

 

An OpenStack VM interchange specification should be drafted and reviewed by
the OpenStack community.

 

2.      OpenStack should *not* specify a "preferred" or "default" virtual
disk format. Just as a core tenet is hypervisor agnosticism another key
position will be VM Image agnosticism. The message is that each deployment
should choose the best hypervisor and image format for their particular
goals. This allows companies like Rackspace or others to choose the systems
and formats that best support the features they want to take to the market.

 

This also sidesteps the potentially divisive topic of support for VHD and
the Microsoft Open Promise licensing. Since support will be optional for all
formats if someone is adamant about not supporting VHD in their OpenStack
deployment they are free to not include it. Others, including Rackspace, may
choose to support VHD for the features it provides.

 

3.      The OpenStack Image Repository project ("Glance") should be extended
to provide native image conversion capabilities. This will hide the
differences in formats from the deployment and operational decisions and
allow workloads to easily be imported and exported between both OpenStack
installations, and also to/from native virtualization installations (such as
VMware).

 

A blueprint and initial vision specification should be drafted and submitted
to the OpenStack Launchpad repository. This will trigger a discussion
amongst the OpenStack community and the Glance committers specifically.

 

Comments welcome.

 

John