openstack team mailing list archive

[Scott Moser <ssmoser2@xxxxxxxxx>]

On Wed, 15 Dec 2010, Thierry Carrez wrote:

> Chris Behrens wrote:
> > Ed and I were discussing this earlier.  Since I've become the owner of the current Rackspace linux guest agent, I've had some plans to make it easier to change the communication layer so that it can support more than just running under Xen and communicating through XenStore.
> >
> > On the agent side of things, I'm planning to make the communication layer pluggable.  This means that that XenStore communication will move into some sort of module, and other modules could be created to communicate over the network (via a private host<->guest network interface) or however else we may decide to make it communicate in the future.  The pieces that do the real work like updating the linux distro configuration files for network, etc, shouldn't need to change based on the hypervisor or communication layer.  But, those should be somewhat pluggable as well, because different distros use different configuration files. :)
>
> There is some overlap here with the discussion on initial instance
> setup. How for example you inject the authorized_keys in the instance.
>
> Soren was discussing this yesterday with pvo, and their most
> cross-platform option seemed to be to emulate a CD in the instance, that
> OS images can be modified to take advantage of.

If you're thinking that you should present the guest with data via an
attached device with an ISO9660 filesystem on it, then you should probably
heavily consider using the format described in the OVF ISO "Transport".
It would seem silly to use a basic transport mechanism so similar to a
developing standard.  One might then make an accusation of NIH syndrome.

The primary thing I do not like about either "cdrom" and the current
implementation of the EC2 metadata service is that it is read-only from
the guests perspective.  The xen store (I think) has the advantage of
being read-write, and obviously a web api such as EC2 metadata could be.

The largest thing I dislike about the EC2 metadata service is that there
is no way to disable it.  Because its a network resource, you can't apply
simple unix permissions on it and limit sensitive data to root-only in the
instance.  In cloud-init, we have the ability to write a null-route to the
169.254.169.254, which essentially prevents anyone other than root to
getting at it.  I would prefer an API call either from external or
internal that removed all access to it.

While nothing in EC2 promises you that EC2 "userdata" is a secure
transport mechanism, it would just be so much more functional if it was.
You could pass information to the first boot of an instance that could be
read, used, and then destroyed.

The thing that I think Amazon did extremely well was to generally take a
hands off approach.  They provide resources that the instance can get
metadata or user data from, but make no assumptions that they know what
the instance is going to do with that data, or even that they know the
filesystem or operating system inside.

The reason I like that so much is that it frees me, as an Image provider
able to change just about anything inside my image (ie, the filesystem,
the kernel, the OS....).  I have complete freedom.  The only restriction
is that in order to get user data and some other data about my instance, I
have to be able to be able to speak http over IPv4 to a well known
address.

> The official Ubuntu cloud images use the EC2 metadata service: they
> query it at boot-time to get user-data that is then used to customize
> the instance behavior. If you add Xenstore, that makes already three
> different means for the IaaS layer to communicate with specific guests.
>
> Maybe we could use the same kind of pluggable communication layer in
> both cases: the boot-time agent (getting info once at boot-time, usually
> provided by the OS image builder), and the active agent (polling info
> regularly, usually planted in by your cloud provider). I'd hate to see
> the two use different mechanisms or layers. (Note that the active agent
> would get installed by the boot-time agent, you don't really want to
> inject files directly into filesystems.)

I'm not sure exactly what you're suggesting here, but bug 684804
(https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/684804) was
opened against cloud-init.  Cloud init offers a mechanism for supporting
multi-part content in user-data.  However, if the service provider wants
to put something into that user data (such as guest-agent installation
instructions) then the end user no longer has complete control over its
content.  Ie if they have no interest in cloud-init, they have to accept
that the data they provided would be stuffed inside an multi-part archive
in order for the platform to stuff *their* data in also.

So, the bug opener suggests that there should be 2 different "user-data"
sources, one for the platform, one for the user.  I think its a good idea.

>
> Some pointers on cloud-init, the boot-time agent included in Ubuntu
> cloud images, that Amazon reused in their own linux cloud images:
>
> https://help.ubuntu.com/community/CloudInit
>
> It is currently EC2-metadata-service-specific, but there is no reason we
> can't extend it to support the same "pluggable communication layer".

It does support reading data from either
a.) files in the filesystem
b.) urls referenced from a file in the filesystem.

There are examples of that at
https://help.ubuntu.com/community/UEC/Images (look for 'seed').