openstack team mailing list archive

[Ewan Mellor <Ewan.Mellor@xxxxxxxxxxxxx>]

> -----Original Message-----
> From: Scott Moser [mailto:ssmoser2@xxxxxxxxx] On Behalf Of Scott Moser
> Sent: 06 January 2012 08:15
> To: Ewan Mellor
> Cc: Richard W.M. Jones; openstack@xxxxxxxxxxxxxxxxxxx
> Subject: RE: [Openstack] Metadata and File Injection
> 
> On Sun, 1 Jan 2012, Ewan Mellor wrote:
> 
> > > -----Original Message-----
> > > From: openstack-bounces+ewan.mellor=citrix.com@xxxxxxxxxxxxxxxxxxx
> > > [mailto:openstack-
> bounces+ewan.mellor=citrix.com@xxxxxxxxxxxxxxxxxxx]
> > > On Behalf Of Richard W.M. Jones
> > > Sent: 21 December 2011 10:56
> > > To: Scott Moser
> > > Cc: openstack@xxxxxxxxxxxxxxxxxxx
> > > Subject: Re: [Openstack] Metadata and File Injection
> > >
> > > On Thu, Dec 15, 2011 at 11:43:13AM -0500, Scott Moser wrote:
> > > >  c.) No way to modify contents of the service after instance
> launch.
> > > >      OK, I said 2 features, and really this one is wishlist.  If
> we
> > > had an
> > > >      arbitrary key-value store that was available, the user could
> > > interact
> > > >      with the guest using this service.  It'd have to be poll-
> based,
> > > but a
> > > >      in-guest hypervisor could watch for creation and deletion of
> > > keys.
> > > >      Potentially, the MD might be RW from inside guest, meaning
> it
> > > could
> > > >      even convey information back.
> > >
> > > Sounds like xenstore ...
> >
> > Yes, that's exactly what XenStore is.  Even better, you don't even
> have
> > to poll -- it has an event system where you can block on a file
> > descriptor until a subtree changes.  Oh, and it's a tree, not just a
> > flat keyspace.
> >
> > Do you know, Rich, of any plans to implement something like XenStore
> for
> > KVM?  OpenStack would have no problem abstracting the differences
> > between two similar services, if there was something there to
> abstract
> > over, but up until now people have pushed back hard on any feature
> that
> > needed XenStore-like functionality, because it is completely missing
> > from KVM.  I think that that's a shame, because there are lots of
> > interesting things that you can do when you have a bidirectional,
> live
> > channel into the guest.
> 
> within kvm/qemu, the path that is being worked to getting host-guest
> communication outside of networking is being done via 'qemu-ga'.
> http://wiki.qemu.org/Features/QAPI/GuestAgent
> 
> Thats a guest agent that would run inside.
> 
> It will communicate over a virtual serial (virtio-serial) bus with the
> host.  The goal is to have full remote procedure call rather than just
> key/value storage.
> 
> Its better to shoot for a full 2 way communication channel between host
> and guest than to just settle for key/value.  The issue with key/value
> would just be that you don't know if the other side received the
> message.
> 
> With the existing guest-file-{open,read,write} you can easily implement
> a
> filesystem level tree like xenstore and use inotify from inside the
> guest
> to monitor for changes.
> 
> I'm not suggesting that qemu-qa is either omnipresent nor acceptable at
> the moment, but I generally agree with less hypervisor specific
> mechanism.
> All that it really depends on is a 2 way communication channel between
> host and guest.

The reason that XenStore was set up as a registry rather than an RPC was to make the system easier to debug.  It was felt that a registry would allow you to very easily see the current (or at least the intended) state of the system, whereas with an RPC-based system the messages have all gone by the time you notice anything's wrong.  There are tradeoffs in return of course, not least the fact that you need to build your own interlocks for indicating message receipt, as you say.

The good news is it's not difficult to build a keystore on top of an RPC system, or an RPC system on top of a keystore.  This means that OpenStack would be able to pick a paradigm, and we'd be able to implement those semantics for both KVM and Xen, even if we end up with some grungy translation to do for one of them.

So all we need now is for qemu-qa to mature!

Cheers,

Ewan.