openstack team mailing list archive

[Dong-In David Kang <dkang@xxxxxxx>]

----- Original Message -----
> On Mon, Dec 05, 2011 at 09:07:06PM -0500, Lorin Hochstein wrote:
> >
> >
> > On Dec 4, 2011, at 7:46 AM, Soren Hansen wrote:
> >
> > > 2011/12/4 Lorin Hochstein <lorin@xxxxxxx>:
> > >> Some of the LXC-related issues we've run into:
> > >>
> > >> - The CPU affinity issue on LXC you mention. Running LXC with
> > >> OpenStack, you
> > >> don't get proper "space sharing" out of the box, each instance
> > >> actually sees
> > >> all of the available CPUs. It's possible to restrict this, but
> > >> that
> > >> functionality doesn't seem to be exposed through libvirt, so it
> > >> would have
> > >> to be implemented in nova.
> 
> I recently added support for CPU affinity to the libvirt LXC driver.
> It will
> be in libvirt 0.9.8. I also wired up various other cgroups tunables
> including
> NUMA memory binding, block I/O tuning and CPU quota/period caps.

  Great news! 
 We are also looking forward to seeing SElinux 'sVirt' support for
LXC by libvirt.
When do you think it will be available? 
In libvirt-0.9.8?

> 
> > >> - LXC doesn't currently support volume attachment through
> > >> libvirt. We were
> > >> able to implement a workaround by invoking "lxc-attach" inside of
> > >> OpenStack
> > >> instead (e.g., see
> > >> <https://github.com/usc-isi/nova/blob/hpc-testing/nova/virt/libvirt/connection.py#L482>.
> > >> But to be able to use lxc-attach, we had to upgrade the Linux
> > >> kernel in
> > >> RHEL6.1 from 2.6.32 to 2.6.38. This kernel isn't supported by
> > >> SGI, which
> > >> means that we aren't able to load the SGI numa-related kernel
> > >> modules.
> 
> Can you clarify what you mean by volume attachment ?
> 
> Are you talking about passing through host block devices, or hotplug
> of
> further filesystems for the container ?
> 

 We tried both libvirt-0.9.3 and libvirt-0.9.7.
For both versions, attachvolume called by OpenStack failed when the target instance is an LXC instance.
In nova/virt/libvirt/connection.py, virt_dom.attachDevice(xml) failed for an LXC instance.
virt_dom.attachDevice(xml) is calling libvirt API.

 By volume attachment, yes, we mean passing through host block devices that is dynamically created by 
nova-volume service (using iscsi).


> > > Why not address these couple of issues in libvirt itself?
> 
> If you let me know what issues you have with libvirt + LXC in
> OpenStack,
> I'll put them on my todo list.
> 

 As Lorin said we implemented it using lxc-attach. 
With lxc-attach we could pass the major/minor number of the (dynamically crated) devices to the LXC instance.
And with lxc-attach we could do "mknod" inside of the LXC instance.
I think supporting that by libvirt would be very useful.
However, it needs lxc-attach working for the Linux kernel. 
We had to upgrade and patch Linux kernel for that purpose.
If there is a better way, it would be wonderful.
But I don't know if there is a way other than using lxc-attach.

 Thanks,

 David.


> Regards,
> Daniel
> --
> |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :|
> |: http://libvirt.org -o- http://virt-manager.org :|
> |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
> |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
> _______________________________________________
> DODCS mailing list
> DODCS@xxxxxxxxxxxxxxx
> http://mailman.isi.edu/mailman/listinfo/dodcs