openstack team mailing list archive

[Dong-In David Kang <dkang@xxxxxxx>]

----- Original Message -----
> Hi,
> 
> So I was the developer who added support for LXC support initially, I
> have some comments in line
> 
>  On Tue, 6 Dec 2011 12:04:53 -0800 (PST)
> Dong-In David Kang <dkang@xxxxxxx> wrote:
> 
> > 
> > 
> > ----- Original Message -----
> > > On Mon, Dec 05, 2011 at 09:07:06PM -0500, Lorin Hochstein wrote:
> > > >
> > > >
> > > > On Dec 4, 2011, at 7:46 AM, Soren Hansen wrote:
> > > >
> > > > > 2011/12/4 Lorin Hochstein <lorin@xxxxxxx>:
> > > > >> Some of the LXC-related issues we've run into:
> > > > >>
> > > > >> - The CPU affinity issue on LXC you mention. Running LXC
> > > > >> with
> > > > >> OpenStack, you
> > > > >> don't get proper "space sharing" out of the box, each
> > > > >> instance
> > > > >> actually sees
> > > > >> all of the available CPUs. It's possible to restrict this,
> > > > >> but
> > > > >> that
> > > > >> functionality doesn't seem to be exposed through libvirt, so
> > > > >> it
> > > > >> would have
> > > > >> to be implemented in nova.
> > > 
> > > I recently added support for CPU affinity to the libvirt LXC
> > > driver.
> > > It will
> > > be in libvirt 0.9.8. I also wired up various other cgroups
> > > tunables
> > > including
> > > NUMA memory binding, block I/O tuning and CPU quota/period caps.
> >
> 
> We are running libvirt 0.9.7 in Ubuntu for Pangolin  and I expect to
> have 0.9.8 before Pangolin is released.
> 
> >   Great news!
> >  We are also looking forward to seeing SElinux 'sVirt' support for
> > LXC by libvirt.
> > When do you think it will be available?
> > In libvirt-0.9.8?
> > 
> > > 
> > > > >> - LXC doesn't currently support volume attachment through
> > > > >> libvirt. We were
> > > > >> able to implement a workaround by invoking "lxc-attach"
> > > > >> inside
> > > > >> of OpenStack
> > > > >> instead (e.g., see
> > > > >> <https://github.com/usc-isi/nova/blob/hpc-testing/nova/virt/libvirt/connection.py#L482>.
> > > > >> But to be able to use lxc-attach, we had to upgrade the
> > > > >> Linux
> > > > >> kernel in
> > > > >> RHEL6.1 from 2.6.32 to 2.6.38. This kernel isn't supported
> > > > >> by
> > > > >> SGI, which
> > > > >> means that we aren't able to load the SGI numa-related
> > > > >> kernel
> > > > >> modules.
> > > 
> > > Can you clarify what you mean by volume attachment ?
> > > 
> > > Are you talking about passing through host block devices, or
> > > hotplug
> > > of
> > > further filesystems for the container ?
> > > 
> > 
> >  We tried both libvirt-0.9.3 and libvirt-0.9.7.
> > For both versions, attachvolume called by OpenStack failed when the
> > target instance is an LXC instance. In
> > nova/virt/libvirt/connection.py, virt_dom.attachDevice(xml) failed
> > for an LXC instance. virt_dom.attachDevice(xml) is calling libvirt
> > API.
> > 
> >  By volume attachment, yes, we mean passing through host block
> > devices that is dynamically created by nova-volume service (using
> > iscsi).
> > 
> 
> This is on my todo list for essex.
> 

 Is there something that we can help?
We've implemented volume support for LXC in our branch.
https://github.com/usc-isi/nova/blob/hpc-testing/nova/virt/libvirt/connection.py

It has some extra features and the code needs to be polished.
Please let us know if there is something we can help.


> > > > > Why not address these couple of issues in libvirt itself?
> > > 
> > > If you let me know what issues you have with libvirt + LXC in
> > > OpenStack,
> > > I'll put them on my todo list.
> > > 
> > 
> >  As Lorin said we implemented it using lxc-attach.
> > With lxc-attach we could pass the major/minor number of the
> > (dynamically crated) devices to the LXC instance. And with
> > lxc-attach
> > we could do "mknod" inside of the LXC instance. I think supporting
> > that by libvirt would be very useful. However, it needs lxc-attach
> > working for the Linux kernel. We had to upgrade and patch Linux
> > kernel for that purpose. If there is a better way, it would be
> > wonderful. But I don't know if there is a way other than using
> > lxc-attach.
> > 
> >  Thanks,
> > 
> >  David.
> > 
> > 
> > > Regards,
> > > Daniel
> > > --
> > > |: http://berrange.com -o-
> > > http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org
> > > -o- http://virt-manager.org :| |: http://autobuild.org -o-
> > > http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org
> > > -o- http://live.gnome.org/gtk-vnc :|
> > > _______________________________________________ DODCS mailing
> > > list
> > > DODCS@xxxxxxxxxxxxxxx
> > > http://mailman.isi.edu/mailman/listinfo/dodcs
> > 
> > _______________________________________________
> > Mailing list: https://launchpad.net/~openstack
> > Post to     : openstack@xxxxxxxxxxxxxxxxxxx
> > Unsubscribe : https://launchpad.net/~openstack
> > More help   : https://help.launchpad.net/ListHelp
> 
> 
> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to     : openstack@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
>