openstack team mailing list archive

[Joshua Harlow <harlowja@xxxxxxxxxxxxx>]

Awesome!

Great job :-)

+1 milllion

On 3/8/12 11:03 AM, "Daniel P. Berrange" <berrange@xxxxxxxxxx> wrote:

Back in January Joshua Harlow raised the question of whether we should
replace the usage of Cheetah templates for generating XML in the libvirt
driver:

  https://lists.launchpad.net/openstack/msg06481.html

Since then I have had some time to work on this idea and now have a
working branch available for testing. I don't want to push this to
Gerrit right now, since it isn't really material suitable for the
Essex release, and AFAICT we don't have a separate review/GIT branch
for non-Essex feature dev work.

Thus for now I have pushed it to a private a branch here:

  https://github.com/berrange/nova/tree/libvirt-xml-config-v1

The  foundation for the work is early in the series, where I create
a new nova/virt/libvirt/config.py module with a set of classes for
representing the aspects of libvirt configuration that Nova is interested
in. Each of the config classes implement a format_dom() method for
serializing themselves to an lxml.etree.Element DOM instance.

Currently these objects can be used to generate XML, but in the future
they will also be able to parse the XML. For this they will implement
a parse_dom() method which will de-deserialize the xml.etree.Element
DOM.

Joshua's original posting had talked about having separate layers
for the config objects vs the serialization. IMHO this would be
overkill, just adding abstraction for little real world gain. We
don't need to have pluggable XML serialization impls, one good one
is sufficient.

The rest of the series is simply a piece-by-piece conversion of the
template code to the new object based APIs. I did it in a great many
steps, to make it easier to review & test the changes.

As well as the guest config creation, I also took the opportunity to
change two others places where we generate XML. The host CPU comparison
code and the domain snapshot creation. There is still one place left
to fix, the firewall filter generator.

By the end of the series we have the following benefits

 - No code anywhere outside config.py ever needs to know about XML
   documents

 - We actually have proper XML escaping, making us safe from potential
   exploits in that area

 - There is clean separation of the logic for constructing the
   guest config, from the logic for generating XML.


My next step following on from this is to actually start making the
config generation more flexible, removing alot of hardcoding it
currently does (eg horrible global virtio on/off switch). This will
entail tagging images on import with an operating system identifier,
and then using libosinfo to query exactly what hardware devices the
OS supports & picking the optimal ones.

I tested this on a KVM host and verified the XML generated for the
guest before/after was the same. I've not tested all the possible
block / network driver combinations though, so might have broken
something not covered by the test suite

Diffstat for the whole patch series

 b/nova/tests/fakelibvirt.py         |   11
 b/nova/tests/test_libvirt.py        |   67 +++--
 b/nova/tests/test_libvirt_config.py |  448 +++++++++++++++++++++++++++++++++
 b/nova/tests/test_libvirt_vif.py    |   54 +---
 b/nova/virt/libvirt/config.py       |  420 +++++++++++++++++++++++++++++++
 b/nova/virt/libvirt/connection.py   |  476 ++++++++++++++++++++++--------------
 b/nova/virt/libvirt/vif.py          |  102 ++++---
 b/nova/virt/libvirt/volume.py       |   52 ++-
 nova/virt/cpuinfo.xml.template      |    9
 nova/virt/libvirt.xml.template      |  188 --------------
 10 files changed, 1323 insertions(+), 504 deletions(-)


Regards,
Daniel
--
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|