savanna-all team mailing list archive

[Alexander Ignatov <aignatov@xxxxxxxxxxxx>]

Hi Jon,

I've updated change request of our implemetation of vanilla Hadoop provider:
https://review.openstack.org/#/c/31532/1

Please note that it is draft version and there are a lot of code to 
improve. Though you can find the main points there.

Just in case I describe tips and tricks to operate with objects inside 
the plugin:

1. Instances and remote calls.
    Instance objects represents actual VMs on OpenStack. Once all 
instances are provisioned Savanna controller calls configure_cluster() 
of some plugin and cluster object contains all that instances. Note, all 
instances are stored inside cirtain node group.
Examples:
a. You want iterate over all instances to get its hostnames:
def configure_cluster(self, cluster):
        for ng in cluster.node_groups:
                for instance in ng.instances:
                        print instance.hostname

Please refer to available fields and properties of Instance object here:
https://github.com/stackforge/savanna/blob/master/savanna/db/models.py#L150-L177

b. You want to execute some command via ssh on each instance:
def configure_cluster(self, cluster):
        for ng in cluster.node_groups:
                for instance in ng.instances:
                        retcode, buf instance.remote.excute_command('ls 
-la')
                        print ret, buf

where ret is returned code and buf is output of the executed command

If you want to copy some data to the some file on the VM you can use:
        data = 'line1\nline2\n'
        instance.remote.write_file_to('/etc/hadoop/config.xml', data)
to read some file
       data = instance.remote.read_file_from('remote_file')

Please see _push_configs_to_nodes() method in plugin.py in my change 
request(link above).


2. Cluster, NodeGroup and Instnce extra field.
Once your plugin recieves 'cluster' object  you can use 'extra' field 
for any needs. Extra fields will be available in 'cluster' object always.
For example you want to remember any object to operate with it directly 
not iterating over and over all instances.
In my plugin implementation I want to remeber NameNode management_ip and 
hostname to push some configs only to this VM.
cluster.extra['nn_instance'] = None
for node_group in cluster.node_groups:
      ng_proc = node_group.node_processes
      for instance in node_group.instances:
          if 'namenode' in ng_proc:
              cluster.extra['nn_instance'] = instance

#then we can format namenode, for example
cluster.extra['nn_instance'].remote.execute_command("su -c 'hadoop 
namenode -format' hadoop")

3. Nova
Just import it:
from savanna.utils.openstack import nova

Examples:
# list of images
nova.client().images.list()

# flavors
nova.client().flavors.list()

You can find more nova usages here:
https://github.com/stackforge/savanna/blob/master/savanna/service/instances.py


I hope this will help to integrate your provider. Please feel free to 
ask any questions about invocations and object usage.

--
Regards,
Alexander Ignatov


On 6/6/2013 12:07 AM, Jon Maron wrote:
> Hi,
>
>   Can the Plugin SPI document 
> (https://wiki.openstack.org/wiki/Savanna/PluggableProvisioning/PluginAPI) 
> be updated with some information regarding the context object and its 
> associated services/APIs?  In our implementation we are still 
> interacting directly with nova/ssh etc and we need an understanding of 
> how to migrate those invocations with the context of the savanna server.
>
> -- Jon