savanna-all team mailing list archive

["Alexander Ignatov" <aignatov@xxxxxxxxxxxx>]

Hi Matthew,

That's great. It will be really good to have a set of images or instructions
with different distributions to be able working with Savanna.
We will commit our scripts and instructions soon. Not sure that savanna's
repo is a good place to store such instructions. I think we should create a
new repo on the stackforge for that.

Regards,
Alexander Ignatov

-----Original Message-----
From: Matthew Farrellee [mailto:matt@xxxxxxxxxx] 
Sent: Tuesday, May 21, 2013 1:54 AM
To: Alexander Ignatov
Cc: 'Sergey Lukjanov'; savanna-all@xxxxxxxxxxxxxxxxxxx
Subject: How to construct the savanna-0.1-hdp-img

Thanks Alexander and Sergey.

I'm going to try out these instructions on a Fedora / RHEL image and see if
I'm able to get a working Savanna system.

In the meantime, will you checkin the scripts to the savanna repository?

Best,


matt

On 05/14/2013 01:32 PM, Alexander Ignatov wrote:
> Matt,
>
> We didn't check images creation in the latest commit in the trunk of 
> diskimage-builder project.
> Here is commit id where created images were worked with Savanna:
> f13570d322e6b06ed945d310c51e68aac94ee7b8
>
> Regards,
> Alexander Ignatov
>
>
> -----Original Message-----
> From: Alexander Ignatov [mailto:aignatov@xxxxxxxxxxxx]
> Sent: Tuesday, May 14, 2013 7:28 PM
> To: 'Sergey Lukjanov'; 'Matthew Farrellee'
> Cc: 'savanna-team@xxxxxxxxxxxx'
> Subject: RE: 0.1.1 cluster remains Starting
>
> Hi Matt,
>
> Actually savanna-0.1-hdp-img was created manually in our OpenStack 
> Lab. Now we are working on automatization of the Savanna image 
> preparation using diskimage-builder project: 
> https://github.com/stackforge/diskimage-builder
>
> Here are manual and automated descriptions. Please note that now we 
> are working with Ubuntu cloud image only in Savanna.
>
> Manual steps (for Ubuntu):
> 1. Run 1 VM instance from cloud image in the OpenStack Lab.
> 2. Install ssh-server on this instance.
> 3. Install Oracle JDK. We are using jdk-6u43-linux-x64 in our image.
> 4. Create groop and user "hadoop" with disabled password.
> 5. Install some Hadoop version 1.1.X (we are using 1.1.1 version) from 
> http://archive.apache.org/dist/hadoop/core/ . We are installing Hadoop 
> from deb packages only. Configuration files related to Hadoop will be 
> placed in /etc/hadoop dir after installation.
> 6. Change .bashrc file for in /home/hadoop. Add the following variables:
>     HADOOP_HOME=/usr/share/hadoop/ and PATH=$PATH:/usr/sbin/
>     Source it.
> 7. Genereate ssh key for hadoop user using ssh-keygen and move 
> id_rsa.pub from .ssh to authorized_keys 8. chmod 600
/home/hadoop/.ssh/authorized_keys
>      chown hadoop:hadoop /run/
>      chown hadoop:hadoop /mnt/
> 9. Setup password 'swordfish' for root. :) 10. Change etc/ssh/sshd_config:
>       Set the following params:
>       PasswordAuthentication yes
>       AuthorizedKeysFile .ssh/authorized_keys
>       UseDNS no
>       PermitTunnel yes
> 11. Change /etc/ssh/ssh_config:
>      StrictHostKeyChecking no
>      GSSAPIAuthentication no
>
> 12. Now you can make snapshot from the running instance and use it's 
> id in the cluster creation requests in Savanna.
>
> -----
>
> Not sure if I didn't miss something in manual steps. Anyway attached 
> script "80-setup-hadoop" contain all steps repeating manual steps above.
>
> Here are the steps how to create cloud image based on Ubuntu with 
> Apache Hadoop installed using diskimage-builder project:
>
> 1. Clone the repository "https://github.com/stackforge/diskimage-builder";
> locally
>
> 2. Add ~/diskimage-builder/bin/ directory to your path (for example, 
> PATH=$PATH:/home/$USER/diskimage-builder/bin/ ).
>
> 3. Export the following variable
> ELEMENTS_PATH=/home/$USER/diskimage-builder/elements/ to your .bashrc. 
> Then source it.
>
> 4. Copy file "img-build-sudoers" from ~/disk-image-builder/sudoers.d/ 
> to your /etc/sudoers.d/
>      Then "chmod 440 /etc/sudoers.d/img-build-sudoers"
>
> 5. Move hadoop/ directory from archive hadoop.zip(attached) to 
> disk-image-builder/elements/
>      mv hadoop/  
> /path_to_disk_image_builder/diskimage-builder/elements/
>
> 6. Just call the following command to create cloud image is able to 
> run on
> OpenStack:
>      hadoop_version=1.1.2 disk-image-create base vm hadoop ubuntu -o
> hadoop_1_1_2
>
>     In this command  'hadoop_version' parameter  is version of hadoop 
> needs to be installed.
>
> After Hadoop installation complete, get hadoop_1_1_2.qcow2 image and 
> deploy it into OpenStack Glance. Diskimage-builder instructions are 
> not well tested yet but it should work with Savanna.
>
> Regards,
> Alexander Ignatov
>
> -----Original Message-----
> From: Sergey Lukjanov [mailto:slukjanov@xxxxxxxxxxxx]
> Sent: Tuesday, May 14, 2013 11:17 AM
> To: Matthew Farrellee
> Cc: savanna-team@xxxxxxxxxxxx
> Subject: Re: 0.1.1 cluster remains Starting
>
> Hi Matt,
>
> Briefly there are several requirements for image:
> * hadoop should be installed to the specific folders;
> * root should be accessible using login/password;
> * some Savanna-specific files and settings should be applied to the 
> operation system.
>
> Alex Ignatov can deeply describe the Savanna image preparation process.
> Currently we have no public instruction about how to do it, but we are 
> working on scripts that will automaticly build images using 
> diskimage-builder.
>
> Sincerely yours,
> Sergey Lukjanov
> Software Engineer
> Mirantis Inc.
> GTalk: me@xxxxxxxxxxx
> Skype: lukjanovsv