yellow team mailing list archive
-
yellow team
-
Mailing list archive
-
Message #00294
[Merge] lp:~yellow/launchpad/lxcsetup into lp:launchpad
Francesco Banconi has proposed merging lp:~yellow/launchpad/lxcsetup into lp:launchpad.
Requested reviews:
Launchpad code reviewers (launchpad-reviewers)
For more details, see:
https://code.launchpad.net/~yellow/launchpad/lxcsetup/+merge/89660
= Summary =
This branch adds a script that can be used to set up a Launchpad environment
inside a LXC, useful for testing Launchpad.
== Proposed fix ==
https://dev.launchpad.net/ParallelTests describes how LXC containers offer
a cheap way to run tests in parallel using ephemeral instances, obtaining the
required isolation to workaround the existing globals (shared work dirs,
hardcoded tcp ports, etc.). The `setuplxc.py` script creates a Launchpad
environment from scratch, ready to be used by ephemerals (e.g. in a buildbot
slave context).
== Pre-implementation notes ==
During development we realized that the same approach (with few changes)
should work to set up a Launchpad development environment.
The developer can just:
- run the script (passing his current local username as user), e.g.::
./setuplxc.py -u username -e user@xxxxxxxxxxxxx -n 'Firstname Lastname'
-c lp-devel -d /home/username/lp-deps /home/username/lp-branches/
- ssh into the container (in the example above: ssh lp-devel)
- cd ~/lp-branches/devel
- make schema
- make run
- start hacking
== Implementation details ==
utilities/setuplxc.py:
* the script is implemented in Python
* requires Python 2.7
* must be run as root
* help on required arguments: utilities/setuplxc.py -h
utilities/launchpad-database-setup
* refactored so that it can be run by root
== Tests ==
The script uses internal helper functions providing doctests. To run them::
python -m doctest -v utilities/setuplxc.py
A more extensive and complete testing approach would require to setup
a precise KVM.
== Demo and Q/A ==
To demo and Q/A this change, do the following:
* Install precise in a virtual machine (e.g. KVM)
* Copy the script inside the virtual machine, e.g. for kvm::
kvm -hda precise_HDA.img -boot c -m 2000 -redir tcp:2222::22 &
scp -P 2222 /utilities/setuplxc.py root@localhost:/tmp/
* Run the script, e.g.::
ssh -p 2222 root@localhost
cd /tmp/
./setuplxc [arguments]
== lint ==
= Launchpad lint =
Checking for conflicts and issues in changed files.
Linting changed files:
utilities/setuplxc.py
--
https://code.launchpad.net/~yellow/launchpad/lxcsetup/+merge/89660
Your team Launchpad Yellow Squad is subscribed to branch lp:~yellow/launchpad/lxcsetup.
=== modified file 'utilities/launchpad-database-setup'
--- utilities/launchpad-database-setup 2011-09-06 01:38:11 +0000
+++ utilities/launchpad-database-setup 2012-01-23 11:06:17 +0000
@@ -18,9 +18,15 @@
# https://dev.launchpad.net/DatabaseSetup which are intended for
# initial Launchpad setup on an otherwise unconfigured postgresql instance
+if [ $(id -u) = 0 ]; then
+ SUDO=''
+else
+ SUDO='sudo'
+fi
+
for pgversion in 8.4 8.3 8.2
do
- sudo grep -q "^auto" /etc/postgresql/$pgversion/main/start.conf
+ $SUDO grep -q "^auto" /etc/postgresql/$pgversion/main/start.conf
if [ $? -eq 0 ]; then
break
fi
@@ -35,31 +41,31 @@
echo "Using postgres $pgversion"
# Make sure that we have the correct version running on port 5432
-sudo grep -q "port.*5432" /etc/postgresql/$pgversion/main/postgresql.conf
+$SUDO grep -q "port.*5432" /etc/postgresql/$pgversion/main/postgresql.conf
if [ $? -ne 0 ]; then
echo "Please check /etc/postgresql/$pgversion/main/postgresql.conf and"
echo "ensure postgres is running on port 5432."
fi;
if [ -e /etc/init.d/postgresql-$pgversion ]; then
- sudo /etc/init.d/postgresql-$pgversion stop
+ $SUDO /etc/init.d/postgresql-$pgversion stop
else
# This is Maverick.
- sudo /etc/init.d/postgresql stop $pgversion
+ $SUDO /etc/init.d/postgresql stop $pgversion
fi
echo Purging postgresql data...
-sudo pg_dropcluster $pgversion main --stop-server
+$SUDO pg_dropcluster $pgversion main --stop-server
echo Re-creating postgresql database...
# Setting locale to C to make the server run in that locale.
-LC_ALL=C sudo pg_createcluster $pgversion main --encoding UNICODE
+LC_ALL=C $SUDO pg_createcluster $pgversion main --encoding UNICODE
echo Applying postgresql configuration changes...
-sudo cp -a /etc/postgresql/$pgversion/main/pg_hba.conf \
+$SUDO cp -a /etc/postgresql/$pgversion/main/pg_hba.conf \
/etc/postgresql/$pgversion/main/pg_hba.conf.old
-sudo grep -q Launchpad /etc/postgresql/$pgversion/main/pg_hba.conf || \
-sudo patch /etc/postgresql/$pgversion/main/pg_hba.conf <<'EOF'
+$SUDO grep -q Launchpad /etc/postgresql/$pgversion/main/pg_hba.conf || \
+$SUDO patch /etc/postgresql/$pgversion/main/pg_hba.conf <<'EOF'
--- pg_hba.conf 2005-11-02 17:33:08.000000000 -0800
+++ /tmp/pg_hba.conf 2005-11-03 07:32:46.932400423 -0800
@@ -58,7 +58,10 @@
@@ -75,13 +81,13 @@
EOF
-sudo chown --reference=/etc/postgresql/$pgversion/main/pg_hba.conf.old \
+$SUDO chown --reference=/etc/postgresql/$pgversion/main/pg_hba.conf.old \
/etc/postgresql/$pgversion/main/pg_hba.conf
-sudo chmod --reference=/etc/postgresql/$pgversion/main/pg_hba.conf.old \
+$SUDO chmod --reference=/etc/postgresql/$pgversion/main/pg_hba.conf.old \
/etc/postgresql/$pgversion/main/pg_hba.conf
-sudo grep -q Launchpad /etc/postgresql/$pgversion/main/postgresql.conf || \
-sudo tee -a /etc/postgresql/$pgversion/main/postgresql.conf <<'EOF'
+$SUDO grep -q Launchpad /etc/postgresql/$pgversion/main/postgresql.conf || \
+$SUDO tee -a /etc/postgresql/$pgversion/main/postgresql.conf <<'EOF'
##
## Launchpad configuration
@@ -98,25 +104,29 @@
if [ "$pgversion" = 8.2 -o "$pgversion" = 8.3 ]
then
- sudo grep -q '^[[:space:]]*max_fsm_relations' /etc/postgresql/$pgversion/main/postgresql.conf || \
- sudo tee -a /etc/postgresql/$pgversion/main/postgresql.conf <<'EOF'
+ $SUDO grep -q '^[[:space:]]*max_fsm_relations' /etc/postgresql/$pgversion/main/postgresql.conf || \
+ $SUDO tee -a /etc/postgresql/$pgversion/main/postgresql.conf <<'EOF'
max_fsm_relations=2000
EOF
fi
if [ -e /etc/init.d/postgresql-$pgversion ]; then
- sudo /etc/init.d/postgresql-$pgversion start
+ $SUDO /etc/init.d/postgresql-$pgversion start
else
# This is Maverick.
- sudo /etc/init.d/postgresql start $pgversion
+ $SUDO /etc/init.d/postgresql start $pgversion
fi
echo Waiting 10 seconds for postgresql to come up...
sleep 10
echo Creating postgresql user $USER
-sudo -u postgres /usr/lib/postgresql/$pgversion/bin/createuser -a -d $USER
+if [ $(id -u) = 0 ]; then
+ su postgres -c "/usr/lib/postgresql/$pgversion/bin/createuser -a -d $USER"
+else
+ sudo -u postgres /usr/lib/postgresql/$pgversion/bin/createuser -a -d $USER
+fi
echo
echo Looks like everything went ok.
=== added file 'utilities/setuplxc.py'
--- utilities/setuplxc.py 1970-01-01 00:00:00 +0000
+++ utilities/setuplxc.py 2012-01-23 11:06:17 +0000
@@ -0,0 +1,554 @@
+#!/usr/bin/env python
+# Copyright 2012 Canonical Ltd. This software is licensed under the
+# GNU Affero General Public License version 3 (see the file LICENSE).
+
+"""Create an LXC test environment for Launchpad testing."""
+
+__metaclass__ = type
+__all__ = [
+ 'cd',
+ 'create_lxc',
+ 'error',
+ 'file_append',
+ 'file_insert',
+ 'get_container_path',
+ 'get_user_ids',
+ 'initialize_host',
+ 'initialize_lxc',
+ 'Namespace',
+ 'ssh',
+ 'stop_lxc',
+ 'su',
+ 'user_exists',
+ ]
+
+# This script is run as root.
+# To run doctests: python -m doctest -v setuplxc.py
+
+from collections import namedtuple, OrderedDict
+from contextlib import contextmanager
+import argparse
+import os
+import pwd
+import shutil
+import subprocess
+import sys
+import time
+
+
+DEPENDENCIES_DIR = '~/dependencies'
+DHCP_FILE = '/etc/dhcp/dhclient.conf'
+HOST_PACKAGES = ['ssh', 'lxc', 'libvirt-bin', 'bzr', 'language-pack-en']
+HOSTS_FILE = '/etc/hosts'
+LP_APACHE_MODULES = 'proxy proxy_http rewrite ssl deflate headers'
+LP_APACHE_ROOTS = (
+ '/var/tmp/bazaar.launchpad.dev/static',
+ '/var/tmp/archive',
+ '/var/tmp/ppa',
+ )
+LP_CHECKOUT = 'devel'
+LP_REPOSITORY = 'lp:launchpad'
+LP_SOURCE_DEPS = (
+ 'http://bazaar.launchpad.net/~launchpad/lp-source-dependencies/trunk')
+LXC_CONFIG_TEMPLATE = '/etc/lxc/local.conf'
+LXC_GATEWAY = '10.0.3.1'
+LXC_GUEST_OS = 'lucid'
+LXC_HOSTS_CONTENT = (
+ ('127.0.0.88',
+ 'launchpad.dev answers.launchpad.dev archive.launchpad.dev '
+ 'api.launchpad.dev bazaar-internal.launchpad.dev beta.launchpad.dev '
+ 'blueprints.launchpad.dev bugs.launchpad.dev code.launchpad.dev '
+ 'feeds.launchpad.dev id.launchpad.dev keyserver.launchpad.dev '
+ 'lists.launchpad.dev openid.launchpad.dev '
+ 'ubuntu-openid.launchpad.dev ppa.launchpad.dev '
+ 'private-ppa.launchpad.dev testopenid.dev translations.launchpad.dev '
+ 'xmlrpc-private.launchpad.dev xmlrpc.launchpad.dev'),
+ ('127.0.0.99', 'bazaar.launchpad.dev'),
+ )
+LXC_NAME = 'lptests'
+LXC_OPTIONS = (
+ ('lxc.network.type', 'veth'),
+ ('lxc.network.link', 'lxcbr0'),
+ ('lxc.network.flags', 'up'),
+ )
+LXC_PATH = '/var/lib/lxc/'
+LXC_REPOS = (
+ 'deb http://archive.ubuntu.com/ubuntu '
+ 'lucid main universe multiverse',
+ 'deb http://archive.ubuntu.com/ubuntu '
+ 'lucid-updates main universe multiverse',
+ 'deb http://archive.ubuntu.com/ubuntu '
+ 'lucid-security main universe multiverse',
+ 'deb http://ppa.launchpad.net/launchpad/ppa/ubuntu lucid main',
+ 'deb http://ppa.launchpad.net/bzr/ppa/ubuntu lucid main',
+ )
+RESOLV_FILE = '/etc/resolv.conf'
+
+
+Env = namedtuple('Env', 'uid gid home')
+
+
+@contextmanager
+def ssh(location, user=None):
+ """Return a callable that can be used to run shell commands into another
+ host using ssh.
+
+ The ssh `location` and, optionally, `user` must be given.
+ If the user is None then the current user is used for the connection.
+ """
+ if user is not None:
+ location = '%s@%s' % (user, location)
+
+ def _sshcall(cmd):
+ sshcmd = (
+ 'ssh',
+ '-o', 'StrictHostKeyChecking=no',
+ '-o', 'UserKnownHostsFile=/dev/null',
+ location,
+ '--', cmd,
+ )
+ return subprocess.call(sshcmd)
+
+ yield _sshcall
+
+
+def get_user_ids(user):
+ """Return the uid and gid of given `user`, e.g.::
+
+ >>> get_user_ids('root')
+ (0, 0)
+ """
+ userdata = pwd.getpwnam(user)
+ return userdata.pw_uid, userdata.pw_gid
+
+
+@contextmanager
+def su(user):
+ """A context manager to temporary run the Python interpreter as a
+ different user.
+ """
+ uid, gid = get_user_ids(user)
+ os.setegid(gid)
+ os.seteuid(uid)
+ current_home = os.getenv('HOME')
+ home = os.path.join(os.path.sep, 'home', user)
+ os.environ['HOME'] = home
+ yield Env(uid, gid, home)
+ os.setegid(os.getgid())
+ os.seteuid(os.getuid())
+ os.environ['HOME'] = current_home
+
+
+@contextmanager
+def cd(directory):
+ """A context manager to temporary change current working dir, e.g.::
+
+ >>> import os
+ >>> os.chdir('/tmp')
+ >>> with cd('/bin'): print os.getcwd()
+ /bin
+ >>> os.getcwd()
+ '/tmp'
+ """
+ cwd = os.getcwd()
+ os.chdir(directory)
+ yield
+ os.chdir(cwd)
+
+
+def get_container_path(lxcname, path='', base_path=LXC_PATH):
+ """Return the path of LXC container called `lxcname`.
+ If a `path` is given, return that path inside the container, e.g.::
+
+ >>> get_container_path('mycontainer')
+ '/var/lib/lxc/mycontainer/rootfs/'
+ >>> get_container_path('mycontainer', '/etc/apt/')
+ '/var/lib/lxc/mycontainer/rootfs/etc/apt/'
+ >>> get_container_path('mycontainer', 'home')
+ '/var/lib/lxc/mycontainer/rootfs/home'
+ """
+ return os.path.join(base_path, lxcname, 'rootfs', path.lstrip('/'))
+
+
+def file_insert(filename, line):
+ """Insert given `line`, if not present, at the beginning of `filename`,
+ e.g.::
+
+ >>> import tempfile
+ >>> f = tempfile.NamedTemporaryFile('w', delete=False)
+ >>> f.write('line1\\n')
+ >>> f.close()
+ >>> file_insert(f.name, 'line0\\n')
+ >>> open(f.name).read()
+ 'line0\\nline1\\n'
+ >>> file_insert(f.name, 'line0\\n')
+ >>> open(f.name).read()
+ 'line0\\nline1\\n'
+ """
+ with open(filename, 'r+') as f:
+ lines = f.readlines()
+ if lines[0] != line:
+ lines.insert(0, line)
+ f.seek(0)
+ f.writelines(lines)
+
+
+def file_append(filename, content):
+ """Append given `content`, if not present, at the end of `filename`,
+ e.g.::
+
+ >>> import tempfile
+ >>> f = tempfile.NamedTemporaryFile('w', delete=False)
+ >>> f.write('line1\\n')
+ >>> f.close()
+ >>> file_append(f.name, 'new content\\n')
+ >>> open(f.name).read()
+ 'line1\\nnew content\\n'
+ >>> file_append(f.name, 'content')
+ >>> open(f.name).read()
+ 'line1\\nnew content\\n'
+ """
+ with open(filename, 'r+') as f:
+ current_content = f.read()
+ if content not in current_content:
+ f.seek(0)
+ f.write(current_content + content)
+
+
+def user_exists(username):
+ """Return True if given `username` exists, e.g.::
+
+ >>> user_exists('root')
+ True
+ >>> user_exists('_this_user_does_not_exist_')
+ False
+ """
+ try:
+ pwd.getpwnam(username)
+ except KeyError:
+ return False
+ return True
+
+
+def error(msg):
+ """Print out the error message and quit the script."""
+ print 'ERROR: %s' % msg
+ sys.exit(1)
+
+
+parser = argparse.ArgumentParser(description=__doc__)
+parser.add_argument(
+ '-u', '--user', required=True,
+ help='The name of the system user to be created or updated.')
+parser.add_argument(
+ '-e', '--email', required=True,
+ help='The email of the user, used for bzr whoami.')
+parser.add_argument(
+ '-n', '--name', required=True,
+ help='The full name of the user, used fo bzr whoami.')
+parser.add_argument(
+ '-l', '--lpuser',
+ help=('The name of the Launchpad user that will be used to check out '
+ 'dependencies. If not provided, the system user name is used.'))
+parser.add_argument(
+ '-v', '--private-key',
+ help=('The SSH private key for the Launchpad user (without passphrase). '
+ 'If the system user already exists with SSH key pair set up, '
+ 'this argument can be omitted.'))
+parser.add_argument(
+ '-b', '--public-key',
+ help=('The SSH public key for the Launchpad user. '
+ 'If the system user already exists with SSH key pair set up, '
+ 'this argument can be omitted.'))
+parser.add_argument(
+ '-a', '--actions', nargs='+',
+ choices=('initialize_host', 'create_lxc', 'initialize_lxc', 'stop_lxc'),
+ help='Only for debugging. Call one or more internal functions.')
+parser.add_argument(
+ '-c', '--lxc-name', default=LXC_NAME,
+ metavar='LXC_NAME (default=%s)' % LXC_NAME,
+ help='The LXC container name.')
+parser.add_argument(
+ '-d', '--dependencies-dir', default=DEPENDENCIES_DIR,
+ metavar='DEPENDENCIES_DIR (default=%s)' % DEPENDENCIES_DIR,
+ help=('The directory of the Launchpad dependencies to be created. '
+ 'The directory must reside under the home directory of the '
+ 'given user (see -u argument).'))
+parser.add_argument(
+ 'directory',
+ help=('The directory of the Launchpad repository to be created. '
+ 'The directory must reside under the home directory of the '
+ 'given user (see -u argument).'))
+
+
+def initialize_host(
+ user, fullname, email, lpuser, private_key, public_key,
+ dependencies_dir, directory):
+ """Initialize host machine."""
+ # Install necessary deb packages. This requires Oneiric or later.
+ subprocess.call(['apt-get', 'update'])
+ subprocess.call(['apt-get', '-y', 'install'] + HOST_PACKAGES)
+ # Create the user (if he does not exist).
+ if not user_exists(user):
+ subprocess.call(['useradd', '-m', '-s', '/bin/bash', '-U', user])
+ # Generate root ssh keys if they do not exist.
+ if not os.path.exists('/root/.ssh/id_rsa.pub'):
+ subprocess.call([
+ 'ssh-keygen', '-q', '-t', 'rsa', '-N', '',
+ '-f', '/root/.ssh/id_rsa'])
+ with su(user) as env:
+ # Set up the user's ssh directory. The ssh key must be associated
+ # with the lpuser's Launchpad account.
+ ssh_dir = os.path.join(env.home, '.ssh')
+ if not os.path.exists(ssh_dir):
+ os.makedirs(ssh_dir)
+ priv_file = os.path.join(ssh_dir, 'id_rsa')
+ pub_file = os.path.join(ssh_dir, 'id_rsa.pub')
+ auth_file = os.path.join(ssh_dir, 'authorized_keys')
+ known_hosts = os.path.join(ssh_dir, 'known_hosts')
+ known_host_content = subprocess.check_output([
+ 'ssh-keyscan', '-t', 'rsa', 'bazaar.launchpad.net'])
+ for filename, contents, mode in [
+ (priv_file, private_key, 'w'),
+ (pub_file, public_key, 'w'),
+ (auth_file, public_key, 'a'),
+ (known_hosts, known_host_content, 'a'),
+ ]:
+ with open(filename, mode) as f:
+ f.write('%s\n' % contents)
+ os.chmod(filename, 0644)
+ os.chmod(priv_file, 0600)
+ # Set up bzr and Launchpad authentication.
+ subprocess.call(['bzr', 'whoami', '"%s <%s>"' % (fullname, email)])
+ subprocess.call(['bzr', 'lp-login', lpuser])
+ # Set up the repository.
+ if not os.path.exists(directory):
+ os.makedirs(directory)
+ subprocess.call(['bzr', 'init-repo', directory])
+ checkout_dir = os.path.join(directory, LP_CHECKOUT)
+ # bzr branch does not work well with seteuid.
+ os.system(
+ "su - %s -c 'bzr branch %s %s'" % (user, LP_REPOSITORY, checkout_dir))
+ with su(user) as env:
+ # Set up source dependencies.
+ os.makedirs('%s/eggs' % dependencies_dir)
+ os.makedirs('%s/yui' % dependencies_dir)
+ os.makedirs('%s/sourcecode' % dependencies_dir)
+ with cd(dependencies_dir):
+ subprocess.call([
+ 'bzr', 'co', '--lightweight',
+ LP_SOURCE_DEPS, 'download-cache'])
+ # Update resolv file in order to get the ability to ssh into the LXC
+ # container using its name.
+ file_insert(RESOLV_FILE, 'nameserver %s\n' % LXC_GATEWAY)
+ file_append(DHCP_FILE, 'prepend domain-name-servers %s;\n' % LXC_GATEWAY)
+
+
+def create_lxc(user, lxcname):
+ """Create the LXC container that will be used for ephemeral instances."""
+ # Container configuration template.
+ content = ''.join('%s=%s\n' % i for i in LXC_OPTIONS)
+ with open(LXC_CONFIG_TEMPLATE, 'w') as f:
+ f.write(content)
+ # Creating container.
+ exit_code = subprocess.call([
+ 'lxc-create',
+ '-t', 'ubuntu',
+ '-n', lxcname,
+ '-f', LXC_CONFIG_TEMPLATE,
+ '--',
+ '-r %s -a i386 -b %s' % (LXC_GUEST_OS, user)
+ ])
+ if exit_code:
+ error('Unable to create the LXC container.')
+ subprocess.call(['lxc-start', '-n', lxcname, '-d'])
+ # Set up root ssh key.
+ user_authorized_keys = os.path.join(
+ os.path.sep, 'home', user, '.ssh/authorized_keys')
+ with open(user_authorized_keys, 'a') as f:
+ f.write(open('/root/.ssh/id_rsa.pub').read())
+ dst = get_container_path(lxcname, '/root/.ssh/')
+ if not os.path.exists(dst):
+ os.makedirs(dst)
+ shutil.copy(user_authorized_keys, dst)
+ # SSH into the container.
+ with ssh(lxcname, user) as sshcall:
+ timeout = 60
+ while timeout:
+ if not sshcall('true'):
+ break
+ timeout -= 1
+ time.sleep(1)
+ else:
+ error('Unable to SSH into LXC.')
+
+
+def initialize_lxc(user, dependencies_dir, directory, lxcname):
+ """Set up the Launchpad development environment inside the LXC container.
+ """
+ with ssh(lxcname) as sshcall:
+ # APT repository update.
+ sources = get_container_path(lxcname, '/etc/apt/sources.list')
+ with open(sources, 'w') as f:
+ f.write('\n'.join(LXC_REPOS))
+ # XXX frankban 2012-01-13 - Bug 892892: upgrading mountall in LXC
+ # containers currently does not work.
+ sshcall("echo 'mountall hold' | dpkg --set-selections")
+ # Upgrading packages.
+ sshcall(
+ 'apt-get update && '
+ 'DEBIAN_FRONTEND=noninteractive '
+ 'apt-get -y --allow-unauthenticated install language-pack-en')
+ sshcall(
+ 'DEBIAN_FRONTEND=noninteractive '
+ 'apt-get -y --allow-unauthenticated install '
+ 'bzr launchpad-developer-dependencies apache2 apache2-mpm-worker')
+ # User configuration.
+ sshcall('adduser %s sudo' % user)
+ pygetgid = 'import pwd; print pwd.getpwnam("%s").pw_gid' % user
+ gid = "`python -c '%s'`" % pygetgid
+ sshcall('addgroup --gid %s %s' % (gid, user))
+ with ssh(lxcname, user) as sshcall:
+ # Set up Launchpad dependencies.
+ checkout_dir = os.path.join(directory, LP_CHECKOUT)
+ sshcall(
+ 'cd %s && utilities/update-sourcecode %s/sourcecode' % (
+ checkout_dir, dependencies_dir))
+ sshcall(
+ 'cd %s && utilities/link-external-sourcecode %s' % (
+ checkout_dir, dependencies_dir))
+ # Create Apache document roots, to avoid warnings.
+ sshcall(' && '.join('mkdir -p %s' % i for i in LP_APACHE_ROOTS))
+ with ssh(lxcname) as sshcall:
+ # Set up Apache modules.
+ for module in LP_APACHE_MODULES.split():
+ sshcall('a2enmod %s' % module)
+ # Launchpad database setup.
+ sshcall(
+ 'cd %s && utilities/launchpad-database-setup %s' % (
+ checkout_dir, user))
+ with ssh(lxcname, user) as sshcall:
+ sshcall('cd %s && make' % checkout_dir)
+ # Set up container hosts file.
+ lines = ['%s\t%s' % (ip, names) for ip, names in LXC_HOSTS_CONTENT]
+ lxc_hosts_file = get_container_path(lxcname, HOSTS_FILE)
+ file_append(lxc_hosts_file, '\n'.join(lines))
+ # Make and install launchpad.
+ with ssh(lxcname) as sshcall:
+ sshcall('cd %s && make install' % checkout_dir)
+
+
+def stop_lxc(lxcname):
+ """Stop the lxc instance named `lxcname`."""
+ with ssh(lxcname) as sshcall:
+ sshcall('poweroff')
+ time.sleep(5)
+ subprocess.call(['lxc-stop', '-n', lxcname])
+
+
+def main(
+ user, fullname, email, lpuser, private_key, public_key, actions,
+ lxc_name, dependencies_dir, directory):
+ function_args_map = OrderedDict((
+ ('initialize_host', (user, fullname, email, lpuser, private_key,
+ public_key, dependencies_dir, directory)),
+ ('create_lxc', (user, lxc_name)),
+ ('initialize_lxc', (user, dependencies_dir, directory, lxc_name)),
+ ('stop_lxc', (lxc_name,)),
+ ))
+ if actions is None:
+ actions = function_args_map.keys()
+ scope = globals()
+ for action in actions:
+ scope[action](*function_args_map[action])
+
+
+class Namespace(object):
+ """A namespace for argparse.
+
+ Add methods for further arguments validation.
+ This class implements ssh key validation, e.g.::
+
+ >>> args = parser.parse_args('-u example_user -e example@xxxxxxxxxxx '
+ ... '-n exampleuser -v PRIVATE -b PUBLIC '
+ ... '/home/example_user/launchpad/'.split(),
+ ... namespace=Namespace())
+ >>> args.are_valid()
+ True
+ >>> args = parser.parse_args('-u example_user -e example@xxxxxxxxxxx '
+ ... '-n exampleuser -b PUBLIC '
+ ... '/home/example_user/launchpad/'.split(),
+ ... namespace=Namespace())
+ >>> args.are_valid()
+ False
+ >>> args.error_message # doctest: +ELLIPSIS
+ 'argument private_key ...'
+
+ and directory validation::
+
+ >>> args = parser.parse_args('-u example_user -e example@xxxxxxxxxxx '
+ ... '-n exampleuser -v PRIVATE -b PUBLIC '
+ ... '/home/'.split(),
+ ... namespace=Namespace())
+ >>> args.are_valid()
+ False
+ >>> args.error_message # doctest: +ELLIPSIS
+ 'argument directory ...'
+ """
+ _errors = None
+
+ def __repr__(self):
+ return repr(vars(self))
+
+ @property
+ def error_message(self):
+ return '\n'.join(self._errors)
+
+ def _get_ssh_key(self, attr, filename):
+ value = getattr(self, attr)
+ if value:
+ return value.decode('string-escape')
+ try:
+ return open(filename).read()
+ except IOError:
+ self._errors.append(
+ 'argument %s is required if the system user '
+ 'does not exists with SSH key pair set up.' % attr)
+
+ def _get_directory(self, attr, home_dir):
+ directory = getattr(self, attr).replace('~', home_dir)
+ if not directory.startswith(home_dir + os.path.sep):
+ self._errors.append('argument %s does not reside under the home '
+ 'directory of the system user.' % attr)
+ return directory
+
+ def are_valid(self):
+ self._errors = []
+ home_dir = os.path.join(os.path.sep, 'home', self.user)
+ if self.lpuser is None:
+ self.lpuser = self.user
+ self.private_key = self._get_ssh_key(
+ 'private_key', os.path.join(home_dir, '.ssh', 'id_rsa'))
+ self.public_key = self._get_ssh_key(
+ 'public_key', os.path.join(home_dir, '.ssh', 'id_rsa.pub'))
+ self.directory = self._get_directory('directory', home_dir)
+ self.dependencies_dir = self._get_directory(
+ 'dependencies_dir', home_dir)
+ return not self._errors
+
+
+if __name__ == '__main__':
+ args = parser.parse_args(namespace=Namespace())
+ if args.are_valid():
+ main(args.user,
+ args.name,
+ args.email,
+ args.lpuser,
+ args.private_key,
+ args.public_key,
+ args.actions,
+ args.lxc_name,
+ args.dependencies_dir,
+ args.directory)
+ else:
+ parser.error(args.error_message)
Follow ups
