widelands-dev team mailing list archive

[SirVer <SirVer@xxxxxx>]

SirVer has proposed merging lp:~widelands-dev/widelands-website/wlwebsite_docker into lp:widelands-website.

Requested reviews:
  Widelands Developers (widelands-dev)

For more details, see:
https://code.launchpad.net/~widelands-dev/widelands-website/wlwebsite_docker/+merge/317036

Experimental branch, not sure if this should be merged.

I set up a docker workflow to run the website against a real mysql database on any development system (windows, mac, linux) easily. The goals here are

1) get rid of installing dependencies (like the python modules) on dev machines.
2) make sure all devs have always the exactly same set of python modules and python version,
3) dev workflows can use external tools (like wl_map_info) also no their dev machine, i.e. less differences between production and dev,
4) quicker setup for new devs.

How does this work? The flow is based around a tool named Docker (https://www.docker.com/) which is the new hotness since a few years. Docker runs containers and links them together. You can think of a container as a zero-cost virtual Linux machine (when running on Linux). 

A container is described by a Dockerfile - which is a Build rule for assembling these virtual machines. And the core of this patch is a Dockerfile for the website. The idea is now to run a Dockercontainer that runs a mysql database[1], then build the website docker container and run it against this mysql. This means you run the full setup of the website on your dev machine.

Read the patch starting from the Makefile, then into the Dockerfile.

The workflow to get this up and running is this:

1) Install docker and do the docker tutorial[2] to get a basic understanding of docker.
2) bring up in one terminal the mysql database (make run_mysql).
3) build an image from the dockerfile: make docker
4) initialize the database and create a superuser: make init
5) run the website and try to upload a map: make serve

[1] https://hub.docker.com/_/mysql/
[2] https://docs.docker.com/engine/getstarted/

-- 
Your team Widelands Developers is requested to review the proposed merge of lp:~widelands-dev/widelands-website/wlwebsite_docker into lp:widelands-website.

=== added file 'Dockerfile'
--- Dockerfile	1970-01-01 00:00:00 +0000
+++ Dockerfile	2017-02-11 21:23:11 +0000
@@ -0,0 +1,37 @@
+FROM ubuntu:zesty
+ENV PYTHONUNBUFFERED 1
+
+ARG UID
+ARG GID
+RUN useradd user -u $UID -g $GID -m -s /bin/bash 
+
+RUN apt-get update && apt-get install -y \
+   software-properties-common
+
+RUN add-apt-repository ppa:widelands-dev/widelands-daily
+
+RUN apt-get update && apt-get install -y \
+   git \
+   libjpeg-dev \
+   libmysqlclient-dev \
+   libpng-dev \
+   python \
+   python-pip \
+   widelands \
+   xvfb \
+   && rm -rf /var/lib/apt/lists/*
+
+RUN mkdir /src/ && mkdir /code && chown -R user /src /code
+
+ADD pip_requirements.txt /code
+RUN pip install -r /code/pip_requirements.txt
+
+ENV PATH /usr/games:$PATH
+ADD run_docker.sh /code
+
+EXPOSE 8000
+# TODO(sirver): this should be enabled, but leads to wrong writes
+# USER user
+WORKDIR /code
+
+CMD ["sh", "run_docker.sh"]

=== added file 'Makefile'
--- Makefile	1970-01-01 00:00:00 +0000
+++ Makefile	2017-02-11 21:23:11 +0000
@@ -0,0 +1,34 @@
+DOCKER_RUN:=docker run -it --rm \
+	--link db:db \
+	-p 8000:8000 \
+	-v $(shell pwd):/code \
+	wlwebsite
+
+# Bring up a mysql server. The database will be completely empty.
+run_mysql:
+	docker run -it \
+		--name db \
+		-e MYSQL_DATABASE=wl_django \
+		-e MYSQL_ROOT_PASSWORD=bluba \
+		-e MYSQL_USER=widelands \
+		-e MYSQL_PASSWORD=widelands \
+		-p 3306:3306 \
+		mysql
+
+# Creates a super user and runs ./manage.py migrate
+init:
+	$(DOCKER_RUN) ./manage.py migrate
+	$(DOCKER_RUN) ./manage.py createsuperuser
+
+
+docker:
+	docker build . -t wlwebsite \
+		--build-arg UID=$(shell id -u) \
+		--build-arg GID=$(shell id -g)
+
+# This will execute the CMD in the Dockerfile which runs the django debug server.
+serve:
+	$(DOCKER_RUN)
+
+cmd:
+	$(DOCKER_RUN) ${CMD}

=== modified file 'pip_requirements.txt'
--- pip_requirements.txt	2017-01-21 12:39:23 +0000
+++ pip_requirements.txt	2017-02-11 21:23:11 +0000
@@ -24,4 +24,3 @@
 untokenize==0.1.1
 -e git://github.com/kerin/django-sphinx.git#egg=django-sphinx
 bleach==1.4.3
-

=== added file 'run_docker.sh'
--- run_docker.sh	1970-01-01 00:00:00 +0000
+++ run_docker.sh	2017-02-11 21:23:11 +0000
@@ -0,0 +1,6 @@
+#!/bin/sh
+
+Xvfb :99 -screen 0 1024x768x16 &> xvfb.log &
+export DISPLAY=:99.0
+
+python manage.py runserver 0.0.0.0:8000

=== modified file 'settings.py'
--- settings.py	2017-01-21 12:39:23 +0000
+++ settings.py	2017-02-11 21:23:11 +0000
@@ -12,19 +12,6 @@
 
 MANAGERS = ADMINS
 
-DATABASES = {
-    'default': {
-        'ENGINE': 'django.db.backends.sqlite3',
-        'NAME': 'dev.db',
-        'USER': '',      # Not used with sqlite3.
-        'PASSWORD': '',  # Not used with sqlite3.
-        # Set to empty string for localhost. Not used with sqlite3.
-        'HOST': '',
-        # Set to empty string for default. Not used with sqlite3.
-        'PORT': '',
-    }
-}
-
 # Local time zone for this installation. Choices can be found here:
 # http://en.wikipedia.org/wiki/List_of_tz_zones_by_name
 # although not all choices may be available on all operating systems.
@@ -48,7 +35,7 @@
 
 # Absolute path to the directory that holds media.
 # Example: "/home/media/media.lawrence.com/"
-MEDIA_ROOT = ''
+MEDIA_ROOT = '/code/media_root'
 
 # URL that handles the media served from MEDIA_ROOT. Make sure to use a
 # trailing slash if there is a path component (optional in other cases).
@@ -245,7 +232,7 @@
 ###############################
 # Sphinx (Search prog) Config #
 ###############################
-USE_SPHINX = True
+USE_SPHINX = False
 SPHINX_API_VERSION = 0x116
 
 ############