duplicity-team team mailing list archive
-
duplicity-team team
-
Mailing list archive
-
Message #04335
[Merge] lp:~dernils/duplicity/Dockerfile into lp:duplicity
nils has proposed merging lp:~dernils/duplicity/Dockerfile into lp:duplicity.
Requested reviews:
duplicity-team (duplicity-team)
For more details, see:
https://code.launchpad.net/~dernils/duplicity/Dockerfile/+merge/324027
I added another backend to the docker test infrastructure, updated the setup for testing and updated the documentation.
--
Your team duplicity-team is requested to review the proposed merge of lp:~dernils/duplicity/Dockerfile into lp:duplicity.
=== modified file 'README-TESTING'
--- README-TESTING 2017-04-23 15:04:49 +0000
+++ README-TESTING 2017-05-14 07:45:09 +0000
@@ -39,7 +39,6 @@
code against different supported environments, including the versions of
dependencies used by the Launchpad build system.
-
A tox run can be started simply by typing
‘tox‘
@@ -95,6 +94,45 @@
The report will be generated and stored in the folder htmlcov.
-
-
+## The wider picture - supporting containers for testing
+Testing duplicity invokes backends. The backends are the places where the backup data is actually stored (e.g. an ftp server). In order to have the highest degree of control over the testing process, backends that can be set up locally are also operated in separated docker containers. The whole test infrastructure is shown in the following picture.
+
+┌─────────────────────┐ ┌──────────────────────────────────────────┐
+│docker image │ │docker image │
+│dernils/duplicitytest│ │dernils/duplicity_testinfrastructure_ssh │
+├─────────────────────┴────┐ ├──────────────────────────────────────────┴──┐
+│ │ │ │
+│ ┌──────────────────┐ │ │ ┌──────────────────┐ │
+│ │ tox │ │ ┌──┼──▶│ sshd │ │
+│ └──────────────────┘ │ │ │ └──────────────────┘ │
+│ │ │ │ │ │
+│ ▼ │ │ └─────────────────────────────────────────────┘
+│ ┌──────────────────┐ │ │ ┌──────────────────────────────────────────┐
+│ │ unittests │ │ │ │docker image │
+│ └──────────────────┘ │ │ │dernils/duplicity_testinfrastructure_ftp │
+│ │ │ │ ├──────────────────────────────────────────┴──┐
+│ ▼ │ │ │ │
+│ ┌──────────────────┐ │ │ │ ┌──────────────────┐ │
+│ │ duplicity │◀───┼─┴──┼──▶│ pure-ftpd │ │
+│ └──────────────────┘ │ │ └──────────────────┘ │
+│ │ │ │ │
+└────────────┼─────────────┘ └─────────────────────────────────────────────┘
+ │
+ │
+ │
+ └────────────┐
+ │
+ Internet .─────────┼─────────.
+ _.────' │ `─────.
+ _.─' │ `──.
+ ,' ▼ `.
+ ; ┌──────────────────┐ :
+ : │ Dropbox │ ;
+ ╲ └──────────────────┘ ╱
+ `. ,'
+ `──. _.─'
+ `─────. _.────'
+ `─────────────────'
+
+The docker images that contain the test infrastructure are defined in the folder /testing/infrastructure. There is a build script to compile the Dockerfile into actual images (build-duplicitiy-test.sh). However, as all images are also published on the docker hub, it is not necessary to build the images before starting testing. Testing can directly be started by using the script setup.sh. If the required docker images are not yet existing, locally, they will be downloaded by Docker.
=== modified file 'testing/infrastructure/build-duplicity_test.sh'
--- testing/infrastructure/build-duplicity_test.sh 2017-05-06 20:09:20 +0000
+++ testing/infrastructure/build-duplicity_test.sh 2017-05-14 07:45:09 +0000
@@ -2,5 +2,20 @@
cd duplicity_test
cp -p ../../../requirements.txt .
+cp -p ../id_rsa .
+cp -p ../id_rsa.pub .
+
docker build --tag dernils/duplicitytest:latest .
rm requirements.txt
+rm id_rsa
+rm id_rsa.pub
+
+cd ..
+cd ftp_server
+docker build --tag dernils/duplicity_testinfrastructure_ftp:latest .
+
+cd ..
+cd ssh_server
+cp -p ../id_rsa.pub .
+docker build --tag dernils/duplicity_testinfrastructure_ssh:latest .
+rm id_rsa.pub
=== modified file 'testing/infrastructure/duplicity_test/Dockerfile'
--- testing/infrastructure/duplicity_test/Dockerfile 2017-05-06 19:41:30 +0000
+++ testing/infrastructure/duplicity_test/Dockerfile 2017-05-14 07:45:09 +0000
@@ -66,6 +66,14 @@
# Setting a working directory to home
WORKDIR /home/test
+# Copy a SSH key to the users folder that is used for some test cases
+USER root
+COPY ./id_rsa /home/test/.ssh/
+COPY ./id_rsa.pub /home/test/.ssh/
+RUN chown -R test:test /home/test/.ssh
+RUN chmod 400 /home/test/.ssh/id_rsa
+USER test
+
# Branch the duplicity repo for testing
RUN bzr branch lp:duplicity duplicity
=== added file 'testing/infrastructure/id_rsa'
--- testing/infrastructure/id_rsa 1970-01-01 00:00:00 +0000
+++ testing/infrastructure/id_rsa 2017-05-14 07:45:09 +0000
@@ -0,0 +1,27 @@
+-----BEGIN RSA PRIVATE KEY-----
+MIIEowIBAAKCAQEA4ZBvr/4i0X0HMPLcTz5F11qCn84GnymquLlR2s/5U8HDpxSK
+2ZLiZ482yNRkxQkV6JMRIXk9EFQ5XeyAZIPildheqVy17DPhUjyiMf/0OB9OFVUj
+dPAytTU1uicYsmJyBbel/g7wBhOEwFqUNzNZ5qDdWgVIGZVXfr6PS+1lr5oAG+Z8
+cdOpuxJg56sCxYvh+wyBIJ70J/6S79xFRNbRUjANsSCReq8Zq5kpXbH4itZN44Zh
+TCkqgdeYf4p7A1lmNtEfzPGi0voXORxCsTcjJERS4yg4iOA55Wi9bRuOo+ltd+lw
+ByEEq/0f8mhvMrqGa6gfsIwXQQfC/m/nxOygbwIDAQABAoIBAFTwlg5ZDfWDNn0J
+R0sT8Srqa3wQElBHWYGcT6WtYX9X3PyC8NKDzHwg4wl3accE59of33iSdhs7DCtB
+TwwK2f2BcAwo3FzpIHVeXdmkP+wOZdIEsIhtlzrDocrKxDMQfU2H3ZsxNtOi7/+b
+XGr5DGabtx8aFA+GPF9/FOkLqZoZF8OdNHTMdBnVCfS0SPcHLDnljC6aplJE7kyh
+RGmASW60X4W8rzdLbrX1W6PDCFMavLLToVqiecm1lU2/0DIGzij96JYv/0QXe34T
+B8PhBRt+jQUr7Wx5mSS7IX0VFbqM+LkyicblLn9BVHSsKVyxva3R4aqjma2+dsfp
+aYBEqqkCgYEA+pXsTZHg6g0c66LKdbBJKnhccUbfzI0bcjazgPcucxZXuim/Gerz
+aEGXuKg5KWjIv3HDpFuH8LttaQWOg2OqLJ2hBmN2DB2r7OiT927Lqnx5RNUcCm7y
+FSg51IWVaQWqXS1NIzs0INrJjIgY3NzXIt4mZ5bkcxV0KO8GBbouPb0CgYEA5nAc
+ZKY+PEoq0eopPU9ZW999GPzfQAjkbc+fJ9YmuYHwToceh4FZZYe2t0ltmuxe/O8L
+cR6bBKRDK/eHn0Q+qd6axWtYCPp3TJ+SGSsSbY+ohw+f9fUYReNn1eirY39+MSmP
+EGY8xREHjhVbvTBLcujeM5PJOfSeL/g85olwq5sCgYEAxa+86j0piLurkyGsQ0g9
+kGm/1AWrzP2NJ3+MATP/yv9XWiBRCKtYN2l3vk/IOKjXpmCjjWhASGQ0nmOp2HGp
+8OSyAg45NLm7xnx9LviTZ/4m1WyiYIDCLnq7lLOF+YJksnlon2C3cSvpURqVMrhu
+aK2E8rgRpdDJAPoAcEg+7qkCgYBo0s7XqhfdbQ+VdaNU1Gedt0g8uasEggcHPM71
+XCE5ebZKwoRMt7ZmzRl8vtYxBx0eic72oil6Lw6vTWZQOMcYlr5o16+wRd9G57+Z
+21ICrLrI1X66b8ee4xc04yYzmKdazcDagOdb7ac2JtkYcoOfvfZ2F+ANkQz0XUO2
+ywaJyQKBgCe09+E3nHQkCQfhOJO4t46bEHeN2gf4OEqoajAPizK58c8rluodg7xx
+ttBFCuMcRLLXNgo/t5SS7DAmWC5zjVsr9T1ooWzBhZiegsatsPY+f30xaAKl1/ET
+q6Rx6FuLjSv8t98sIu3l2sWuTvSzujbtxRGjy0Z4CF/tmbrAbQRF
+-----END RSA PRIVATE KEY-----
=== added file 'testing/infrastructure/id_rsa.pub'
--- testing/infrastructure/id_rsa.pub 1970-01-01 00:00:00 +0000
+++ testing/infrastructure/id_rsa.pub 2017-05-14 07:45:09 +0000
@@ -0,0 +1,1 @@
+ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDhkG+v/iLRfQcw8txPPkXXWoKfzgafKaq4uVHaz/lTwcOnFIrZkuJnjzbI1GTFCRXokxEheT0QVDld7IBkg+KV2F6pXLXsM+FSPKIx//Q4H04VVSN08DK1NTW6JxiyYnIFt6X+DvAGE4TAWpQ3M1nmoN1aBUgZlVd+vo9L7WWvmgAb5nxx06m7EmDnqwLFi+H7DIEgnvQn/pLv3EVE1tFSMA2xIJF6rxmrmSldsfiK1k3jhmFMKSqB15h/insDWWY20R/M8aLS+hc5HEKxNyMkRFLjKDiI4DnlaL1tG46j6W136XAHIQSr/R/yaG8yuoZrqB+wjBdBB8L+b+fE7KBv nils@xxxxxxxxxxxxxxxxxxx
=== modified file 'testing/infrastructure/setup.sh'
--- testing/infrastructure/setup.sh 2017-05-06 19:17:29 +0000
+++ testing/infrastructure/setup.sh 2017-05-14 07:45:09 +0000
@@ -34,10 +34,12 @@
# Remove all running instances of the test system and also remove the containers. This ensure
# that the test infrastructure is frehshly started.
-docker rm $(docker stop $(docker ps -a -q --filter name=ftpd_server --format="{{.ID}}"))
-docker rm $(docker stop $(docker ps -a -q --filter name=duplicity_test --format="{{.ID}}"))
+docker rm -f $(docker stop $(docker ps -a -q --filter name=d70c0e18-37d5-11e7-a919-92ebcb67fe33-ftpd_server --format="{{.ID}}"))
+docker rm -f $(docker stop $(docker ps -a -q --filter name=ee681ee4-37d5-11e7-a919-92ebcb67fe33-duplicity_ssh_server --format="{{.ID}}"))
+docker rm -f $(docker stop $(docker ps -a -q --filter name=f3c09128-37d5-11e7-a919-92ebcb67fe33-duplicity_test --format="{{.ID}}"))
# Start the containers. Docker run will automatically download the image if necessary
-docker run -d --net testnetwork --ip 10.10.10.3 --name ftpd_server -p 21:21 -p 30000-30009:30000-30009 -e "PUBLICHOST=localhost" dernils/duplicity_testinfrastructure_ftp
-docker run --name duplicity_test --net testnetwork --ip 10.10.10.2 -it dernils/duplicitytest:latest
+docker run -d --net testnetwork --ip 10.10.10.3 --name d70c0e18-37d5-11e7-a919-92ebcb67fe33-ftpd_server -p 21:21 -p 30000-30009:30000-30009 -e "PUBLICHOST=localhost" dernils/duplicity_testinfrastructure_ftp
+docker run -d --net testnetwork --ip 10.10.10.4 --name ee681ee4-37d5-11e7-a919-92ebcb67fe33-duplicity_ssh_server -p 22:22 -e "PUBLICHOST=localhost" dernils/duplicity_testinfrastructure_ssh:latest
+docker run --name f3c09128-37d5-11e7-a919-92ebcb67fe33-duplicity_test --net testnetwork --ip 10.10.10.2 -it dernils/duplicitytest:latest
=== added directory 'testing/infrastructure/ssh_server'
=== added file 'testing/infrastructure/ssh_server/Dockerfile'
--- testing/infrastructure/ssh_server/Dockerfile 1970-01-01 00:00:00 +0000
+++ testing/infrastructure/ssh_server/Dockerfile 2017-05-14 07:45:09 +0000
@@ -0,0 +1,63 @@
+# -*- Mode:Python; indent-tabs-mode:nil; tab-width:4 -*-
+#
+# Copyright 2017 Nils Tekampe <nils@xxxxxxxxxxx>
+# Thanks to Aleksandar Diklic "https://github.com/rastasheep";
+#
+# This file is part of duplicity.
+# It is the Dockerfile of a simple ftp server that is used for backend testing
+#
+# Duplicity is free software; you can redistribute it and/or modify it
+# under the terms of the GNU General Public License as published by the
+# Free Software Foundation; either version 2 of the License, or (at your
+# option) any later version.
+#
+# Duplicity is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with duplicity; if not, write to the Free Software Foundation,
+# Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+#
+FROM ubuntu:16.04
+
+RUN apt-get update
+
+RUN apt-get install -y openssh-server
+RUN mkdir /var/run/sshd
+
+RUN echo 'root:root' |chpasswd
+
+RUN sed -ri 's/^PermitRootLogin\s+.*/PermitRootLogin yes/' /etc/ssh/sshd_config
+RUN sed -ri 's/UsePAM yes/#UsePAM yes/g' /etc/ssh/sshd_config
+RUN echo "Match User userWithOnlyKeyAccess" >> /etc/ssh/sshd_config
+RUN echo " PasswordAuthentication no" >> /etc/ssh/sshd_config
+
+RUN useradd -ms /bin/bash userWithKeyAccess
+RUN useradd -ms /bin/bash userWithPasswordAccess
+RUN useradd -ms /bin/bash userWithOnlyKeyAccess
+
+USER userWithKeyAccess
+WORKDIR /home/userWithKeyAccess
+COPY ./id_rsa.pub /home/userWithKeyAccess/.ssh/authorized_keys
+
+USER userWithOnlyKeyAccess
+WORKDIR /home/userWithOnlyKeyAccess
+COPY ./id_rsa.pub /home/userWithOnlyKeyAccess/.ssh/authorized_keys
+
+USER userWithPasswordAccess
+WORKDIR /home/userWithPasswordAccess
+
+USER root
+
+RUN echo 'userWithKeyAccess:userWithKeyAccess' |chpasswd
+RUN echo 'userWithPasswordAccess:userWithPasswordAccess' |chpasswd
+
+RUN chown -R userWithKeyAccess:userWithKeyAccess /home/userWithKeyAccess/.ssh
+RUN chown -R userWithOnlyKeyAccess:userWithOnlyKeyAccess /home/userWithOnlyKeyAccess/.ssh/
+
+
+EXPOSE 22
+
+CMD ["/usr/sbin/sshd", "-D"]
\ No newline at end of file
Follow ups
