ubuntu-touch-coreapps-reviewers team mailing list archive

[Nicholas Skaggs <nskaggs@xxxxxxxxxx>]

Nicholas Skaggs has proposed merging lp:~nskaggs/help-app/functional-test-template into lp:help-app.

Commit message:
Add basic skeleton structure for adding functional tests

Requested reviews:
  Ubuntu Help app developers (help-app-dev)
Related bugs:
  Bug #1456924 in Ubuntu Help App: "Add initial functional tests to the app"
  https://bugs.launchpad.net/help-app/+bug/1456924

For more details, see:
https://code.launchpad.net/~nskaggs/help-app/functional-test-template/+merge/260206

Add basic skeleton structure for adding functional tests
-- 
Your team Ubuntu Help app developers is requested to review the proposed merge of lp:~nskaggs/help-app/functional-test-template into lp:help-app.

=== modified file 'HACKING'
--- HACKING	2015-05-13 16:32:48 +0000
+++ HACKING	2015-05-26 19:24:25 +0000
@@ -22,7 +22,7 @@
 -----------------------
 
 Our work-flow is a bit special, so bear with us for a while. If you just want
-to start adding content, skip this section. The sections below should 
+to start adding content, skip this section. The sections below should
 probably be enough for that.
 
 We had a couple of requirements when we started this project:
@@ -36,8 +36,8 @@
 pelican and po4a on the content side, the Ubuntu HTML5 UI toolkit and
 the Ubuntu Web Guidelines on the display side.
 
- * po4a is used to extract translatable messages from the Markdown 
-   content, so that we can get .pot and .po files which are understood by 
+ * po4a is used to extract translatable messages from the Markdown
+   content, so that we can get .pot and .po files which are understood by
    Launchpad. We also use po4a to generate translated Markdown files.
  * pelican takes care of generating HTML from the translated Markdown
    files. It can build into different output directories, using different
@@ -73,7 +73,7 @@
 Prerequisites
 -------------
 
-Note: if you are still on anything before vivid (15.04) (ie: trusty, utopic, 
+Note: if you are still on anything before vivid (15.04) (ie: trusty, utopic,
 etc.), please add the following PPA:
 
   sudo add-apt-repository ppa:ubuntu-touch-coreapps-drivers/collection
@@ -85,8 +85,8 @@
 Before you get started, run:
 
   sudo apt install python-pelican po4a make bzrtools \
-	ubuntu-html5-ui-toolkit python3-polib python3-magic \
-	python3-markdown pep8 pyflakes python-magic python-polib
+    ubuntu-html5-ui-toolkit python3-polib python3-magic \
+    python3-markdown pep8 pyflakes python-magic python-polib
 
 This will install the necessary files, so you can build the app or web build
 locally and check if your changes actually make sense and look and work well.
@@ -98,15 +98,15 @@
 This app is structured in a very simple way. All the content is and all your
 edits happen in
 
-	./content/pages/
+    ./content/pages/
 
-The markup for the text is Markdown, which is very easy to learn. Just have 
-a look at http://daringfireball.net/projects/markdown/ and some of the 
+The markup for the text is Markdown, which is very easy to learn. Just have
+a look at http://daringfireball.net/projects/markdown/ and some of the
 existing files to get started.
 
 Once you're happy with your edits, change to the top-level directory and run
 
-	make app
+    make app
 
 This will also generate translated pages.
 
@@ -115,7 +115,7 @@
 To launch the app, you can use ubuntu-html5-app-launcher in the www dir, or
 just call
 
-	make launch
+    make launch
 
 (This will also run the 'make app' command for you.)
 
@@ -126,7 +126,7 @@
 To create a click package (for example, if you want to test it on your
 device), just run
 
-	make click
+    make click
 
 in the top-level directory.
 
@@ -137,48 +137,66 @@
 We also want to publish this information on the web. To start a 'web' build
 of the content, just run
 
-	make web
+    make web
 
 Find the resulting files in the ./build/web directory.
 
-To update the publishing branch (you need to be member of ~help-app-dev), 
+To update the publishing branch (you need to be member of ~help-app-dev),
 run
 
-	make web-publish
+    make web-publish
 
-We plan to make use of 
+We plan to make use of
 http://www.w3.org/International/questions/qa-apache-lang-neg for publishing
 this on the web.
 
 Testing
 -------
 
-We love automated testing! We added a couple of test cases to 
-
-	./internals/tests/
-
-and it would be great if you added more. We want to run these tests 
+We love automated testing! We added a couple of test cases to
+
+    ./internals/tests/
+
+and it would be great if you added more. We want to run these tests
 whenever we build the package or publish the docs or whatever else. Just
 to make sure that everything works fine. To run the test suite, run:
 
-	make check
+    make check
+
+
+Functional (autopilot / selenium) tests
+---------------------------------------
+
+Make sure that you have oxide-chromedriver and selenium installed:
+
+    sudo apt-get install oxideqt-chromedriver
+    sudo apt-add-repository ppa:canonical-platform-qa/selenium
+    sudo apt-get update
+    sudo apt-get install python3-selenium
+
+Run the tests:
+
+    cd /internals/tests/functional
+    autopilot3 list help_app
+    autopilot3 run help_app
+
 
 Debugging
 ---------
 
-Did you find a bug and you're not sure how to debug it? 
+Did you find a bug and you're not sure how to debug it?
 
 Adding DEBUG=1 to the make command will give you additional information
-in a few cases. For example will 
+in a few cases. For example will
 
  - pelican print out debug information, which is very useful if you
    are changing pelican configuration options or are playing around
    with markdown extensions.
- - 'make launch' add the --inspector argument to ubuntu-html5-app-launcher. 
-   This will allow you to point your webkit-enabled browser to  
-    http://<yourip>:9221  
+ - 'make launch' add the --inspector argument to ubuntu-html5-app-launcher.
+   This will allow you to point your webkit-enabled browser to
+    http://<yourip>:9221
    to examine some of the web attributes of the app.
-  
+
 
 Running the app on a phone
 --------------------------
@@ -199,12 +217,12 @@
 To file bugs, or view/submit merge proposals, simply use this project in
 Launchpad:
 
-	https://launchpad.net/help-app
+    https://launchpad.net/help-app
 
 To update the .pot file, simply run the following command in the top-level
 directory:
 
-	make update-pot
+    make update-pot
 
 If you want to get help with the app or anything else, ask the fine folks
 in #ubuntu-app-devel - for example dholbach, balloons or dpm.

=== added directory 'internals/tests/functional'
=== added directory 'internals/tests/functional/help_app'
=== added file 'internals/tests/functional/help_app/__init__.py'
--- internals/tests/functional/help_app/__init__.py	1970-01-01 00:00:00 +0000
+++ internals/tests/functional/help_app/__init__.py	2015-05-26 19:24:25 +0000
@@ -0,0 +1,4 @@
+# -*- Mode: Python; coding: utf-8; indent-tabs-mode: nil; tab-width: 4 -*-
+
+""" autopilot tests """
+

=== added directory 'internals/tests/functional/help_app/tests'
=== added file 'internals/tests/functional/help_app/tests/__init__.py'
--- internals/tests/functional/help_app/tests/__init__.py	1970-01-01 00:00:00 +0000
+++ internals/tests/functional/help_app/tests/__init__.py	2015-05-26 19:24:25 +0000
@@ -0,0 +1,105 @@
+# -*- Mode: Python; coding: utf-8; indent-tabs-mode: nil; tab-width: 4 -*-
+
+""" Autopilot tests """
+
+import os
+import subprocess
+
+from selenium import webdriver
+from selenium.webdriver.chrome.options import Options
+
+import ubuntuuitoolkit as uitk
+from ubuntuuitoolkit import fixture_setup
+
+from autopilot import platform
+
+from testtools.matchers import NotEquals, EndsWith
+from autopilot.matchers import Eventually
+
+from autopilot.testcase import AutopilotTestCase
+
+CURRENT_ARCHITECTURE = subprocess.check_output(
+    ["dpkg-architecture", "-qDEB_HOST_MULTIARCH"]).strip()
+CHROMEDRIVER_EXEC_PATH = \
+    "/usr/lib/{}/oxide-qt/chromedriver".format(
+        CURRENT_ARCHITECTURE.decode("utf-8"))
+DEFAULT_WEBVIEW_INSPECTOR_IP = '127.0.0.1'
+DEFAULT_WEBVIEW_INSPECTOR_PORT = 9221
+
+
+class HTML5TestCaseBase(AutopilotTestCase):
+    def setUp(self):
+        self.driver = None
+        self.app_proxy = None
+        super().setUp()
+        if platform.model() == 'Desktop':
+            self.patch_environment(
+                'UBUNTU_WEBVIEW_DEVTOOLS_HOST',
+                DEFAULT_WEBVIEW_INSPECTOR_IP)
+            self.patch_environment(
+                'UBUNTU_WEBVIEW_DEVTOOLS_PORT',
+                str(DEFAULT_WEBVIEW_INSPECTOR_PORT))
+        else:
+            self.useFixture(fixture_setup.InitctlEnvironmentVariable(
+                global_=True,
+                UBUNTU_WEBVIEW_DEVTOOLS_HOST=DEFAULT_WEBVIEW_INSPECTOR_IP,
+                UBUNTU_WEBVIEW_DEVTOOLS_PORT=str(
+                    DEFAULT_WEBVIEW_INSPECTOR_PORT)
+            ))
+
+    def tearDown(self):
+        if self.driver:
+            self.page.close()
+            self.page.quit()
+        # XXX: This should not be there but AP hangs
+        # if we dont extra force the process to be killed
+        # (although AP already tries to kill part of its teardown)
+        if platform.model() == 'Desktop' \
+           and self.app_proxy \
+           and self.app_proxy.process:
+            self.app_proxy.process.kill()
+        super(HTML5TestCaseBase, self).tearDown()
+
+    def get_webview(self):
+        return self.app_proxy.select_single('UbuntuWebView02')
+
+    def launch_webdriver(self):
+        options = Options()
+        options.binary_location = ''
+        options.debugger_address = '{}:{}'.format(
+            DEFAULT_WEBVIEW_INSPECTOR_IP,
+            DEFAULT_WEBVIEW_INSPECTOR_PORT)
+
+        self.driver = webdriver.Chrome(
+            executable_path=CHROMEDRIVER_EXEC_PATH,
+            chrome_options=options)
+
+        self.assertThat(self.driver, NotEquals(None))
+
+    @property
+    def page(self):
+        return self.driver
+
+    def launch_html5_app_inline(self, args):
+        return self.launch_test_application(
+            'ubuntu-html5-app-launcher',
+            *args,
+            emulator_base=uitk.UbuntuUIToolkitCustomProxyObjectBase)
+
+    def launch_html5_app(self):
+        self.app_proxy = self.launch_html5_app_inline(
+            ['--www={}/{}'.format(
+                os.path.dirname(
+                    os.path.realpath(__file__)),
+                '../../../../../build/app/www')])
+        self.wait_for_app()
+
+        self.launch_webdriver()
+
+    def wait_for_app(self):
+        self.assertThat(self.app_proxy, NotEquals(None))
+        webview = self.get_webview()
+        self.assertThat(
+            lambda: webview.url,
+            Eventually(EndsWith("/index.html")))
+

=== added file 'internals/tests/functional/help_app/tests/test_app_launch.py'
--- internals/tests/functional/help_app/tests/test_app_launch.py	1970-01-01 00:00:00 +0000
+++ internals/tests/functional/help_app/tests/test_app_launch.py	2015-05-26 19:24:25 +0000
@@ -0,0 +1,14 @@
+# -*- Mode: Python; coding: utf-8; indent-tabs-mode: nil; tab-width: 4 -*-
+
+from help_app.tests import HTML5TestCaseBase
+
+
+class LaunchTestCaseBase(HTML5TestCaseBase):
+    def setUp(self):
+        super(LaunchTestCaseBase, self).setUp()
+
+    def test_basic_launch(self):
+        self.launch_html5_app()
+
+        # <add assert here>
+