launchpad-reviewers team mailing list archive

[Richard Harding <rick.harding@xxxxxxxxxxxxx>]

Richard Harding has proposed merging lp:~rharding/launchpad/watch_jsbuild into lp:launchpad with lp:~rharding/launchpad/use_lpjsmin as a prerequisite.

Requested reviews:
  Launchpad code reviewers (launchpad-reviewers)

For more details, see:
https://code.launchpad.net/~rharding/launchpad/watch_jsbuild/+merge/94379

= Summary =
In order to test and work with JS changes, you need to rebuild the JS files into the combo root, minimize, etc. This is done from the command `make jsbuild`, but it's a manual command that severly slows down development.

This adds a new make command `make jsbuild_watch` that will run a constant process, watching for changed JS files, and auto building them on the fly. In this way you shouldn't have to manually run `make jsbuild` while the process is running.

The only works for the combo loader and so isn't helpful if that feature flag isn't enabled.

== Proposed Fix ==
Adds a watch script and a make command to start it up manually. It runs and is killed with just a ctrl-c for now.

== Implementation Details ==
The watch_jsbuild.py script watching any changed files in lib. It checks them to see if they're non-test JS files. If so, it copies and minifies them into the build directory for the combo loader.

The script uses pyinotify to watch the lib directory and all sub directories for changed files.
-- 
https://code.launchpad.net/~rharding/launchpad/watch_jsbuild/+merge/94379
Your team Launchpad code reviewers is requested to review the proposed merge of lp:~rharding/launchpad/watch_jsbuild into lp:launchpad.

=== modified file 'Makefile'
--- Makefile	2012-02-23 16:50:33 +0000
+++ Makefile	2012-02-23 16:50:33 +0000
@@ -63,7 +63,8 @@
     bin/harness bin/iharness bin/ipy bin/jsbuild bin/lpjsmin\
     bin/killservice bin/kill-test-services bin/lint.sh bin/retest \
     bin/run bin/run-testapp bin/sprite-util bin/start_librarian bin/stxdocs \
-    bin/tags bin/test bin/tracereport bin/twistd bin/update-download-cache
+    bin/tags bin/test bin/tracereport bin/twistd bin/update-download-cache \
+	bin/watch_jsbuild.py
 
 BUILDOUT_TEMPLATES = buildout-templates/_pythonpath.py.in
 
@@ -179,6 +180,9 @@
 	    --srcdir lib/lp/app/javascript \
 	    --builddir $(LP_BUILT_JS_ROOT)
 
+jsbuild_watch:
+	$(PY) bin/watch_jsbuild.py
+
 $(JS_LP): jsbuild_widget_css
 $(JS_YUI):
 	cp -a lib/canonical/launchpad/icing/yui_2.7.0b/build build/js/yui2

=== added file 'buildout-templates/bin/watch_jsbuild.py.in'
--- buildout-templates/bin/watch_jsbuild.py.in	1970-01-01 00:00:00 +0000
+++ buildout-templates/bin/watch_jsbuild.py.in	2012-02-23 16:50:33 +0000
@@ -0,0 +1,79 @@
+#!/usr/bin/env python
+
+# Using ionotify we watch our sources of JavaScript in order to know we should
+# build when the files change.
+
+import lpjsmin
+import pyinotify
+import os
+import re
+import shutil
+
+BUILD_DIR = 'build/js/lp'
+WATCH_DIR = 'lib'
+
+# to start out let's assume your CWD is where we're referencing things from
+CWD = os.getcwd()
+JSDIR = os.path.join(CWD, BUILD_DIR)
+FILTERS = [
+    # ignore test files
+    lambda x: x.startswith('test_'),
+    # ignore directories
+    lambda x: os.path.isdir(x),
+    # ignore files that don't end in .js
+    lambda x: not x.endswith('.js'),
+]
+RENAME = re.compile("^.*lib/lp/(.*)/javascript")
+
+
+def build_path(path):
+    match = RENAME.search(path)
+    js_dir = match.groups()[0]
+    return os.path.join(JSDIR, RENAME.sub(js_dir, path))
+
+
+def copyfile(src, dst):
+    # make sure the directory for it exists
+    d = os.path.dirname(dst)
+    if not os.path.exists(d):
+        os.makedirs(d)
+    shutil.copyfile(src, dst)
+
+
+class HandleChanges(pyinotify.ProcessEvent):
+    def process_IN_CREATE(self, event):
+        if event.mask == pyinotify.IN_MODIFY:
+            needs_filtering = [f(event.name) for f in FILTERS]
+            if True in needs_filtering:
+                pass
+            else:
+                print "Was created:", event.pathname
+
+    def process_IN_MODIFY(self, event):
+        if event.mask == pyinotify.IN_MODIFY:
+            needs_filtering = [f(event.name) for f in FILTERS]
+            if True in needs_filtering:
+                pass
+            else:
+                new_path = build_path(event.pathname)
+                copyfile(event.pathname, new_path)
+                lpjsmin.minify(new_path)
+
+
+if __name__ == "__main__":
+    print "Watching '%s' for JS changes" % WATCH_DIR
+    # Instanciate a new WatchManager (will be used to store watches).
+    wm = pyinotify.WatchManager()
+
+    # watched events
+    mask = pyinotify.IN_MODIFY | pyinotify.IN_CREATE
+
+    # Associate this WatchManager with a Notifier (will be used to report and
+    # process events).
+    handler = HandleChanges()
+    notifier = pyinotify.Notifier(wm, handler)
+
+    # Add a new watch on cwd for ALL_EVENTS.
+    # Loop forever and handle events.
+    wm.add_watch(WATCH_DIR, mask, rec=True)
+    notifier.loop()

=== modified file 'buildout.cfg'
--- buildout.cfg	2012-02-23 16:50:33 +0000
+++ buildout.cfg	2012-02-23 16:50:33 +0000
@@ -68,6 +68,7 @@
     funkload
     zc.zservertracelog
     lpjsmin
+    pyinotify
 # XXX gary 2009-5-12 bug 375751:
 # Make mailman built and installed in a more normal way.
 extra-paths =

=== modified file 'versions.cfg'
--- versions.cfg	2012-02-23 16:50:33 +0000
+++ versions.cfg	2012-02-23 16:50:33 +0000
@@ -73,6 +73,7 @@
 # bug 834427).  Can be replaced with Pygments 1.5 once it is released.
 Pygments = 1.4dev-20111007
 pygpgme = 0.2
+pyinotify = 0.9.3
 pymongo = 2.1.1
 pyOpenSSL = 0.10
 pystache = 0.3.1