yellow team mailing list archive

[Benji York <benji.york@xxxxxxxxxxxxx>]

Benji York has proposed merging lp:~benji/juju-gui/bug-1083618 into lp:juju-gui.

Requested reviews:
  Juju GUI Hackers (juju-gui)
Related bugs:
  Bug #1083618 in juju-gui: "Automatically generate releases"
  https://bugs.launchpad.net/juju-gui/+bug/1083618

For more details, see:
https://code.launchpad.net/~benji/juju-gui/bug-1083618/+merge/138740

Add the ability to make releases to the Makefile

The "dist" target creates a (gzipped) tar file which contains the entire
branch sans the ".bzr" and "release" directories.

This branch also introduces a CHANGES.yaml file.  We are experementing with
YAML (instead of text or reST) to see if it has any advantages, especially in
the relm of displaying recent changes in the app itself.

There are two kinds of releases that can be made and they can be made to
either the "real" Launchpad site or to the staging site (for trial-runs).
Full details have been added to the procedures documentation.

https://codereview.appspot.com/6909046/

-- 
https://code.launchpad.net/~benji/juju-gui/bug-1083618/+merge/138740
Your team Juju GUI Hackers is requested to review the proposed merge of lp:~benji/juju-gui/bug-1083618 into lp:juju-gui.

=== modified file '.bzrignore'
--- .bzrignore	2012-12-06 19:27:37 +0000
+++ .bzrignore	2012-12-07 14:29:25 +0000
@@ -14,5 +14,7 @@
 virtualenv
 yuidoc
 build
+upload_release.py
+releases
 build-prod
 build-debug

=== added file 'CHANGES.yaml'
--- CHANGES.yaml	1970-01-01 00:00:00 +0000
+++ CHANGES.yaml	2012-12-07 14:29:25 +0000
@@ -0,0 +1,26 @@
+# This file keeps track of the changes to the file.  We use it both to
+# document the changes, and to find the most recent version numbers.  See the
+# release process documented in docs/process.rst for details.
+
+# This file uses the YAML format because it is easy to write and it is easy to
+# convert to JSON, which seemed like a reasonable goal in case we want to use
+# this data in the GUI itself at some point.  Here's an example of how to
+# write the changes data.
+
+# - 0.5.0:
+#   - Did something
+#   - Did something else
+#   - >
+#     Another thing that spans multiple lines
+#     like this
+# - 0.4.0:
+#   - Something!
+#   - Ooh!
+
+# Release identifiers in this file should always be "unreleased" or
+# [NUMBER].[NUMBER].[NUMBER] per http://semver.org/ .
+
+- unreleased:
+    - First tarball release
+- 0.1.0:
+    - OpenStack Summit 2012 demo

=== modified file 'Makefile'
--- Makefile	2012-12-07 11:48:03 +0000
+++ Makefile	2012-12-07 14:29:25 +0000
@@ -21,7 +21,67 @@
 	node_modules/node-minify node_modules/node-spritesheet \
 	node_modules/rimraf node_modules/should node_modules/yui \
 	node_modules/yuidocjs
-EXPECTED_NODE_TARGETS=$(shell echo "$(NODE_TARGETS)" | tr ' ' '\n' | sort | tr '\n' ' ')
+EXPECTED_NODE_TARGETS=$(shell echo "$(NODE_TARGETS)" | tr ' ' '\n' | sort \
+	| tr '\n' ' ')
+
+### Relase-specific variables - see docs/process.rst for an overview. ###
+BZR_REVNO=$(shell bzr revno)
+# Figure out the two most recent version numbers.
+ULTIMATE_VERSION=$(shell grep '^-' CHANGES.yaml | head -n 1 | sed 's/[ :-]//g')
+PENULTIMATE_VERSION=$(shell grep '^-' CHANGES.yaml | head -n 2 | tail -n 1 \
+    | sed 's/[ :-]//g')
+# If the user specified (via setting an environment variable on the command
+# line) that this is a final (non-development) release, set the version number
+# and series appropriately.
+ifdef FINAL
+# If this is a FINAL (non-development) release, then the most recent version
+# number should not be "unreleased".
+ifeq ($(ULTIMATE_VERSION), unreleased)
+    $(error FINAL releases must have a most-recent version number other than \
+	"unreleased" in CHANGES.yaml)
+endif
+VERSION=$(ULTIMATE_VERSION)
+SERIES=stable
+else
+# If this is development (non-FINAL) release, then the most recent version
+# number must be "unreleased".
+ifneq ($(ULTIMATE_VERSION), unreleased)
+    $(error non-FINAL releases must have a most-recent version number of \
+	"unreleased" in CHANGES.yaml)
+endif
+RELEASE_VERSION=$(PENULTIMATE_VERSION)+build.$(BZR_REVNO)
+SERIES=trunk
+endif
+# If we are doing a production release (as opposed to a trial-run release) we
+# use the "real" Launchpad site, if not we use the Launchpad staging site.
+ifndef PROD
+LAUNCHPAD_API_ROOT=staging
+endif
+RELEASE_NAME=juju-gui-$(RELEASE_VERSION)
+RELEASE_FILE=releases/$(RELEASE_NAME).tgz
+RELEASE_SIGNATURE=releases/$(RELEASE_NAME).asc
+# Is the branch being released a branch of trunk?
+ifndef IS_TRUNK_BRANCH
+IS_TRUNK_BRANCH=$(shell bzr info | grep \
+	'parent branch: bzr+ssh://bazaar.launchpad.net/+branch/juju-gui/' \
+	> /dev/null && echo 1)
+endif
+# Does the branch on disk have uncomitted/unpushed changes?
+ifndef BRANCH_IS_CLEAN
+BRANCH_IS_CLEAN=$(shell [ -z "`bzr status`" ] && bzr missing --this && echo 1)
+endif
+# Is it safe to do a release of the branch?  For trial-run releases you can
+# override this check on the command line by setting the BRANCH_IS_GOOD
+# environment variable.
+ifndef BRANCH_IS_GOOD
+ifneq ($(strip $(IS_TRUNK_BRANCH)),)
+ifneq ($(strip $(BRANCH_IS_CLEAN)),)
+BRANCH_IS_GOOD=1
+endif
+endif
+endif
+### End of relase-specific variables ###
+
 TEMPLATE_TARGETS=$(shell bzr ls -k file app/templates)
 
 SPRITE_SOURCE_FILES=$(shell bzr ls -R -k file app/assets/images)
@@ -254,6 +314,49 @@
 	cp manifest.appcache.in $(APPCACHE)
 	sed -re 's/^\# TIMESTAMP .+$$/\# TIMESTAMP $(DATE)/' -i $(APPCACHE)
 
+# This really depends on CHANGES.yaml, the bzr revno changing, and the build
+# /juju-ui directory existing.  We are vaguely trying to approximate the second
+# one by connecting it to our pertinent versioned files.  The appcache target
+# creates the third, and directories are a bit tricky with Makefiles so we are
+# OK with that.
+build/juju-ui/version.js: appcache CHANGES.yaml $(JSFILES) $(TEMPLATE_TARGETS) \
+		$(SPRITE_SOURCE_FILES)
+	echo "var jujuGuiVersionName='$(RELEASE_VERSION)';" \
+	    > build/juju-ui/version.js
+
+upload_release.py:
+	bzr cat lp:launchpadlib/contrib/upload_release_tarball.py \
+	    > upload_release.py
+
+$(RELEASE_FILE): build
+	@echo "$(BRANCH_IS_CLEAN)"
+ifdef BRANCH_IS_GOOD
+	mkdir -p releases
+	tar c --auto-compress --exclude-vcs --exclude releases \
+	    --transform "s|^|$(RELEASE_NAME)/|" -f $(RELEASE_FILE) *
+	@echo "Release was created in $(RELEASE_FILE)."
+else
+	@echo "**************************************************************"
+	@echo "*********************** RELEASE FAILED ***********************"
+	@echo "**************************************************************"
+	@echo
+	@echo "To make a release, you must either be in a branch of"
+	@echo "lp:juju-gui without uncommitted/unpushed changes, or you must"
+	@echo "override one of the pertinent variable names to force a "
+	@echo "release."
+	@echo
+	@echo "See the README for more information."
+	@echo
+	@false
+endif
+
+$(RELEASE_SIGNATURE): $(RELEASE_FILE)
+	gpg --armor --sign --detach-sig $(RELEASE_FILE)
+
+dist: $(RELEASE_FILE) $(RELEASE_SIGNATURE) upload_release.py
+	python2 upload_release.py juju-gui $(SERIES) $(RELEASE_VERSION) \
+	    $(RELEASE_FILE) $(LAUNCHPAD_API_ROOT)
+
 appcache: $(APPCACHE)
 
 # A target used only for forcibly updating the appcache.
@@ -268,6 +371,6 @@
 	appcache-touch appcache-force yuidoc spritegen yuidoc-lint \
 	build-files javascript-libraries build build-debug help \
 	build-prod clean clean-deps clean-docs clean-all devel debug \
-	prod link-debug-files link-prod-files doc
+	prod link-debug-files link-prod-files doc dist
 
 .DEFAULT_GOAL := all

=== modified file 'README'
--- README	2012-11-07 18:41:11 +0000
+++ README	2012-12-07 14:29:25 +0000
@@ -26,6 +26,10 @@
 
   $ sudo apt-get install python-sphinx
 
+To make releases, you'll need pytz::
+
+  $ sudo apt-get install python-tz
+
 The gui frontend can be installed with::
 
   $ bzr branch lp:juju-ui trunk
@@ -113,6 +117,69 @@
 
   $ jshint --config=jshint.config `bzr ls -RV -k file | grep -v assets/`
 
+Making releases
+===============
+
+To make a release, you must either be in a checkout of lp:juju-gui
+without uncommitted changes, or you must override one of the
+`pertinent variable names`_ to force a release.
+
+.. _`pertinent variable names`: `Potentially Useful Release-Oriented Makefile Variables`_
+
+You also need to have a gpg key, and the python-pytz package installed (as
+well as launchpadlib, but that is installed by default in Ubuntu).
+
+See the Process document (docs/process.rst) for step-by-step checklists to
+make developer and stable releases.
+
+Potentially Useful Release-Oriented Makefile Variables
+------------------------------------------------------
+
+The following is a list of pertinent Makefile variables.
+
+FINAL
+  Set FINAL to any non-empty value to make a final release. This will cause
+  the bzr revno to be omitted from the tarball name, and (if you use the
+  release target) will cause the release to be uploaded to the stable series
+  rather than the trunk series. Example usage::
+
+    $ FINAL=1 make release
+
+PROD
+  By default, releases will be uploaded to staging.launchpad.net, which is a
+  separate version of Launchpad that uses a temporary database.  This can be
+  convenient for trying out the release process in the Makefile without
+  affecting our actual production releases.  Set PROD to any non-empty value
+  to send uploads to launchpad.net, the production version of Launchpad, when
+  you are ready to make a real release.
+
+  Note that you may need to ask the webops to turn off the two-factor
+  authentication on your Launchpad staging account in order for the staging to
+  work. Go to the #launchpad-ops channel on the canonical IRC server and ask
+  something like "webops, could you disable 2FA on my staging account?"
+
+  Example usage::
+
+    $ PROD=1 make release
+
+IS_TRUNK_CHECKOUT
+  Set this to any non-empty value to force the Makefile to believe it is
+  working with a trunk checkout. Example usage::
+
+    $ IS_TRUNK_CHECKOUT=1 make release
+
+HAS_NO_CHANGES
+  Set this to any non-empty value to force the Makefile to believe that the
+  current code tree has no changes. Example usage::
+
+    $ HAS_NO_CHANGES=1 make release
+
+IS_SAFE_RELEASE
+  Set this to any non-empty value to force the Makefile to bypass checks of
+  IS_TRUNK_CHECKOUT and HAS_NO_CHANGES. Example usage::
+
+    $ IS_SAFE_RELEASE=1 make release
+
 More documentation
 ==================
 

=== modified file 'docs/process.rst'
--- docs/process.rst	2012-11-01 10:33:30 +0000
+++ docs/process.rst	2012-12-07 14:29:25 +0000
@@ -13,6 +13,10 @@
   are not sure of your solution, prototype before the pre-implementation call.
 - Use TDD.  Your prototype might be perfect, but you can still move it aside
   and rebuild it gradually as you add tests.  It can go quickly.
+- Update the CHANGES.yaml file with your changes.  The newest (topmost)
+  section should have the version "unreleased".  If not and you are
+  making changes, add an "unreleased" section at the top.  All other
+  version numbers follow Semantic Versioning (http://semver.org/).
 
 Checklist for Preparing for a Review
 ====================================
@@ -40,6 +44,8 @@
   - Treat this as an opportunity to learn.  Consider what you could have
     done differently.
 
+- Update the CHANGES.yaml file with a description of the changes you
+  made.  Reference Launchpad bugs if appropriate.
 - If the branch is very minor, such as a documentation change, feel free to
   self-review.  However, *don't neglect to consider your responsibilities
   above*, especially the diff review and running tests (even if you think
@@ -86,6 +92,116 @@
   want to re-review after the changes, or if you automatically approve if the
   changes are made.
 
+Checklist for Making a Stable Release
+=====================================
+
+- Get a checkout of the trunk:: ``bzr co lp:juju-gui``.
+- If you are using a pre-existing checkout, make sure it is up-to-date:: ``bzr
+  up``.
+- Verify that the top-most version in CHANGES.yaml specifies the
+  expected version string.  It should be bigger than the most recent
+  version found on https://launchpad.net/juju-gui/stable .
+- Run the tests and verify they pass: ``make test``.
+- Create the tarball: ``FINAL=1 make tarball``.  The process will end by
+  reporting the name of the tarball it made.
+- In an empty temporary directory somewhere else on your system, expand the
+  tarball: ``tar xvzf PATH_TO_TARBALL``
+- In that directory, start a server: ``python -m SimpleHTTPServer 8888``
+- In Chrome and Firefox, QA the application.  XXX EXPLICIT QA STEPS GO HERE!
+- For now, we will assume you would like to verify the release on the
+  Launchpad staging server.  As we become more confident with this process,
+  this step may become unnecessary.  In the checkout, run ``FINAL=1 make
+  release``.  This will step you through signing the tarball, connecting
+  to Launchpad, and uploading the release.
+
+  * Note that you may need to ask the webops to turn off the two-factor
+    authentication on your Launchpad staging account in order for this to
+    work. Go to the #launchpad-ops channel on the canonical IRC server and ask
+    something like "webops, could you disable 2FA on my staging account?"
+  * When Launchpad asks you what level of permissions to grant, assuming you
+    are running on your own computer that you manage securely, the easiest
+    thing to do is hopefully also reasonably safe: accept that the computer
+    may perform all actions, indefinitely.
+
+- Go to https://staging.launchpad.net/juju-gui/stable and verify that you see
+  a new release and a new download file.
+- Download the file, expand it in a temporary directory, run ``python -m
+  SimpleHTTPServer 8888``, and do a quick double-check in the browser that it
+  is what you expect.  Looking at juju-ui/version.js should also show you the
+  version you expect.
+- This is a final release.  Consider asking others to verify the package on staging.
+- Now it is time for the actual, real release.  Head back to your checkout and
+  run ``FINAL=1 PROD=1 make release``.  The computer will again walk you
+  through the process.
+
+  * Note that, one time per computer, you will again have to accept the
+    Launchpadlib security token: In Launchpad, the staging site and the
+    production have fully separate databases, including authentication.  What
+    is done in production will in many cases eventually be copied over to
+    staging, but never vice versa.  Staging data is destroyed periodically.
+
+- Go to https://launchpad.net/juju-gui/stable and verify that you see
+  a new release and a new download file.
+
+You are done!
+
+Checklist for Making a Developer Release
+========================================
+
+- Get a checkout of the trunk:: ``bzr co lp:juju-gui``.
+- If you are using a pre-existing checkout, make sure it is up-to-date::
+  ``bzr up``.
+- Verify that the top-most version in CHANGES.yaml specifies the expected
+  version string.  Run ``bzr revno``.  These two values, combined, should be
+  bigger than the most recent version found on
+  https://launchpad.net/juju-gui/trunk .  To be clear, the version should be
+  the same or greater as the most recent developer release, and the revno
+  should be greater.
+- Run the tests and verify they pass: ``make test``.
+- Create the tarball: ``make tarball``.  It will end by reporting the name of
+  the tarball it made.
+- In an empty temporary directory somewhere else on your system, expand the
+  tarball: ``tar xvzf PATH_TO_TARBALL``
+- Looking at juju-ui/version.js should show you a version string that combines
+  the value in the checkout's CHANGES.yaml with the checkout's revno.
+- In that directory, start a server: ``python -m SimpleHTTPServer 8888``
+- In Chrome and Firefox, QA the application.  XXX EXPLICIT QA STEPS GO HERE!
+- For now, we will assume you would like to verify the release on the
+  Launchpad staging server.  As we become more confident with this process,
+  this step may become unnecessary.  In the checkout, run ``make release``.
+  This will step you through signing the tarball, connecting to
+  Launchpad, and uploading the release.
+
+  * Note that you may need to ask the webops to turn off the two-factor
+    authentication on your Launchpad staging account in order for this to
+    work. Go to the #launchpad-ops channel on the canonical IRC server and ask
+    something like "webops, could you disable 2FA on my staging account?"
+  * When Launchpad asks you what level of permissions to grant, assuming you
+    are running on your own computer that you manage securely, the easiest
+    thing to do is hopefully also reasonably safe: accept that the computer
+    may perform all actions, indefinitely.
+
+- Go to https://staging.launchpad.net/juju-gui/trunk and verify that you see
+  a new release and a new download file.
+- Download the file, expand it in a temporary directory, run ``python -m
+  SimpleHTTPServer 8888``, and do a quick double-check in the browser that it
+  is what you expect.  Looking at juju-ui/version.js should also show you the
+  version you expect, as seen in the similar earlier step above.
+- Now it is time for the actual, real release.  Head back to your checkout and
+  run ``PROD=1 make release``.  The computer will again walk you through the
+  process.
+
+  * Note that, one time per computer, you will again have to accept the
+    Launchpadlib security token: In Launchpad, the staging site and the
+    production have fully separate databases, including authentication.  What
+    is done in production will in many cases eventually be copied over to
+    staging, but never vice versa.  Staging data is destroyed periodically.
+
+- Go to https://launchpad.net/juju-gui/trunk and verify that you see
+  a new release and a new download file.
+
+You are done!
+
 Checklist for Running a Daily Meeting
 =====================================