launchpad-reviewers team mailing list archive

[Colin Watson <mp+426611@xxxxxxxxxxxxxxxxxx>]

Colin Watson has proposed merging ~cjwatson/launchpad:doc-framework into launchpad:master.

Commit message:
Document Launchpad's application framework

Requested reviews:
  Launchpad code reviewers (launchpad-reviewers)

For more details, see:
https://code.launchpad.net/~cjwatson/launchpad/+git/launchpad/+merge/426611

Since several aspects of the framework are quite custom, I often find myself explaining parts of it, and it would be good to have this written down more clearly.
-- 
Your team Launchpad code reviewers is requested to review the proposed merge of ~cjwatson/launchpad:doc-framework into launchpad:master.

diff --git a/doc/explanation/framework.rst b/doc/explanation/framework.rst
new file mode 100644
index 0000000..ea8b1f7
--- /dev/null
+++ b/doc/explanation/framework.rst
@@ -0,0 +1,106 @@
+=====================
+Application framework
+=====================
+
+Launchpad is built around Zope components.  Zope has meant substantially
+different things over the years (`Wikipedia's Zope article
+<https://en.wikipedia.org/wiki/Zope>`_ has a good overview, and `The World
+of Zope <https://zope.dev/world.html>`_ is also a useful introduction to
+some terminology), so this document tries to be a little more specific.  In
+modern terms, Launchpad uses the Zope Toolkit, but not the full Zope
+application server.
+
+Component architecture
+----------------------
+
+Launchpad uses the Zope Component Architecture (`zope.interface
+<https://zopeinterface.readthedocs.io/>`_ and `zope.component
+<https://zopecomponent.readthedocs.io/>`_) extensively and pervasively.
+``zope.interface`` defines most of Launchpad's internal interfaces, and is
+also built upon by `lazr.restful <https://lazrrestful.readthedocs.io/>`_ to
+define its `external web service API <https://help.launchpad.net/API>`_.
+``zope.component`` provides facilities for registering and looking up
+components, making it easier to decouple implementations from interfaces and
+reducing the need for circular imports.
+
+The `zope.schema <https://zopeschema.readthedocs.io/>`_ package from the
+Zope Toolkit allows specifying more detailed types for attributes of
+interfaces, and `zope.security <https://zopesecurity.readthedocs.io/>`_
+allows enforcing interfaces using security policies.  The latter is
+particularly important to Launchpad; it is the foundation of our ability to
+operate a complex multi-tenanted service in which users' privileges often
+overlap in ways that cannot be accurately enforced purely at API boundaries,
+and must instead be enforced object-by-object.  Most objects passed across
+interface boundaries within Launchpad are wrapped in Zope security proxies
+so that the security policy in ``lp.services.webapp.authorization`` is
+consulted on all attribute accesses.
+
+URL traversal and publishing
+----------------------------
+
+Launchpad uses `zope.traversing <https://zopetraversing.readthedocs.io/>`_
+to "traverse" URLs (i.e. work segment-by-segment along them until reaching a
+model object with an associated view), and `zope.publisher
+<https://zopepublisher.readthedocs.io/>`_ to define the details of how
+objects are "published" (i.e. the process of turning an HTTP request into an
+appropriate response, including various hooks that are run before and after
+calling the object itself).
+
+Many of the details here are handled by custom code in
+``lp.services.webapp``, especially ``lp.services.webapp.publication`` which
+provides many application-specific hooks, and
+``lp.services.webapp.publishing`` which defines much of Launchpad's
+traversal framework.  The URL structure is defined in various
+``configure.zcml`` files: ``browser:url`` tags are used to build up the
+canonical URL for an object, and ``browser:navigation`` tags register
+classes called as part of traversal to resolve the next step from each
+intermediate object in the URL (starting at
+``lp.app.browser.launchpad.LaunchpadRootNavigation``).
+
+Page templating
+---------------
+
+Launchpad uses `Page Templates <https://pagetemplates.readthedocs.io/>`_ as
+its HTML templating mechanism.  While systems like `Jinja
+<https://jinja.palletsprojects.com/>`_ are more popular elsewhere, the Page
+Templates language has the benefit that source files are themselves valid
+XML rather than being a combination of two syntaxes, and it is not generally
+possible to write well-formed input that produces ill-formed output.
+
+Page Templates were originally invented by Zope, and we still happen to use
+`Zope's implementation of them <https://zopepagetemplate.readthedocs.io/>`_,
+but other implementations such as `Chameleon
+<https://chameleon.readthedocs.io/>`_ exist.
+
+Views and forms
+---------------
+
+Launchpad's view layer (used for the web UI) is custom code, starting at
+``lp.services.webapp.publisher.LaunchpadView``.  However, many views are
+built around "forms", defined by building up a data schema using
+``zope.interface`` and ``zope.schema`` and then using `zope.formlib
+<https://zopeformlib.readthedocs.io/>`_ to generate HTML forms and to parse
+data from HTTP ``POST`` requests.  ``zope.formlib`` also provides "widgets"
+whose job it is to define the precise rendering and parsing of particular
+form elements, and Launchpad defines a number of custom widgets for cases
+where ``zope.formlib``'s widgets are insufficient.
+
+Events
+------
+
+It's often useful for parts of Launchpad to be able to subscribe to events
+generated by other parts of Launchpad.  For example, a number of subscribers
+watch for objects such as comments being created in order to be able to
+assign "karma" to their creators.  This is coordinated using `zope.event
+<https://zopeevent.readthedocs.io/>`_, as well as ``subscriber`` tags in
+various ``configure.zcml`` files.  `lazr.lifecycle
+<https://lazrlifecycle.readthedocs.io/>`_ enhances this with common events
+for object creation, modification, and deletion.
+
+WSGI server
+-----------
+
+Launchpad's application server runs within `Gunicorn
+<https://docs.gunicorn.org/>`_, wrapped using `Talisker
+<https://pypi.org/project/talisker/>`_ to add several useful operational
+facilities.
diff --git a/doc/explanation/index.rst b/doc/explanation/index.rst
index 7ea00e8..bb30f5a 100644
--- a/doc/explanation/index.rst
+++ b/doc/explanation/index.rst
@@ -7,6 +7,7 @@ Explanation
    :maxdepth: 1
 
    running-details
+   framework
    architecture
    pip
    favicon