ubuntu-appstore-developers team mailing list archive

[Colin Watson <cjwatson@xxxxxxxxxx>]

On Thu, Jul 18, 2013 at 12:23:21PM -0500, Jamie Strandboge wrote:
> Marc, Colin, Steve, and Ted: are we all in agreement as to the new format?
> 
> If yes, to make sure we are all on the same page, for application confinement:
>  * aa-clicktool updated to dereference symlinks and handle new json format
>  * apparmor-easyprof-ubuntu updated for CLICK_DIR
>  * manifest wiki updated for new format
> 
> Steve, you handle the first and I'll handle the other two.

Well, I proposed it so I'm obviously in agreement :-)  I made a couple
of refinements to my draft in light of this thread, though.  It seems
that there are still some things to be solved, but nothing that's a
blocker and I don't think we're painting ourselves into a corner.

Here's my current diff to the hooks specification, for reference.  I'm
trying to get this implemented before I go away for the weekend, but if
I can't then I'll push work in progress to a branch.

=== modified file 'doc/hooks.rst'
--- doc/hooks.rst	2013-06-27 14:30:15 +0000
+++ doc/hooks.rst	2013-07-18 17:30:21 +0000
@@ -2,6 +2,9 @@
 Hooks
 =====
 
+Rationale
+---------
+
 Of course, any sensible packaging format needs a hook mechanism of some
 kind; just unpacking a filesystem tarball isn't going to cut it.  But part
 of the point of Click packages is to make packages easier to audit by
@@ -36,79 +39,146 @@ directories.  The second is that process
 on the system dpkg database, which is large and therefore slow.
 
 Let us consider an example of the sort that might in future be delivered as
-a Click package, and one which is simple but not too simple.
-unity-scope-manpages (ignoring its dependencies) delivers a plugin for the
-unity help lens (/usr/share/unity/lenses/help/unity-scope-manpages.scope)
-and a D-Bus service
-(/usr/share/dbus-1/services/unity-scope-manpages.service).  These are
-consumed by unity-lens-help and dbus respectively, and each lists the
-corresponding directory looking for files to consume.
+a Click package, and one which is simple but not too simple.  Our example
+package (com.ubuntu.example) delivers an AppArmor profile and two .desktop
+files.  These are consumed by apparmor and desktop-integration (TBD)
+respectively, and each lists the corresponding directory looking for files
+to consume.
 
 We must assume that in the general case it will be at least inconvenient to
 cause the integrated-with packages to look in multiple directories,
 especially when the list of possible directories is not fixed, so we need a
 way to cause files to exist in those directories.  On the other hand, we
 cannot unpack directly into those directories, because that takes us back to
-using dpkg itself.
-
-What we can do with reasonable safety is populate symlink farms.  As a
-strawman proposal, consider the following:
+using dpkg itself, and is incompatible with system image updates where the
+root file system is read-only.  What we can do with reasonable safety is
+populate symlink farms.
+
+Specification
+-------------
+
+ * Only system packages (i.e. .debs) may declare hooks.  Click packages must
+   be declarative in that they may not include code executed outside
+   AppArmor confinement, which precludes declaring hooks.
+
+ * "System-level hooks" are those which operate on the full set of installed
+   package/version combinations.  They may run as any (system) user.
+   (Example: AppArmor profile handling.)
+
+ * "User-level hooks" are those which operate on the set of packages
+   registered by a given user.  They run as that user, and thus would
+   generally be expected to keep their state in the user's home directory or
+   some similar user-owned file system location.  (Example: desktop file
+   handling.)
+
+ * A Click package may contain one or more applications (the common case
+   will be only one).  Each application has a name.
+
+ * An "application ID" is a string unique to each application instance: it
+   is made up of the Click package name, the application name, and the Click
+   package version joined by underscores, e.g.
+   ``com.ubuntu.clock_alarm_0.1``.
 
  * An integrated-with system package may add ``*.hook`` files to
-   /usr/share/click/hooks/.  These are standard Debian-style control files
-   with the following keys:
-
-     Pattern: <file-pattern>    (required)
-     Exec: <program>            (optional)
-     Trigger: yes               (optional)
-
-   The value of Pattern is a printf format string which must contain exactly
-   one %s substitution: the package manager substitutes the unique Click
-   package name (e.g. com.example.app) into it to get the target path.  On
-   install, it creates the target path as a symlink to a path provided by
-   the Click package; on upgrade, it changes the target path to be a symlink
-   to the path in the new version of the Click package; on removal, it
-   unlinks the target path.
-
-   If the Exec key is present, its value is executed as if passed to the
-   shell after the above symlink is created.  A non-zero exit status is an
-   error; hook implementors must be careful to make commands in Exec fields
-   robust.  Note that this command intentionally takes no arguments, and
-   will be run on install, upgrade, and removal; it must be written such
-   that it causes the system to catch up with the current state of all
-   installed hooks.  Exec commands must be idempotent.
-
-   For the optional Trigger key, see below.
-
- * A Click package may include a "hooks" entry in its manifest (exact format
-   TBD).  If present, it must contain a mapping of keys to values.  The keys
-   are used to look up ``*.hook`` files with matching base names.  The
-   values are symlink target paths used by the package manager when creating
-   symlinks according to the Pattern field in ``*.hook`` files.
+   ``/usr/share/click/system-hooks/`` (system-level hooks) or
+   ``/usr/share/click/user-hooks/`` (user-level hooks).  These are standard
+   Debian-style control files with the following keys:
+
+   Pattern: <file-pattern> (required)
+     The value of ``Pattern`` is a string containing one or more
+     substitution placeholders, as follows:
+
+     ``${id}``
+       The application ID.
+
+     ``${user}``
+       The user name (user-level hooks only).
+
+     ``${home}``
+       The user's home directory (user-level hooks only).
+
+     ``$$``
+       The character '``$``'.
+
+     At least one ``${id}`` substitution is required.  For user-level hooks,
+     at least one of ``${user}`` and ``${home}`` must be present.
+
+     On install, the package manager creates the target path as a symlink to
+     a path provided by the Click package; on upgrade, it changes the target
+     path to be a symlink to the path in the new version of the Click
+     package; on removal, it unlinks the target path.
+
+     The terms "install", "upgrade", and "removal" are taken to refer to the
+     status of the hook rather than of the package.  That is, when upgrading
+     between two versions of a package, if the old version uses a given hook
+     but the new version does not, then that is a removal; if the old
+     version does not use a given hook but the new version does, then that
+     is an install; if both versions use a given hook, then that is an
+     upgrade.
+
+     For system-level hooks, one target path exists for each unpacked
+     version, unless "``Single-Version: yes``" is used (see below).  For
+     user-level hooks, a target path exists only for the current version
+     registered by each user for each package.
+
+     Upgrades of user-level hooks may leave the symlink pointed at the same
+     target (since the target will itself be via a ``current`` symlink in
+     the user registration directory).  ``Exec`` commands in hooks should
+     take care to check the modification timestamp of the target.
+
+   Exec: <program> (optional)
+     If the ``Exec`` key is present, its value is executed as if passed to
+     the shell after the above symlink is modified.  A non-zero exit status
+     is an error; hook implementors must be careful to make commands in
+     ``Exec`` fields robust.  Note that this command intentionally takes no
+     arguments, and will be run on install, upgrade, and removal; it must be
+     written such that it causes the system to catch up with the current
+     state of all installed hooks.  ``Exec`` commands must be idempotent.
+
+   Trigger: yes (optional)
+     It will often be valuable to execute a dpkg trigger after installing a
+     Click package to avoid code duplication between system and Click
+     package handling, although we must do so asynchronously and any errors
+     must not block the installation of Click packages.  If "``Trigger:
+     yes``" is set in a ``*.hook`` file, then "``click install``" will
+     activate an asynchronous D-Bus service at the end of installation,
+     passing the names of all the changed paths resulting from Pattern key
+     expansions; this will activate any file triggers matching those paths,
+     and process all the packages that enter the triggers-pending state as a
+     result.
+
+   User: <username> (required, system-level hooks only)
+     System-level hooks are run as the user whose name is specified as the
+     value of ``User``.  There is intentionally no default for this key, to
+     encourage hook authors to run their hooks with the least appropriate
+     privilege.
+
+   Single-Version: yes (optional, system-level hooks only)
+     By default, system-level hooks support multiple versions of packages,
+     so target paths may exist at multiple versions.  "``Single-Version:
+     yes``" causes only the current version of each package to have a target
+     path.
+
+ * A Click package may attach to zero or more hooks, by including a "hooks"
+   entry in its manifest.  If present, this must be a dictionary mapping
+   application names to hook sets; each hook set is itself a dictionary
+   mapping hook names to paths.  The hook names are used to look up
+   ``*.hook`` files with matching base names.  The paths are relative to the
+   directory where the Click package is unpacked, and are used as symlink
+   targets by the package manager when creating symlinks according to the
+   ``Pattern`` field in ``*.hook`` files.
 
  * There is a dh_click program which installs the ``*.hook`` files in system
    packages and adds maintainer script fragments to cause click to catch up
    with any newly-provided hooks.  It may be invoked using ``dh $@ --with
    click``.
 
- * It will often be valuable to execute a dpkg trigger after installing a
-   Click package to avoid code duplication between system and Click package
-   handling, although we must do so asynchronously and any errors must not
-   block the installation of Click packages.  If "Trigger: yes" is set in a
-   ``*.hook`` file, then "click install" will activate an asynchronous D-Bus
-   service at the end of installation, passing the names of all the changed
-   paths resulting from Pattern key expansions; this will activate any file
-   triggers matching those paths, and process all the packages that enter
-   the triggers-pending state as a result.
-
- * The terms "install", "upgrade", and "removal" are taken to refer to the
-   status of the hook rather than of the package.  That is, when upgrading
-   between two versions of a package, if the old version uses a given hook
-   but the new version does not, then that is a removal; if the old version
-   does not use a given hook but the new version does, then that is an
-   install; if both versions use a given hook, then that is an upgrade.
+Examples
+--------
+
+TODO: update this for AppArmor/desktop examples
 
-Thus, a worked example would have::
+::
 
   /usr/share/click/hooks/unity-lens-help.hook
     Pattern: /usr/share/unity/lenses/help/click-%s.scope
@@ -120,8 +190,10 @@ Thus, a worked example would have::
 
   com.ubuntu.unity-scope-manpages/manifest.json:
     "hooks": {
+      "unity-scope-manpages": {
         "unity-lens-help": "help/unity-scope-manpages.scope",
         "dbus-service": "services/unity-scope-manpages.service",
+      }
     }
 
 TODO: copy rather than symlink, for additional robustness?

Thanks,

-- 
Colin Watson                                       [cjwatson@xxxxxxxxxx]