ubuntu-packaging-guide-team team mailing list archive

[Dmitry Shachnev <mitya57@xxxxxxxxx>]

Dmitry Shachnev has proposed merging lp:~mitya57/ubuntu-packaging-guide/add-python into lp:ubuntu-packaging-guide.

Requested reviews:
  Ubuntu Packaging Guide Team (ubuntu-packaging-guide-team)
Related bugs:
  Bug #702008 in Ubuntu Packaging Guide: "Add article "Python packaging""
  https://bugs.launchpad.net/ubuntu-packaging-guide/+bug/702008

For more details, see:
https://code.launchpad.net/~mitya57/ubuntu-packaging-guide/add-python/+merge/117875

Added article "Packaging Python modules and apps" (python-packaging.rst).

Initially it was based on http://wiki.ubuntu.com/PackagingGuide/Python, but that page was very outdated, so I've rewritten it almost from scratch.
-- 
https://code.launchpad.net/~mitya57/ubuntu-packaging-guide/add-python/+merge/117875
Your team Ubuntu Packaging Guide Team is requested to review the proposed merge of lp:~mitya57/ubuntu-packaging-guide/add-python into lp:ubuntu-packaging-guide.

=== modified file 'debian/changelog'
--- debian/changelog	2012-07-08 06:59:21 +0000
+++ debian/changelog	2012-08-02 11:33:23 +0000
@@ -27,6 +27,9 @@
     process). (LP: #996096)
   * Refer to the backports process (LP: #1017984).
 
+  [ Dmitry Shachnev ]
+  * Add python-packaging.rst (LP: #702008)
+
  -- Daniel Holbach <daniel.holbach@xxxxxxxxxx>  Wed, 06 Jun 2012 09:28:00 +0200
 
 ubuntu-packaging-guide (0.1) quantal; urgency=low

=== modified file 'themes/ubuntu/layout.html'
--- themes/ubuntu/layout.html	2012-05-16 18:21:30 +0000
+++ themes/ubuntu/layout.html	2012-08-02 11:33:23 +0000
@@ -67,6 +67,7 @@
     <link rel="stylesheet" href="{{ pathto('_static/960.css', 1) }}" type="text/css" />
     <link rel="stylesheet" href="{{ pathto('_static/base.css', 1) }}" type="text/css" />
     <link rel="stylesheet" href="{{ pathto('_static/home.css', 1) }}" type="text/css" />
+    <link rel="stylesheet" href="{{ pathto('_static/pygments.css', 1) }}" type="text/css" />
     {%- for cssfile in css_files %}
     <link rel="stylesheet" href="{{ pathto(cssfile, 1) }}" type="text/css" />
     {%- endfor %}

=== added file 'themes/ubuntu/static/pygments.css'
--- themes/ubuntu/static/pygments.css	1970-01-01 00:00:00 +0000
+++ themes/ubuntu/static/pygments.css	2012-08-02 11:33:23 +0000
@@ -0,0 +1,62 @@
+.highlight .hll { background-color: #ffffcc }
+.highlight  { background: #f8f8f8; }
+.highlight .c { color: #408080; font-style: italic } /* Comment */
+.highlight .err { border: 1px solid #FF0000 } /* Error */
+.highlight .k { color: #008000; font-weight: bold } /* Keyword */
+.highlight .o { color: #666666 } /* Operator */
+.highlight .cm { color: #408080; font-style: italic } /* Comment.Multiline */
+.highlight .cp { color: #BC7A00 } /* Comment.Preproc */
+.highlight .c1 { color: #408080; font-style: italic } /* Comment.Single */
+.highlight .cs { color: #408080; font-style: italic } /* Comment.Special */
+.highlight .gd { color: #A00000 } /* Generic.Deleted */
+.highlight .ge { font-style: italic } /* Generic.Emph */
+.highlight .gr { color: #FF0000 } /* Generic.Error */
+.highlight .gh { color: #000080; font-weight: bold } /* Generic.Heading */
+.highlight .gi { color: #00A000 } /* Generic.Inserted */
+.highlight .go { color: #808080 } /* Generic.Output */
+.highlight .gp { color: #000080; font-weight: bold } /* Generic.Prompt */
+.highlight .gs { font-weight: bold } /* Generic.Strong */
+.highlight .gu { color: #800080; font-weight: bold } /* Generic.Subheading */
+.highlight .gt { color: #0040D0 } /* Generic.Traceback */
+.highlight .kc { color: #008000; font-weight: bold } /* Keyword.Constant */
+.highlight .kd { color: #008000; font-weight: bold } /* Keyword.Declaration */
+.highlight .kn { color: #008000; font-weight: bold } /* Keyword.Namespace */
+.highlight .kp { color: #008000 } /* Keyword.Pseudo */
+.highlight .kr { color: #008000; font-weight: bold } /* Keyword.Reserved */
+.highlight .kt { color: #B00040 } /* Keyword.Type */
+.highlight .m { color: #666666 } /* Literal.Number */
+.highlight .s { color: #BA2121 } /* Literal.String */
+.highlight .na { color: #7D9029 } /* Name.Attribute */
+.highlight .nb { color: #008000 } /* Name.Builtin */
+.highlight .nc { color: #0000FF; font-weight: bold } /* Name.Class */
+.highlight .no { color: #880000 } /* Name.Constant */
+.highlight .nd { color: #AA22FF } /* Name.Decorator */
+.highlight .ni { color: #999999; font-weight: bold } /* Name.Entity */
+.highlight .ne { color: #D2413A; font-weight: bold } /* Name.Exception */
+.highlight .nf { color: #0000FF } /* Name.Function */
+.highlight .nl { color: #A0A000 } /* Name.Label */
+.highlight .nn { color: #0000FF; font-weight: bold } /* Name.Namespace */
+.highlight .nt { color: #008000; font-weight: bold } /* Name.Tag */
+.highlight .nv { color: #19177C } /* Name.Variable */
+.highlight .ow { color: #AA22FF; font-weight: bold } /* Operator.Word */
+.highlight .w { color: #bbbbbb } /* Text.Whitespace */
+.highlight .mf { color: #666666 } /* Literal.Number.Float */
+.highlight .mh { color: #666666 } /* Literal.Number.Hex */
+.highlight .mi { color: #666666 } /* Literal.Number.Integer */
+.highlight .mo { color: #666666 } /* Literal.Number.Oct */
+.highlight .sb { color: #BA2121 } /* Literal.String.Backtick */
+.highlight .sc { color: #BA2121 } /* Literal.String.Char */
+.highlight .sd { color: #BA2121; font-style: italic } /* Literal.String.Doc */
+.highlight .s2 { color: #BA2121 } /* Literal.String.Double */
+.highlight .se { color: #BB6622; font-weight: bold } /* Literal.String.Escape */
+.highlight .sh { color: #BA2121 } /* Literal.String.Heredoc */
+.highlight .si { color: #BB6688; font-weight: bold } /* Literal.String.Interpol */
+.highlight .sx { color: #008000 } /* Literal.String.Other */
+.highlight .sr { color: #BB6688 } /* Literal.String.Regex */
+.highlight .s1 { color: #BA2121 } /* Literal.String.Single */
+.highlight .ss { color: #19177C } /* Literal.String.Symbol */
+.highlight .bp { color: #008000 } /* Name.Builtin.Pseudo */
+.highlight .vc { color: #19177C } /* Name.Variable.Class */
+.highlight .vg { color: #19177C } /* Name.Variable.Global */
+.highlight .vi { color: #19177C } /* Name.Variable.Instance */
+.highlight .il { color: #666666 } /* Literal.Number.Integer.Long */

=== modified file 'ubuntu-packaging-guide/index.rst'
--- ubuntu-packaging-guide/index.rst	2012-06-26 16:02:09 +0000
+++ ubuntu-packaging-guide/index.rst	2012-08-02 11:33:23 +0000
@@ -63,4 +63,5 @@
    udd-newpackage
    chroots
    traditional-packaging
+   python-packaging
    kde

=== added file 'ubuntu-packaging-guide/python-packaging.rst'
--- ubuntu-packaging-guide/python-packaging.rst	1970-01-01 00:00:00 +0000
+++ ubuntu-packaging-guide/python-packaging.rst	2012-08-02 11:33:23 +0000
@@ -0,0 +1,144 @@
+=========================================
+Packaging Python modules and applications
+=========================================
+
+Our packaging follows Debian’s `Python policy`_. We will use `python-markdown`_ package as an example, which can be downloaded from `PyPI`_. You can look at its packaging at its `Subversion repository`_.
+
+There are two types of Python packages — *modules* and *apps*.
+
+At the moment of writing this, Ubuntu has two incompatible versions of Python — *2.x* and *3.x*. ``/usr/bin/python`` is a symbolic link to a default Python 2.x version, and ``/usr/bin/python3`` — to a default Python 2.x version. Python modules should be built against all supported Python versions.
+
+If you are going to package a new Python module, you can find ``py2dsc`` tool useful (available in `python-stdeb`_ package).
+
+debian/control
+--------------
+
+Python 2.x and 3.x versions of the package should be in separate binary packages. Names should have ``python{,3}-modulename`` format (like: ``python3-dbus.mainloop.qt``). Here, we will use ``python-markdown`` and ``python3-markdown`` for module packages and ``python-markdown-doc`` for the documentation package.
+
+Things in ``debian/control`` that are specific for a Python package:
+
+- Section of module packages should be ``python``, of the documentation package — ``doc``. For an application, a single binary package will be enough.
+- We should add build dependencies on ``python-all (>= 2.6.6-3~)`` and ``python3-all (>= 3.1.2-7~)`` to make sure Python helpers are available (see the next section for details).
+- It’s recommended to add ``X-Python-Version`` and ``X-Python3-Version`` fields — see “`Specifying Supported Versions`_” section of the Policy for details. For example:
+  ::
+  
+    X-Python-Version: >= 2.6
+    X-Python3-Version: >= 3.1
+  
+  If your package works only with Python 2.x or 3.x, build depend only on one ``-all`` package and use only one ``-Version`` field.
+- Module packages should have ``{python:Depends}`` and ``{python3:Depends}`` substitution variables (respectively) in their dependency lists.
+
+debian/rules
+------------
+
+The recommended helpers for python modules are ``dh_python2`` and ``dh_python3``. Unfortunately, ``debhelper`` doesn’t yet build Python 3.x packages automatically (see `bug 597105`_ in Debian BTS), so we’ll need to do that manually in override sections (skip this if your package doesn’t support Python 3.x).
+
+Here’s our ``debian/rules`` file (with annotations):
+
+.. code-block:: makefile
+
+   # This command builds the list of supported Python 3 versions
+   PYTHON3=$(shell py3versions -r)
+   
+   %:
+       # Adding the required helpers
+       dh $@ --with python2,python3
+
+   override_dh_auto_clean:
+       dh_auto_clean
+       rm -rf build/
+   
+   override_dh_auto_build:
+       # Build for each Python 3 version
+       set -ex; for python in $(PYTHON3); do \
+           $$python setup.py build; \
+       done
+       dh_auto_build
+   
+   override_dh_auto_install:
+       # The same for install; note the --install-layout=deb option
+       set -ex; for python in $(PYTHON3); do \
+           $$python setup.py install --install-layout=deb --root=debian/tmp; \
+       done
+       dh_auto_install
+
+It is also a good practice to run tests during the build, if they are shipped by upstream. Usually tests can be invoked using ``setup.py test`` or ``setup.py check``.
+
+debian/\*.install
+-----------------
+
+Python 2.x modules are installed into ``/usr/share/pyshared/`` directory, and symbolic links are created ``/usr/lib/python2.x/dist-packages/`` for every interpreter version, while Python 3.x ones are all installed into ``/usr/lib/python3/dist-packages/``.
+
+If your package is an application and has private Python modules, they should be installed in ``/usr/share/module``, or ``/usr/lib/module`` if the modules are architecture-dependent (e.g. extensions) (see “`Programs Shipping Private Modules`_” section of the Policy).
+
+So, our ``python-markdown.install`` file will look like this (we’ll also want to install a ``markdown_py`` executable):
+
+::
+
+  usr/lib/python2.*/
+  usr/bin/
+
+and ``python3-markdown.install`` will only have one line:
+
+::
+
+  usr/lib/python3/
+
+The ``-doc`` package
+--------------------
+
+Tool that is most commonly used for building Python docs is `Sphinx`_. To add Sphinx documentation to your package (using ``dh_sphinxdoc`` helper), you should:
+
+* Add build-dependency on ``python-sphinx`` or ``python3-sphinx`` package (depending on what Python version do you want to use);
+* Append ``sphinxdoc`` to ``dh --with`` line;
+* Run ``setup.py build_sphinx`` in ``override_dh_auto_build`` (sometimes not needed);
+* Add ``{sphinxdoc:Depends}`` to dependency list of your ``-doc`` package;
+* Add path to built docs directory (usually ``build/sphinx/html``) to your ``.docs`` file.
+
+In our case, docs are automatically built in ``build/docs/`` directory (when we run ``setup.py build``), so we can simply put this in the ``python-markdown-doc.docs`` file:
+
+::
+
+  build/docs/
+
+Because docs also contain source ``.txt`` files, we’ll also tell ``dh_compress`` to not compress them — and add this to ``debian/rules``:
+
+.. code-block:: makefile
+
+   override_dh_compress:
+       dh_compress -X.txt
+
+Checking for packaging mistakes
+-------------------------------
+
+Along with ``lintian``, there is a special tool for checking Python packages — ``lintian4py``. It is available in `lintian4python`_ package. For example, this command invokes both ``lintian`` and ``lintian4py`` and checks source and binary packages:
+
+::
+
+  lintian{,4py} -EI --pedantic *.dsc *.deb
+
+Here, ``-EI`` option is used to enable experimental and informational tags.
+
+See also
+--------
+
+* The `Python policy`_;
+* `Python/Packaging`_ article on Debian wiki;
+* `Python/LibraryStyleGuide`_ and `Python/AppStyleGuide`_ articles on Debian wiki;
+* Debian `python-modules`_ and `python-apps`_ teams.
+
+.. _`Python policy`: http://www.debian.org/doc/packaging-manuals/python-policy/
+.. _`python-markdown`: http://packages.python.org/Markdown/
+.. _`PyPI`: http://pypi.python.org/pypi/Markdown/
+.. _`Subversion repository`: http://anonscm.debian.org/viewvc/python-modules/packages/python-markdown/trunk/debian/
+.. _`python-stdeb`: https://launchpad.net/ubuntu/+source/stdeb
+.. _`bug 597105`: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=597105
+.. _`Specifying Supported Versions`: http://www.debian.org/doc/packaging-manuals/python-policy/ch-module_packages.html#s-specifying_versions
+.. _`Programs Shipping Private Modules`: http://www.debian.org/doc/packaging-manuals/python-policy/ch-programs.html#s-current_version_progs
+.. _`Sphinx`: http://sphinx.pocoo.org/
+.. _`lintian4python`: https://launchpad.net/ubuntu/+source/lintian4python
+.. _`Python/Packaging`: http://wiki.debian.org/Python/Packaging
+.. _`Python/LibraryStyleGuide`: http://wiki.debian.org/Python/LibraryStyleGuide
+.. _`Python/AppStyleGuide`: http://wiki.debian.org/Python/AppStyleGuide
+.. _`python-modules`: http://wiki.debian.org/Teams/PythonModulesTeam/
+.. _`python-apps`: http://wiki.debian.org/Teams/PythonAppsPackagingTeam/