launchpad-reviewers team mailing list archive

[Colin Watson <cjwatson@xxxxxxxxxxxxx>]

Colin Watson has proposed merging ~cjwatson/launchpad:py3-readline-libedit-workaround into launchpad:master.

Commit message:
Work around readline crash in Python 3 on 18.04

Requested reviews:
  Launchpad code reviewers (launchpad-reviewers)

For more details, see:
https://code.launchpad.net/~cjwatson/launchpad/+git/launchpad/+merge/392028

On 18.04, importing GTK (which we do in lp.testing.html5browser) indirectly loads libedit, and versions of Python 3 before 3.8.1 crash when importing readline in a process that has libedit loaded (https://bugs.python.org/issue38634, https://bugs.launchpad.net/bugs/1899076).  As a result the test suite crashes quite early on.

There's unfortunately no reasonable way to avoid this, because we aren't loading libedit directly and nor are we importing readline directly; they're both quite deeply-buried side-effects of other things.  We have to take an unreasonable way instead: stub out the readline module in the test suite on affected systems.  This is deliberately kept as narrow as possible, and we should drop it if and when the Python bug is fixed in Ubuntu 18.04 or we no longer care about trying to get Launchpad to run on Python 3 on 18.04.
-- 
Your team Launchpad code reviewers is requested to review the proposed merge of ~cjwatson/launchpad:py3-readline-libedit-workaround into launchpad:master.

diff --git a/lib/lp/scripts/utilities/test.py b/lib/lp/scripts/utilities/test.py
index e1e67b8..0b5fa75 100755
--- a/lib/lp/scripts/utilities/test.py
+++ b/lib/lp/scripts/utilities/test.py
@@ -25,6 +25,7 @@ import sys
 import time
 import warnings
 
+import distro
 import six
 from zope.testrunner import options
 from zope.testrunner.feature import Feature
@@ -34,6 +35,7 @@ from lp.scripts.utilities import (
     importpedant,
     warninghandler,
     )
+from lp.services.compat import mock
 from lp.services.config import config
 
 
@@ -91,6 +93,21 @@ def configure_environment():
     # Suppress accessibility warning because the test runner does not have UI.
     os.environ['GTK_MODULES'] = ''
 
+    if distro.linux_distribution()[:2] == ('Ubuntu', '18.04') and six.PY3:
+        # XXX cjwatson 2020-10-09: Certain versions of Python crash when
+        # importing readline into a process that has libedit loaded
+        # (https://bugs.python.org/issue38634,
+        # https://bugs.launchpad.net/bugs/1899076), so stub out readline to
+        # prevent this.  This unfortunately makes debugging less pleasant.
+        #
+        # So far the only LTS version of Ubuntu that exhibits this behaviour
+        # is 18.04; 16.04 doesn't seem to end up loading libedit because its
+        # libGL is laid out differently in a way that doesn't end up loading
+        # libedit, and 20.04 has the Python bug fixed.  We should drop this
+        # once 18.04 is fixed or once we no longer care about it, since this
+        # workaround is pretty nasty.
+        sys.modules['readline'] = mock.Mock()
+
 
 def filter_warnings():
     # Silence spurious warnings. Note that this does not propagate to
diff --git a/setup.py b/setup.py
index e88b299..4e76a25 100644
--- a/setup.py
+++ b/setup.py
@@ -163,6 +163,7 @@ setup(
         'cssselect',
         'cssutils',
         'defusedxml',
+        'distro',
         'dkimpy[ed25519]',
         'dulwich',
         'feedparser',