launchpad-reviewers team mailing list archive

[Colin Watson <cjwatson@xxxxxxxxxxxxx>]

Colin Watson has proposed merging ~cjwatson/launchpad:fix-clean-build into launchpad:master.

Commit message:
Fix clean build without cached wheels

Requested reviews:
  Launchpad code reviewers (launchpad-reviewers)

For more details, see:
https://code.launchpad.net/~cjwatson/launchpad/+git/launchpad/+merge/378670

Upgrade to setuptools 44.0.0 and wheel 0.34.2.  setuptools now uses pip rather than easy_install to handle setup_requires; it still supports the easy_install find_links option in .pydistutils.cfg, but no longer needs or supports the allow_hosts option.

Install cffi earlier.  This is cumbersome, but it's needed because of conflicts with the system-installed python-cffi-backend package in system-site-packages mode; the recent pip upgrade caused us to trip over this if we didn't have a cached wheel for PyNaCl.

I've tested this (slowly) by removing download-cache/wheels/ before running a build from a clean tree.

Dependencies MP: https://code.launchpad.net/~cjwatson/lp-source-dependencies/+git/lp-source-dependencies/+merge/378669
-- 
Your team Launchpad code reviewers is requested to review the proposed merge of ~cjwatson/launchpad:fix-clean-build into launchpad:master.

diff --git a/Makefile b/Makefile
index b7374bc..828765b 100644
--- a/Makefile
+++ b/Makefile
@@ -245,7 +245,6 @@ $(PY): download-cache constraints.txt setup.py
 	rm -rf env
 	mkdir -p env
 	(echo '[easy_install]'; \
-	 echo "allow_hosts = ''"; \
 	 echo 'find_links = file://$(WD)/download-cache/dist/') \
 		>env/.pydistutils.cfg
 	$(VIRTUALENV) \
@@ -254,11 +253,11 @@ $(PY): download-cache constraints.txt setup.py
 		env
 	ln -sfn env/bin bin
 	$(SHHH) $(PIP) install $(PIP_INSTALL_ARGS) \
-		-r pip-requirements.txt
+		-r setup-requirements.txt
 	$(SHHH) LPCONFIG=$(LPCONFIG) $(PIP) \
 		--cache-dir=$(WD)/download-cache/ \
 		install $(PIP_INSTALL_ARGS) \
-		-c pip-requirements.txt -c constraints.txt -e . \
+		-c setup-requirements.txt -c constraints.txt -e . \
 		|| { code=$$?; rm -f $@; exit $$code; }
 	touch $@
 
diff --git a/constraints.txt b/constraints.txt
index 315e440..33b8872 100644
--- a/constraints.txt
+++ b/constraints.txt
@@ -74,14 +74,16 @@ BTrees==4.5.1
 persistent==4.4.3
 python-gettext==4.0
 pytz==2018.9
-setuptools==41.0.0
+# Handled in setup-requirements.txt instead.
+#setuptools==41.0.0
 six==1.12.0
 transaction==2.4.0
 
 # zope.password needs these
 bcrypt==3.1.6
-cffi==1.12.2
-pycparser==2.19
+# Handled in setup-requirements.txt instead.
+#cffi==1.12.2
+#pycparser==2.19
 
 # Python2-only
 zope.untrustedpython==4.0.0
@@ -312,7 +314,6 @@ waitress==1.3.1
 webencodings==0.5.1
 WebOb==1.8.5
 WebTest==2.0.33
-wheel==0.29.0
 wsgi-intercept==1.9.2
 WSGIProxy2==0.4.6
 wsgiref==0.1.2
diff --git a/doc/pip.txt b/doc/pip.txt
index 3774b0b..b570ab8 100644
--- a/doc/pip.txt
+++ b/doc/pip.txt
@@ -102,11 +102,12 @@ pip:
     information beyond these recipes, see the setuptools and distutils
     documentation.
 
-``pip-requirements.txt``
+``setup-requirements.txt``
     This is a `requirements file`_ used to upgrade pip itself to a
-    reasonably recent version.  We use this so that we aren't confined to
-    features of pip supported by the version supplied by the operating
-    system.
+    reasonably recent version and install a few other packages that must be
+    installed before installing anything else.  We use this so that we
+    aren't confined to features of pip supported by the version supplied by
+    the operating system.
 
 ``constraints.txt``
     This is a `constraints file`_ that specifies the precise versions of the
diff --git a/pip-requirements.txt b/pip-requirements.txt
deleted file mode 100644
index 9e1ff37..0000000
--- a/pip-requirements.txt
+++ /dev/null
@@ -1 +0,0 @@
-pip==20.0.2
diff --git a/setup-requirements.txt b/setup-requirements.txt
new file mode 100644
index 0000000..11ef58a
--- /dev/null
+++ b/setup-requirements.txt
@@ -0,0 +1,10 @@
+pip==20.0.2
+setuptools==44.0.0
+wheel==0.34.2
+
+# Install these early to avoid mismatches with a system-installed
+# python-cffi-backend package.
+# XXX cjwatson 2020-02-06: This would all be much easier if we could remove
+# the need for virtualenv --system-site-packages.
+cffi==1.12.2
+pycparser==2.19