launchpad-reviewers team mailing list archive
-
launchpad-reviewers team
-
Mailing list archive
-
Message #24831
[Merge] ~cjwatson/lp-mailman:merge-lp into lp-mailman:master
Colin Watson has proposed merging ~cjwatson/lp-mailman:merge-lp into lp-mailman:master.
Commit message:
Merge changes from Launchpad
Requested reviews:
Launchpad code reviewers (launchpad-reviewers)
For more details, see:
https://code.launchpad.net/~cjwatson/lp-mailman/+git/lp-mailman/+merge/385263
It's occasionally useful to merge Launchpad into lp-mailman, since there's still some common code and this lets lp-mailman pick up fixes.
Substantive changes introduced with this merge:
* Import xmlrpclib from six.moves.xmlrpc_client
* Import ifilter*/imap/izip* from six.moves
* Update README to refer to the Launchpad Git repo
* Upgrade to unittest2 1.1.0+lp1
* Build a wheelhouse with relocation-safe paths
* Port contrib.glock to Python 3 print functions
* Use next(iterator) rather than iterator.next()
* Port utilities to print_function
* Use open() rather than file()
* Remove pre-2.2 compatibility from contrib.glock
* Remove support for running Launchpad from bzr
* Port lp.services.log.logger to print_function
* Make format-imports compatible with Python
* Use /usr/bin/python2 rather than /usr/bin/python
* Upgrade ZTK packages to 2020-04-03 versions
* Use six for dict iteration
--
Your team Launchpad code reviewers is requested to review the proposed merge of ~cjwatson/lp-mailman:merge-lp into lp-mailman:master.
diff --git a/.gitignore b/.gitignore
index 6d58221..4eedbe6 100644
--- a/.gitignore
+++ b/.gitignore
@@ -34,3 +34,4 @@ run.gdb
callgrind.out.*
!logs/README.txt
/logs
+/wheelhouse
diff --git a/Makefile b/Makefile
index a50bdff..b5a3abb 100644
--- a/Makefile
+++ b/Makefile
@@ -16,15 +16,14 @@ PIP_ENV := LC_ALL=C.UTF-8
# be reviewed/merged/deployed.
PIP_NO_INDEX := 1
PIP_ENV += PIP_NO_INDEX=$(PIP_NO_INDEX)
-# Although --ignore-installed is slower, we need it to avoid confusion with
-# system-installed Python packages. If we ever manage to remove the need
-# for virtualenv --system-site-packages, then we can remove this too.
-PIP_ENV += PIP_IGNORE_INSTALLED=1
-PIP_ENV += PIP_FIND_LINKS=file://$(WD)/download-cache/dist/
+PIP_ENV += PIP_FIND_LINKS="file://$(WD)/wheelhouse/ file://$(WD)/download-cache/dist/"
VIRTUALENV := $(PIP_ENV) virtualenv
PIP := PYTHONPATH= $(PIP_ENV) env/bin/pip --cache-dir=$(WD)/download-cache/
+SITE_PACKAGES := \
+ $$(env/bin/python -c 'from distutils.sysconfig import get_python_lib; print(get_python_lib())')
+
TESTOPTS=
SHHH=utilities/shhh.py
@@ -89,10 +88,20 @@ endif
# This target is used by LOSAs to prepare a build to be pushed out to
# destination machines. We only want wheels: they are the expensive bits,
-# and the other bits might run into problems like bug 575037. This
-# target runs pip, and then removes everything created except for the
-# wheels.
-build_wheels: $(PIP_BIN) clean_pip
+# and the other bits might run into problems like bug 575037. This target
+# runs pip, builds a wheelhouse with predictable paths that can be used even
+# if the build is pushed to a different path on the destination machines,
+# and then removes everything created except for the wheels.
+#
+# It doesn't seem to be straightforward to build a wheelhouse of all our
+# dependencies without also building a useless wheel of lp-mailman itself;
+# fortunately that doesn't take too long, and we just remove it afterwards.
+build_wheels: $(PIP_BIN)
+ $(RM) -r wheelhouse
+ $(SHHH) $(PIP) wheel \
+ -c setup-requirements.txt -c constraints.txt -w wheelhouse .
+ $(RM) wheelhouse/lp_mailman-[0-9]*.whl
+ $(MAKE) clean_pip
# setuptools won't touch files that would have the same contents, but for
# Make's sake we need them to get fresh timestamps, so we touch them after
@@ -105,8 +114,9 @@ $(PY): download-cache constraints.txt setup.py
rm -rf env
mkdir -p env
$(VIRTUALENV) \
- --python=$(PYTHON) --system-site-packages --never-download \
+ --python=$(PYTHON) --never-download \
--extra-search-dir=$(WD)/download-cache/dist/ \
+ --extra-search-dir=$(WD)/wheelhouse/ \
env
ln -sfn env/bin bin
$(SHHH) $(PIP) install -r setup-requirements.txt
@@ -179,7 +189,7 @@ clean: clean_mailman clean_pip clean_logs
if test -f sourcecode/mailman/Makefile; then \
$(MAKE) -C sourcecode/mailman clean; \
fi
- $(RM) -r env
+ $(RM) -r env wheelhouse
$(RM) -r build
$(RM) $(VERSION_INFO)
$(RM) -r /var/tmp/lperr \
@@ -193,13 +203,13 @@ realclean: clean
TAGS: compile
# emacs tags
ctags -R -e --languages=-JavaScript --python-kinds=-i -f $@.new \
- $(CURDIR)/lib $(CURDIR)/env/lib/$(PYTHON)/site-packages
+ $(CURDIR)/lib "$(SITE_PACKAGES)"
mv $@.new $@
tags: compile
# vi tags
ctags -R --languages=-JavaScript --python-kinds=-i -f $@.new \
- $(CURDIR)/lib $(CURDIR)/env/lib/$(PYTHON)/site-packages
+ $(CURDIR)/lib "$(SITE_PACKAGES)"
mv $@.new $@
.PHONY: build_wheels check clean clean_logs clean_pip compile \
diff --git a/README b/README
index fd11bc0..899707a 100644
--- a/README
+++ b/README
@@ -30,10 +30,10 @@ There's a full guide for getting up-and-running with a development Launchpad
environment at <https://dev.launchpad.net/Getting>. When you are ready to
submit a patch, please consult <https://dev.launchpad.net/PatchSubmission>.
-Our bug tracker is at <https://bugs.launchpad.net/launchpad/> and you can get
+Our bug tracker is at <https://bugs.launchpad.net/lp-mailman/> and you can get
the source code any time by doing:
- $ bzr branch lp:launchpad
+ $ git clone https://git.launchpad.net/lp-mailman
Navigating the tree
diff --git a/buildmailman.py b/buildmailman.py
index 68beb86..62fc844 100644
--- a/buildmailman.py
+++ b/buildmailman.py
@@ -1,4 +1,4 @@
-#! /usr/bin/python
+#! /usr/bin/python2
#
# Copyright 2009, 2010 Canonical Ltd. This software is licensed under the
# GNU Affero General Public License version 3 (see the file LICENSE).
diff --git a/constraints.txt b/constraints.txt
index 86616ca..d19662a 100644
--- a/constraints.txt
+++ b/constraints.txt
@@ -1,88 +1,93 @@
# ztk-versions.cfg from ZTK a9eb2093b5 (2019-10-23), with some upgrades
# ---------------------------------------------------------------------
+# XXX cjwatson 2020-02-21: Cases where we've upgraded relative to
+# ztk-versions.cfg are denoted by the upstream version being commented out
+# and followed by the real version we want to use. This is a bit
+# cumbersome, and we should build this file dynamically instead.
+
zope.annotation==4.7.0
zope.applicationcontrol==4.2.0
zope.authentication==4.4.0
zope.browser==2.3
zope.browsermenu==4.4
-zope.browserpage==4.3.0
-zope.browserresource==4.3
+zope.browserpage==4.4.0
+zope.browserresource==4.4
zope.cachedescriptors==4.3.1
zope.catalog==4.2.1
-zope.component==4.5
+zope.component==4.6.1
zope.componentvocabulary==2.2.0
-zope.configuration==4.3.1
-zope.container==4.2.2
+zope.configuration==4.4.0
+zope.container==4.4.0
zope.contentprovider==4.2.1
-zope.contenttype==4.4
+zope.contenttype==4.5.0
zope.copy==4.2
zope.copypastemove==4.1.0
zope.datetime==4.2.0
-zope.deferredimport==4.3
+zope.deferredimport==4.3.1
zope.deprecation==4.4.0
zope.dottedname==4.3
zope.dublincore==4.2.0
zope.error==4.5.0
zope.event==4.4
zope.exceptions==4.3
-zope.filerepresentation==4.2.0
-zope.formlib==4.6.0
-zope.hookable==4.2.0
-zope.i18n==4.6.2
-zope.i18nmessageid==4.3.1
-zope.index==4.4.0
-zope.interface==4.6.0
+zope.filerepresentation==5.0.0
+zope.formlib==4.7.1
+zope.hookable==5.0.1
+zope.i18n==4.7.0
+zope.i18nmessageid==5.0.1
+zope.index==5.0.0
+zope.interface==5.0.2
zope.intid==4.3.0
zope.keyreference==4.2.0
zope.lifecycleevent==4.3
zope.location==4.2
zope.login==2.1.0
-zope.mimetype==2.4.0
+zope.mimetype==2.5.0
zope.minmax==2.2.0
-zope.pagetemplate==4.4.1
+zope.pagetemplate==4.5.0
zope.password==4.3.1
zope.pluggableauth==2.3.0
zope.principalannotation==4.3.0
zope.principalregistry==4.2.0
zope.processlifetime==2.3.0
-zope.proxy==4.3.1
+zope.proxy==4.3.5
zope.ptresource==4.2.0
-zope.publisher==5.0.1
+zope.publisher==5.2.0
zope.ramcache==2.3
-zope.schema==4.9.3
-zope.security==4.3.1
+zope.schema==6.0.0
+zope.security==5.1.1
zope.securitypolicy==4.3.1
zope.sendmail==5.0
#zope.session==4.3.0
# lp:~launchpad-committers/zope.session:launchpad
zope.session==4.3.0+lp1
-zope.site==4.2.2
+zope.site==4.3.0
zope.size==4.3
zope.structuredtext==4.3
zope.tal==4.4
-zope.tales==5.0
+zope.tales==5.0.2
zope.testing==4.7
-#zope.testrunner==5.0
+#zope.testrunner==5.1
# lp:~launchpad-committers/zope.testrunner:launchpad
zope.testrunner==5.1+lp2
-zope.traversing==4.3.1
+zope.traversing==4.4.1
zope.viewlet==4.2.1
# Direct dependencies
-BTrees==4.5.1
-persistent==4.4.3
+BTrees==4.7.1
+persistent==4.6.4
python-gettext==4.0
-pytz==2018.9
+pytz==2019.3
# Handled in setup-requirements.txt instead.
-#setuptools==41.0.0
-six==1.12.0
-transaction==2.4.0
+#setuptools==44.0.0
+six==1.14.0
+transaction==3.0.0
# zope.password needs these
-bcrypt==3.1.6
-cffi==1.12.2
-pycparser==2.19
+bcrypt==3.1.7
+cffi==1.14.0
+pycparser==2.20
# Python2-only
zope.untrustedpython==4.0.0
@@ -90,55 +95,60 @@ zope.untrustedpython==4.0.0
RestrictedPython==3.6.0
# Testing dependencies
-ZConfig==3.4.0
+ZConfig==3.5.0
ZODB==5.5.1
argparse==1.4.0
-colorama==0.4.1
+colorama==0.4.3
extras==1.0.0
fixtures==3.0.0
linecache2==1.0.0
manuel==1.10.1
-pbr==5.1.3
-pyparsing==2.4.0
+pbr==5.4.4
+pyparsing==2.4.6
python-mimeparse==1.6.0
-python-subunit==1.3.0
-testtools==2.3.0
+python-subunit==1.4.0
+testtools==2.4.0
traceback2==1.4.0
-unittest2==1.1.0
-zc.lockfile==1.4
+#unittest2==1.1.0
+# lp1 Set version directly in setup.py to fix wheel building.
+unittest2==1.1.0+lp1
+zc.lockfile==2.0
zdaemon==4.3
-zodbpickle==1.0.3
+zodbpickle==2.0.0
# Testing tools
-coverage==4.5.3
+coverage==5.0.4
nose==1.3.7
# Documentation dependencies
-# We have to keep a version < 2 to still support Python 2
Sphinx==1.8.5
-docutils==0.14
-imagesize==1.1.0
+docutils==0.16
+imagesize==1.2.0
alabaster==0.7.12
-babel==2.6.0
-Jinja2==2.10.1
+babel==2.8.0
+Jinja2==2.11.1
MarkupSafe==1.1.1
-Pygments==2.3.1
-snowballstemmer==1.2.1
-lxml==4.4.1
+Pygments==2.5.2
+snowballstemmer==2.0.0
+lxml==4.5.0
repoze.sphinx.autointerface==0.8
-#requests==2.21.0
-requests==2.22.0
-certifi==2019.3.9
-#urllib3==1.24.1
-urllib3==1.25.3
-idna==2.8
+requests==2.23.0
+certifi==2019.11.28
+urllib3==1.25.8
+idna==2.9
chardet==3.0.4
-sphinxcontrib-programoutput==0.14
-sphinxcontrib-websupport==1.1.0
+sphinxcontrib-applehelp==1.0.2
+sphinxcontrib-devhelp==1.0.2
+sphinxcontrib-htmlhelp==1.0.3
+sphinxcontrib-jsmath==1.0.1
+sphinxcontrib-programoutput==0.16
+sphinxcontrib-qthelp==1.0.3
+sphinxcontrib-serializinghtml==1.1.4
+sphinxcontrib-websupport==1.1.2
sphinx-rtd-theme==0.4.3
-packaging==19.0
-typing==3.6.6
+packaging==20.3
+typing==3.7.4.1
z3c.recipe.sphinxdoc==1.1.0
# ZTK buildout dependencies
@@ -146,9 +156,9 @@ collective.recipe.cmd==0.11
mr.developer==2.0.0
z3c.checkversions==1.1
z3c.recipe.compattest==1.0
-zc.buildout==2.13.1
+zc.buildout==2.13.3
zc.recipe.egg==2.0.7
-zc.recipe.testrunner==2.0.0
+zc.recipe.testrunner==2.1
# Launchpad
# ---------
diff --git a/lib/contrib/glock.py b/lib/contrib/glock.py
index 36a3061..d1ced09 100644
--- a/lib/contrib/glock.py
+++ b/lib/contrib/glock.py
@@ -20,6 +20,9 @@ Unix.
@see: class L{GlobalLock} for more details.
'''
+
+from __future__ import absolute_import, print_function
+
__version__ = '0.2.' + '$Revision: #5 $'[12:-2]
__author__ = 'Richard Gruet', 'rjgruet@xxxxxxxxx'
__date__ = '$Date: 2005/06/19 $'[7:-2], '$Author: rgruet $'[9:-2]
@@ -70,11 +73,6 @@ class LockAlreadyAcquired(GlobalLockError):
pass
-# Constants
-# ---------:
-if sys.version[:3] < '2.2':
- True, False = 1, 0 # built-in in Python 2.2+
-
#----------------------------------------------------------------------------
class GlobalLock:
#----------------------------------------------------------------------------
@@ -132,7 +130,7 @@ class GlobalLock:
self.acquire()
def __del__(self):
- #print '__del__ called' ##
+ #print('__del__ called') ##
try: self.release()
except: pass
if _windows:
@@ -185,7 +183,7 @@ class GlobalLock:
else:
raise GlobalLockError('Cannot acquire lock on "file" '
'%s: %s\n' % (self.name, message))
- #print 'got file lock.' ##
+ #print('got file lock.') ##
# Then acquire the local (inter-thread) lock:
if not self.threadLock.acquire(blocking):
@@ -194,7 +192,7 @@ class GlobalLock:
'someone else' % self.name)
if self.previous_lockfile_present and self.logger:
self.logger.warn("Stale lockfile detected and claimed.")
- #print 'got thread lock.' ##
+ #print('got thread lock.') ##
self.is_locked = True
@@ -229,7 +227,7 @@ class GlobalLock:
else:
try:
win32event.ReleaseMutex(self.mutex)
- #print "released mutex"
+ #print("released mutex")
except pywintypes.error as e:
errCode, fctName, errMsg = e.args
if errCode == 288:
@@ -264,7 +262,7 @@ def test():
#----------------------------------------------------------------------------
##TODO: a more serious test with distinct processes !
- print 'Testing glock.py...'
+ print('Testing glock.py...')
# unfortunately can't test inter-process lock here!
lockName = 'myFirstLock'
@@ -283,31 +281,31 @@ def test():
# Check that <> threads of same process do block:
import threading, time
thread = threading.Thread(target=threadMain, args=(l,))
- print 'main: locking...',
+ print('main: locking...', end='')
l.acquire()
- print ' done.'
+ print(' done.')
thread.start()
time.sleep(3)
- print '\nmain: unlocking...',
+ print('\nmain: unlocking...', end='')
l.release()
- print ' done.'
+ print(' done.')
time.sleep(0.1)
- print '=> Test of glock.py passed.'
+ print('=> Test of glock.py passed.')
return l
def threadMain(lock):
- print 'thread started(%s).' % lock
+ print('thread started(%s).' % lock)
try: lock.acquire(blocking=False)
except LockAlreadyAcquired: pass
else: raise Exception('should have raised LockAlreadyAcquired')
- print 'thread: locking (should stay blocked for ~ 3 sec)...',
+ print('thread: locking (should stay blocked for ~ 3 sec)...', end='')
lock.acquire()
- print 'thread: locking done.'
- print 'thread: unlocking...',
+ print('thread: locking done.')
+ print('thread: unlocking...', end='')
lock.release()
- print ' done.'
- print 'thread ended.'
+ print(' done.')
+ print('thread ended.')
#----------------------------------------------------------------------------
# M A I N
diff --git a/lib/lp/scripts/utilities/importpedant.py b/lib/lp/scripts/utilities/importpedant.py
index e2f38ab..b08da9b 100644
--- a/lib/lp/scripts/utilities/importpedant.py
+++ b/lib/lp/scripts/utilities/importpedant.py
@@ -1,4 +1,4 @@
-# Copyright 2009-2016 Canonical Ltd. This software is licensed under the
+# Copyright 2009-2020 Canonical Ltd. This software is licensed under the
# GNU Affero General Public License version 3 (see the file LICENSE).
from __future__ import absolute_import, print_function, unicode_literals
diff --git a/lib/lp/scripts/utilities/warninghandler.py b/lib/lp/scripts/utilities/warninghandler.py
index 6acc9ce..bd167c8 100644
--- a/lib/lp/scripts/utilities/warninghandler.py
+++ b/lib/lp/scripts/utilities/warninghandler.py
@@ -11,6 +11,8 @@ import StringIO
import sys
import warnings
+import six
+
class WarningReport:
@@ -44,7 +46,7 @@ def report_other_warnings():
if other_warnings:
print(file=sys.stderr)
print("General warnings.", file=sys.stderr)
- for warninginfo in other_warnings.itervalues():
+ for warninginfo in six.itervalues(other_warnings):
print(file=sys.stderr)
print(warninginfo, file=sys.stderr)
diff --git a/lib/lp/services/config/__init__.py b/lib/lp/services/config/__init__.py
index 79fdab8..6bdc4b9 100644
--- a/lib/lp/services/config/__init__.py
+++ b/lib/lp/services/config/__init__.py
@@ -60,8 +60,8 @@ def find_instance_name():
if instance_name is None:
for config_lookup_file in CONFIG_LOOKUP_FILES:
if os.path.exists(config_lookup_file):
- instance_name = file(
- config_lookup_file, 'r').read()[:80].strip()
+ with open(config_lookup_file) as f:
+ instance_name = f.read()[:80].strip()
break
# Of instance_name falls back for developers.
diff --git a/lib/lp/services/log/logger.py b/lib/lp/services/log/logger.py
index 2c52e5b..4327e9a 100644
--- a/lib/lp/services/log/logger.py
+++ b/lib/lp/services/log/logger.py
@@ -3,6 +3,8 @@
"""Loggers."""
+from __future__ import absolute_import, print_function
+
__metaclass__ = type
__all__ = [
'BufferLogger',
@@ -110,7 +112,7 @@ class FakeLogger:
else:
output_file = self.output_file
prefix = LEVEL_PREFIXES.get(level, "%d>" % level)
- print >> output_file, prefix, self._format_message(msg, *stuff)
+ print(prefix, self._format_message(msg, *stuff), file=output_file)
if 'exc_info' in kw:
traceback.print_exc(file=output_file)
diff --git a/lib/lp/services/mailman/scripts/mlist_sync.py b/lib/lp/services/mailman/scripts/mlist_sync.py
index 80018f2..d52a436 100644
--- a/lib/lp/services/mailman/scripts/mlist_sync.py
+++ b/lib/lp/services/mailman/scripts/mlist_sync.py
@@ -1,4 +1,4 @@
-#!/usr/bin/python -S
+#!/usr/bin/python2 -S
# Copyright 2009-2019 Canonical Ltd. This software is licensed under the
# GNU Affero General Public License version 3 (see the file LICENSE).
diff --git a/lib/lp/services/tests/test_xmlrpc.py b/lib/lp/services/tests/test_xmlrpc.py
index 3a143f5..bd04f90 100644
--- a/lib/lp/services/tests/test_xmlrpc.py
+++ b/lib/lp/services/tests/test_xmlrpc.py
@@ -12,7 +12,7 @@ from lp.testing import TestCase
class TestTransport(TestCase):
- """Test code that allows xmlrpclib.ServerProxy to have a socket timeout"""
+ """Test code that allows ServerProxy to have a socket timeout."""
def test_default_initialization(self):
transport = Transport()
diff --git a/lib/lp/services/webapp/errorlog.py b/lib/lp/services/webapp/errorlog.py
index bb6d6c7..4846d9e 100644
--- a/lib/lp/services/webapp/errorlog.py
+++ b/lib/lp/services/webapp/errorlog.py
@@ -180,7 +180,7 @@ class ErrorReportingUtility:
:param message: Unicode message.
:returns: Key for this message.
"""
- key = self._oops_message_key_iter.next()
+ key = next(self._oops_message_key_iter)
self._oops_messages[key] = message
return key
diff --git a/lib/lp/services/xmlrpc.py b/lib/lp/services/xmlrpc.py
index dde6103..9eff891 100644
--- a/lib/lp/services/xmlrpc.py
+++ b/lib/lp/services/xmlrpc.py
@@ -10,15 +10,15 @@ __all__ = [
]
import socket
-import xmlrpclib
from defusedxml.xmlrpc import monkey_patch
+from six.moves import xmlrpc_client
# Protect against various XML parsing vulnerabilities.
monkey_patch()
-class LaunchpadFault(xmlrpclib.Fault):
+class LaunchpadFault(xmlrpc_client.Fault):
"""Base class for a Launchpad XMLRPC fault.
Subclasses should define a unique error_code and a msg_template,
@@ -34,7 +34,7 @@ class LaunchpadFault(xmlrpclib.Fault):
assert self.msg_template is not None, (
"Subclasses must define msg_template.")
msg = self.msg_template % kw
- xmlrpclib.Fault.__init__(self, self.error_code, msg)
+ xmlrpc_client.Fault.__init__(self, self.error_code, msg)
def __eq__(self, other):
if not isinstance(other, LaunchpadFault):
@@ -47,19 +47,19 @@ class LaunchpadFault(xmlrpclib.Fault):
return not (self == other)
-class Transport(xmlrpclib.Transport):
- """An xmlrpclib transport that supports a timeout argument.
+class Transport(xmlrpc_client.Transport):
+ """An xmlrpc_client transport that supports a timeout argument.
- Use by passing into the "transport" argument of the xmlrpclib.ServerProxy
- initialization.
+ Use by passing into the "transport" argument of the
+ xmlrpc_client.ServerProxy initialization.
"""
def __init__(self,
use_datetime=0, timeout=socket._GLOBAL_DEFAULT_TIMEOUT):
- xmlrpclib.Transport.__init__(self, use_datetime)
+ xmlrpc_client.Transport.__init__(self, use_datetime)
self.timeout = timeout
def make_connection(self, host):
- conn = xmlrpclib.Transport.make_connection(self, host)
+ conn = xmlrpc_client.Transport.make_connection(self, host)
conn.timeout = self.timeout
return conn
diff --git a/lib/lp/testing/__init__.py b/lib/lp/testing/__init__.py
index f9824ce..687e4e8 100644
--- a/lib/lp/testing/__init__.py
+++ b/lib/lp/testing/__init__.py
@@ -19,6 +19,7 @@ import subprocess
import fixtures
import lp_sitecustomize
+import six
import testtools
from testtools.matchers import (
Equals,
@@ -148,13 +149,13 @@ def monkey_patch(context, **kwargs):
"""
old_values = {}
not_set = object()
- for name, value in kwargs.iteritems():
+ for name, value in six.iteritems(kwargs):
old_values[name] = getattr(context, name, not_set)
setattr(context, name, value)
try:
yield
finally:
- for name, value in old_values.iteritems():
+ for name, value in six.iteritems(old_values):
if value is not_set:
delattr(context, name)
else:
diff --git a/lib/lp/testing/utilities/retest.py b/lib/lp/testing/utilities/retest.py
index 172f536..46906bc 100755
--- a/lib/lp/testing/utilities/retest.py
+++ b/lib/lp/testing/utilities/retest.py
@@ -25,11 +25,13 @@ report (or a part of) can be piped in, for example by pasting it:
from __future__ import print_function
import fileinput
-from itertools import takewhile, imap
+from itertools import takewhile
import os
import re
import sys
+from six.moves import map as imap
+
from lp.services.config import config
diff --git a/scripts/mlist-sync.py b/scripts/mlist-sync.py
index ee19b90..0161190 100755
--- a/scripts/mlist-sync.py
+++ b/scripts/mlist-sync.py
@@ -1,4 +1,4 @@
-#!/usr/bin/python -S
+#!/usr/bin/python2 -S
# Copyright 2009-2019 Canonical Ltd. This software is licensed under the
# GNU Affero General Public License version 3 (see the file LICENSE).
diff --git a/scripts/update-version-info.sh b/scripts/update-version-info.sh
index f925832..54e777e 100755
--- a/scripts/update-version-info.sh
+++ b/scripts/update-version-info.sh
@@ -1,24 +1,27 @@
#!/bin/bash
#
-# Copyright 2009-2016 Canonical Ltd. This software is licensed under the
+# Copyright 2009-2020 Canonical Ltd. This software is licensed under the
# GNU Affero General Public License version 3 (see the file LICENSE).
#
# Update version-info.py -- but only if the revision number has
# changed
-#
newfile=version-info-${RANDOM}.py
-if [ -e .git ]; then
- if ! which git > /dev/null || ! test -x $(which git); then
- echo "No working 'git' executable found" >&2
- exit 1
- fi
+if [ ! -e .git ]; then
+ echo "Not in a Git working tree" >&2
+ exit 1
+fi
- branch_nick="$(git rev-parse --abbrev-ref HEAD | sed "s/'/\\\\'/g")"
- revision_id="$(git rev-parse HEAD)"
- date="$(git show -s --format=%ci HEAD)"
- cat > $newfile <<EOF
+if ! which git > /dev/null || ! test -x $(which git); then
+ echo "No working 'git' executable found" >&2
+ exit 1
+fi
+
+branch_nick="$(git rev-parse --abbrev-ref HEAD | sed "s/'/\\\\'/g")"
+revision_id="$(git rev-parse HEAD)"
+date="$(git show -s --format=%ci HEAD)"
+cat > $newfile <<EOF
#! /usr/bin/env python
from __future__ import print_function
@@ -32,33 +35,17 @@ version_info = {
if __name__ == '__main__':
print('revision id: %(revision_id)s' % version_info)
EOF
-elif [ -d .bzr ]; then
- if ! which bzr > /dev/null || ! test -x $(which bzr); then
- echo "No working 'bzr' executable found" >&2
- exit 1
- fi
-
- bzr version-info --format=python > $newfile 2>/dev/null
-else
- echo "Not in a Git or Bazaar working tree" >&2
- exit 1
-fi
revision_id=$(python $newfile | sed -n 's/^revision id: //p')
if ! [ -f version-info.py ]; then
echo "Creating version-info.py at revision $revision_id"
mv ${newfile} version-info.py
else
- # Here we compare the actual output instead of the contents of the
- # file because bzr includes a build-date that is actually updated
- # every time you run bzr version-info.
- newcontents=$(python $newfile)
- oldcontents=$(python version-info.py)
- if [ "$newcontents" != "$oldcontents" ]; then
- echo "Updating version-info.py to revision $revision_id"
- mv ${newfile} version-info.py
- else
+ if cmp -s version-info.py "$newfile"; then
echo "Skipping version-info.py update; already at revision $revision_id"
rm ${newfile}
+ else
+ echo "Updating version-info.py to revision $revision_id"
+ mv ${newfile} version-info.py
fi
fi
diff --git a/setup.py b/setup.py
index 90cc9a0..676bc73 100644
--- a/setup.py
+++ b/setup.py
@@ -146,8 +146,7 @@ setup(
'fixtures',
'lazr.config',
'lazr.enum',
- # Pin version for now to avoid confusion with system site-packages.
- 'mock==1.0.1',
+ 'mock',
'oops',
'oops_amqp',
'oops_datedir_repo',
diff --git a/utilities/find-changed-files.sh b/utilities/find-changed-files.sh
index 159ebd4..bdaedd2 100755
--- a/utilities/find-changed-files.sh
+++ b/utilities/find-changed-files.sh
@@ -1,6 +1,6 @@
#!/bin/bash
#
-# Copyright 2009-2019 Canonical Ltd. This software is licensed under the
+# Copyright 2009-2020 Canonical Ltd. This software is licensed under the
# GNU Affero General Public License version 3 (see the file LICENSE).
#
# Determine the changed files in the working tree, or if the working tree is
@@ -9,59 +9,23 @@
set -e
set -o pipefail
-if [ -e .git ]; then
- git_diff_files() {
- git diff --name-only -z $@ | perl -l -0 -ne '
- # Only show paths that exist and are not symlinks.
- print if -e and not -l'
- }
+if [ ! -e .git ]; then
+ echo "Not in a Git working tree" >&2
+ exit 1
+fi
- files=$(git_diff_files HEAD)
- if [ -z "$files" ]; then
- # git doesn't give us a way to track the parent branch, so just use
- # master by default and let the user override that using a
- # positional argument.
- files=$(git_diff_files "${1:-master}")
- fi
-elif [ -d .bzr ]; then
- bzr() {
- # PYTHONPATH may point to the ./lib directory in the launchpad tree.
- # This directory includes a bzrlib. When this script calls bzr, we
- # want it to use the system bzrlib, not the one in the launchpad
- # tree.
- PYTHONPATH='' `which bzr` "$@"
- }
+git_diff_files() {
+ git diff --name-only -z $@ | perl -l -0 -ne '
+ # Only show paths that exist and are not symlinks.
+ print if -e and not -l'
+}
- diff_status=0
- bzr diff > /dev/null || diff_status=$?
- if [ $diff_status -eq 0 ] ; then
- # No uncommitted changes in the tree.
- if bzr status | grep -q "^Current thread:"; then
- # This is a loom, lint changes relative to the lower thread.
- rev_option="-r thread:"
- elif [ "$(bzr pipes | sed -n -e "/^\\*/q;p" | wc -l)" -gt 0 ]; then
- # This is a pipeline with at least one pipe before the
- # current, lint changes relative to the previous pipe
- rev_option="-r ancestor::prev"
- else
- # Lint changes relative to the parent.
- rev=`bzr info | sed \
- '/parent branch:/!d; s/ *parent branch: /ancestor:/'`
- rev_option="-r $rev"
- fi
- elif [ $diff_status -eq 1 ] ; then
- # Uncommitted changes in the tree, return those files.
- rev_option=""
- else
- # bzr diff failed
- exit 1
- fi
- # Extract filename from status line. Skip symlinks.
- files=`bzr st --short $rev_option |
- sed -e '/^.[MN]/!d; s/.* //' -e '/@$/d'`
-else
- echo "Not in a Git or Bazaar working tree" >&2
- exit 1
+files=$(git_diff_files HEAD)
+if [ -z "$files" ]; then
+ # git doesn't give us a way to track the parent branch, so just use
+ # master by default and let the user override that using a
+ # positional argument.
+ files=$(git_diff_files "${1:-master}")
fi
echo $files
diff --git a/utilities/format-imports b/utilities/format-imports
index 043b91c..18d9403 100755
--- a/utilities/format-imports
+++ b/utilities/format-imports
@@ -1,4 +1,4 @@
-#!/usr/bin/python
+#!/usr/bin/python2
#
# Copyright 2010-2012 Canonical Ltd. This software is licensed under the
# GNU Affero General Public License version 3 (see the file LICENSE).
@@ -125,6 +125,8 @@ is over the length limit.
}}}
"""
+from __future__ import absolute_import, print_function
+
__metaclass__ = type
# SKIP this file when reformatting.
@@ -301,7 +303,7 @@ def format_imports(imports):
thirdparty_section = {}
local_section = {}
# Group modules into sections.
- for module, statement in imports.iteritems():
+ for module, statement in imports.items():
module_base = module_base_regex.findall(module)[0]
comment = statement.comment
if module_base == '_pythonpath':
@@ -342,7 +344,8 @@ def format_imports(imports):
def reformat_importsection(filename):
"""Replace the given file with a reformatted version of it."""
- pyfile = file(filename).read()
+ with open(filename) as f:
+ pyfile = f.read()
import_start, import_end = find_imports_section(pyfile)
if import_start is None:
# Skip files with no import section.
@@ -377,7 +380,7 @@ def process_file(fpath):
"""Process the file with the given path."""
changed = reformat_importsection(fpath)
if changed:
- print fpath
+ print(fpath)
def process_tree(dpath):
diff --git a/utilities/link-external-sourcecode b/utilities/link-external-sourcecode
index d35ef02..0a2bb90 100755
--- a/utilities/link-external-sourcecode
+++ b/utilities/link-external-sourcecode
@@ -1,4 +1,4 @@
-#!/usr/bin/python
+#!/usr/bin/python2
#
# Copyright 2009-2017 Canonical Ltd. This software is licensed under the GNU
# Affero General Public License version 3 (see the file LICENSE).
diff --git a/utilities/run-as b/utilities/run-as
index aee679d..27c14b5 100755
--- a/utilities/run-as
+++ b/utilities/run-as
@@ -1,4 +1,4 @@
-#! /usr/bin/python
+#! /usr/bin/python2
#
# Copyright 2017 Canonical Ltd. This software is licensed under the
# GNU Affero General Public License version 3 (see the file LICENSE).
diff --git a/utilities/shhh.py b/utilities/shhh.py
index 6b9c1f6..63ce9ed 100755
--- a/utilities/shhh.py
+++ b/utilities/shhh.py
@@ -1,4 +1,4 @@
-#! /usr/bin/python -S
+#! /usr/bin/python2 -S
#
# Copyright 2009-2017 Canonical Ltd. This software is licensed under the
# GNU Affero General Public License version 3 (see the file LICENSE).
@@ -48,7 +48,7 @@ def shhh(cmd):
>>> shhh_script(cmd)
('', '', 1)
- >>> cmd = [python, "-c", "import sys; print 666; sys.exit(42)"]
+ >>> cmd = [python, "-c", "import sys; print(666); sys.exit(42)"]
>>> shhh(cmd)
666
42
@@ -57,7 +57,9 @@ def shhh(cmd):
>>> cmd = [
... python, "-c",
- ... "import sys; print 666; print >> sys.stderr, 667; sys.exit(42)",
+ ... "from __future__ import print_function; "
+ ... "import sys; "
+ ... "print(666); print(667, file=sys.stderr); sys.exit(42)",
... ]
>>> shhh_script(cmd)
('666\n', '667\n', 42)
diff --git a/utilities/update-copyright b/utilities/update-copyright
index c2e0236..dcca9db 100755
--- a/utilities/update-copyright
+++ b/utilities/update-copyright
@@ -1,6 +1,6 @@
-#!/usr/bin/python
+#!/usr/bin/python2
#
-# Copyright 2010-2013 Canonical Ltd. This software is licensed under the
+# Copyright 2010-2020 Canonical Ltd. This software is licensed under the
# GNU Affero General Public License version 3 (see the file LICENSE).
"""Update the year in copyright notices.
@@ -10,6 +10,8 @@ notice to reflect the current year. Looks for the notice in the first three
lines of the file and leaves the file unchanged if it finds none.
"""
+from __future__ import absolute_import, print_function
+
from datetime import date
import os
import re
@@ -49,27 +51,30 @@ def update_files(filenames):
"""Open the files with the given file names and update them."""
for filename in filenames:
if not os.path.isfile(filename):
- print "Skipped: %s does not exist or is not a regular file." %(
- filename)
+ print("Skipped: %s does not exist or is not a regular file." %(
+ filename))
continue
if not os.access(filename, os.W_OK):
- print "Skipped: %s is not writeable." % filename
+ print("Skipped: %s is not writeable." % filename)
continue
- lines = file(filename).readlines()
+ with open(filename) as f:
+ lines = f.readlines()
changed = update_copyright(lines)
if changed:
newfile = open(filename, 'w')
newfile.write(''.join(lines))
newfile.close()
- print "Updated: %s" % filename
+ print("Updated: %s" % filename)
else:
- print "Unchanged: %s" % filename
+ print("Unchanged: %s" % filename)
def find_changed_files():
"""Use the find-changed-files.sh script."""
find_changed_files_cmd = [
os.path.join(UTILITIES_DIR, 'find-changed-files.sh')]
- filenames = Popen(find_changed_files_cmd, stdout=PIPE).communicate()[0]
+ filenames = Popen(
+ find_changed_files_cmd, stdout=PIPE,
+ universal_newlines=True).communicate()[0]
return filenames.strip()
def find_and_update():
diff --git a/utilities/update-sourcecode b/utilities/update-sourcecode
index 4b2f644..a10eb01 100755
--- a/utilities/update-sourcecode
+++ b/utilities/update-sourcecode
@@ -1,4 +1,4 @@
-#!/usr/bin/python -u
+#!/usr/bin/python2 -u
#
# Copyright 2009 Canonical Ltd. This software is licensed under the
# GNU Affero General Public License version 3 (see the file LICENSE).
