launchpad-reviewers team mailing list archive
-
launchpad-reviewers team
-
Mailing list archive
-
Message #26656
[Merge] ~cjwatson/launchpad:wsgi-ppa-auth into launchpad:master
Colin Watson has proposed merging ~cjwatson/launchpad:wsgi-ppa-auth into launchpad:master.
Commit message:
Add a WSGI authenticator for private PPAs
Requested reviews:
Launchpad code reviewers (launchpad-reviewers)
For more details, see:
https://code.launchpad.net/~cjwatson/launchpad/+git/launchpad/+merge/399617
I got sufficiently annoyed in the process of fixing bug 1722209 to see how hard it would be to fix it properly, since the existing htpasswd scheme has been a thorn in our side for a while now. The answer appears to be "a bit, but not very".
This will let us remove our reliance on htpasswd files, and probably eventually make it easier to do things like issue time-limited tokens (or even macaroons?) to builders.
There are obviously various deployment issues to sort out, and generate-ppa-htaccess still deals with things like deactivation and cancellation emails which we'll need to move elsewhere; it may be possible to just inline the relevant checks into the DB queries in ArchiveAuthTokenSet, since we're always looking at a single subscriber or named token name and so the result set sizes are very small.
The way that mod_wsgi loads the entry point is peculiar. I considered separating the client code off entirely from Launchpad, but this turns out to be light enough in terms of memory use and startup time that I don't think it's worth the effort at the moment.
We would have to start doing a graceful Apache restart on code upgrades. Is this acceptable?
One remaining worry is that it's not obvious how to deal with a migration to Python 3. This code should be fine either on Python 2 or 3, but the libapache-mod-wsgi and libapache-mod-wsgi-py3 packages conflict, which would make it awkward to deal with the actual upgrade without downtime. It's possible that we might need to put off using this until we've finished moving to Python 3, but I think this upgrade plan might work too:
* Before deploying the switch to Python 3, change WSGIAuthUserScript to refer to a path that will remain Python 2 across the upgrade, and do a graceful Apache restart. At this point, libapache2-mod-wsgi is installed.
* Deploy a Python 3 tree. Other parts of Launchpad will start running on Python 3 at this point, but the WSGI authenticator will temporarily remain on Python 2. Make sure a Python 2 tree stays around.
* Change WSGIAuthUserScript to refer to the Python 3 tree. Don't restart Apache yet.
* Install libapache2-mod-wsgi-py3. I *believe* this will do a full (not graceful) Apache restart in its postinst, so a small amount of downtime is likely and we'd need to announce a downtime window in advance.
* If we need to roll back, then reverse this procedure, again incurring downtime.
I also considered using other authentication mechanisms, but haven't found anything that's clearly suitable. The only RADIUS module that appears to be still somewhat maintained is mod_auth_radius, but its caching strategy is to set a client-side cookie, which isn't suitable for PPAs where the client is typically apt, and it doesn't seem to support mod_authn_socache. I suppose we could try to implement enough of the LDAP protocol to satisfy mod_authnz_ldap, but that sounds pretty grim. None of the other authentication modules I could find (at least as packaged in xenial) seem to have a very attractive model for this application. The best plan B I can think of would be to use mod_authnz_external in combination with mod_authn_socache; I think that would work, and it would avoid relying on a matching Python version running in Apache processes, but for both performance and security reasons I'm not sure I love using external authenticator processes here. I'm open to alternative suggestions.
This is essentially the same as https://code.launchpad.net/~cjwatson/launchpad/wsgi-ppa-auth/+merge/332125, converted to git, rebased on master, and generally modernized a bit.
--
Your team Launchpad code reviewers is requested to review the proposed merge of ~cjwatson/launchpad:wsgi-ppa-auth into launchpad:master.
diff --git a/Makefile b/Makefile
index 5779007..49ced64 100644
--- a/Makefile
+++ b/Makefile
@@ -489,6 +489,7 @@ copy-apache-config: codehosting-dir
base=local-launchpad; \
fi; \
sed -e 's,%BRANCH_REWRITE%,$(shell pwd)/scripts/branch-rewrite.py,' \
+ -e 's,%WSGI_ARCHIVE_AUTH%,$(shell pwd)/scripts/wsgi-archive-auth.py,' \
-e 's,%LISTEN_ADDRESS%,$(LISTEN_ADDRESS),' \
configs/$(LPCONFIG)/local-launchpad-apache > \
/etc/apache2/sites-available/$$base
diff --git a/configs/development/local-launchpad-apache b/configs/development/local-launchpad-apache
index bc81928..f15d1fb 100644
--- a/configs/development/local-launchpad-apache
+++ b/configs/development/local-launchpad-apache
@@ -133,7 +133,6 @@
<VirtualHost %LISTEN_ADDRESS%:80>
ServerName ppa.launchpad.test
- ServerAlias private-ppa.launchpad.test
LogLevel debug
DocumentRoot /var/tmp/ppa
@@ -146,8 +145,36 @@
Deny from all
Allow from 127.0.0.0/255.0.0.0
</IfVersion>
- AllowOverride AuthConfig
+ AllowOverride None
+ Options Indexes
+ </Directory>
+</VirtualHost>
+
+<VirtualHost %LISTEN_ADDRESS%:80>
+ ServerName private-ppa.launchpad.test
+ LogLevel debug
+
+ DocumentRoot /var/tmp/ppa
+ <Directory /var/tmp/ppa/>
+ <IfVersion >= 2.4>
+ <RequireAll>
+ Require ip 127.0.0.0/255.0.0.0
+ Require valid-user
+ </RequireAll>
+ </IfVersion>
+ <IfVersion < 2.4>
+ Order Deny,Allow
+ Deny from all
+ Allow from 127.0.0.0/255.0.0.0
+ Require valid-user
+ Satisfy All
+ </IfVersion>
+ AllowOverride None
Options Indexes
+ AuthType Basic
+ AuthName "Token Required"
+ AuthBasicProvider wsgi
+ WSGIAuthUserScript %WSGI_ARCHIVE_AUTH% application-group=lp
</Directory>
</VirtualHost>
diff --git a/lib/lp/services/config/schema-lazr.conf b/lib/lp/services/config/schema-lazr.conf
index fe28a8c..d542f4a 100644
--- a/lib/lp/services/config/schema-lazr.conf
+++ b/lib/lp/services/config/schema-lazr.conf
@@ -1419,6 +1419,11 @@ signing_keys_root: /var/tmp/ppa-signing-keys
# datatype: boolean
require_signing_keys: false
+# The URL to the internal archive API endpoint. This should implement
+# IArchiveAPI.
+# datatype: string
+archive_api_endpoint: http://xmlrpc-private.launchpad.test:8087/archive
+
[ppa_apache_log_parser]
logs_root: /srv/ppa.launchpad.net-logs
diff --git a/lib/lp/soyuz/doc/archiveauthtoken.txt b/lib/lp/soyuz/doc/archiveauthtoken.txt
index 2b0fecd..c6c3fa3 100644
--- a/lib/lp/soyuz/doc/archiveauthtoken.txt
+++ b/lib/lp/soyuz/doc/archiveauthtoken.txt
@@ -140,12 +140,35 @@ It's also possible to retrieve a set of all the tokens for an archive.
... print(token.person.name)
bradsmith
-Tokens can also be retreived by archive and person:
+Tokens can also be retrieved by archive and person:
>>> print(token_set.getActiveTokenForArchiveAndPerson(
... new_token.archive, new_token.person).token)
testtoken
+Or by archive and person name:
+
+ >>> print(token_set.getActiveTokenForArchiveAndPersonName(
+ ... new_token.archive, "bradsmith").token)
+ testtoken
+
+Tokens are only returned if they match a current subscription:
+
+ >>> from zope.security.proxy import removeSecurityProxy
+ >>> from lp.soyuz.enums import ArchiveSubscriberStatus
+ >>> removeSecurityProxy(subscription_to_joe_private_ppa).status = (
+ ... ArchiveSubscriberStatus.EXPIRED)
+
+ >>> print(token_set.getActiveTokenForArchiveAndPerson(
+ ... new_token.archive, new_token.person))
+ None
+ >>> print(token_set.getActiveTokenForArchiveAndPersonName(
+ ... new_token.archive, "bradsmith"))
+ None
+
+ >>> removeSecurityProxy(subscription_to_joe_private_ppa).status = (
+ ... ArchiveSubscriberStatus.CURRENT)
+
== Amending Tokens ==
diff --git a/lib/lp/soyuz/interfaces/archiveapi.py b/lib/lp/soyuz/interfaces/archiveapi.py
new file mode 100644
index 0000000..f54b958
--- /dev/null
+++ b/lib/lp/soyuz/interfaces/archiveapi.py
@@ -0,0 +1,46 @@
+# Copyright 2017-2021 Canonical Ltd. This software is licensed under the
+# GNU Affero General Public License version 3 (see the file LICENSE).
+
+"""Interfaces for internal archive APIs."""
+
+from __future__ import absolute_import, print_function, unicode_literals
+
+__metaclass__ = type
+__all__ = [
+ 'IArchiveAPI',
+ 'IArchiveApplication',
+ ]
+
+from zope.interface import Interface
+
+from lp.services.webapp.interfaces import ILaunchpadApplication
+
+
+class IArchiveApplication(ILaunchpadApplication):
+ """Archive application root."""
+
+
+class IArchiveAPI(Interface):
+ """The Soyuz archive XML-RPC interface to Launchpad.
+
+ Published at "archive" on the private XML-RPC server.
+
+ PPA frontends use this to check archive authorization tokens.
+ """
+
+ def checkArchiveAuthToken(archive_reference, username, password):
+ """Check an archive authorization token.
+
+ :param archive_reference: The reference form of the archive to check.
+ :param username: The username sent using HTTP Basic Authentication;
+ this should either be a `Person.name` or "+" followed by the
+ name of a named authorization token.
+ :param password: The password sent using HTTP Basic Authentication;
+ this should be a corresponding `ArchiveAuthToken.token`.
+
+ :returns: A `NotFound` fault if `archive_reference` does not
+ identify an archive or the username does not identify a valid
+ token for this archive; an `Unauthorized` fault if the password
+ is not equal to the selected token for this archive; otherwise
+ None.
+ """
diff --git a/lib/lp/soyuz/interfaces/archiveauthtoken.py b/lib/lp/soyuz/interfaces/archiveauthtoken.py
index 18069f4..d8156f4 100644
--- a/lib/lp/soyuz/interfaces/archiveauthtoken.py
+++ b/lib/lp/soyuz/interfaces/archiveauthtoken.py
@@ -1,4 +1,4 @@
-# Copyright 2009 Canonical Ltd. This software is licensed under the
+# Copyright 2009-2021 Canonical Ltd. This software is licensed under the
# GNU Affero General Public License version 3 (see the file LICENSE).
"""ArchiveAuthToken interface."""
@@ -96,23 +96,33 @@ class IArchiveAuthTokenSet(Interface):
:return: An object conforming to `IArchiveAuthToken`.
"""
- def getByArchive(archive):
+ def getByArchive(archive, valid=False):
"""Retrieve all the tokens for an archive.
:param archive: The context archive.
+ :param valid: If True, only return valid tokens.
:return: A result set containing `IArchiveAuthToken`s.
"""
def getActiveTokenForArchiveAndPerson(archive, person):
- """Retrieve an active token for the given archive and person.
+ """Retrieve a valid active token for the given archive and person.
:param archive: The archive to which the token corresponds.
:param person: The person to which the token corresponds.
:return: An `IArchiveAuthToken` or None.
"""
+ def getActiveTokenForArchiveAndPersonName(archive, person_name):
+ """Retrieve a valid active token for the given archive and person name.
+
+ :param archive: The archive to which the token corresponds.
+ :param person_name: The name of the person to which the token
+ corresponds.
+ :return: An `IArchiveAuthToken` or None.
+ """
+
def getActiveNamedTokenForArchive(archive, name):
- """Retrieve an active named token for the given archive and name.
+ """Retrieve a valid active named token for the given archive and name.
:param archive: The archive to which the token corresponds.
:param name: The name of a named authorization token.
@@ -120,9 +130,9 @@ class IArchiveAuthTokenSet(Interface):
"""
def getActiveNamedTokensForArchive(archive, names=None):
- """Retrieve a subset of active named tokens for the given archive if
- `names` is specified, or all active named tokens for the archive if
- `names` is null.
+ """Retrieve a subset of valid active named tokens for the given
+ archive if `names` is specified, or all valid active named tokens
+ for the archive if `names` is null.
:param archive: The archive to which the tokens correspond.
:param names: An optional list of token names.
diff --git a/lib/lp/soyuz/model/archiveauthtoken.py b/lib/lp/soyuz/model/archiveauthtoken.py
index 8033b31..f4f7deb 100644
--- a/lib/lp/soyuz/model/archiveauthtoken.py
+++ b/lib/lp/soyuz/model/archiveauthtoken.py
@@ -1,4 +1,4 @@
-# Copyright 2009 Canonical Ltd. This software is licensed under the
+# Copyright 2009-2021 Canonical Ltd. This software is licensed under the
# GNU Affero General Public License version 3 (see the file LICENSE).
"""Database class for table ArchiveAuthToken."""
@@ -21,8 +21,10 @@ from storm.locals import (
from storm.store import Store
from zope.interface import implementer
+from lp.registry.model.teammembership import TeamParticipation
from lp.services.database.constants import UTC_NOW
from lp.services.database.interfaces import IStore
+from lp.soyuz.enums import ArchiveSubscriberStatus
from lp.soyuz.interfaces.archiveauthtoken import (
IArchiveAuthToken,
IArchiveAuthTokenSet,
@@ -85,19 +87,37 @@ class ArchiveAuthTokenSet:
return IStore(ArchiveAuthToken).find(
ArchiveAuthToken, ArchiveAuthToken.token == token).one()
- def getByArchive(self, archive):
+ def getByArchive(self, archive, valid=False):
"""See `IArchiveAuthTokenSet`."""
+ # Circular import.
+ from lp.soyuz.model.archivesubscriber import ArchiveSubscriber
store = Store.of(archive)
- return store.find(
- ArchiveAuthToken,
+ clauses = [
ArchiveAuthToken.archive == archive,
- ArchiveAuthToken.date_deactivated == None)
+ ArchiveAuthToken.date_deactivated == None,
+ ]
+ if valid:
+ clauses.extend([
+ ArchiveAuthToken.archive_id == ArchiveSubscriber.archive_id,
+ ArchiveSubscriber.status == ArchiveSubscriberStatus.CURRENT,
+ ArchiveSubscriber.subscriber_id == TeamParticipation.teamID,
+ TeamParticipation.personID == ArchiveAuthToken.person_id,
+ ])
+ return store.find(ArchiveAuthToken, *clauses)
def getActiveTokenForArchiveAndPerson(self, archive, person):
"""See `IArchiveAuthTokenSet`."""
- return self.getByArchive(archive).find(
+ return self.getByArchive(archive, valid=True).find(
ArchiveAuthToken.person == person).one()
+ def getActiveTokenForArchiveAndPersonName(self, archive, person_name):
+ """See `IArchiveAuthTokenSet`."""
+ # Circular import.
+ from lp.registry.model.person import Person
+ return self.getByArchive(archive, valid=True).find(
+ ArchiveAuthToken.person == Person.id,
+ Person.name == person_name).one()
+
def getActiveNamedTokenForArchive(self, archive, name):
"""See `IArchiveAuthTokenSet`."""
return self.getByArchive(archive).find(
diff --git a/lib/lp/soyuz/wsgi/__init__.py b/lib/lp/soyuz/wsgi/__init__.py
new file mode 100644
index 0000000..e69de29
--- /dev/null
+++ b/lib/lp/soyuz/wsgi/__init__.py
diff --git a/lib/lp/soyuz/wsgi/archiveauth.py b/lib/lp/soyuz/wsgi/archiveauth.py
new file mode 100644
index 0000000..fd34161
--- /dev/null
+++ b/lib/lp/soyuz/wsgi/archiveauth.py
@@ -0,0 +1,78 @@
+# Copyright 2017-2021 Canonical Ltd. This software is licensed under the
+# GNU Affero General Public License version 3 (see the file LICENSE).
+
+"""WSGI archive authorisation provider.
+
+This is as lightweight as possible, as it runs on PPA frontends.
+"""
+
+from __future__ import absolute_import, print_function, unicode_literals
+
+__metaclass__ = type
+__all__ = [
+ 'check_password',
+ ]
+
+import crypt
+from random import SystemRandom
+import string
+import time
+
+import six
+from six.moves.xmlrpc_client import (
+ Fault,
+ ServerProxy,
+ )
+
+from lp.services.config import config
+from lp.services.memcache.client import memcache_client_factory
+
+
+def _get_archive_reference(environ):
+ # Reconstruct the relevant part of the URL. We don't care about where
+ # we're installed.
+ path = six.ensure_text(environ.get("SCRIPT_NAME") or "/", "ISO-8859-1")
+ path_info = six.ensure_text(environ.get("PATH_INFO", ""), "ISO-8859-1")
+ path += (path_info if path else path_info[1:])
+ # Extract the first three segments of the path, and rearrange them to
+ # form an archive reference.
+ path_parts = path.lstrip("/").split("/")
+ if len(path_parts) >= 3:
+ return "~%s/%s/%s" % (path_parts[0], path_parts[2], path_parts[1])
+
+
+_sr = SystemRandom()
+
+
+def _crypt_sha256(word):
+ """crypt.crypt(word, crypt.METHOD_SHA256), backported from Python 3.5."""
+ saltchars = string.ascii_letters + string.digits + "./"
+ salt = "$5$" + "".join(_sr.choice(saltchars) for _ in range(16))
+ return crypt.crypt(word, salt)
+
+
+_memcache_client = memcache_client_factory(timeline=False)
+
+
+def check_password(environ, user, password):
+ archive_reference = _get_archive_reference(environ)
+ if archive_reference is None:
+ return None
+ memcache_key = "archive-auth:%s:%s" % (archive_reference, user)
+ crypted_password = _memcache_client.get(memcache_key)
+ if (crypted_password and
+ crypt.crypt(password, crypted_password) == crypted_password):
+ return True
+ proxy = ServerProxy(config.personalpackagearchive.archive_api_endpoint)
+ try:
+ proxy.checkArchiveAuthToken(archive_reference, user, password)
+ # Cache positive responses for a minute to reduce database load.
+ _memcache_client.set(
+ memcache_key, _crypt_sha256(password), time.time() + 60)
+ return True
+ except Fault as e:
+ if e.faultCode == 410: # Unauthorized
+ return False
+ else:
+ # Interpret any other fault as NotFound (320).
+ return None
diff --git a/lib/lp/soyuz/wsgi/tests/__init__.py b/lib/lp/soyuz/wsgi/tests/__init__.py
new file mode 100644
index 0000000..e69de29
--- /dev/null
+++ b/lib/lp/soyuz/wsgi/tests/__init__.py
diff --git a/lib/lp/soyuz/wsgi/tests/test_archiveauth.py b/lib/lp/soyuz/wsgi/tests/test_archiveauth.py
new file mode 100644
index 0000000..a41f9d7
--- /dev/null
+++ b/lib/lp/soyuz/wsgi/tests/test_archiveauth.py
@@ -0,0 +1,149 @@
+# Copyright 2017-2021 Canonical Ltd. This software is licensed under the
+# GNU Affero General Public License version 3 (see the file LICENSE).
+
+"""Tests for the WSGI archive authorisation provider."""
+
+from __future__ import absolute_import, print_function, unicode_literals
+
+__metaclass__ = type
+
+import crypt
+import os.path
+import subprocess
+import time
+
+from fixtures import MonkeyPatch
+import transaction
+
+from lp.services.config import config
+from lp.services.memcache.testing import MemcacheFixture
+from lp.soyuz.wsgi import archiveauth
+from lp.testing import TestCaseWithFactory
+from lp.testing.layers import ZopelessAppServerLayer
+from lp.xmlrpc import faults
+
+
+class TestWSGIArchiveAuth(TestCaseWithFactory):
+
+ layer = ZopelessAppServerLayer
+
+ def setUp(self):
+ super(TestWSGIArchiveAuth, self).setUp()
+ self.now = time.time()
+ self.useFixture(MonkeyPatch("time.time", lambda: self.now))
+ self.memcache_fixture = self.useFixture(MemcacheFixture())
+ # The WSGI provider doesn't use Zope, so we can't rely on the
+ # fixture substituting a Zope utility.
+ self.useFixture(MonkeyPatch(
+ "lp.soyuz.wsgi.archiveauth._memcache_client",
+ self.memcache_fixture))
+
+ def test_get_archive_reference_short_url(self):
+ self.assertIsNone(archiveauth._get_archive_reference(
+ {"SCRIPT_NAME": "/foo"}))
+
+ def test_get_archive_reference_archive_base(self):
+ self.assertEqual(
+ "~user/ubuntu/ppa",
+ archiveauth._get_archive_reference(
+ {"SCRIPT_NAME": "/user/ppa/ubuntu"}))
+
+ def test_get_archive_reference_inside_archive(self):
+ self.assertEqual(
+ "~user/ubuntu/ppa",
+ archiveauth._get_archive_reference(
+ {"SCRIPT_NAME": "/user/ppa/ubuntu/dists"}))
+
+ def test_check_password_short_url(self):
+ self.assertIsNone(archiveauth.check_password(
+ {"SCRIPT_NAME": "/foo"}, "user", ""))
+ self.assertEqual({}, self.memcache_fixture._cache)
+
+ def test_check_password_not_found(self):
+ self.assertIsNone(archiveauth.check_password(
+ {"SCRIPT_NAME": "/nonexistent/bad/unknown"}, "user", ""))
+ self.assertEqual({}, self.memcache_fixture._cache)
+
+ def test_crypt_sha256(self):
+ crypted_password = archiveauth._crypt_sha256("secret")
+ self.assertEqual(
+ crypted_password, crypt.crypt("secret", crypted_password))
+
+ def makeArchiveAndToken(self):
+ archive = self.factory.makeArchive(private=True)
+ archive_path = "/%s/%s/ubuntu" % (archive.owner.name, archive.name)
+ subscriber = self.factory.makePerson()
+ archive.newSubscription(subscriber, archive.owner)
+ token = archive.newAuthToken(subscriber)
+ transaction.commit()
+ return archive, archive_path, subscriber.name, token.token
+
+ def test_check_password_unauthorized(self):
+ _, archive_path, username, password = self.makeArchiveAndToken()
+ # Test that this returns False, not merely something falsy (e.g.
+ # None).
+ self.assertIs(
+ False,
+ archiveauth.check_password(
+ {"SCRIPT_NAME": archive_path}, username, password + "-bad"))
+ self.assertEqual({}, self.memcache_fixture._cache)
+
+ def test_check_password_success(self):
+ archive, archive_path, username, password = self.makeArchiveAndToken()
+ self.assertIs(
+ True,
+ archiveauth.check_password(
+ {"SCRIPT_NAME": archive_path}, username, password))
+ crypted_password = self.memcache_fixture.get(
+ "archive-auth:%s:%s" % (archive.reference, username))
+ self.assertEqual(
+ crypted_password, crypt.crypt(password, crypted_password))
+
+ def test_check_password_considers_cache(self):
+ class FakeProxy:
+ def __init__(self, uri):
+ pass
+
+ def checkArchiveAuthToken(self, archive_reference, username,
+ password):
+ raise faults.Unauthorized()
+
+ _, archive_path, username, password = self.makeArchiveAndToken()
+ self.assertIs(
+ True,
+ archiveauth.check_password(
+ {"SCRIPT_NAME": archive_path}, username, password))
+ self.useFixture(
+ MonkeyPatch("lp.soyuz.wsgi.archiveauth.ServerProxy", FakeProxy))
+ # A subsequent check honours the cache.
+ self.assertIs(
+ False,
+ archiveauth.check_password(
+ {"SCRIPT_NAME": archive_path}, username, password + "-bad"))
+ self.assertIs(
+ True,
+ archiveauth.check_password(
+ {"SCRIPT_NAME": archive_path}, username, password))
+ # If we advance time far enough, then the cached result expires.
+ self.now += 60
+ self.assertIs(
+ False,
+ archiveauth.check_password(
+ {"SCRIPT_NAME": archive_path}, username, password))
+
+ def test_script(self):
+ _, archive_path, username, password = self.makeArchiveAndToken()
+ script_path = os.path.join(
+ config.root, "scripts", "wsgi-archive-auth.py")
+
+ def check_via_script(archive_path, username, password):
+ with open(os.devnull, "w") as devnull:
+ return subprocess.call(
+ [script_path, archive_path, username, password],
+ stderr=devnull)
+
+ self.assertEqual(0, check_via_script(archive_path, username, password))
+ self.assertEqual(
+ 1, check_via_script(archive_path, username, password + "-bad"))
+ self.assertEqual(
+ 2, check_via_script("/nonexistent/bad/unknown", "user", ""))
diff --git a/lib/lp/soyuz/xmlrpc/archive.py b/lib/lp/soyuz/xmlrpc/archive.py
new file mode 100644
index 0000000..6050c8c
--- /dev/null
+++ b/lib/lp/soyuz/xmlrpc/archive.py
@@ -0,0 +1,62 @@
+# Copyright 2017-2021 Canonical Ltd. This software is licensed under the
+# GNU Affero General Public License version 3 (see the file LICENSE).
+
+"""Implementations of the XML-RPC APIs for Soyuz archives."""
+
+from __future__ import absolute_import, print_function, unicode_literals
+
+__metaclass__ = type
+__all__ = [
+ 'ArchiveAPI',
+ ]
+
+from zope.component import getUtility
+from zope.interface import implementer
+from zope.security.proxy import removeSecurityProxy
+
+from lp.services.webapp import LaunchpadXMLRPCView
+from lp.soyuz.interfaces.archive import IArchiveSet
+from lp.soyuz.interfaces.archiveapi import IArchiveAPI
+from lp.soyuz.interfaces.archiveauthtoken import IArchiveAuthTokenSet
+from lp.xmlrpc import faults
+from lp.xmlrpc.helpers import return_fault
+
+
+BUILDD_USER_NAME = "buildd"
+
+
+@implementer(IArchiveAPI)
+class ArchiveAPI(LaunchpadXMLRPCView):
+ """See `IArchiveAPI`."""
+
+ @return_fault
+ def _checkArchiveAuthToken(self, archive_reference, username, password):
+ archive = getUtility(IArchiveSet).getByReference(archive_reference)
+ if archive is None:
+ raise faults.NotFound(
+ message="No archive found for '%s'." % archive_reference)
+ archive = removeSecurityProxy(archive)
+ token_set = getUtility(IArchiveAuthTokenSet)
+ if username == BUILDD_USER_NAME:
+ secret = archive.buildd_secret
+ else:
+ if username.startswith("+"):
+ token = token_set.getActiveNamedTokenForArchive(
+ archive, username[1:])
+ else:
+ token = token_set.getActiveTokenForArchiveAndPersonName(
+ archive, username)
+ if token is None:
+ raise faults.NotFound(
+ message="No valid tokens for '%s' in '%s'." % (
+ username, archive_reference))
+ secret = removeSecurityProxy(token).token
+ if password != secret:
+ raise faults.Unauthorized()
+
+ def checkArchiveAuthToken(self, archive_reference, username, password):
+ """See `IArchiveAPI`."""
+ # This thunk exists because you can't use a decorated function as
+ # the implementation of a method exported over XML-RPC.
+ return self._checkArchiveAuthToken(
+ archive_reference, username, password)
diff --git a/lib/lp/soyuz/xmlrpc/tests/__init__.py b/lib/lp/soyuz/xmlrpc/tests/__init__.py
new file mode 100644
index 0000000..e69de29
--- /dev/null
+++ b/lib/lp/soyuz/xmlrpc/tests/__init__.py
diff --git a/lib/lp/soyuz/xmlrpc/tests/test_archive.py b/lib/lp/soyuz/xmlrpc/tests/test_archive.py
new file mode 100644
index 0000000..71c9150
--- /dev/null
+++ b/lib/lp/soyuz/xmlrpc/tests/test_archive.py
@@ -0,0 +1,124 @@
+# Copyright 2017-2021 Canonical Ltd. This software is licensed under the
+# GNU Affero General Public License version 3 (see the file LICENSE).
+
+"""Tests for the internal Soyuz archive API."""
+
+from __future__ import absolute_import, print_function, unicode_literals
+
+__metaclass__ = type
+
+from zope.security.proxy import removeSecurityProxy
+
+from lp.services.features.testing import FeatureFixture
+from lp.soyuz.interfaces.archive import NAMED_AUTH_TOKEN_FEATURE_FLAG
+from lp.soyuz.xmlrpc.archive import ArchiveAPI
+from lp.testing import TestCaseWithFactory
+from lp.testing.layers import LaunchpadFunctionalLayer
+from lp.xmlrpc import faults
+
+
+class TestArchiveAPI(TestCaseWithFactory):
+
+ layer = LaunchpadFunctionalLayer
+
+ def setUp(self):
+ super(TestArchiveAPI, self).setUp()
+ self.useFixture(FeatureFixture({NAMED_AUTH_TOKEN_FEATURE_FLAG: "on"}))
+ self.archive_api = ArchiveAPI(None, None)
+
+ def assertNotFound(self, archive_reference, username, password, message):
+ """Assert that an archive auth token check returns NotFound."""
+ fault = self.archive_api.checkArchiveAuthToken(
+ archive_reference, username, password)
+ self.assertEqual(faults.NotFound(message), fault)
+
+ def assertUnauthorized(self, archive_reference, username, password):
+ """Assert that an archive auth token check returns Unauthorized."""
+ fault = self.archive_api.checkArchiveAuthToken(
+ archive_reference, username, password)
+ self.assertEqual(faults.Unauthorized("Authorisation required."), fault)
+
+ def test_checkArchiveAuthToken_unknown_archive(self):
+ self.assertNotFound(
+ "~nonexistent/unknown/bad", "user", "",
+ "No archive found for '~nonexistent/unknown/bad'.")
+
+ def test_checkArchiveAuthToken_no_tokens(self):
+ archive = removeSecurityProxy(self.factory.makeArchive(private=True))
+ self.assertNotFound(
+ archive.reference, "nobody", "",
+ "No valid tokens for 'nobody' in '%s'." % archive.reference)
+
+ def test_checkArchiveAuthToken_no_named_tokens(self):
+ archive = removeSecurityProxy(self.factory.makeArchive(private=True))
+ self.assertNotFound(
+ archive.reference, "+missing", "",
+ "No valid tokens for '+missing' in '%s'." % archive.reference)
+
+ def test_checkArchiveAuthToken_buildd_wrong_password(self):
+ archive = removeSecurityProxy(self.factory.makeArchive(private=True))
+ self.assertUnauthorized(
+ archive.reference, "buildd", archive.buildd_secret + "-bad")
+
+ def test_checkArchiveAuthToken_buildd_correct_password(self):
+ archive = removeSecurityProxy(self.factory.makeArchive(private=True))
+ self.assertIsNone(self.archive_api.checkArchiveAuthToken(
+ archive.reference, "buildd", archive.buildd_secret))
+
+ def test_checkArchiveAuthToken_named_token_wrong_password(self):
+ archive = removeSecurityProxy(self.factory.makeArchive(private=True))
+ token = archive.newNamedAuthToken("special")
+ removeSecurityProxy(token).deactivate()
+ self.assertNotFound(
+ archive.reference, "+special", token.token,
+ "No valid tokens for '+special' in '%s'." % archive.reference)
+
+ def test_checkArchiveAuthToken_named_token_deactivated(self):
+ archive = removeSecurityProxy(self.factory.makeArchive(private=True))
+ token = archive.newNamedAuthToken("special")
+ self.assertIsNone(self.archive_api.checkArchiveAuthToken(
+ archive.reference, "+special", token.token))
+
+ def test_checkArchiveAuthToken_named_token_correct_password(self):
+ archive = removeSecurityProxy(self.factory.makeArchive(private=True))
+ token = archive.newNamedAuthToken("special")
+ self.assertIsNone(self.archive_api.checkArchiveAuthToken(
+ archive.reference, "+special", token.token))
+
+ def test_checkArchiveAuthToken_personal_token_wrong_password(self):
+ archive = removeSecurityProxy(self.factory.makeArchive(private=True))
+ subscriber = self.factory.makePerson()
+ archive.newSubscription(subscriber, archive.owner)
+ token = archive.newAuthToken(subscriber)
+ self.assertUnauthorized(
+ archive.reference, subscriber.name, token.token + "-bad")
+
+ def test_checkArchiveAuthToken_personal_token_deactivated(self):
+ archive = removeSecurityProxy(self.factory.makeArchive(private=True))
+ subscriber = self.factory.makePerson()
+ archive.newSubscription(subscriber, archive.owner)
+ token = archive.newAuthToken(subscriber)
+ removeSecurityProxy(token).deactivate()
+ self.assertNotFound(
+ archive.reference, subscriber.name, token.token,
+ "No valid tokens for '%s' in '%s'." % (
+ subscriber.name, archive.reference))
+
+ def test_checkArchiveAuthToken_personal_token_cancelled(self):
+ archive = removeSecurityProxy(self.factory.makeArchive(private=True))
+ subscriber = self.factory.makePerson()
+ subscription = archive.newSubscription(subscriber, archive.owner)
+ token = archive.newAuthToken(subscriber)
+ removeSecurityProxy(subscription).cancel(archive.owner)
+ self.assertNotFound(
+ archive.reference, subscriber.name, token.token,
+ "No valid tokens for '%s' in '%s'." % (
+ subscriber.name, archive.reference))
+
+ def test_checkArchiveAuthToken_personal_token_correct_password(self):
+ archive = removeSecurityProxy(self.factory.makeArchive(private=True))
+ subscriber = self.factory.makePerson()
+ archive.newSubscription(subscriber, archive.owner)
+ token = archive.newAuthToken(subscriber)
+ self.assertIsNone(self.archive_api.checkArchiveAuthToken(
+ archive.reference, subscriber.name, token.token))
diff --git a/lib/lp/systemhomes.py b/lib/lp/systemhomes.py
index f886587..84babd1 100644
--- a/lib/lp/systemhomes.py
+++ b/lib/lp/systemhomes.py
@@ -1,4 +1,4 @@
-# Copyright 2009-2017 Canonical Ltd. This software is licensed under the
+# Copyright 2009-2021 Canonical Ltd. This software is licensed under the
# GNU Affero General Public License version 3 (see the file LICENSE).
"""Content classes for the 'home pages' of the subsystems of Launchpad."""
@@ -62,6 +62,7 @@ from lp.services.webapp.interfaces import (
from lp.services.webapp.publisher import canonical_url
from lp.services.webservice.interfaces import IWebServiceApplication
from lp.services.worlddata.interfaces.language import ILanguageSet
+from lp.soyuz.interfaces.archiveapi import IArchiveApplication
from lp.testopenid.interfaces.server import ITestOpenIDApplication
from lp.translations.interfaces.translationgroup import ITranslationGroupSet
from lp.translations.interfaces.translations import IRosettaApplication
@@ -70,6 +71,12 @@ from lp.translations.interfaces.translationsoverview import (
)
+@implementer(IArchiveApplication)
+class ArchiveApplication:
+
+ title = "Archive API"
+
+
@implementer(ICodehostingApplication)
class CodehostingApplication:
"""Codehosting End-Point."""
diff --git a/lib/lp/xmlrpc/application.py b/lib/lp/xmlrpc/application.py
index a84706b..20d6dbf 100644
--- a/lib/lp/xmlrpc/application.py
+++ b/lib/lp/xmlrpc/application.py
@@ -1,4 +1,4 @@
-# Copyright 2009-2015 Canonical Ltd. This software is licensed under the
+# Copyright 2009-2021 Canonical Ltd. This software is licensed under the
# GNU Affero General Public License version 3 (see the file LICENSE).
"""XML-RPC API to the application roots."""
@@ -30,11 +30,12 @@ from lp.services.authserver.interfaces import IAuthServerApplication
from lp.services.features.xmlrpc import IFeatureFlagApplication
from lp.services.webapp import LaunchpadXMLRPCView
from lp.services.webapp.interfaces import ILaunchBag
+from lp.soyuz.interfaces.archiveapi import IArchiveApplication
from lp.xmlrpc.interfaces import IPrivateApplication
# NOTE: If you add a traversal here, you should update
-# the regular expression in utilities/page-performance-report.ini
+# the regular expression in lp:lp-dev-utils page-performance-report.ini.
@implementer(IPrivateApplication)
class PrivateApplication:
@@ -44,6 +45,11 @@ class PrivateApplication:
return getUtility(IMailingListApplication)
@property
+ def archive(self):
+ """See `IPrivateApplication`."""
+ return getUtility(IArchiveApplication)
+
+ @property
def authserver(self):
"""See `IPrivateApplication`."""
return getUtility(IAuthServerApplication)
diff --git a/lib/lp/xmlrpc/configure.zcml b/lib/lp/xmlrpc/configure.zcml
index 47a3100..d387eab 100644
--- a/lib/lp/xmlrpc/configure.zcml
+++ b/lib/lp/xmlrpc/configure.zcml
@@ -1,4 +1,4 @@
-<!-- Copyright 2009-2015 Canonical Ltd. This software is licensed under the
+<!-- Copyright 2009-2021 Canonical Ltd. This software is licensed under the
GNU Affero General Public License version 3 (see the file LICENSE).
-->
@@ -22,6 +22,19 @@
/>
<securedutility
+ class="lp.systemhomes.ArchiveApplication"
+ provides="lp.soyuz.interfaces.archiveapi.IArchiveApplication">
+ <allow interface="lp.soyuz.interfaces.archiveapi.IArchiveApplication"/>
+ </securedutility>
+
+ <xmlrpc:view
+ for="lp.soyuz.interfaces.archiveapi.IArchiveApplication"
+ interface="lp.soyuz.interfaces.archiveapi.IArchiveAPI"
+ class="lp.soyuz.xmlrpc.archive.ArchiveAPI"
+ permission="zope.Public"
+ />
+
+ <securedutility
class="lp.systemhomes.CodehostingApplication"
provides="lp.code.interfaces.codehosting.ICodehostingApplication">
<allow interface="lp.code.interfaces.codehosting.ICodehostingApplication"/>
diff --git a/lib/lp/xmlrpc/interfaces.py b/lib/lp/xmlrpc/interfaces.py
index 035447a..403745c 100644
--- a/lib/lp/xmlrpc/interfaces.py
+++ b/lib/lp/xmlrpc/interfaces.py
@@ -1,4 +1,4 @@
-# Copyright 2011-2015 Canonical Ltd. This software is licensed under the
+# Copyright 2011-2021 Canonical Ltd. This software is licensed under the
# GNU Affero General Public License version 3 (see the file LICENSE).
"""Interfaces for the Launchpad application."""
@@ -17,6 +17,8 @@ from lp.services.webapp.interfaces import ILaunchpadApplication
class IPrivateApplication(ILaunchpadApplication):
"""Launchpad private XML-RPC application root."""
+ archive = Attribute("Archive XML-RPC end point.""")
+
authserver = Attribute("""Old Authserver API end point.""")
codeimportscheduler = Attribute("""Code import scheduler end point.""")
diff --git a/scripts/wsgi-archive-auth.py b/scripts/wsgi-archive-auth.py
new file mode 100755
index 0000000..3640ed1
--- /dev/null
+++ b/scripts/wsgi-archive-auth.py
@@ -0,0 +1,71 @@
+#!/usr/bin/python2
+#
+# Copyright 2017-2021 Canonical Ltd. This software is licensed under the
+# GNU Affero General Public License version 3 (see the file LICENSE).
+
+"""WSGI archive authorisation provider entry point.
+
+Unlike most Launchpad scripts, the #! line of this script does not use -S.
+This is because it is only executed (as opposed to imported) for testing,
+and mod_wsgi does not disable the automatic import of the site module when
+importing this script, so we want the test to imitate mod_wsgi's behaviour
+as closely as possible.
+"""
+
+from __future__ import absolute_import, print_function, unicode_literals
+
+__metaclass__ = type
+__all__ = [
+ 'check_password',
+ ]
+
+# mod_wsgi imports this file without a useful sys.path, so we need some
+# acrobatics to set ourselves up properly.
+import os.path
+import sys
+
+scripts_dir = os.path.dirname(os.path.abspath(os.path.realpath(__file__)))
+if scripts_dir not in sys.path:
+ sys.path.insert(0, scripts_dir)
+top = os.path.dirname(scripts_dir)
+
+# We can't stop mod_wsgi importing the site module. Cross fingers and
+# arrange for it to be re-imported.
+sys.modules.pop("site", None)
+sys.modules.pop("sitecustomize", None)
+
+import _pythonpath
+
+from lp.soyuz.wsgi.archiveauth import check_password
+
+
+def main():
+ """Hook for testing, not used by WSGI."""
+ from argparse import ArgumentParser
+
+ from lp.services.memcache.testing import MemcacheFixture
+ from lp.soyuz.wsgi import archiveauth
+
+ parser = ArgumentParser()
+ parser.add_argument("archive_path")
+ parser.add_argument("username")
+ parser.add_argument("password")
+ args = parser.parse_args()
+ archiveauth._memcache_client = MemcacheFixture()
+ result = check_password(
+ {"SCRIPT_NAME": args.archive_path}, args.username, args.password)
+ if result is None:
+ print("Archive or user does not exist.", file=sys.stderr)
+ return 2
+ elif result is False:
+ print("Password does not match.", file=sys.stderr)
+ return 1
+ elif result is True:
+ return 0
+ else:
+ print("Unexpected result from check_password: %s" % result)
+ return 3
+
+
+if __name__ == "__main__":
+ sys.exit(main())
diff --git a/utilities/rocketfuel-setup b/utilities/rocketfuel-setup
index 6d7c19a..43d3807 100755
--- a/utilities/rocketfuel-setup
+++ b/utilities/rocketfuel-setup
@@ -1,6 +1,6 @@
#! /bin/bash
#
-# Copyright 2009-2012 Canonical Ltd. This software is licensed under the
+# Copyright 2009-2021 Canonical Ltd. This software is licensed under the
# GNU Affero General Public License version 3 (see the file LICENSE).
#
# This script will set up a brand new Ubuntu machine as a LP developer
@@ -191,6 +191,12 @@ if [ $? -ne 0 ]; then
exit 1
fi
+sudo a2enmod wsgi > /dev/null
+if [ $? -ne 0 ]; then
+ echo "ERROR: Unable to enable wsgi module in Apache2"
+ exit 1
+fi
+
if [ $DO_WORKSPACE == 0 ]; then
cat <<EOT
Branches have not been created, as requested. You will need to do some or all
