launchpad-reviewers team mailing list archive
-
launchpad-reviewers team
-
Mailing list archive
-
Message #21920
[Merge] lp:~cjwatson/launchpad/wsgi-ppa-auth into lp:launchpad
Colin Watson has proposed merging lp:~cjwatson/launchpad/wsgi-ppa-auth into lp:launchpad with lp:~cjwatson/launchpad/virtualenv-pip as a prerequisite.
Commit message:
Add a WSGI authenticator for private PPAs.
Requested reviews:
Launchpad code reviewers (launchpad-reviewers)
For more details, see:
https://code.launchpad.net/~cjwatson/launchpad/wsgi-ppa-auth/+merge/332125
I got sufficiently annoyed in the process of fixing bug 1722209 to see how hard it would be to fix it properly, since the existing htpasswd scheme has been a thorn in our side for a while now. The answer appears to be "a bit, but not very".
This will let us remove our reliance on htpasswd files, and probably eventually make it easier to do things like issue time-limited tokens (or even macaroons?) to builders.
There are obviously various deployment issues to sort out, and generate-ppa-htaccess still deals with things like deactivation and cancellation emails which we'll need to move elsewhere; it may be possible to just inline the relevant checks into the DB queries in ArchiveAuthTokenSet, since we're always looking at a single subscriber or named token name and so the result set sizes are very small.
The way that mod_wsgi loads the entry point is peculiar. I tried to handle it with the existing buildout-based build system, but that ended up being too fiddly, so I decided to just depend on my virtualenv/pip conversion branch. It would probably be a good idea to land https://code.launchpad.net/~cjwatson/launchpad/optimise-bin-py/+merge/331863 first as well to lighten the load from lp_sitecustomize. I considered separating the client code off entirely from Launchpad, but this turns out to be light enough in terms of memory use and startup time that I don't think it's worth the effort at the moment.
--
Your team Launchpad code reviewers is requested to review the proposed merge of lp:~cjwatson/launchpad/wsgi-ppa-auth into lp:launchpad.
=== modified file 'Makefile'
--- Makefile 2017-10-11 14:54:19 +0000
+++ Makefile 2017-10-11 14:54:21 +0000
@@ -461,6 +461,7 @@
base=local-launchpad; \
fi; \
sed -e 's,%BRANCH_REWRITE%,$(shell pwd)/scripts/branch-rewrite.py,' \
+ -e 's,%WSGI_ARCHIVE_AUTH%,$(shell pwd)/scripts/wsgi-archive-auth.py,' \
-e 's,%LISTEN_ADDRESS%,$(LISTEN_ADDRESS),' \
configs/development/local-launchpad-apache > \
/etc/apache2/sites-available/$$base
=== modified file 'configs/development/local-launchpad-apache'
--- configs/development/local-launchpad-apache 2017-01-10 17:26:29 +0000
+++ configs/development/local-launchpad-apache 2017-10-11 14:54:21 +0000
@@ -134,7 +134,6 @@
<VirtualHost %LISTEN_ADDRESS%:80>
ServerName ppa.launchpad.dev
- ServerAlias private-ppa.launchpad.dev
LogLevel debug
DocumentRoot /var/tmp/ppa
@@ -147,8 +146,36 @@
Deny from all
Allow from 127.0.0.0/255.0.0.0
</IfVersion>
- AllowOverride AuthConfig
- Options Indexes
+ AllowOverride None
+ Options Indexes
+ </Directory>
+</VirtualHost>
+
+<VirtualHost %LISTEN_ADDRESS%:80>
+ ServerName private-ppa.launchpad.dev
+ LogLevel debug
+
+ DocumentRoot /var/tmp/ppa
+ <Directory /var/tmp/ppa/>
+ <IfVersion >= 2.4>
+ <RequireAll>
+ Require ip 127.0.0.0/255.0.0.0
+ Require valid-user
+ </RequireAll>
+ </IfVersion>
+ <IfVersion < 2.4>
+ Order Deny,Allow
+ Deny from all
+ Allow from 127.0.0.0/255.0.0.0
+ Require valid-user
+ Satisfy All
+ </IfVersion>
+ AllowOverride None
+ Options Indexes
+ AuthType Basic
+ AuthName "Token Required"
+ AuthBasicProvider wsgi
+ WSGIAuthUserScript %WSGI_ARCHIVE_AUTH% application-group=lp
</Directory>
</VirtualHost>
=== modified file 'constraints.txt'
--- constraints.txt 2017-10-11 14:54:19 +0000
+++ constraints.txt 2017-10-11 14:54:21 +0000
@@ -334,6 +334,7 @@
subvertpy==0.9.1
testresources==0.2.7
testscenarios==0.4
+timedcache==1.1
timeline==0.0.3
# Build of lp:~canonical-launchpad-branches/twisted:lp-backport.
# p1 Support diffie-hellman-group14-sha1 key exchange in conch.ssh.
=== modified file 'lib/lp/services/config/schema-lazr.conf'
--- lib/lp/services/config/schema-lazr.conf 2017-09-07 13:25:13 +0000
+++ lib/lp/services/config/schema-lazr.conf 2017-10-11 14:54:21 +0000
@@ -1460,6 +1460,11 @@
# datatype: boolean
require_signing_keys: false
+# The URL to the internal archive API endpoint. This should implement
+# IArchiveAPI.
+# datatype: string
+archive_api_endpoint: http://xmlrpc-private.launchpad.dev:8087/archive
+
[ppa_apache_log_parser]
logs_root: /srv/ppa.launchpad.net-logs
=== modified file 'lib/lp/soyuz/doc/archiveauthtoken.txt'
--- lib/lp/soyuz/doc/archiveauthtoken.txt 2012-04-10 14:01:17 +0000
+++ lib/lp/soyuz/doc/archiveauthtoken.txt 2017-10-11 14:54:21 +0000
@@ -139,12 +139,35 @@
... print token.person.name
bradsmith
-Tokens can also be retreived by archive and person:
+Tokens can also be retrieved by archive and person:
>>> print token_set.getActiveTokenForArchiveAndPerson(
... new_token.archive, new_token.person).token
testtoken
+Or by archive and person name:
+
+ >>> print token_set.getActiveTokenForArchiveAndPersonName(
+ ... new_token.archive, "bradsmith").token
+ testtoken
+
+Tokens are only returned if they match a current subscription:
+
+ >>> from zope.security.proxy import removeSecurityProxy
+ >>> from lp.soyuz.enums import ArchiveSubscriberStatus
+ >>> removeSecurityProxy(subscription_to_joe_private_ppa).status = (
+ ... ArchiveSubscriberStatus.EXPIRED)
+
+ >>> print token_set.getActiveTokenForArchiveAndPerson(
+ ... new_token.archive, new_token.person)
+ None
+ >>> print token_set.getActiveTokenForArchiveAndPersonName(
+ ... new_token.archive, "bradsmith")
+ None
+
+ >>> removeSecurityProxy(subscription_to_joe_private_ppa).status = (
+ ... ArchiveSubscriberStatus.CURRENT)
+
== Amending Tokens ==
=== added file 'lib/lp/soyuz/interfaces/archiveapi.py'
--- lib/lp/soyuz/interfaces/archiveapi.py 1970-01-01 00:00:00 +0000
+++ lib/lp/soyuz/interfaces/archiveapi.py 2017-10-11 14:54:21 +0000
@@ -0,0 +1,46 @@
+# Copyright 2017 Canonical Ltd. This software is licensed under the
+# GNU Affero General Public License version 3 (see the file LICENSE).
+
+"""Interfaces for internal archive APIs."""
+
+from __future__ import absolute_import, print_function, unicode_literals
+
+__metaclass__ = type
+__all__ = [
+ 'IArchiveAPI',
+ 'IArchiveApplication',
+ ]
+
+from zope.interface import Interface
+
+from lp.services.webapp.interfaces import ILaunchpadApplication
+
+
+class IArchiveApplication(ILaunchpadApplication):
+ """Archive application root."""
+
+
+class IArchiveAPI(Interface):
+ """The Soyuz archive XML-RPC interface to Launchpad.
+
+ Published at "archive" on the private XML-RPC server.
+
+ PPA frontends use this to check archive authorization tokens.
+ """
+
+ def checkArchiveAuthToken(archive_reference, username, password):
+ """Check an archive authorization token.
+
+ :param archive_reference: The reference form of the archive to check.
+ :param username: The username sent using HTTP Basic Authentication;
+ this should either be a `Person.name` or "+" followed by the
+ name of a named authorization token.
+ :param password: The password sent using HTTP Basic Authentication;
+ this should be a corresponding `ArchiveAuthToken.token`.
+
+ :returns: A `NotFound` fault if `archive_reference` does not
+ identify an archive or the username does not identify a valid
+ token for this archive; an `Unauthorized` fault if the password
+ is not equal to the selected token for this archive; otherwise
+ None.
+ """
=== modified file 'lib/lp/soyuz/interfaces/archiveauthtoken.py'
--- lib/lp/soyuz/interfaces/archiveauthtoken.py 2016-07-14 16:06:01 +0000
+++ lib/lp/soyuz/interfaces/archiveauthtoken.py 2017-10-11 14:54:21 +0000
@@ -96,23 +96,33 @@
:return: An object conforming to `IArchiveAuthToken`.
"""
- def getByArchive(archive):
+ def getByArchive(archive, valid=False):
"""Retrieve all the tokens for an archive.
:param archive: The context archive.
+ :param valid: If True, only return valid tokens.
:return: A result set containing `IArchiveAuthToken`s.
"""
def getActiveTokenForArchiveAndPerson(archive, person):
- """Retrieve an active token for the given archive and person.
+ """Retrieve a valid active token for the given archive and person.
:param archive: The archive to which the token corresponds.
:param person: The person to which the token corresponds.
:return: An `IArchiveAuthToken` or None.
"""
+ def getActiveTokenForArchiveAndPersonName(archive, person_name):
+ """Retrieve a valid active token for the given archive and person name.
+
+ :param archive: The archive to which the token corresponds.
+ :param person_name: The name of the person to which the token
+ corresponds.
+ :return: An `IArchiveAuthToken` or None.
+ """
+
def getActiveNamedTokenForArchive(archive, name):
- """Retrieve an active named token for the given archive and name.
+ """Retrieve a valid active named token for the given archive and name.
:param archive: The archive to which the token corresponds.
:param name: The name of a named authorization token.
@@ -120,9 +130,9 @@
"""
def getActiveNamedTokensForArchive(archive, names=None):
- """Retrieve a subset of active named tokens for the given archive if
- `names` is specified, or all active named tokens for the archive if
- `names` is null.
+ """Retrieve a subset of valid active named tokens for the given
+ archive if `names` is specified, or all valid active named tokens
+ for the archive if `names` is null.
:param archive: The archive to which the tokens correspond.
:param names: An optional list of token names.
=== modified file 'lib/lp/soyuz/model/archiveauthtoken.py'
--- lib/lp/soyuz/model/archiveauthtoken.py 2016-07-14 16:06:01 +0000
+++ lib/lp/soyuz/model/archiveauthtoken.py 2017-10-11 14:54:21 +0000
@@ -21,8 +21,10 @@
from storm.store import Store
from zope.interface import implementer
+from lp.registry.model.teammembership import TeamParticipation
from lp.services.database.constants import UTC_NOW
from lp.services.database.interfaces import IStore
+from lp.soyuz.enums import ArchiveSubscriberStatus
from lp.soyuz.interfaces.archiveauthtoken import (
IArchiveAuthToken,
IArchiveAuthTokenSet,
@@ -85,19 +87,37 @@
return IStore(ArchiveAuthToken).find(
ArchiveAuthToken, ArchiveAuthToken.token == token).one()
- def getByArchive(self, archive):
+ def getByArchive(self, archive, valid=False):
"""See `IArchiveAuthTokenSet`."""
+ # Circular import.
+ from lp.soyuz.model.archivesubscriber import ArchiveSubscriber
store = Store.of(archive)
- return store.find(
- ArchiveAuthToken,
+ clauses = [
ArchiveAuthToken.archive == archive,
- ArchiveAuthToken.date_deactivated == None)
+ ArchiveAuthToken.date_deactivated == None,
+ ]
+ if valid:
+ clauses.extend([
+ ArchiveAuthToken.archive_id == ArchiveSubscriber.archive_id,
+ ArchiveSubscriber.status == ArchiveSubscriberStatus.CURRENT,
+ ArchiveSubscriber.subscriber_id == TeamParticipation.teamID,
+ TeamParticipation.personID == ArchiveAuthToken.person_id,
+ ])
+ return store.find(ArchiveAuthToken, *clauses)
def getActiveTokenForArchiveAndPerson(self, archive, person):
"""See `IArchiveAuthTokenSet`."""
- return self.getByArchive(archive).find(
+ return self.getByArchive(archive, valid=True).find(
ArchiveAuthToken.person == person).one()
+ def getActiveTokenForArchiveAndPersonName(self, archive, person_name):
+ """See `IArchiveAuthTokenSet`."""
+ # Circular import.
+ from lp.registry.model.person import Person
+ return self.getByArchive(archive, valid=True).find(
+ ArchiveAuthToken.person == Person.id,
+ Person.name == person_name).one()
+
def getActiveNamedTokenForArchive(self, archive, name):
"""See `IArchiveAuthTokenSet`."""
return self.getByArchive(archive).find(
=== added directory 'lib/lp/soyuz/wsgi'
=== added file 'lib/lp/soyuz/wsgi/__init__.py'
=== added file 'lib/lp/soyuz/wsgi/archiveauth.py'
--- lib/lp/soyuz/wsgi/archiveauth.py 1970-01-01 00:00:00 +0000
+++ lib/lp/soyuz/wsgi/archiveauth.py 2017-10-11 14:54:21 +0000
@@ -0,0 +1,69 @@
+# Copyright 2017 Canonical Ltd. This software is licensed under the
+# GNU Affero General Public License version 3 (see the file LICENSE).
+
+"""WSGI archive authorisation provider.
+
+This is as lightweight as possible, as it runs on PPA frontends.
+"""
+
+from __future__ import absolute_import, print_function, unicode_literals
+
+__metaclass__ = type
+__all__ = [
+ 'check_password',
+ ]
+
+try:
+ from xmlrpc.client import (
+ Fault,
+ ServerProxy,
+ )
+except ImportError:
+ from xmlrpclib import (
+ Fault,
+ ServerProxy,
+ )
+
+from timedcache import TimedCache
+
+from lp.services.config import config
+
+
+# A simple in-process cache with a short time limit. Although it isn't
+# shared between processes, this should be enough to alleviate load without
+# having to worry about the security of caching authentication tokens in
+# something like memcached.
+_cache = TimedCache(ttl=60)
+
+
+def _get_archive_reference(environ):
+ # Reconstruct the relevant part of the URL. We don't care about where
+ # we're installed.
+ path = environ.get("SCRIPT_NAME") or "/"
+ path_info = environ.get("PATH_INFO", "")
+ path += (path_info if path else path_info[1:])
+ # Extract the first three segments of the path, and rearrange them to
+ # form an archive reference.
+ path_parts = path.lstrip("/").split("/")
+ if len(path_parts) >= 3:
+ return "~%s/%s/%s" % (path_parts[0], path_parts[2], path_parts[1])
+
+
+def check_password(environ, user, password):
+ archive_reference = _get_archive_reference(environ)
+ if archive_reference is None:
+ return None
+ if _cache.get((archive_reference, user, password)):
+ return True
+ proxy = ServerProxy(config.personalpackagearchive.archive_api_endpoint)
+ try:
+ proxy.checkArchiveAuthToken(archive_reference, user, password)
+ # Cache positive responses for a minute to reduce database load.
+ _cache[(archive_reference, user, password)] = True
+ return True
+ except Fault as e:
+ if e.faultCode == 410: # Unauthorized
+ return False
+ else:
+ # Interpret any other fault as NotFound (320).
+ return None
=== added directory 'lib/lp/soyuz/wsgi/tests'
=== added file 'lib/lp/soyuz/wsgi/tests/__init__.py'
=== added file 'lib/lp/soyuz/wsgi/tests/test_archiveauth.py'
--- lib/lp/soyuz/wsgi/tests/test_archiveauth.py 1970-01-01 00:00:00 +0000
+++ lib/lp/soyuz/wsgi/tests/test_archiveauth.py 2017-10-11 14:54:21 +0000
@@ -0,0 +1,137 @@
+# Copyright 2017 Canonical Ltd. This software is licensed under the
+# GNU Affero General Public License version 3 (see the file LICENSE).
+
+"""Tests for the WSGI archive authorisation provider."""
+
+from __future__ import absolute_import, print_function, unicode_literals
+
+__metaclass__ = type
+
+import os.path
+import subprocess
+import time
+
+from fixtures import MonkeyPatch
+from testtools.matchers import Is
+import transaction
+
+from lp.services.config import config
+from lp.soyuz.wsgi import archiveauth
+from lp.testing import TestCaseWithFactory
+from lp.testing.layers import ZopelessAppServerLayer
+from lp.xmlrpc import faults
+
+
+class TestWSGIArchiveAuth(TestCaseWithFactory):
+
+ layer = ZopelessAppServerLayer
+
+ def setUp(self):
+ super(TestWSGIArchiveAuth, self).setUp()
+ self.now = time.time()
+ self.useFixture(MonkeyPatch("time.time", lambda: self.now))
+ self.addCleanup(archiveauth._cache.clear)
+
+ def test_get_archive_reference_short_url(self):
+ self.assertIsNone(archiveauth._get_archive_reference(
+ {"SCRIPT_NAME": "/foo"}))
+
+ def test_get_archive_reference_archive_base(self):
+ self.assertEqual(
+ "~user/ubuntu/ppa",
+ archiveauth._get_archive_reference(
+ {"SCRIPT_NAME": "/user/ppa/ubuntu"}))
+
+ def test_get_archive_reference_inside_archive(self):
+ self.assertEqual(
+ "~user/ubuntu/ppa",
+ archiveauth._get_archive_reference(
+ {"SCRIPT_NAME": "/user/ppa/ubuntu/dists"}))
+
+ def test_check_password_short_url(self):
+ self.assertIsNone(archiveauth.check_password(
+ {"SCRIPT_NAME": "/foo"}, "user", ""))
+ self.assertEqual({}, archiveauth._cache)
+
+ def test_check_password_not_found(self):
+ self.assertIsNone(archiveauth.check_password(
+ {"SCRIPT_NAME": "/nonexistent/bad/unknown"}, "user", ""))
+ self.assertEqual({}, archiveauth._cache)
+
+ def makeArchiveAndToken(self):
+ archive = self.factory.makeArchive(private=True)
+ archive_path = "/%s/%s/ubuntu" % (archive.owner.name, archive.name)
+ subscriber = self.factory.makePerson()
+ archive.newSubscription(subscriber, archive.owner)
+ token = archive.newAuthToken(subscriber)
+ transaction.commit()
+ return archive, archive_path, subscriber.name, token.token
+
+ def test_check_password_unauthorized(self):
+ _, archive_path, username, password = self.makeArchiveAndToken()
+ # Test that this returns False, not merely something falsy (e.g.
+ # None).
+ self.assertThat(
+ archiveauth.check_password(
+ {"SCRIPT_NAME": archive_path}, username, password + "-bad"),
+ Is(False))
+ self.assertEqual({}, archiveauth._cache)
+
+ def test_check_password_success(self):
+ archive, archive_path, username, password = self.makeArchiveAndToken()
+ self.assertThat(
+ archiveauth.check_password(
+ {"SCRIPT_NAME": archive_path}, username, password),
+ Is(True))
+ self.assertEqual(
+ {(archive.reference, username, password): True},
+ archiveauth._cache)
+
+ def test_check_password_considers_cache(self):
+ class FakeProxy:
+ def __init__(self, uri):
+ pass
+
+ def checkArchiveAuthToken(self, archive_reference, username,
+ password):
+ raise faults.Unauthorized()
+
+ _, archive_path, username, password = self.makeArchiveAndToken()
+ self.assertThat(
+ archiveauth.check_password(
+ {"SCRIPT_NAME": archive_path}, username, password),
+ Is(True))
+ self.useFixture(
+ MonkeyPatch("lp.soyuz.wsgi.archiveauth.ServerProxy", FakeProxy))
+ # A subsequent check honours the cache.
+ self.assertThat(
+ archiveauth.check_password(
+ {"SCRIPT_NAME": archive_path}, username, password + "-bad"),
+ Is(False))
+ self.assertThat(
+ archiveauth.check_password(
+ {"SCRIPT_NAME": archive_path}, username, password),
+ Is(True))
+ # If we advance time far enough, then the cached result expires.
+ self.now += 60
+ self.assertThat(
+ archiveauth.check_password(
+ {"SCRIPT_NAME": archive_path}, username, password),
+ Is(False))
+
+ def test_script(self):
+ _, archive_path, username, password = self.makeArchiveAndToken()
+ script_path = os.path.join(
+ config.root, "scripts", "wsgi-archive-auth.py")
+
+ def check_via_script(archive_path, username, password):
+ with open(os.devnull, "w") as devnull:
+ return subprocess.call(
+ [script_path, archive_path, username, password],
+ stderr=devnull)
+
+ self.assertEqual(0, check_via_script(archive_path, username, password))
+ self.assertEqual(
+ 1, check_via_script(archive_path, username, password + "-bad"))
+ self.assertEqual(
+ 2, check_via_script("/nonexistent/bad/unknown", "user", ""))
=== added file 'lib/lp/soyuz/xmlrpc/archive.py'
--- lib/lp/soyuz/xmlrpc/archive.py 1970-01-01 00:00:00 +0000
+++ lib/lp/soyuz/xmlrpc/archive.py 2017-10-11 14:54:21 +0000
@@ -0,0 +1,62 @@
+# Copyright 2017 Canonical Ltd. This software is licensed under the
+# GNU Affero General Public License version 3 (see the file LICENSE).
+
+"""Implementations of the XML-RPC APIs for Soyuz archives."""
+
+from __future__ import absolute_import, print_function, unicode_literals
+
+__metaclass__ = type
+__all__ = [
+ 'ArchiveAPI',
+ ]
+
+from zope.component import getUtility
+from zope.interface import implementer
+from zope.security.proxy import removeSecurityProxy
+
+from lp.soyuz.interfaces.archive import IArchiveSet
+from lp.soyuz.interfaces.archiveapi import IArchiveAPI
+from lp.soyuz.interfaces.archiveauthtoken import IArchiveAuthTokenSet
+from lp.services.webapp import LaunchpadXMLRPCView
+from lp.xmlrpc import faults
+from lp.xmlrpc.helpers import return_fault
+
+
+BUILDD_USER_NAME = "buildd"
+
+
+@implementer(IArchiveAPI)
+class ArchiveAPI(LaunchpadXMLRPCView):
+ """See `IArchiveAPI`."""
+
+ @return_fault
+ def _checkArchiveAuthToken(self, archive_reference, username, password):
+ archive = getUtility(IArchiveSet).getByReference(archive_reference)
+ if archive is None:
+ raise faults.NotFound(
+ message="No archive found for '%s'." % archive_reference)
+ archive = removeSecurityProxy(archive)
+ token_set = getUtility(IArchiveAuthTokenSet)
+ if username == BUILDD_USER_NAME:
+ secret = archive.buildd_secret
+ else:
+ if username.startswith("+"):
+ token = token_set.getActiveNamedTokenForArchive(
+ archive, username[1:])
+ else:
+ token = token_set.getActiveTokenForArchiveAndPersonName(
+ archive, username)
+ if token is None:
+ raise faults.NotFound(
+ message="No valid tokens for '%s' in '%s'." % (
+ username, archive_reference))
+ secret = removeSecurityProxy(token).token
+ if password != secret:
+ raise faults.Unauthorized()
+
+ def checkArchiveAuthToken(self, archive_reference, username, password):
+ """See `IArchiveAPI`."""
+ # This thunk exists because you can't use a decorated function as
+ # the implementation of a method exported over XML-RPC.
+ return self._checkArchiveAuthToken(
+ archive_reference, username, password)
=== added directory 'lib/lp/soyuz/xmlrpc/tests'
=== added file 'lib/lp/soyuz/xmlrpc/tests/__init__.py'
=== added file 'lib/lp/soyuz/xmlrpc/tests/test_archive.py'
--- lib/lp/soyuz/xmlrpc/tests/test_archive.py 1970-01-01 00:00:00 +0000
+++ lib/lp/soyuz/xmlrpc/tests/test_archive.py 2017-10-11 14:54:21 +0000
@@ -0,0 +1,124 @@
+# Copyright 2017 Canonical Ltd. This software is licensed under the
+# GNU Affero General Public License version 3 (see the file LICENSE).
+
+"""Tests for the internal Soyuz archive API."""
+
+from __future__ import absolute_import, print_function, unicode_literals
+
+__metaclass__ = type
+
+from zope.security.proxy import removeSecurityProxy
+
+from lp.services.features.testing import FeatureFixture
+from lp.soyuz.interfaces.archive import NAMED_AUTH_TOKEN_FEATURE_FLAG
+from lp.soyuz.xmlrpc.archive import ArchiveAPI
+from lp.testing import TestCaseWithFactory
+from lp.testing.layers import LaunchpadFunctionalLayer
+from lp.xmlrpc import faults
+
+
+class TestArchiveAPI(TestCaseWithFactory):
+
+ layer = LaunchpadFunctionalLayer
+
+ def setUp(self):
+ super(TestArchiveAPI, self).setUp()
+ self.useFixture(FeatureFixture({NAMED_AUTH_TOKEN_FEATURE_FLAG: "on"}))
+ self.archive_api = ArchiveAPI(None, None)
+
+ def assertNotFound(self, archive_reference, username, password, message):
+ """Assert that an archive auth token check returns NotFound."""
+ fault = self.archive_api.checkArchiveAuthToken(
+ archive_reference, username, password)
+ self.assertEqual(faults.NotFound(message), fault)
+
+ def assertUnauthorized(self, archive_reference, username, password):
+ """Assert that an archive auth token check returns Unauthorized."""
+ fault = self.archive_api.checkArchiveAuthToken(
+ archive_reference, username, password)
+ self.assertEqual(faults.Unauthorized("Authorisation required."), fault)
+
+ def test_checkArchiveAuthToken_unknown_archive(self):
+ self.assertNotFound(
+ "~nonexistent/unknown/bad", "user", "",
+ "No archive found for '~nonexistent/unknown/bad'.")
+
+ def test_checkArchiveAuthToken_no_tokens(self):
+ archive = removeSecurityProxy(self.factory.makeArchive(private=True))
+ self.assertNotFound(
+ archive.reference, "nobody", "",
+ "No valid tokens for 'nobody' in '%s'." % archive.reference)
+
+ def test_checkArchiveAuthToken_no_named_tokens(self):
+ archive = removeSecurityProxy(self.factory.makeArchive(private=True))
+ self.assertNotFound(
+ archive.reference, "+missing", "",
+ "No valid tokens for '+missing' in '%s'." % archive.reference)
+
+ def test_checkArchiveAuthToken_buildd_wrong_password(self):
+ archive = removeSecurityProxy(self.factory.makeArchive(private=True))
+ self.assertUnauthorized(
+ archive.reference, "buildd", archive.buildd_secret + "-bad")
+
+ def test_checkArchiveAuthToken_buildd_correct_password(self):
+ archive = removeSecurityProxy(self.factory.makeArchive(private=True))
+ self.assertIsNone(self.archive_api.checkArchiveAuthToken(
+ archive.reference, "buildd", archive.buildd_secret))
+
+ def test_checkArchiveAuthToken_named_token_wrong_password(self):
+ archive = removeSecurityProxy(self.factory.makeArchive(private=True))
+ token = archive.newNamedAuthToken("special")
+ removeSecurityProxy(token).deactivate()
+ self.assertNotFound(
+ archive.reference, "+special", token.token,
+ "No valid tokens for '+special' in '%s'." % archive.reference)
+
+ def test_checkArchiveAuthToken_named_token_deactivated(self):
+ archive = removeSecurityProxy(self.factory.makeArchive(private=True))
+ token = archive.newNamedAuthToken("special")
+ self.assertIsNone(self.archive_api.checkArchiveAuthToken(
+ archive.reference, "+special", token.token))
+
+ def test_checkArchiveAuthToken_named_token_correct_password(self):
+ archive = removeSecurityProxy(self.factory.makeArchive(private=True))
+ token = archive.newNamedAuthToken("special")
+ self.assertIsNone(self.archive_api.checkArchiveAuthToken(
+ archive.reference, "+special", token.token))
+
+ def test_checkArchiveAuthToken_personal_token_wrong_password(self):
+ archive = removeSecurityProxy(self.factory.makeArchive(private=True))
+ subscriber = self.factory.makePerson()
+ archive.newSubscription(subscriber, archive.owner)
+ token = archive.newAuthToken(subscriber)
+ self.assertUnauthorized(
+ archive.reference, subscriber.name, token.token + "-bad")
+
+ def test_checkArchiveAuthToken_personal_token_deactivated(self):
+ archive = removeSecurityProxy(self.factory.makeArchive(private=True))
+ subscriber = self.factory.makePerson()
+ archive.newSubscription(subscriber, archive.owner)
+ token = archive.newAuthToken(subscriber)
+ removeSecurityProxy(token).deactivate()
+ self.assertNotFound(
+ archive.reference, subscriber.name, token.token,
+ "No valid tokens for '%s' in '%s'." % (
+ subscriber.name, archive.reference))
+
+ def test_checkArchiveAuthToken_personal_token_cancelled(self):
+ archive = removeSecurityProxy(self.factory.makeArchive(private=True))
+ subscriber = self.factory.makePerson()
+ subscription = archive.newSubscription(subscriber, archive.owner)
+ token = archive.newAuthToken(subscriber)
+ removeSecurityProxy(subscription).cancel(archive.owner)
+ self.assertNotFound(
+ archive.reference, subscriber.name, token.token,
+ "No valid tokens for '%s' in '%s'." % (
+ subscriber.name, archive.reference))
+
+ def test_checkArchiveAuthToken_personal_token_correct_password(self):
+ archive = removeSecurityProxy(self.factory.makeArchive(private=True))
+ subscriber = self.factory.makePerson()
+ archive.newSubscription(subscriber, archive.owner)
+ token = archive.newAuthToken(subscriber)
+ self.assertIsNone(self.archive_api.checkArchiveAuthToken(
+ archive.reference, subscriber.name, token.token))
=== modified file 'lib/lp/systemhomes.py'
--- lib/lp/systemhomes.py 2017-05-16 16:33:53 +0000
+++ lib/lp/systemhomes.py 2017-10-11 14:54:21 +0000
@@ -72,6 +72,7 @@
from lp.services.webapp.publisher import canonical_url
from lp.services.webservice.interfaces import IWebServiceApplication
from lp.services.worlddata.interfaces.language import ILanguageSet
+from lp.soyuz.interfaces.archiveapi import IArchiveApplication
from lp.testopenid.interfaces.server import ITestOpenIDApplication
from lp.translations.interfaces.translationgroup import ITranslationGroupSet
from lp.translations.interfaces.translations import IRosettaApplication
@@ -80,6 +81,12 @@
)
+@implementer(IArchiveApplication)
+class ArchiveApplication:
+
+ title = "Archive API"
+
+
@implementer(ICodehostingApplication)
class CodehostingApplication:
"""Codehosting End-Point."""
=== modified file 'lib/lp/xmlrpc/application.py'
--- lib/lp/xmlrpc/application.py 2015-10-26 14:54:43 +0000
+++ lib/lp/xmlrpc/application.py 2017-10-11 14:54:21 +0000
@@ -31,11 +31,12 @@
from lp.services.features.xmlrpc import IFeatureFlagApplication
from lp.services.webapp import LaunchpadXMLRPCView
from lp.services.webapp.interfaces import ILaunchBag
+from lp.soyuz.interfaces.archiveapi import IArchiveApplication
from lp.xmlrpc.interfaces import IPrivateApplication
# NOTE: If you add a traversal here, you should update
-# the regular expression in utilities/page-performance-report.ini
+# the regular expression in lp:lp-dev-utils page-performance-report.ini.
@implementer(IPrivateApplication)
class PrivateApplication:
@@ -45,6 +46,11 @@
return getUtility(IMailingListApplication)
@property
+ def archive(self):
+ """See `IPrivateApplication`."""
+ return getUtility(IArchiveApplication)
+
+ @property
def authserver(self):
"""See `IPrivateApplication`."""
return getUtility(IAuthServerApplication)
=== modified file 'lib/lp/xmlrpc/configure.zcml'
--- lib/lp/xmlrpc/configure.zcml 2015-05-04 14:56:58 +0000
+++ lib/lp/xmlrpc/configure.zcml 2017-10-11 14:54:21 +0000
@@ -22,6 +22,19 @@
/>
<securedutility
+ class="lp.systemhomes.ArchiveApplication"
+ provides="lp.soyuz.interfaces.archiveapi.IArchiveApplication">
+ <allow interface="lp.soyuz.interfaces.archiveapi.IArchiveApplication"/>
+ </securedutility>
+
+ <xmlrpc:view
+ for="lp.soyuz.interfaces.archiveapi.IArchiveApplication"
+ interface="lp.soyuz.interfaces.archiveapi.IArchiveAPI"
+ class="lp.soyuz.xmlrpc.archive.ArchiveAPI"
+ permission="zope.Public"
+ />
+
+ <securedutility
class="lp.systemhomes.CodehostingApplication"
provides="lp.code.interfaces.codehosting.ICodehostingApplication">
<allow interface="lp.code.interfaces.codehosting.ICodehostingApplication"/>
=== modified file 'lib/lp/xmlrpc/interfaces.py'
--- lib/lp/xmlrpc/interfaces.py 2015-05-04 14:56:58 +0000
+++ lib/lp/xmlrpc/interfaces.py 2017-10-11 14:54:21 +0000
@@ -17,6 +17,8 @@
class IPrivateApplication(ILaunchpadApplication):
"""Launchpad private XML-RPC application root."""
+ archive = Attribute("Archive XML-RPC end point.""")
+
authserver = Attribute("""Old Authserver API end point.""")
codeimportscheduler = Attribute("""Code import scheduler end point.""")
=== added file 'scripts/wsgi-archive-auth.py'
--- scripts/wsgi-archive-auth.py 1970-01-01 00:00:00 +0000
+++ scripts/wsgi-archive-auth.py 2017-10-11 14:54:21 +0000
@@ -0,0 +1,66 @@
+#!/usr/bin/python
+#
+# Copyright 2017 Canonical Ltd. This software is licensed under the
+# GNU Affero General Public License version 3 (see the file LICENSE).
+
+"""WSGI archive authorisation provider entry point.
+
+Unlike most Launchpad scripts, the #! line of this script does not use -S.
+This is because it is only executed (as opposed to imported) for testing,
+and mod_wsgi does not disable the automatic import of the site module when
+importing this script, so we want the test to imitate mod_wsgi's behaviour
+as closely as possible.
+"""
+
+from __future__ import absolute_import, print_function, unicode_literals
+
+__metaclass__ = type
+__all__ = [
+ 'check_password',
+ ]
+
+# mod_wsgi imports this file without a useful sys.path, so we need some
+# acrobatics to set ourselves up properly.
+import os.path
+import sys
+
+scripts_dir = os.path.dirname(os.path.abspath(os.path.realpath(__file__)))
+if scripts_dir not in sys.path:
+ sys.path.insert(0, scripts_dir)
+top = os.path.dirname(scripts_dir)
+
+# We can't stop mod_wsgi importing the site module. Cross fingers and
+# arrange for it to be re-imported.
+sys.modules.pop("site", None)
+sys.modules.pop("sitecustomize", None)
+
+import _pythonpath
+
+from lp.soyuz.wsgi.archiveauth import check_password
+
+
+def main():
+ # Hook for testing, not used by WSGI.
+ from argparse import ArgumentParser
+ parser = ArgumentParser()
+ parser.add_argument("archive_path")
+ parser.add_argument("username")
+ parser.add_argument("password")
+ args = parser.parse_args()
+ result = check_password(
+ {"SCRIPT_NAME": args.archive_path}, args.username, args.password)
+ if result is None:
+ print("Archive or user does not exist.", file=sys.stderr)
+ return 2
+ elif result is False:
+ print("Password does not match.", file=sys.stderr)
+ return 1
+ elif result is True:
+ return 0
+ else:
+ print("Unexpected result from check_password: %s" % result)
+ return 3
+
+
+if __name__ == "__main__":
+ sys.exit(main())
=== modified file 'setup.py'
--- setup.py 2017-10-11 14:54:19 +0000
+++ setup.py 2017-10-11 14:54:21 +0000
@@ -217,6 +217,7 @@
'subvertpy',
'testscenarios',
'testtools',
+ 'timedcache',
'timeline',
'transaction',
'Twisted',
=== modified file 'utilities/rocketfuel-setup'
--- utilities/rocketfuel-setup 2017-01-10 17:24:08 +0000
+++ utilities/rocketfuel-setup 2017-10-11 14:54:21 +0000
@@ -189,6 +189,12 @@
exit 1
fi
+sudo a2enmod wsgi > /dev/null
+if [ $? -ne 0 ]; then
+ echo "ERROR: Unable to enable wsgi module in Apache2"
+ exit 1
+fi
+
if [ $DO_WORKSPACE == 0 ]; then
cat <<EOT
Branches have not been created, as requested. You will need to do some or all
Follow ups
