launchpad-dev team mailing list archive

[Robert Collins <robertc@xxxxxxxxxxxxxxxxx>]

Bah. Keyboard fail - mailus interruptus.

On Fri, Jun 3, 2011 at 2:33 PM, Robert Collins
<robertc@xxxxxxxxxxxxxxxxx> wrote:

Up to this bit was fine.

> That means we're now selecting amongst HTTP based protocols for a
> default protocol to talk to microservices.

Now starting over...

I have some thoughts here (specific to the default protocol/stack):

Must:
* handle binary data (librarian, gpg signing requests,
* support multiple client languages (e.g. cannot be bound even over
the network to just zope clients, or even just python)
* Support graceful resets (where we have a status url that controls
whether HAProxy feeds traffic to the service

Should:
* Allow easy backwards compatible evolution of apis. (Nb: I say should
here because one can always publish a new API to do migrations. But
there may be nicer ways).
* Permit OOPS integration

Must Not:
* Require client side caching
* Require client side run-time compilation (but a static thing during
deploy is ok).

Beyond these, I think the performance of the stack is important to us.
While we're not shipping massive volumes of data around the faster
things are, all else considered, the better.

Now, theres some debate around REST vs XMLRPC vs google protobufs as
options - but these things aren't all on the same spectrum.

Specifically, REST is compatible with XML - so we actually have
several dimensions to consider:
 * data serialisation (xml/json/...)
 * object access style (REST aka url-to-object/RPC aka
namespace-defines-functions)
 * parameter encoding for queries (url parameters/formdata-in-a-POST)
 * mapping to HTTP verbs (GET/POST/PUT/PATCH)

On some of these I have some opinions :) I seek plenty of discussion
though! Also note that whatever we choose as a default may not affect
what we do for things talking to the core service surrounding our big
postgresql server - that has enough stacks already running on it; I
worry about the idea of adding in *yet another* protocol mapping
there.

So generally speaking, I think json or bson is probably preferrable to
xml; I'm inclined towards RPC url mapping because working
object-at-a-time is terribly terribly slow and we have /so many/
multiple-object-access cases that doing url mapping to talk about
things seems like a pessimism (but perhaps lazr.restful is just too
fresh in my mind).
For parameter encoding, its really nice being able to bench test
readonly calls by hand with crafted URLs. This doesn't help with
mutating calls, or at least, not so much. And I think using GET for
readonly requests ties nicely into adhoc experimentation; for mutation
I think that for most of our use cases POST - that is, making a method
call, maps most cleanly into our domain.

Anyhow, even with all that said I think we should do some experiments
and bring up https://dev.launchpad.net/ArchitectureGuide/ServicesRoadmap#GPG%20validation
as a microservice in a couple of different styles - with the goal of
being able to see if the dev environment & productivity may outweight
any niggles around a particular combination of these dimensions.

I invite anyone interested to take that service description, and the
development-relevant aspects of
https://dev.launchpad.net/ArchitectureGuide/ServicesRequirements (e.g.
that it logs in apache common log format) and put together a prototype
in your favorite stack.

Have a great weekend!

-Rob