launchpad-dev team mailing list archive

[Robert Collins <robertc@xxxxxxxxxxxxxxxxx>]

Francis and I went through the services roadmap with James Troup
yesterday, identifying things that need particular care/caution from a
Canonical IS perspective.

>From this conversation a few things became clearer and I've updated
the document accordingly.

One of the changes was to decide how to do HTTP services.

Essentially the internal LAN is trusted - its locked down extremely carefully.

So, for internal services we'll be safe with:
 - on each host running a service an apache instance
 - the service binds to localhost only
 - apache provides basic authentication as a front end
 - we have haproxy providing load balancing for the service
 - and iptables on both the haproxy hosts and the service hosts
controlling access to the apache instances (haproxy + nagios only) and
to the haproxy instance (expected service clients + nagios only).

This means that *by default* for microservices we don't need to worry
about access control to the service as part of its code: we do for
*deployment* but not for *development*.

Rabbit has an awkward high availability story; specifically its not
trivial to get the reliability we have out of HTTP services, This is
partly because rabbit clusters don't distribute the queues and because
its a more stateful and complex system than HTTP. Long story short we
won't be in a position to use queues for persistence and its simpler
to use HTTP to gracefully handle a single backend node dying.

That means we're now selecting amongst HTTP based protocols for a
default protocol to talk to microservices.

I have some thoughts here (specific to the default):

Must:
* handle binary data (librarian, gpg signing requests,
* support multiple client languages (e.g. cannot be bound even over
the network to just zope clients, or even just python)
*

Must Not:

XMLRPC: pros: already deploys, batteries included in Python and many
other languages. cons: XML, RPC model rather than restful - no
opportunity for caching, URLs can be opaque when debugging.
adhoc restful json based apis. pros: nice to look at by hand, easy to
interact with manually. cons: not included in the Python standard
library, optimises for things that don't really affect us.
google protobufs: pros: clear contracts, wire level upgrading
built-in. cons: not well understood within the LP team, currently
somewhat slow [in python].