launchpad-dev team mailing list archive

[Robert Collins <robertc@xxxxxxxxxxxxxxxxx>]

It feels like an age since I wrote a mail about the services project.
Probably it was only a month or something :)

The url for reference: https://dev.launchpad.net/ArchitectureGuide/Services

tl;dr:
 - new projects should take a services approach where sensible
 - existing things will migrate as part of other feature work

Current status
===========

Francis and Elliot (Francis' boss :P) have signed off on the project
as a whole: architecturally we're now moving to a microservice
approach. All new datastorage / processing / integration facilities
should be designed with an eye to small reusable services (e.g. like
the software centre agent). As a team we have a *lot* to learn about
how to do this well, about where it makes sense and where it doesn't.

We're still learning about the best lean-stacks to use for these
components as well, and the recommended protocols to join them. That
means that for a prototype microservice you can choose anything that
IS and the team will be comfortable working with. Thats a bit vague,
but there you go :). Microservices are deliberately small, so if when
turning a prototype model into a production model we end up redoing
stuff - thats fine, we will have *learnt* something on the way: Write
one to throw away.

We have some inventory: the Epic put together a browser push system
based on a rabbit component the Canonical Landscape team wrote. This
work is *nearly* complete but not 'live' yet because we've set a bar
for microservices that ensures we can administer them and diagnose
failures in a sensible fashion. Red will be bring that project up to
production standards before they rotate onto maintenance. This
involves: integrating the twisted daemon with the OOPS system; getting
error reporting in some fashion from rabbit (be that log sniffing, a
management plugin, whatever) as well as key metric alerting and
graphing.

I've recently extracted *most* of the OOPS system from lp:launchpad
into small components:
https://lists.launchpad.net/launchpad-dev/msg07825.html discusses
that. This makes integrating OOPSes into a wsgi stack very easy, and
provides a clear path for extracting other aspects of it (like the
python logging -> OOPS mapping) as well as future improvements (moving
to a bson serializer, and push-publishing rather than cron jobs to
copy OOPSes around).

Future things
=========

Once we're live with *a* microservice (probably one of the browser
notification service or a gpg verification microservice) we'll
probably need to start ramping up our production deployment processes
and tooling: we'll have multiple different codebases that we need a
systematic view on, and possibly even want a deployment report on each
of them. (The larger the service the more likelyhood two people are
hacking on it concurrently). This will warrant some infrastructure
around this - probably moving to a data driven system - we may want to
productise that internall and allow other Canonical departments to
reuse the same logic.

In the near term there are no plans for an aggressive
migrate-to-services project: rather we plan to do things organically
as part of solving other problems or tasks. The barriers to entry
should be low enough that we can do this - and if we can't, we will
need to fund enough development that the barriers *become* low enough,
otherwise this won't take off :)

Cheers,
Rob