launchpad-dev team mailing list archive

[Martin Pool <mbp@xxxxxxxxxxxxx>]

Those are awesome results.  I also really admire the way you keep on
so consistently celebrating the progress and pointing the way forward.

> https://dev.launchpad.net/ArchitectureGuide/ServicesAnalysis
>
> Please read this and do one of:
>  - comment in it
>  - reply to this thread
>  - reply to me privately
>
> depending your personal preferences.

Some thoughts, not all linked up or fully cooked:

* I'm happy to see this kind of change possibly moving forward.

* The lp api is really good in some ways (mostly, coverage) but
limited in others, and people speak of lazr.restful and co as being
hard to change or to represent particular things.  So it seems like
this project could either snag on that or be a powerful motivator to
improve it.

 * "low wtf factor on changes": some people with SOA seem to hit
(even) worse WTF moments; I suppose this is covered by some of your
list of benefits from the current design, but it may also need to be
addressed as a specific risk

 * lp api clients are hard to test now.  (there are some bugs open.)
The recommended thing is either: run up your own lp dev; or test
against production.  (Previously, it was recommended to test against
staging, but my unscientific impression is it is down or timing out
often enough to make this difficult.)  You talk about requiring
services to provide a test fake implementation.  I wonder if that is
waste: extra code to write, and a chance for them to get out of sync.
Perhaps it is simpler to just have the actual services be reliable,
fast, and simple enough to run up that people can actually test
against them; then only if that cannot be achieved make a fake
implementation.  This can still make the test layering more clear, and
it leaves the path open to doing a fake if you want to later.

 * So a specific step towards this could be using loggerhead's json
api to display some branch content inline in the main application?  By
talking to it from the client, or by having the webapp templates proxy
it?

 * Maybe it needs a better less generic (even more exciting?) name
than "services analysis".

 * As you say, lplib is problematic.  External clients need a better
solution too.  It does seem to me that xmlrpc is the way of the past
though, and that the trend is towards things that are json and restful
but a bit more reality-based than lplib's current approach.  It meshes
with what other client programs will expect, and it lets us get better
mileage and help out of http-level services, like caches.  (Not that
we want to rely too much on caching at that level, but it's nice not
to rule it out.)  Starting new work based on xmlrpc today would seem
unfortunate.

 * The section on coupling and cohesion of source trees to
manifestations is good.

> However, if we treat the templating and api engine as the entry-service rather than as part of the core data access service, we can dramatically simplify the testing story: a clean contract between template rendering/public api and model manipulation/optimisation/refactoring.

Can you explain that more?

 * I've said separately I think the api needs to move a bit more
towards thinking about what people are likely to want in a particular
request, much as we think about what ought to be on a particular
webapp page, rather than just exposing model objects.  (This is not to
say every api ought to be hand crafted.)  A headline case would be
shipping the bug tasks inline with the bug object; in some ways doing
this specifically for cases where it makes sense is more valuable than
allowing users to ask for them.

> One thing that would make this service easier to implement is to stop rendering templates in API calls (at all) - and instead generate those things client side if they are being served out in an API response.

I wasn't aware Launchpad did that at the moment.  What kind of thing
renders templates from API calls.

 * I recall you saying previously it would be really infeasible to do
a stepwise migration from a lazr.restful based api to a new way of
publishing apis.  I can't recall the reasons, but it might be worth
discussing in there because it seems like a substantial issue in
moving to this model: either there needs to be a way to add a
next-generation api piecewise, or lazr.restful gets improved, or
something else.

 * log management and correlation seems to get more important if there
are more independently moving parts.

hth
Martin