launchpad-dev team mailing list archive

[Robert Collins <robertc@xxxxxxxxxxxxxxxxx>]

On Tue, Dec 13, 2011 at 12:09 AM, Gavin Panella
<gavin.panella@xxxxxxxxxxxxx> wrote:
> On 12 December 2011 04:53, Robert Collins wrote:
> [...]
>> I propose the following scheme for our SOA ids:
>> $service$instance:$type:$typeid
>
> Seems good, though it reminds me of namespaced-OOPS hell, unfairly
> perhaps. Do we need to include $service?

So the OOPS thing was painful because:
 - each instance of identical services needed unique identifiers
 - the identifiers were not opaque (oops-tools made assumptions about
their internal structure)

Neither of those will apply here :)

Yes, we do need to include service, because - for instance, both SSO
and Launchpad both have a 'person' concept and requiring cross-team
coordination on every new type defined would not scale well.

>> Where service is one of ('lp', 'oops-tools', 'sso') today, with more
>> services allocated as we bring up additional microservices.
>> $instance is one of ('', '-qastaging', '-staging') and is used to
>> deal with collisions with objects created on staging services. It
>> also allows us to consume both a production and a staging service
>> from a staging service, if we want to. (And we will, for the
>> librarian, eventually).
>
> Using -staging to namespace staging, for example, is confusing with
> regards to service names like oops-tools. A different delimiter
> between service and instance like "/" or "." would be unambiguous.

Sure, we can use / or .. I don't think it makes much difference though:
oops-tools-staging. Are you confused ? :)

>> $type is allocated within the service itself. Examples are 'person',
>> 'archive' etc. Types that are exported, basically.  $typeid is any
>> string value that the service wants to use to get back to the
>> original object. For instance, a UUID, or a row id, or a pg object
>> id would all be valid. For things where no-guessability matters,
>> using a UUID is appropriate, for other things using the DB primary
>> key is fine, as long as it meets the stability and uniqueness
>> constraints.
>
> What constraints will there be on each part of the ID, or am I jumping
> the gun? I would imagine something like a strict subset of printable
> ASCII would make sense, but perhaps there are arguments for having
> wider options. I strongly feel we should avoid encoding (or escaping)
> issues; sticking with ASCII is probably a good way of doing that.

We could say 'utf8' and leave it at that. Or we could say 'the
printable subset of ascii' or some such. I'd just say non whitespace
utf8, as strings are easier to deal with, and avoiding whitespace
avoids most likely encoding issues.

>> As an example - 'lp:person:2' - would be able to be passed around to
>> multiple services (team membership, audit trail, notification
>> service). Only LP itself could interpret the string however to mean
>> Person 2.
>
> Using the lp: prefix in particular may cause confusion between objects
> and bzr URLs. For greppability (and linkability) it might be worth
> using a common prefix like "obj" or just "o":
>
>  obj:lp:person:1234

We could; OTOH linking these things, if its doable, would suggest we
just use urls (longer but well known rules already).

>> Unless folk can poke holes in this, I will update the SOA docs to
>> reflect this as the overall approach, and we can see how it goes.
>
> Is there value in adopting URNs [1] for the basic structure and rules
> of these IDs? There may already be some tool support for dealing with
> these, though I've never knowingly stumbled upon any.

Urn's are basically dead; everything is urls now. I may be biased ;).

> Also, do we want to make objects externally referencable? I'm thinking
> of being able to use, say, an HTTP URL that contains an object ID.
> That would impact the IDs constraints.

As long as the ids are unicode straight forward rules will get them to
a webserver - unicode is at least a solved problem. The question of
external references is interesting.

Will there be services we consume that are external?  Perhaps, but not
part of the core SOA - our confidentiality requirements mean we'll be
self hosted for the foreseeable future. And where we consume an
external service, it will either:
 - have its own convention we need to thunk across to
 - or allow us to supply our own soa ids, in which case any opaque string is ok.

Conversely, will we want to return an SOA id in a response to a
publically sourced request (or supply it when doing a webhook or
similar).
When folk consume us as a service we are providing a facade across the
backend services, and using our existing RESTful approach of URLs in
the public API space is pretty much ideal.
For webhooks, we'll want to let people come back to use via our public
API again, and so the same URLs will be appropriate.

I think the question this raises is 'why not use urls for these
backend services?' For which we could, but I wouldn't want use to
confuse issues by using urls that are addressable on our public API,
unless they truely reflect the same thing. And given that our url
structure is extremely mutable, that would be a problem, because
stability is a criteria for soa ids.

Consider a team membership service. It may have a restful API where it
issues urls for referring to its teams. They would include the soa id
for the team in the URL. We would *populate* the teams with persons
using their person soa id. -> soa ids are frequently components of
urls but not urls themselves.

Hmm, I think there is enough meat here that it will need some
reflection over the shutdown period; will come back to it in Jan.

-Rob