openstack team mailing list archive

[Mark Nottingham <mnot@xxxxxxxx>]

On 11/06/2012, at 6:58 AM, Joseph Heck wrote:

> First - what's the current thought of support for PATCH vs PUT in updating REST resources? Are there any issues with clients being able to use a PATCH verb? It's not something I'm super familiar with, so I'm looking for feedback from the community here. Ideally, I'd like to support the semantics of the PATCH HTTP verb, and possibly just assert no support for the PUT verb to be clear about intended functionality. Is that going to throw anyone for a loop?

I answered a question about PATCH before; don't want to repeat myself, but it should be workable. Happy to chat more about it if you have specific questions.

> Second - filtering/searching for resources. The draft includes a section labelled "Query By Name", which is probably mis-labelled, as it's intended to cover the general idea of passing in query parameters to general listing resource endpoints to filter the result set. The API endpoints across all the resources are defined as plurals, with the idea that specificity comes later in the URI (for referencing a single resource), or that we could add on these query parameters to restrict/filter by resource type. 

I'm in the middle of doing some log analysis and other research about how the APIs are used at Rackspace. It's too early to share results (although I do intend to, in some form, because the idea is to inform future API design), but one of the things that's very noticeable is how (extremely!) little pagination and filtering seem to be used in anger.

In fact, if you take a look at the libraries, you'll find that they often don't use or even support filtering or pagination; e.g., libcloud doesn't, AFAICT.

So, it's worth having a think about what the use cases actually are; both filtering and pagination are usually ways to save one or more of:
  a) client-side work
  b) server-side work
  c) bandwidth / latency

One interesting exercise would be to estimate the largest number of users (or whatever else you'd be listing) that a reasonable deployment would put in a single response, triple it, do a dummy serialisation in JSON, and then gzip it, so that you can estimate the size, see how long it takes to parse on the client, etc. 

From what I've seen (in OpenStack as well as in other APIs that have nothing to do with Cloud), API designers tend to overestimate the utility of pagination and especially filtering ("somebody might use it"), but users just ignore them, doing all of the work on the client side, except in extreme circumstances (e.g., VERY large responses / very high latency).

Unless you have strong use cases for them, I'd be inclined to drop them; they increase implementation, QA, and documentation complexity, as well as making the API harder to understand. YMMV, of course :)

The other issue with pagination is that a relative paged approach (like you're taking) means that readers' views of the complete set of items can be corrupted by simultaneous writers. While in some instances this is just an annoying UI bug (missing or duplicated entries on different pages, lower cache hit rates), in some circumstances it can be more serious (clients not understanding the true state of the system, and making bad decisions as a result). 

Again, a source of bugs and complexity (we came up with one approach to this with archived feeds in RFC5005, but it's pretty heavyweight, especially for use cases like this).

Hope this helps,

P.S. the X-Subject-Token stuff is breaking HTTP; you need to either put the token (or a facsimile for it) in the URL, or put Vary: Subject-Token in EVERY response those resources generate. The former is preferred; this is over TLS, right? Sorry I didn't see that earlier.

P.P.S If it's not too late, drop the X- from that header! <http://tools.ietf.org/html/draft-ietf-appsawg-xdash-05>


--
Mark Nottingham   http://www.mnot.net/