openstack team mailing list archive

["Nguyen, Liem Manh" <liem_m_nguyen@xxxxxx>]

Keystone is using XSD1.1 already.

Mark, I totally agree with you in that we need a "contract" to a service, may it WADL or whatever IDL you use...  For us, we have used WADL as a mere contract (among our services and QA), and that's about it.  We don't generate code from WADL, nor try to bake code to the WADL.  Perhaps, this is an area where things can go off the chart, as you mentioned...  But, we need a contract.  Particularly, an API contract:


*         Should not expose language-specifics...  For example, if the contract takes into account the dynamism of Python, that's wrong, IMHO, because someone else may (and will) try to implement the contract in Java or C and will find it awkward.

*         Should allow some level of automated validation; so, I know that I just checked something in that breaks the API.  For this reason, I like XSD validation and Scala Specs<http://code.google.com/p/specs/> for full specs-driven coding.

*         Most importantly of all, should be "precise, unambiguous, and easy to understand and maintain" like you mentioned, since we are ultimately consumers of the contract.

In the past, WADL + XSD have served us well (to the extent that we use them for); but, I am definitely open for any other approach out there that meets the above use-cases...

Cheers,
Liem

From: Christopher B Ferris [mailto:chrisfer@xxxxxxxxxx]
Sent: Friday, June 15, 2012 6:57 AM
To: Mark Nottingham
Cc: Nguyen, Liem Manh; openstack@xxxxxxxxxxxxxxxxxxx
Subject: [Openstack] WADL [was: v3 API draft (update and questions to the community)]

+1

Over-reliance on WADL will only make it more challenging to gracefully evolve the APIs such that implementations can be forwards and/or backwards compatible, especially when exchanging XML based on an XSD that is not carefully crafted with proper extensibility points incorporated throughout the schema design, unless we were to adopt XSD1.1 which has an optional open content model (but which has not yet seen wide adoption, sadly).

Cheers,

Christopher Ferris
IBM Distinguished Engineer, CTO Industry and Cloud Standards
Member, IBM Academy of Technology
IBM Software Group, Standards Strategy
email: chrisfer@xxxxxxxxxx
Twitter: christo4ferris
phone: +1 508 234 2986


-----openstack-bounces+chrisfer=us.ibm.com@xxxxxxxxxxxxxxxxxxx wrote: -----
To: "Nguyen, Liem Manh" <liem_m_nguyen@xxxxxx>
From: Mark Nottingham
Sent by: openstack-bounces+chrisfer=us.ibm.com@xxxxxxxxxxxxxxxxxxx
Date: 06/14/2012 08:34PM
Cc: "openstack@xxxxxxxxxxxxxxxxxxx" <openstack@xxxxxxxxxxxxxxxxxxx>
Subject: [Openstack] WADL [was: v3 API draft (update and questions to the community)]
Hi Liem,

I'm one of the folks who helped Marc get WADL off of the ground. At the time, my use cases were exactly as you describe: documentation (e.g., <https://github.com/mnot/wadl_stylesheets>) and testing.

Even back then, there was a lot of discussion in the community; e.g., see:
   http://bitworking.org/news/193/Do-we-need-WADL
   http://old.nabble.com/Is-it-a-good-idea-to-make-your-WADL-available--tc6087155r1.html
   http://www.25hoursaday.com/weblog/CommentView.aspx?guid=f88dc5a6-0aff-44ca-ba42-38c651612092

I think many of the concerns that were expressed then are still valid -- some even within these limited uses. In no particular order:

* People can and will use WADL to represent a "contract" to a service (really, an IDL), and "bake" client code to a snapshot of it in time. While it's true that the client and server need to have agreement about what goes on the wire and what it means, the assumptions around what guarantees WADL makes are not well-thought-out (in a manner similar to WSDL), making clients generated from it very tightly bound to the snapshot of the server they saw at some point in the past. This, in turn, makes evolution / extension of the API a lot harder than it needs to be.

* WADL's primitives are XML Schema datatypes. This is a horrible match for dynamic languages like Python.

* WADL itself embodies certain patterns of use that tend to show through if you design for it; these may or may not be the best patterns for a particular use case. This is because HTTP and URLs are very flexible things, and it isn't expressive enough to cover all of that space. As a result, you can end up with convoluted APIs that are designed to fit WADL, rather than do the task at hand.

>From what I've seen, many developers in OpenStack are profoundly uninterested in working with WADL. YMMV, but AFAICT this results in the WADL being done by other folks, and not matching the reality of the implementation; not a good situation for anyone.

What we need, I think, is a specification of the API that's precise, unambiguous, and easy to understand and maintain. I personally don't think WADL is up to that task (at least as a primary artefact), so (as I mentioned), I'm going to be proposing another approach.

Cheers,



On 15/06/2012, at 2:08 AM, Nguyen, Liem Manh wrote:

> IMHO, a well-documented WADL + XSD would say a thousand words (maybe more)...  And can serve as a basis for automated testing as well.  I understand that the v3 API draft is perhaps not at that stage yet; but, would like to see a WADL + XSD set as soon as the concepts are solidified.
>
> Liem
>
> -----Original Message-----
> From: openstack-bounces+liem_m_nguyen=hp.com@xxxxxxxxxxxxxxxxxxx [mailto:openstack-bounces+liem_m_nguyen=hp.com@xxxxxxxxxxxxxxxxxxx] On Behalf Of Mark Nottingham
> Sent: Tuesday, June 12, 2012 8:43 PM
> To: Gabriel Hurley
> Cc: openstack@xxxxxxxxxxxxxxxxxxx
> Subject: Re: [Openstack] [keystone] v3 API draft (update and questions to the community)
>
>
> On 13/06/2012, at 1:24 PM, Gabriel Hurley wrote:
>
>> Totally agree with all of Jay's points, and I also couldn't agree more with Mark on the importance of being crystal clear, and not operating on just a "common understanding" which is quickly misunderstood or forgotten.
>>
>> Ideally I'd like to see an OpenStack API feature contract of some sort... essentially a document describing the FULL list of features, how those parameters are controlled and how they would interact, and what a project should do if they do not implement an API feature (hopefully only for technical reasons such as Keystone paging with LDAP or swift with complex DB-esque operations). This isn't saying we should have a unified API spec, I'm talking solely about a contract for the features all APIs should strive to support.
>>
>> This would be a big project, but everyone would then have a common agreement about what the user experience of interacting with OpenStack should be. The project APIs as they stand are siloed and stunningly inconsistent, and I'd love to work toward fixing that.
>
> Absolutely.
>
> One of my other projects is to rewrite the API as a proper specification (in a style similar to an Internet-Draft, not that we'd necessarily publish it as one).
>
> I should have something to show soon; if you're interested in helping out, that'd be great.
>
> Cheers,
>
>
>> My two cents,
>>
>>   - Gabriel
>>
>>> -----Original Message-----
>>> From: openstack-bounces+gabriel.hurley=nebula.com@xxxxxxxxxxxxxxxxxxx
>>> [mailto:openstack-
>>> bounces+gabriel.hurley=nebula.com@xxxxxxxxxxxxxxxxxxx] On Behalf Of
>>> Mark Nottingham
>>> Sent: Tuesday, June 12, 2012 7:20 PM
>>> To: Jay Pipes
>>> Cc: openstack@xxxxxxxxxxxxxxxxxxx
>>> Subject: Re: [Openstack] [keystone] v3 API draft (update and questions to
>>> the community)
>>>
>>>
>>> On 13/06/2012, at 3:31 AM, Jay Pipes wrote:
>>>
>>>> This isn't necessarily true. Nova's compute layer goes through a number of
>>> steps to ensure a semi-transactional nature to certain operations like
>>> resizing. Certain times a query needs to indicate that it intends to make a
>>> reservation of resources (see quota/reservation system now .. this is the
>>> SELECT FOR UPDATE paradigm) and other times, the query doesn't care
>>> about such things. In the latter case, there aren't expectations that the list
>>> returned is 100% accurate according to the state of the database at a
>>> particular timestamp of when the transaction occurred. In this case, filters
>>> and optimistic pagination works perfectly fine, IMHO.
>>>
>>> That might work, but we need to be crystal-clear about the semantics of
>>> what we're giving back; having it understood between OpenStack projects
>>> isn't good enough.
>>>
>>> I.e., we're not building the APIs just for Horizon; they're for lots of folks, and
>>> subtle semantics -- even when well-documented, much less when they're
>>> not -- are often misunderstood.
>>>
>>> Cheers,
>>>
>>> --
>>> Mark Nottingham   http://www.mnot.net/
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> Mailing list: https://launchpad.net/~openstack
>>> Post to     : openstack@xxxxxxxxxxxxxxxxxxx
>>> Unsubscribe : https://launchpad.net/~openstack
>>> More help   : https://help.launchpad.net/ListHelp
>>
>>
>>
>> _______________________________________________
>> Mailing list: https://launchpad.net/~openstack
>> Post to     : openstack@xxxxxxxxxxxxxxxxxxx
>> Unsubscribe : https://launchpad.net/~openstack
>> More help   : https://help.launchpad.net/ListHelp
>
> --
> Mark Nottingham   http://www.mnot.net/
>
>
>
>
> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to     : openstack@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp

--
Mark Nottingham   http://www.mnot.net/




_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to     : openstack@xxxxxxxxxxxxxxxxxxx
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp