openstack team mailing list archive

[Jorge Williams <jorge.williams@xxxxxxxxxxxxx>]

Comments inline

On Aug 22, 2011, at 9:05 PM, Vishvananda Ishaya wrote:

> Inline
> On Aug 22, 2011, at 4:59 PM, Jorge Williams wrote:
> 
>> Hi Vish,
>> 
>> I don't have a problem moving the spec out of docs manuals and into another project even the nova repo.   But, I do have a number of issues with the approach that you're proposing. First, I think that fundamentally there should be a decoupling of the spec and the implementation.   If you have the spec generated from the code than essentially the spec is whatever the code does. It's very difficult to interoperate with specs that are generated this way as the specs tend to be very brittle and opaque (since you have to study the code). If you introduce a  bug in the code that bug filters it's way all the way to the spec (this was a big problem with SOAP and CORBA). It's difficult to detect errors because you cant validate. By keeping the implementation and the spec separate you can validate one against the other. 
> 
> The spec SHOULD BE exactly what the code does, verifiably.  I'm proposing that we have exactly the existing document, only that the xml and json examples in the spec are actually generated from the code so that we know they are accurate.  This is a minor point to me though, and could be accomplished by testing the spec against the code as well.

Let's say you generate the samples from the code.  Everything works great.  The samples represent exactly what the code is doing.  A client developer takes a look at the spec samples and develops an app against it.  Then a merge request goes through that changes the format of the JSON inadvertently in a subtle way.  Now the client is out of synch and breaks. The client dev looks back at the spec,  not only has the code changed but the spec has changed too since it's generated, -- the client therefore  assumes that the bug is his bad -- so he changes the code to match the spec.  Things work great, until the service developer notices that the merge changed the format of the JSON and changes it back to what it used to be -- now the client is broken again....Anyway that illustrates why these type of approaches are brittle. 

Here's what I think is a better approach.  Design your API and write the samples by hand. You may design schema for your XML representation and verify that your samples validate. Write tests that check the generated XML and JSON  that comes out of your service against the hand written versions and, if applicable, against the schema.  You know the API is done when the tests pass.  If a merge  comes in that changes the representation then the tests will fail. Either way there's not much you can do in your code that can change's the spec from under your client's feet because the spec and the code are separate.  Sure during early development the spec and the code may be out of synch, but at least the client knows where you're headed and can work to meet you there.

So yes, you should be using those samples for testing. You just shouldn't be generating the samples from the code. The code SHOULD DO what the spec says verifiably -- not the other way around.


> 
>> 
>> Second, I don't think that the core OpenStack API should change with every OpenStack release. There are a number of efforts to provide multiple implementation of an existing OpenStack API.  We should encourage this, but it becomes difficult if the core spec is in constant flux.  Certainly you can use the extension mechanism to bring functionality out to market quickly, but the process of deciding what goes into the core should be more deliberate. Really good specs, shouldn't need to change very often, think HTTP, X11, SMTP, etc. We need to encourage clients to write support for our spec and we need to also encourage other implementors to write implementations for it. These efforts become very difficult if the spec is in constant flux.
> 
> Incrementing the version number is only a problem if we fail to support the old versions.  At the rate we are adding new functionality, I think we will easily need a new spec every six months for the foreseeable future.  If there are no reasonable changes in a six month period, we can skip a release and not have a new version of the spec
> 

Things have been changing because we've been working hard to realize what the code API should be.  Once we have this settled, I don't see a big reason why the core spec should change every 6 months. In fact, functionality wise the core API you see in 1.1 today hasn't really changed all that much in comparison to the 1.0 API.


-jOrGe W.

This email may include confidential information. If you received it in error, please delete it.