openstack team mailing list archive

[Jorge Williams <jorge.williams@xxxxxxxxxxxxx>]

On Feb 15, 2011, at 1:06 PM, Justin Santa Barbara wrote:


How would this work if someone didn't run  a volume service or glance? Should the api listen for that?

My expectation is that if someone didn't run a volume service, we should expose that just as if there were insufficient resources (because that's not far from the case.)  We'd return an error like "no resources to satisfy your request".  That way there's only one code path for quota exhausted / zero quota / no volume service / all disks full / no 'HIPAA compliant' or 'earthquake proof' volumes available when a user requests that.


A better approach is to simply provide a service catalog with list of endpoints, you can easily detect whether a volume service  is available etc. This allows you to detect what services are available with a single request, rather than polling for multiple failures.  Think of writing a control panel and having a long list of services (image, volumes, network, etc).  Do you want to make separate calls??  Do you have images?  volumes?  networks?  etc etc.  A service catalog allows you to make a single call and that gives you an inventory of what's available you can then decide what to enable and disable in your control panel.

For glance, I don't know - how is it even possible to boot an instance without an image?

You may have a list of stock images that you support within compute and do without a full image service implementation (translation, cataloging, etc).


 We shouldn't be relying on extensions for Cactus.  In fact, I'd rather leave out extensions until we have a solid use case.  You may be saying that volumes will be our test-use case, but I think that will yield a sub-optimal API.


I see extensions doing a few things. First, it gives a way for other developers to work on and promote additions to the api without fighting to get them into core at first.

While I agree that's a good goal, I don't think we should rely on it for our core services, because it will give a sub-optimal experience.  I also think that this extension element may well be worse than simply having separate APIs.  Right now I think we're designing in a vacuum, as you say.

I don't see our core services relying on extensions.  Once we design APIs for our core services, the core APIs should have enough functionality to support core implementation.  Extensions are there to support functionality that is out of the core.  New features etc.  It allows us to innovate.


Can you explain how it would yield a sub-optimal api?
It would yield a sub-optimal API precisely because the process of fighting to get things into core makes them better.  If you don't believe that, then we should shut down the mailing list and develop closed-source.


I totally believe that fighting to get things into core will make things better.  Having extensions doesn't prevent this from happing I would argue that it encourages folks to develop stuff and show it off. If you have a great idea, you can show it working, if clients like it they will code against it and you can create incentive from getting it into the core.

Another thing I would note is that not everything belongs in the core -- there's always a need  for niche functionality that may be applicable only to a single operator or group of operators.  Troy gave a really great example for Rackspace with backup schedules -- and that's just one example there are others -- for Rackspace there are  features will likely never make it to the core because they require a very specific support infrastructure behind them.  With extensions we can add these features without breaking clients.


A less meta reasoning would be that when we design two things together, we're able to ensure they work together.  The screw and the screwdriver didn't evolve independently.  If we're designing them together, we shouldn't complicate things by use of extensions.

Again, our core services should stand on their own -- without extensions -- extensions are there to support new features in a backwards compatible way, to allow operators to differentiate themselves, and to offer support for niche functionality.



I don't think that anyone is proposing that a volume API be entirely defined as an extension to OpenStack compute. The volume extension servers simply as an example and it  covers the case for mounting and un-mounting  a volume.  If we can figure out a way of doing this in a general way we can always promote the functionality to the core.

I don't disagree that there should be core apis for each service, but that in the long run, there may not be a single api. Glance already doesn't have an api in the openstack 1.1 spec. What about Swift?

OK - so it sounds like volumes are going to be in the core API (?) - good.

No more like, there will be a core API for managing volumes that is different from the Compute API.


Let's get that into the API spec.  It also sounds like extensions (swift / glance?) are not going to be in the same API long-term.  So why do we have the extensions mechanism?

Until we have an implemented use case (i.e. a patch) that uses the extensions element, I don't see how we can spec it out or approve it.  So if you want it in v1.1, we better find a team that wants to use it and write code.  If there is such a patch, I stand corrected and let's get it reviewed and merged.


I would reiterate what Troy said, we have a need for the extension mechanism today in Rackspace.


I would actually expect that the majority of the use cases that we want in the API but don't _want_ to go through core would be more simply addressed by well-known metadata (e.g. RAID-5, multi-continent replication, HPC, HIPAA).



In general, I like the organic model with promotion of useful extensions into core.  I'm 100% opposed to extensions being used for our initial functionality.  I'm also very concerned by the idea that we think we can radically change these APIs.  If we're talking about spinning images out into a separate API in future, or moving volumes into the core, that would be a major revision, because we'd break clients beyond simply requiring them to tolerate additional elements.  Versioning helps here, but note that AWS has _never_ deprecated a call afaik.  Versioning is a purely linear evolution, and so if I want a new feature, that means I have to use the refactored APIs?  It sounds like we're designing the OpenStack API v0.1, otherwise we're going to have some very angry developers here.


I'm not sure what your concern is here.  Sure you can change an API though extensions, but extensions allow those changes to occur in a compatible manner that won't break clients.  Extensions allow versions to change less frequently, so there's less of a need for a client to change especially if the client isn't interested in the new extension. Note that extensions don't modify the core API, they extend it.


One way to mitigate this is to create C, PHP, Python, Ruby and Java wrappers as part of OpenStack, and make _that_ the stable API for now.  The wrappers would have to hide any API changes, and thus we're not externalizing the pain of API changes.  I think we desperately need this anyway (well, at least one language binding) to support simple integration testing.

I totally agree that we need to have good language binding support -- and that a good portion of our clients will perceive the language binding as the API.  So I'm all for the development of good language bindings.  We still have to go through the process of designing the protocol as an API not only because we'll always have clients that will want to interact with us at that level, but because providing access via REST interfaces gives us a lot of advantages in terms us scalability.

-jOrGe W.