openstack team mailing list archive

[Ziad Sawalha <ziad.sawalha@xxxxxxxxxxxxx>]

Hi Paul - thank you for the good feedback.

I'm going to address your points individually below, but before I want to
to set some context and address some of your broader concerns.

The 2.0 API for Keystone is released and multiple implementers are already
working on it (in fact, we at Rackspace have just released ours). There
were many calls for comments on the API throughout the year, but we locked
down the spec finally in September to try to deliver an implementation in
time for Diablo.

The focus for Essex, as voiced by the community in the summit in Boston,
is on maturing the OpenStack implementation and adding some extensions
like role based access control (RBAC). Therefore, there is not much of a
discussion going right now about the next version of the API (although
this could be it starting!). So the silence is not the symptom of a closed
discussion, but the absence of one at this time.


See below...



On 11/23/11 4:21 PM, "Paul Querna" <pquerna@xxxxxxxxxx> wrote:

>Hello Y'all,
>
>I'm writing the list with some of my thoughts as an user of the
>Keystone 2.0 API.
>
>Generally, I believe the API is too complicated, has too many 'hacks'
>for backwards compatibility put into the wrong places, and pushes too
>much logic into consumers and service implementers.
>
>My experience with Keystone comes from several separate projects using
>the API:
>
>1) A new Rackspace Service, not yet publicly announced, which uses the
>Keystone API to validate tokens. (We wrote our own internal library in
>Node.js for interacting with Keystone)
>
>2) In Apache Libcloud, I implemented support for the Keystone API,
>specifically to get tokens for a service like OpenStack Nova,
>Rackspace Cloud Servers, Load Balancers or Cloud Files.
>
>3) I also work with the team implementing a new Rackspace Control
>Panel project. This project uses Libcloud for it's interaction with
>Keystone, but has several more use cases beyond simple Username and
>API Key validation.
>
>
>Part 1: Specific Issues
>
>A) The Token Validation API is fail deadly, because of support for
>Tokens without a Tenant ID scope:
>
>  
><http://docs.openstack.org/api/openstack-identity-service/2.0/content/GET_
>validateToken_v2.0_tokens__tokenId__Admin_API_Service_Developer_Operations
>-d1e1356.html>
>
>When you are implementing a service that needs to validates tokens,
>you pass in the tenant scope as the belongsTo parameter with the
>Tenant ID.  However, this parameter is optional.  If a malicious
>Tenant Id is passed in, for example if a service doesn't perform
>sufficient validation, like letting a user pass in a & into the
>tenantId, a token is considered valid for _all_ contexts.  Now, in
>theory, you should be looking at the roles provided under the user,
>and the examples given in the OpenStack documentation echo back the
>validated Tentant ID to you, however in practice, and as seen in
>production environments, this response body includes a default
>identity role, and does not echo back the validated Tenant ID.

Tokens without scope are supported by the API - we had requests with use
cases for it - but it is not required. In fact, the Rackspace
implementation always returns a scoped token.

This is one of those examples I think you refer to where we had to keep
the spec loose enough to meet the needs of all parties in the discussion
and hit the dates we were aiming for. But we are always working on
improving things and if you have a suggestion for how to improve this one
we're listening. We accept contributions voraciously and have been know to
implement other peoples good ideas even when they don't come with code :-)

In fact, I know at least one $BigCo that is working on a proposal to
improve this. But as a user, your input will be weighted highly so feel
free to email, blueprint, or submit a proposal.

>
>
>B) Requiring consumers to pass Tenant IDs around is not a common
>pattern in other cloud APIs.  A consumer was already keeping track of
>their username, apikey, and temporal token, and now they essentially
>need to keep another piece of information around, the Tenant ID.
>This seems like it is an unneeded variable.  For example, Amazon
>implements AWS Identity and Access Management by changing the API key
>& secret that is used against the API depending on the role of the
>account -- this hides the abstraction away from both client libraries
>and validating services -- they still just care about the API key and
>secret, and do not need to pass around an extra Tenant ID.

This sounds like a concern with the OpenStack implementation and not the
API spec.

The Keystone API spec doesn't require consumers to pass Tenant IDs around.
It even allows for a full implementation without the consumer having to
know or manage their tenant IDs. We've done that at Rackspace where you
auth with your credentials, get URLs back for the services you have, and
then you call those URLs using your token. Granted, the tenant ID (a.k.a
account numbers) is embedded in the URL, but this comes from the Rackspace
Cloud Servers and Swift API and is not a Keystone API design requirement.

Can you give an example of where you are having to maintain tenant IDs?

>
>
>C) Requiring Services to encode the Tenant ID into their URLs is not a
>common design, and can cause issues.  By encoding identity both into
>the Token and the Tenant in the URL, there are now multiple attack
>vectors from a security perspective, and can make URL routing in some
>frameworks more painful.

As I mention above, that's not a Keystone requirement and was not driven
by Keystone. As user who has multiple accounts myself I find the model
works for me, but it is certainly not forced or required upon services in
order to work with Keystone.

I know that much thought went into that design taking into consideration
things like rate limiting, caching, etcŠ

This is probably a broader OpenStack API conversation. Fair warning, those
conversations have historically lead to bike and blood shedding :-)


>
>
>D) The Service Catalog makes it difficult to run testing, beta, and
>staging environments.  This is because in practice many services are
>not registered in the service catalog.  To work around this, we
>commonly see that a regex is used to extract the Tenant Id from
>another URL, and then the client builds a new URL.  You can see this
>even being recommended by Rackspace in the Disqus comments ont he
>Cloud DNS API here:
><http://docs.rackspace.com/cdns/api/v1.0/cdns-devguide/content/Service_Acc
>ess_Endpoints-d1e753.html>

Is the problem "in practice many services are not registered in the
service catalog"?

I'm wondering if we need to make it easier for services to register
themselves in the catalog or evolve the service catalog to handle use
cases like staging, beta, and testing environments?

Specific to the link you post up there, we're working with the DNS team to
get their endpoints registered in the Rackspace auth servers. That has
historically not been as easy it should.

>
>
>E) The Service catalog should make it easy to expose many tenants to
>the same service.  Currently you need to build a unique tentant
>specific URL for each service.   The most common use case is enabling
>all or a large set of tenants to access the same service, and it seems
>like this use case is not well covered.

I'm not sure I get the pain point you have here, but I can talk to the
Keystone API aspect of this.

Given that Keystone has support for services that want to embed a tenant
in their URL (as well as those that don't), the spec includes a definition
of the service catalog that doesn't assume either. All the catalog focuses
on is providing endpoints and data about each endpoint.

Are you asking specifically for support to return one endpoint and have a
list of tenants that
can exist under that endpoint? Something like this:


"serviceCatalog":[{
	"name":"Cloud Servers",
	"type":"compute",
	"endpoints":[{
		"publicURL":"https://compute.north.host.com/v1/";,
		"tenants": ["t1000", "t2000", "t3000", etc...]
		}]


It can be done. We've seen (and as you pointed out) the usability
decreases with the number of use cases we support. For example, that use
case above would not work for anyone using tenants in their URLs, so
they;d be limited to one tenant in the array and someone could feasibly
complain about having to put one tenant in an array every time.

But if this is a needed use case for consumers it can be done.

The process we would follow be be to add it as an extension (since the
core API is released) and f it gains traction, propose it for core in the
next API. The next API will likely be a topic brought up in the next
summit in April.

>
>
>F) Features like API Keys, are marked as Extensions to the API, and
>not part of the core API.

Correct. We asserted that all the backends we would support (and that
operators of OpenStack would be running) have support for passwords. Not
all backends have support for API Keys, though. If API Keys go into the
core API they would need to be supported by all implementers and that
raises the bar to deploying OpenStack in some enterprises.

Example: say Rackspace wants to deploy OpenStack internally for employee
use and wants to integrate it with an enterprise directory running LDAP.
Password authentication is a simple tie in. If we also had to find a way
to support issuing employees API keys, that becomes a much more complex
project. As most enterprise directories don't support API keys natively.

>
>
>G) Support for multifactor authentication (MFA) is not currently
>available in any way.  This is very worrisome since the only 'in core'
>Authentication mechanism is a username and a password.  With previous
>systems, where the Username and Password were not 'owned' by a service
>like Keystone, products like the control panel could implement MFA
>themselves, but now that you can authenticate to the backend APIs
>using only a password, Keystone must also support MFA.   Password Auth
>is very different from API Key auth -- yes, they are both in theory
>randomly generated and big, but in practice Passwords are weak, and
>reused widely, while only API keys are big and random.

Same argument as above. The extension mechanism is robust and would allow
any operator to add an MFA extension and offer that to their customers.
Putting it in core would make the prerequisites for deploying OpenStack
heavy.

>
>
>H) There doesn't seem to be an open discussion about where the
>Keystone API is going -- I hear mumbles about $OtherBigCompanies
>wanting to extend the Keystone APIs in various ways, but there is no
>discussion on this mailing list.  I know of at least 4 implementations
>of Keystone within just Rackspace, but all of them are adding custom
>extensions to even accomplish their baseline of use cases.

Correct. As I mentioned initially, all the focus now is on documenting and
stabilizing.

There are blueprints out there and work being done on extensions which
will "show up". IMO, that's a sign of an active community and evidence
that the extension mechanism allows people to work on their features and
evolve the domain without having to drive the core project one way or the
other or be gated by others.

The four implementations within Rackspace is a testament to the level of
innovation going on. My expectation is that the implementation that best
serves our customers will prevail and the learnings should feed back into
the Keystone API to continue evolving it.

>
>
>
>Part 2: My Thoughts / Recommendations
>
>
>A) More open discussion about the issues in Keystone.  Issues won't
>get fixed until there is an open and honest dialog between the various
>groups which depend on Keystone.  Identity, Authentication and
>Authorization must be rock solid -- everything else in OpenStack and
>many other commercial products are built upon it.

I fully support that. Thank you for driving it. There are a number of
threads and items you bring up here that I think could and should drive
blueprints and new features.

>
>
>B)  I believe fundamentally that the data model for service catalogs
>as exposed to the user is over complicated and broken.  I would prefer
>that you hit the auth service with your username + apikey or password.
>This returns to you a list of tenantIds aka billable cloud accounts.
>Each of those records have a unique API Token. When you hit a service
>using that Token, you do not include a TenantId. The service validates
>the token, and its always in the context of a single tenant.
>
>I believe we should consider changing the get Token API to return
>something more like this:
>
>  {access: [
>        {tenantId: "42",
>         token: 'XXXXXX',
>         serviceCatalog: [{type: 'compute', baseURL: 'https://..../',
>region: 'us-west'} ... ]
>        }]};
>The major change here is to make tenants a first level object, instead
>of being a component of the service catalog --  each tenant has their
>potentially unique service catalog, and each tenant has a unique API
>token that is valid for this users's roles on it.

We intentionally made the API token centric (other standards like OAUTH
and SAML have come to that conclusion). That's the most flexible model.
For example, Rackspace has at least two tenants for each customer (cloud
files and cloud servers). We don't want to force customers to have
separate tokens for each. Your proposal would force us to do that.

The current spec lets our customers manage only one token and we, as an
operator, can decide what goes in the token. That's what we were solving
for. Maybe there's a simpler model for the service catalog, but I haven't
seen it yet. But a good topic to keep iterating on.


>
>This would slightly complicate "service users" with access to many
>tenants, but it would massively simplify the common use cases of a
>user with a small number of tenants, and for a service which needs to
>validate the token.  A service user with access to many tenants would
>need to fetch a unique token for each tenant, but this is the place to
>put the complexity -- people writing a service that spans hundreds or
>thousands of tenants are already doing something complicated --
>fetching a unique auth token is the least of their issues.
>
>This reduces the number of variables on both the consumer and the
>service for passing around, and makes it less fail-deadly.
>
>This approach also eliminates the need to encode the tenant ID into
>the URLs of services.
>
>
>C) There needs to be a reduction in the number of implementations of
>Keystone, and a reduction in the number of extensions needed to even
>operate a reasonable baseline of services.   More things, like API
>keys and how one would do a multifactor authentication backend must be
>part of the core specification, and we should strive to have the
>reference implementation actually used by companies in the ecosystem,
>instead of how today pretty much every $BigCo using OpenStack is
>implementing their own Keystone service from scratch -- people will
>always have custom needs, but those should be plugins to a solid
>reference implementation as much as possible, instead of today where
>everyone is rebuilding it from scratch.
>

We can do better, yes. And we can hope to settle towards one
implementation. There's a world of identity solutions out there and we
didn't start Keystone to enter that market, but to enable easier
integration in a cloud paradigm.

And not everyone is going to want to run a Python identity server. We've
split the API spec from the implementation to allow for diversity.

>
>
>
>Thoughts?

Thank you again, Paul. Good topics. And please keep the discussion going.
I'd be happy to continue on this thread and also to work with anyone on
blueprints or enhancements to Keystone and the API.

I have a few questions for you and would like to hear your thoughts on how
to optimize the user experience, especially around the service catalog.

Thank you!

>
>Thanks,
>
>Paul
>
>_______________________________________________
>Mailing list: https://launchpad.net/~openstack
>Post to     : openstack@xxxxxxxxxxxxxxxxxxx
>Unsubscribe : https://launchpad.net/~openstack
>More help   : https://help.launchpad.net/ListHelp