openstack team mailing list archive

[Ryan Lane <rlane@xxxxxxxxxxxxx>]

>> http://bogott.net/misc/osmpuppet.png
>>
>> That's what it looks like when you configure an instance using Open Stack
>> Manager, which is WikiMedia's VM management interface.  My main priority for
>> adding puppet support to Nova is to facilitate the creation and control of a
>> GUI much like that one.
>
>
>
> Can you explain how your solution works now? You want to inject data into
> the VMs in the proposal, but outside of designating the puppet master, all
> the data for variables and classes should be changes to the puppet master,
> not the instances. That's kind of the whole point of the puppet master.
>
> One thing you really seem to want is RBAC for the nova users.
>
> How are you getting the names for the recipes into your system? Is that sync
> with what is on the puppet master somehow or you are going to do data entry
> and it's all string matching?
>

Right now we are adding puppetvar and puppetclass attributes into LDAP
nodes. An instance uses cloud-init, installs puppet, points puppet at
the right puppet master, and does a puppet run. A puppet master waits
for certificate requests, verifies the instance is in LDAP, then signs
the certificate. The instance's catalogue is generated by the puppet
master, based on the LDAP entry.

>> On 1/26/12 5:03 PM, Andrew Clay Shafer wrote:
>>
>>
>> I'd also like to see more of a service oriented approach and avoid adding
>> tables to nova if possible.
>>
>> I'm not sure the best solution is to come up with a generic service for
>> $configuration_manager for nova core. I'd rather see these implemented as
>> optional first class extensions.
>>
>> This sounds intriguing, but I'll plead ignorance here; can you tell me
>> more about what this would look like, or direct me to an existing analogous
>> service?
>
>
> I don't think there is a good existing example, but I know if the defacto
> way to add functionality in nova is to add tables to the db, that's the path
> to operational and maintenance hell.
>
> That's not just for this integration, but in general.
>
> For openstack to become what it should be, Nova shouldn't be a monolithic
> app on a database.
>
> Even if you wanted to run this on the same node, it probably shouldn't be
> tables in the same database. It should be a separate services with it's own
> db user and scheme then be integrated by apis or maybe adding to wsgi.
>

Well, I think part of the problem is there doesn't seem to be any way
to extend nova core right now. The API is extendable, but actions
generated by existing API can't be modified without hacking at the
core code. This includes database code/schema as well.

Having the tables in the same database isn't problematic as long as we
ensure there aren't naming conflicts. Often the way to handle this is
to have a standardized naming prefix.

> I'm confused how you want to run puppet exactly. The site.pp would typically
> live on the puppet master.
>
> Can you explain more about what you are thinking or how your current
> solution works?
>

We are currently using the LDAP approach with a puppet master, but
it's problematic with what we're actually doing.

In our implementation everything is controlled via puppet, and our
puppet repo is kept in a gerrit repo. We have a branch for production
and one for our pre-production cloned environment. One project
contains our pre-production clone (testlabs). Every other project is
meant as a space to build things, and then puppetize them. The
services tested in other projects are moved into testlabs via a
cherry-pick, then tested there, then cherry-picked across to
production.

Currently all projects use the testlabs puppet branch, because the
centralized puppet master must run on a single branch. What we want is
for each project to have their own branch of the repo, so that they
can test all of their changes in their own branch without needing
code-review and without needing to possibly clobber other projects.

The problem we are running into is that each project needs its own
puppet master (and no, puppet environments don't work), as puppet
isn't multi-tenant. The good thing is, we don't really *need* a puppet
master. Since we have all of the manifests in a single repo, we can
just checkout the entire repo, and have the instance run the puppet
configuration directly. We can create a gerrit branch on project
creation, and have all instances configured to pull/push using that
branch.

So, what we really want is a way to stick the repo and the node
configuration into something that can be accessed on instance
creation.

> I think it would be sweet if nova and the dashboard (and probably keystone
> too) had a standardized way to add integrated functionality. I don't believe
> nova core should be reimplementing/duplicating functionality and logic in
> other systems.
>
> The goal of interacting with the instances through a shared interface is a
> good one, I'm not against that, I just want to see less deep coupling to
> accomplish it.
>

Yes, a shared interface is what we'd like as well. My main motivation
for getting this moved into nova is that I want people to be able to
use the cli tools in addition to web tools. This was also my
motivation for wanting DNS support added ;).

>> I may be misunderstanding you, or my blueprint may be unclear.  Available,
>> Unavailable, and Default don't refer to the availability of classes on the
>> puppet master; rather, they refer to whether or not a class is made
>> available to a nova user for a given instance.  An 'available' class would
>> appear in the checklist in my screenshot.  An Unavailable class would not.
>> A 'default' class would appear, and be pre-checked.  In all three cases the
>> class is presumed to be present on the puppet master.
>
>
> I already asked this, but what keeps that in sync with the puppet master?
>

Nothing, this is user-defined and global defined based on the
end-user's knowledge of the puppet repo. The puppet master has no way
of saying which variables are available, at minimum, since it's
possible for templates to require variables that aren't defined in
manifests.

> Personally, I'd rather see an integration that has a per user configuration
> to a puppet master that stays in sync than the RBAC per module.

Well, we aren't aiming at RBAC, but rather aiming at giving a user a
list of classes and variables available in a specific project. Think
of one project mostly wanting to use the CLI, another using horizon,
and another using OpenStackManager (my MediaWiki extension). Without
nova knowing what's available in a project there's no way for each
interface to display this information.

>>> I also think managing a site.pp is going to be inferior to providing an
>>> endpoint that can act as an eternal node tool for the puppet master.
>>> http://docs.puppetlabs.com/guides/external_nodes.html
>>
>> In which case nova would interact directly with the puppet master for
>> configuration purposes?  (I don't hate that idea, just asking for
>> clarification.)
>
>
> That's puppet's model. Whether you use a site.pp, or external nodes. I'm
> unclear how you want to do it. Can you explain how your system works now?
>>

As described above, we feel that using a non-centralized puppet system
is preferable (for us). Of course, we'd like to have broad support.

>>> One other point, that you might have thought of, but I don't see anywhere
>>> on the wiki is how to handle the ca/certs for the instances.
>>
>> I believe this (and your subsequent question) falls under the heading of "
>> Instances are presumed to know any puppet config info they need at creation
>> time (e.g. how to contact the puppet master). "  Important, but outside the
>> scope of this design :)
>
>
> Thinking through this is actually critical for any standardized puppet
> integration in my opinion. The solution is a prerequisite before considering
> anything else.

Yes, this is an issue. We'd need to figure this out. In the case where
you don't have a puppet master, this isn't necessary, but for it to be
generic, it would need to work. It's possible to sign through puppet's
API, though, so it should be easy enough to have the puppet driver
handle that, if needed.

>> So that I understand your terminology... are extensions like the quotas or
>> floating ips considered 'core nova'?
>
>
> There is not a bright line especially with the way things have evolved to
> now, but I would say floating IPs should definitely be core functionality.
> Quotas may be debatable, but I think it is defensible, though part of me
> feels like some of that kind of permission functionality might be better
> decoupled.
>
>> Thanks again for your input!  Clearly it would be best to hash this out at
>> the design summit, but I'm hoping to get at least a bit of coding done
>> before April :)
>
>
> I hope to be there. I do like the idea, I just want to do what's best for
> OpenStack.
>

Agreed. It may be that some form of hook system, or some other way of
extending nova without hacking core would be a more appropriate way of
handling things like this. Of course, an extensions system also makes
compatibility between vendors much more difficult.

- Ryan Lane