openstack team mailing list archive
-
openstack team
-
Mailing list archive
-
Message #02266
Re: Chef Deployment System for Swift - a proposed design - feedback?
Judd,
Proxy servers with the allow_account_management flag will accept PUT
and DELETE requests on accounts. On most deployments I would think
that this functionality would have limited access and locked down.
That is why I was suggesting it be broken out into a different roll.
--J
On Fri, May 6, 2011 at 10:35 AM, Judd Maltin <judd@xxxxxxxxxxxxxx> wrote:
> Hi Joe -
>
> Answers inline:
>
> On Wed, Apr 27, 2011 at 9:55 PM, Jay Payne <letterj@xxxxxxxxx> wrote:
>>
>> Judd,
>>
>> I'm not that familiar with Chef (I'll do some research) but I have a
>> couple of questions and some thoughts:
>>
>> 1. Is this for a multi-server environment?
>
> Yes.
>>
>> 2. Are all your proxy nodes going to have "allow_account_management =
>> true" in the configs? It might be a good idea to have a second proxy
>> config for account management only
>
> I'm not sure - I don't know about the performance impact of such a config.
>>
>> 3. Have you looked at using swauth instead of auth?
>
> Not yet.
>>
>> 4. Have you thought about an admin or client node that has utilities
>> on it like st and stats-report?
>
> Good suggestion. An admin node which is part of the cluster and not subject
> to the performance sensitivity of line of business nodes.
>>
>> 5. How where will you do on-going ring management or changes?
>
> Rings are defined in a Chef "DataBag". When the databag is updated, Chef
> detects the file change. When the chef-client runs on the nodes, they pick
> up the config changes. Scheduling and orchestrating the chef-client runs
> will also be expressed in the data-bag (or from an orchestration server)
>>
>> 6. I would think about including some type of functional test at the
>> end of the deployment process to verify everything was created
>> properly and that all nodes can communicate.
>
> Good point - adding in validation steps and final functional test.
>>
>>
>>
>> --J
>>
>> On Wed, Apr 27, 2011 at 6:18 PM, Judd Maltin <judd@xxxxxxxxxxxxxx> wrote:
>> > Hi Folks,
>> >
>> > I've been hacking away at creating an automated deployment system for
>> > Swift
>> > using Chef. I'd like to drop a design idea on you folks (most of which
>> > I've
>> > already implemented) and get feedback from this esteemed group.
>> >
>> > My end goal is to have a "manifest" (apologies to Puppet) which will
>> > define
>> > an entire swift cluster, deploy it automatically, and allow edits to the
>> > ingredients to manage the cluster. In this case, a "manifest" is a
>> > combination of a chef databag describing the swift settings, and a
>> > spiceweasel infrastructure.yaml file describing the OS configuration.
>> >
>> > Ingredients:
>> > - swift cookbook with base, proxy and server recipes. proxy nodes also
>> > (provisionally) contain auth services. storage nodes handle object,
>> > container and account services.
>> > -- Base recipe handles common package install, OS user creation. Sets
>> > up
>> > keys.
>> > -- Proxy recipe handles proxy nodes: network config, package install,
>> > memcache config, proxy and auth package config, user creation, ring
>> > management (including builder file backup), user management
>> > -- Storage recipe handles storage nodes: network config, storage device
>> > config, package install, ring management.
>> >
>> > - chef databag that describes a swift cluster (eg:
>> > mycluster_databag.json)
>> > -- proxy config settings
>> > -- memcached settings
>> > -- settings for all rings and devices
>> > -- basic user settings
>> > -- account management
>> >
>> > - chef "spiceweasel" file that auto-vivifies the infrastructure: (eg:
>> > mycluster_infra.yaml)
>> > -- uploads cookbooks
>> > -- uploads roles
>> > -- uploads the cluster's databag
>> > -- kicks off node provisioning by requesting from infrastructure API
>> > (ec2 or
>> > what have you) the following:
>> > --- chef roles applied (role[swift:proxy] or role[swift:storage])
>> > --- server flavor
>> > --- storage device configs
>> > --- hostname
>> > --- proxy and storage network details
>> >
>> > By calling this spiceweasel file, the infrastructure can leap into
>> > existence.
>> >
>> > I'm more or less done with all this stuff - and I'd really appreciate
>> > conceptual feedback before I take out all the non-sense code I have in
>> > the
>> > files and publish.
>> >
>> > Many thanks! Happy spring, northern hemispherians!
>> > -judd
>> >
>> > Judd Maltin
>> > T: 917-882-1270
>> > F: 501-694-7809
>> > A loving heart is never wrong.
>> >
>> >
>> >
>> >
>> > _______________________________________________
>> > Mailing list: https://launchpad.net/~openstack
>> > Post to : openstack@xxxxxxxxxxxxxxxxxxx
>> > Unsubscribe : https://launchpad.net/~openstack
>> > More help : https://help.launchpad.net/ListHelp
>> >
>> >
>
>
>
> --
> Judd Maltin
> T: 917-882-1270
> F: 501-694-7809
> A loving heart is never wrong.
>
>
>
>
References