openstack team mailing list archive

[Jason Cannavale <jason.cannavale@xxxxxxxxxxxxx>]

RCB deploy has a set of chef cookbooks we use for diablo all in one testing (minus swift) in addition to the work we have been doing with the crowbar team.

Since we seem to be adding to the problem, we'd be happy to collaborate on the consolidation..

Jason

On Feb 6, 2012, at 20:56, "Jesse Andrews" <anotherjesse@xxxxxxxxx> wrote:

> I know that the RCB deploy team works with the Crowbar team on chef
> recipes for that project.
> 
> Regarding the github.com/ansolabs & github.com/rcb recipes - I'll have
> to delegate to Vishy who worked on those.
> 
> Jesse
> 
> On Mon, Feb 6, 2012 at 6:07 PM, Jay Pipes <jaypipes@xxxxxxxxx> wrote:
>> Hi Stackers,
>> 
>> tl;dr
>> -----
>> 
>> There are myriad Chef cookbooks "out there" in the ecosystem and locked up
>> behind various company firewalls. It would be awesome if we could agree to:
>> 
>> * Align to a single origin repository for OpenStack cookbooks
>> * Consolidate OpenStack Chef-based deployment experience into a single
>> knowledge base
>> * Have branches on the origin OpenStack cookbooks repository that align with
>> core OpenStack projects
>> * Automate the validation and testing of these cookbooks on multiple
>> supported versions of the OpenStack code base
>> 
>> Details
>> -------
>> 
>> Current State of Forks
>> ======================
>> 
>> Matt Ray and I tried to outline the current state of the various OpenStack
>> Chef cookbooks this past Thursday, and we came up with the following state
>> of affairs:
>> 
>> ** The "official" OpenStack Chef cookbooks **
>> 
>> https://github.com/openstack/openstack-chef
>> 
>> These chef cookbooks are the ones maintained mostly by Dan Prince and Brian
>> Lamar and these are the cookbooks used by the SmokeStack project. The
>> cookbooks contained in the above repo can install all the core OpenStack
>> projects with the exception of Swift and Horizon.
>> 
>> This repo is controlled by the Gerrit instance at review.openstack.org just
>> like other core OpenStack projects.
>> 
>> However, these cookbooks DO NOT currently have a stable/diablo branch --
>> they are updated when the development trunks of any OpenStack project merges
>> a commit that requires deployment or configuration-related changes to their
>> associated cookbook.
>> 
>> Important note: it's easy for Dan and Brian to know when updates to these
>> cookbooks are necessary -- SmokeStack will bomb out if a
>> deployment-affecting configuration change hits a core project trunk :)
>> 
>> These cookbooks are the ONLY cookbooks that contain stuff for deploying with
>> XenServer, AFAICT.
>> 
>> ** NTT PF Lab Diablo Chef cookbooks **
>> 
>> https://github.com/ntt-pf-lab/openstack-chef/
>> 
>> So, NTT PF Lab forked the upstream Chef cookbooks back in Nov 11, 2011,
>> because they needed a set of Chef cookbooks for OpenStack that functioned
>> for the Diablo code base.
>> 
>> While Nov 11, 2011, is not the *exact* date of the Diablo release, these
>> cookbooks do in fact work for a Diablo install -- Nati Ueno is using them
>> for the FreeCloud deployment so we know they work...
>> 
>> ** OpsCode OpenStack Chef Cookbooks **
>> 
>> Matt Ray from OpsCode created a set of cookbooks for OpenStack for the
>> Cactus release of OpenStack:
>> 
>> https://github.com/mattray/openstack-cookbooks
>> http://wiki.opscode.com/display/chef/Deploying+OpenStack+with+Chef
>> 
>> These cookbooks were forked from the Anso Labs' original OpenStack cookbooks
>> from the Bexar release and were the basis for the Chef work that Dell did
>> for Crowbar. Crowbar was originally based on Cactus, and according to Matt,
>> the repositories of OpenStack cookbooks that OpsCode houses internally and
>> uses most often are Cactus-based cookbooks. (Matt, please correct me if I am
>> wrong here...)
>> 
>> ** Rackspace CloudBuilders OpenStack Chef Cookbooks **
>> 
>> The RCB team also has a repository of OpenStack Chef cookbooks:
>> 
>> https://github.com/cloudbuilders/openstack-cookbooks
>> 
>> Now, GitHub *says* that these cookbooks were forked from the official
>> upstream cookbooks, but I do not think that is correct. Looking at this
>> repo, I believe that this repo was *actually* forked from the Anso Labs
>> OpenStack Chef Cookbooks, as the list of cookbooks is virtually identical.
>> 
>> ** Anso Labs OpenStack Chef Cookbooks **
>> 
>> These older cookbooks are in this repo:
>> 
>> https://github.com/ansolabs/openstack-cookbooks/tree/master/cookbooks
>> 
>> Interestingly, this repo DOES contain a cookbook for Swift.
>> 
>> Current State of Documentation
>> ==============================
>> 
>> Documentation for best practices on using Chef for your OpenStack
>> deployments is, well, a bit scattered. Matt Ray has some good information on
>> the README on his cookbook repo and the OpsCode wiki:
>> 
>> https://github.com/mattray/openstack-cookbooks/blob/cactus/README.md
>> http://wiki.opscode.com/display/chef/Deploying+OpenStack+with+Chef
>> 
>> But it is unfortunately not going to help people looking to deploy Diablo
>> and later versions of OpenStack.
>> 
>> Most of the other repos contain virtually no documentation on using the
>> cookbooks or how they are written.
>> 
>> I have a suspicion that one of the reasons that there has been such a
>> proliferation of cookbooks has been the lack of documentation pointing
>> people to an appropriate repo, how to use the cookbooks properly, and what
>> the best practices for deployment are. That, and the fact that folks are
>> just trying to stand up complex clouds and Get Things Done, and
>> documentation is annoying to write ;)
>> 
>> Proposal for Alignment
>> ======================
>> 
>> I think the following steps would be good to get done by the time Essex
>> rolls out the door in April:
>> 
>> 1) Create a stable/diablo branch of the openstack/openstack-chef cookbook
>> repo and maintain it in the same way that we maintain stable branches for
>> core OpenStack projects. I propose we use the branch point that NTT PF Lab
>> used to create their fork of the upstream repo.
>> 
>> 2) Work with Matt Ray and other Chef experts to combine any and all best
>> practices that may be contained in the non-official cookbook repos into the
>> upstream official repository. From a cursory overview, there are some
>> differences in how databags are handled, how certs are handled, how certain
>> cookbooks are constructed, and of course differences in the actual cookbooks
>> in the repos themselves.
>> 
>> 3) Consolidate documentation on how to use the cookbooks, the best practices
>> used in constructing the cookbooks, and possibly some videos/tutorials
>> walking folks through this critical piece of the OpenStack puzzle.
>> 
>> 4) Create Jenkins builders for stable branch deployment testing. We
>> currently test the official development cookbooks by way of SmokeStack gates
>> on all core OpenStack projects. Would be great to get the same testing
>> automated for non-development branches of the cookbooks.
>> 
>> Thoughts and criticism most welcome, and apologies in advance if I got any
>> of the above history wrong. Feel free to correct me!
>> 
>> Best,
>> -jay
>> 
>> _______________________________________________
>> Mailing list: https://launchpad.net/~openstack
>> Post to     : openstack@xxxxxxxxxxxxxxxxxxx
>> Unsubscribe : https://launchpad.net/~openstack
>> More help   : https://help.launchpad.net/ListHelp
> 
> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to     : openstack@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp