Hi Stackers,
tl;dr
-----
There are myriad Chef cookbooks "out there" in the ecosystem and locked up
behind various company firewalls. It would be awesome if we could agree to:
* Align to a single origin repository for OpenStack cookbooks
* Consolidate OpenStack Chef-based deployment experience into a single
knowledge base
* Have branches on the origin OpenStack cookbooks repository that align with
core OpenStack projects
* Automate the validation and testing of these cookbooks on multiple
supported versions of the OpenStack code base
Details
-------
Current State of Forks
======================
Matt Ray and I tried to outline the current state of the various OpenStack
Chef cookbooks this past Thursday, and we came up with the following state
of affairs:
** The "official" OpenStack Chef cookbooks **
https://github.com/openstack/openstack-chef
These chef cookbooks are the ones maintained mostly by Dan Prince and Brian
Lamar and these are the cookbooks used by the SmokeStack project. The
cookbooks contained in the above repo can install all the core OpenStack
projects with the exception of Swift and Horizon.
This repo is controlled by the Gerrit instance at review.openstack.org just
like other core OpenStack projects.
However, these cookbooks DO NOT currently have a stable/diablo branch --
they are updated when the development trunks of any OpenStack project merges
a commit that requires deployment or configuration-related changes to their
associated cookbook.
Important note: it's easy for Dan and Brian to know when updates to these
cookbooks are necessary -- SmokeStack will bomb out if a
deployment-affecting configuration change hits a core project trunk :)
These cookbooks are the ONLY cookbooks that contain stuff for deploying with
XenServer, AFAICT.
** NTT PF Lab Diablo Chef cookbooks **
https://github.com/ntt-pf-lab/openstack-chef/
So, NTT PF Lab forked the upstream Chef cookbooks back in Nov 11, 2011,
because they needed a set of Chef cookbooks for OpenStack that functioned
for the Diablo code base.
While Nov 11, 2011, is not the *exact* date of the Diablo release, these
cookbooks do in fact work for a Diablo install -- Nati Ueno is using them
for the FreeCloud deployment so we know they work...
** OpsCode OpenStack Chef Cookbooks **
Matt Ray from OpsCode created a set of cookbooks for OpenStack for the
Cactus release of OpenStack:
https://github.com/mattray/openstack-cookbooks
http://wiki.opscode.com/display/chef/Deploying+OpenStack+with+Chef
These cookbooks were forked from the Anso Labs' original OpenStack cookbooks
from the Bexar release and were the basis for the Chef work that Dell did
for Crowbar. Crowbar was originally based on Cactus, and according to Matt,
the repositories of OpenStack cookbooks that OpsCode houses internally and
uses most often are Cactus-based cookbooks. (Matt, please correct me if I am
wrong here...)
** Rackspace CloudBuilders OpenStack Chef Cookbooks **
The RCB team also has a repository of OpenStack Chef cookbooks:
https://github.com/cloudbuilders/openstack-cookbooks
Now, GitHub *says* that these cookbooks were forked from the official
upstream cookbooks, but I do not think that is correct. Looking at this
repo, I believe that this repo was *actually* forked from the Anso Labs
OpenStack Chef Cookbooks, as the list of cookbooks is virtually identical.
** Anso Labs OpenStack Chef Cookbooks **
These older cookbooks are in this repo:
https://github.com/ansolabs/openstack-cookbooks/tree/master/cookbooks
Interestingly, this repo DOES contain a cookbook for Swift.
Current State of Documentation
==============================
Documentation for best practices on using Chef for your OpenStack
deployments is, well, a bit scattered. Matt Ray has some good information on
the README on his cookbook repo and the OpsCode wiki:
https://github.com/mattray/openstack-cookbooks/blob/cactus/README.md
http://wiki.opscode.com/display/chef/Deploying+OpenStack+with+Chef
But it is unfortunately not going to help people looking to deploy Diablo
and later versions of OpenStack.
Most of the other repos contain virtually no documentation on using the
cookbooks or how they are written.
I have a suspicion that one of the reasons that there has been such a
proliferation of cookbooks has been the lack of documentation pointing
people to an appropriate repo, how to use the cookbooks properly, and what
the best practices for deployment are. That, and the fact that folks are
just trying to stand up complex clouds and Get Things Done, and
documentation is annoying to write ;)
Proposal for Alignment
======================
I think the following steps would be good to get done by the time Essex
rolls out the door in April:
1) Create a stable/diablo branch of the openstack/openstack-chef cookbook
repo and maintain it in the same way that we maintain stable branches for
core OpenStack projects. I propose we use the branch point that NTT PF Lab
used to create their fork of the upstream repo.
2) Work with Matt Ray and other Chef experts to combine any and all best
practices that may be contained in the non-official cookbook repos into the
upstream official repository. From a cursory overview, there are some
differences in how databags are handled, how certs are handled, how certain
cookbooks are constructed, and of course differences in the actual cookbooks
in the repos themselves.
3) Consolidate documentation on how to use the cookbooks, the best practices
used in constructing the cookbooks, and possibly some videos/tutorials
walking folks through this critical piece of the OpenStack puzzle.
4) Create Jenkins builders for stable branch deployment testing. We
currently test the official development cookbooks by way of SmokeStack gates
on all core OpenStack projects. Would be great to get the same testing
automated for non-development branches of the cookbooks.
Thoughts and criticism most welcome, and apologies in advance if I got any
of the above history wrong. Feel free to correct me!
Best,
-jay
_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to : openstack@xxxxxxxxxxxxxxxxxxx
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
