openstack team mailing list archive

[John Garbutt <John.Garbutt@xxxxxxxxxx>]

I should be able to help out getting these working with XenServer/XCP, if that is useful to anyone?

Curiosity leads me to ask: Where do I find the puppet equivalent these days?

Cheers,
John

> -----Original Message-----
> From: openstack-bounces+john.garbutt=eu.citrix.com@xxxxxxxxxxxxxxxxxxx
> [mailto:openstack-
> bounces+john.garbutt=eu.citrix.com@xxxxxxxxxxxxxxxxxxx] On Behalf Of
> Matt Ray
> Sent: 10 July 2012 10:23
> To: Jay Pipes
> Cc: openstack@xxxxxxxxxxxxxxxxxxx; Nati Ueno
> Subject: Re: [Openstack] [CHEF] Clarification on osops/chef-
> repo/roles/nova-compute.rb
> 
> Bluntness appreciated, this process is already in motion.
> http://opscode.com/openstack was launched 2 weeks ago and I promptly
> left for conferences and vacation. I am consolidating GitHub repos
> here:
> 
> https://github.com/opscode/openstack-chef-repo
> https://github.com/opscode-cookbooks/nova
> https://github.com/opscode-cookbooks/glance
> https://github.com/opscode-cookbooks/horizon
> https://github.com/opscode-cookbooks/keystone
> https://github.com/opscode-cookbooks/swift
> 
> Work is being done in my own repos until it's ready for an initial release,
> which will include a Getting Started with Chef and OpenStack document.
> https://github.com/mattray/
> 
> I'm working with quite a few folks already, Rackspace, Dell, DreamHost and
> others and Intel is sponsoring this work.
> 
> Jay and I chatted a bit in IRC, we're quite aligned in how we plan on working
> this and the goal will be to get github.com/openstack/chef-repo gated with
> Gerrit and CI and pulling from Opscode's repos soon. Please feel free to join
> the discussion on our new mailing list focused on this effort here:
> http://groups.google.com/group/opscode-chef-openstack
> 
> And an IRC channel:
> #openstack-chef on irc.freenode.net
> 
> Thanks,
> Matt Ray
> Senior Technical Evangelist | Opscode Inc.
> matt@xxxxxxxxxxx | (512) 731-2218
> Twitter, IRC, GitHub: mattray
> 
> 
> On Tue, Jul 10, 2012 at 11:25 AM, Jay Pipes <jaypipes@xxxxxxxxx> wrote:
> > Apologies in advance for my blunt and somewhat dour response, Matt.
> > I'm not singling you out at all, and I know you've tried your best to
> > get the various Chef stakeholders to work together. Also apologies for
> > top-posting, but there's not a whole lot of use inline posting this.
> >
> > tl;dr
> > -----
> >
> > We need to stop the needless fracturing of the operational knowledge
> > of the Chef community and try working as a team towards some common
> > goals instead of creating fork after fork of repos of Chef cookbooks.
> >
> > There is a ton of wasted effort in this area.
> >
> > Proposal:
> >
> > * Get our act together and treat Chef repos (and puppet/juju/whatever)
> > as we do other OpenStack core and supporting projects -- use Gerrit,
> > use a CI gating system, do real code reviews on it, and in general
> > treat them as a supporting OpenStack project
> > * Mark ALL Chef repos that are not currently maintained with an
> > OBSELETE marker and/or DELETE the repo on Github
> > * Consolidate all *cookbooks* into a repository in
> > github.com/openstack/chef-repo
> > * Use git submodules to manage cookbooks that are upstreamed in
> > github.com/opscode/ that have little to no changes in them
> > * Actually fix the documentation of the dang cookbooks -- right now,
> > half of them include the documentation from the memcache cookbook, as
> > they were lazily copy-pasted around, or the standard example doc that
> > is created when using something like knife.
> > * Put as much variation in deployment philosophy into *roles* and
> > attribute overrides/defaults
> >
> > More/Rant/Details
> > -----------------
> >
> > Maybe it's just the open source developer in me, but I don't
> > understand why there is such an aversion to coordination in the
> > deployment/ops community around the scripts and deployment
> > cookbooks/modules/charms/whatever.
> >
> > Is it that everyone has a different idea of what is best? Is it
> > because deployers/ops folks think that coordinating with other
> > contributors is too time-consuming? Is it because the chef repos are
> > not controlled in the same way as, say, devstack or the core projects,
> > with an automated patch queue manager and code review system that
> > actually encourages debate over patches? A combination of all of the
> above?
> >
> > Over the last 2 years, I've worked at 3 companies in the OpenStack
> > ecosystem. All three companies had their own repos of Chef cookbooks
> > (still do to this day). 50-60% of the content of these cookbooks is
> > identical. 10-20% of the content of these cookbooks is different --
> > but only slightly or cosmetically. And a good portion of the remaining
> > 20-40% are differences that are incorrectly (IMHO) placed in the
> > cookbooks and recipes instead of where they should be: in roles and
> > environments, with cookbooks created that deal with variations in
> > deployments with attributes and the occasional if/else block.
> >
> > In trying to determine the appropriate Chef repo to use for the
> > TryStack project, we found the following repo:
> >
> > https://github.com/osops/chef-repo
> >
> > to have the most up-to-date. I've since been told this repo is no
> > longer maintained. This is very frustrating, not because of this
> > particular repo, but because this is just one in a long line of
> > neglected and forgotten forks of chef cookbook repositories. The fact
> > that the default Chef behaviour and Opscode documentation encourages
> > the copy/pasting of cookbooks all over the place and GitHub itself
> > encourages the random and promiscuous forking of repos doesn't help.
> >
> > Let's get real about the deployment/ops code and
> > cookbooks/modules/charms. Let's treat them the same way we do code in
> > the core projects and supporting projects.
> >
> > Thanks for your time,
> > -jay
> >
> > On 07/10/2012 11:42 AM, Matt Ray wrote:
> >> Just a heads up, I'm working on building unified community-driven
> >> cookbooks over in https://github.com/opscode/openstack-chef-repo
> (and
> >> repos for the individual cookbooks). These are forked from
> >> Rackspace's cookbooks and I'm working with them and others to make
> >> reusable, well-documented and supported Chef cookbooks for
> OpenStack.
> >> I'll make a larger announcement around them once I have a working
> >> quickstart document for them.
> >>
> >> tl;dr; https://github.com/opscode/openstack-chef-repo
> >>
> >> Thanks,
> >> Matt Ray
> >> Senior Technical Evangelist | Opscode Inc.
> >> matt@xxxxxxxxxxx | (512) 731-2218
> >> Twitter, IRC, GitHub: mattray
> >>
> >>
> >> On Tue, Jul 10, 2012 at 10:22 AM, Jay Pipes <jaypipes@xxxxxxxxx> wrote:
> >>> Gah... probably would be good if you guys either shut down the repo
> >>> or made a big notice on the README then :(
> >>>
> >>> -jay
> >>>
> >>> On 07/09/2012 05:25 PM, Joe Breu wrote:
> >>>> Hi Jay,
> >>>>
> >>>> The chef cookbooks at https://github.com/osops are no longer
> maintained.
> >>>>  Our current cookbooks are at
> >>>> https://github.com/rcbops/chef-cookbooks
> >>>>
> >>>>
> >>>>
> >>>> ---
> >>>> Joseph Breu
> >>>> Deployment Engineer
> >>>> Rackspace Cloud Builders
> >>>> 210-312-3508
> >>>>
> >>>> On Jul 9, 2012, at 8:01 AM, Jay Pipes wrote:
> >>>>
> >>>>> Vish and Ron, just getting back to this... see inline continued
> >>>>> questions for you both.
> >>>>>
> >>>>> On 07/02/2012 04:24 PM, Vishvananda Ishaya wrote:
> >>>>>>
> >>>>>> On Jul 2, 2012, at 7:28 AM, Jay Pipes wrote:
> >>>>>>
> >>>>>>> Hi Ron, cc'ing the openstack ML for extra eyes and opinions...
> >>>>>>>
> >>>>>>> So, Nati and I are looking to use either the osops chef-repo or
> >>>>>>> something similar as the basis of the new TryStack zone chef
> deployment.
> >>>>>>> I've been going through the recipes and roles and I have a
> >>>>>>> question on the nova-compute *role*:
> >>>>>>>
> >>>>>>> https://github.com/osops/chef-repo/blob/master/roles/nova-
> comput
> >>>>>>> e.rb
> >>>>>>>
> >>>>>>> I'm wondering why the nova-api recipe is in the runlist for
> >>>>>>> nova-compute?
> >>>>>>
> >>>>>> Because metadata needs to run on the compute hosts in the default
> >>>>>> layout. This should be switched to use nova-api-metadata instead
> >>>>>> of nova-api, but the split out hasn't been done yet.
> >>>>>
> >>>>> OK, I will work on splitting this out a bit more effectively.
> >>>>>
> >>>>> One additional question, though. In opening up the
> >>>>> /cookbooks/nova/recipes/nova/compute.rb file, you will notice this
> >>>>> at the top:
> >>>>>
> >>>>> include_recipe "nova::api"
> >>>>>
> >>>>> Therefore, unless I am mistaken, the nova-compute *role*'s runlist
> >>>>> actually doesn't need to contain both nova-api AND nova-compute
> >>>>> since apparently the nova-compute *recipe* already includes all of
> >>>>> the nova-api recipe.
> >>>>>
> >>>>> Would you agree with that?
> >>>>>
> >>>>>>> In addition, I was wondering if y'all had considered making more
> >>>>>>> use of roles instead of recipes to allow most of the attribute
> >>>>>>> assignment and variation to be in the combination of roles
> >>>>>>> assigned to a host, instead of recipes combined in a role?
> >>>>>>>
> >>>>>>> For example, right now, there is a "nova-controller" role that
> >>>>>>> looks like this:
> >>>>>>>
> >>>>>>> name "nova-controller"
> >>>>>>> description "Nova controller node (vncproxy + rabbit)"
> >>>>>>> run_list(
> >>>>>>> "role[base]",
> >>>>>>> "recipe[nova::controller]"
> >>>>>>> )
> >>>>>>>
> >>>>>>> Because most of the special sauce is in the nova::controller
> >>>>>>> recipe, I have to go into that recipe to see what the role is
> composed of:
> >>>>>>>
> >>>>>>> include_recipe "mysql::server"
> >>>>>>> include_recipe "openssh::default"
> >>>>>>>
> >>>>>>> include_recipe "rabbitmq::default"
> >>>>>>> include_recipe "keystone::server"
> >>>>>>> include_recipe "glance::registry"
> >>>>>>> include_recipe "glance::api"
> >>>>>>> include_recipe "nova::nova-setup"
> >>>>>>> include_recipe "nova::scheduler"
> >>>>>>> include_recipe "nova::api"
> >>>>>>>
> >>>>>>> if platform?(%w{fedora})
> >>>>>>> # Fedora skipping vncproxy for right now else include_recipe
> >>>>>>> "nova::vncproxy"
> >>>>>>> end
> >>>>>>>
> >>>>>>> But what this recipe really is is an opinionated description of
> >>>>>>> a "controller role". If the role was, instead, structured like so:
> >>>>>>>
> >>>>>>> name "nova-controller"
> >>>>>>> description "Nova Controller - All major API services and
> >>>>>>> control servers like MySQL and Rabbit"
> >>>>>>> run_list(
> >>>>>>> "role[base]",
> >>>>>>> "recipe[mysql::server]",
> >>>>>>> "recipe[openssh::default]",
> >>>>>>> "recipe[rabbitmq::default]",
> >>>>>>> "recipe[keystone::server]",
> >>>>>>> "recipe[glance::api]",
> >>>>>>> "recipe[glance::registry]",
> >>>>>>> "recipe[nova::scheduler]",
> >>>>>>> "recipe[nova::api]",
> >>>>>>> )
> >>>>>>>
> >>>>>>> Then the deployer can more easily switch up the way they deploy
> >>>>>>> OpenStack servers by merely changing the role -- say, removing
> >>>>>>> the Rabbit service and putting it somewhere else -- WITHOUT
> >>>>>>> having to modify a recipe in a Git submodule in the upstream
> cookbooks.
> >>>>>>>
> >>>>>>> Furthermore, if we broke out more roles -- such as "control-
> services"
> >>>>>>> which might be MySQL and Rabbit only -- than we could make the
> >>>>>>> "super roles" ,like the nova-controller role above, more of a
> >>>>>>> "put this and that role together, like so:
> >>>>>>>
> >>>>>>> name "nova-controller"
> >>>>>>> description "Nova Controller - All major API services and
> >>>>>>> control servers like MySQL and Rabbit"
> >>>>>>> run_list(
> >>>>>>> "role[base]",
> >>>>>>> "role[control_services]",
> >>>>>>> "recipe[keystone::server]",
> >>>>>>> "recipe[glance::api]",
> >>>>>>> "recipe[glance::registry]",
> >>>>>>> "recipe[nova::scheduler]",
> >>>>>>> "recipe[nova::api]",
> >>>>>>> )
> >>>>>>
> >>>>>> This all makes sense to me.  Ron?
> >>>>>
> >>>>> Ron, any disagreement here?
> >>>>>
> >>>>>>> Finally, I've noticed that there are aren't any HA options in
> >>>>>>> the osops recipes -- specifically around MySQL. Are there plans
> >>>>>>> to do so? We use heartbeat/Pacemaker/DRBD in the original
> >>>>>>> TryStack cookbooks [1] and environments to get simple HA
> >>>>>>> solutions up and would love to see those in the upstream.
> >>>>>
> >>>>> Either of you, any thoughts on this front?
> >>>>>
> >>>>> Thanks!
> >>>>> -jay
> >>>>>
> >>>>>>> Thanks and all the best guys,
> >>>>>>> -jay
> >>>>>>>
> >>>>>>> [1]
> >>>>>>> https://github.com/trystack/openstack-chef/tree/stable/diablo/co
> >>>>>>> okbooks/heartbeat
> >>>>>>
> >>>>>
> >>>>>
> >>>>> _______________________________________________
> >>>>> Mailing list: https://launchpad.net/~openstack
> >>>>> Post to     : openstack@xxxxxxxxxxxxxxxxxxx
> >>>>> <mailto:openstack@xxxxxxxxxxxxxxxxxxx>
> >>>>> Unsubscribe : https://launchpad.net/~openstack
> >>>>> More help   : https://help.launchpad.net/ListHelp
> >>>>
> >>>
> >>>
> >>> _______________________________________________
> >>> Mailing list: https://launchpad.net/~openstack
> >>> Post to     : openstack@xxxxxxxxxxxxxxxxxxx
> >>> Unsubscribe : https://launchpad.net/~openstack
> >>> More help   : https://help.launchpad.net/ListHelp
> >
> 
> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to     : openstack@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp