openstack team mailing list archive

[Andrew Clay Shafer <acs@xxxxxxxxxxxxxxxx>]

>
>
> >
> > If your SAIO/test diverge from production deployments, what are you
> really
> > testing?
>
> there's a big difference, imho, between testing code integration and
> production deployments.
> e.g. if you're trying to write a functional test for swift, that
> requires the proxy server to connect to the account, container and
> object servers - it'd be pretty handy to have a SAIO deployment (e.g.
> to test the new brimring).
>
> Are you trying to say that all deployments should ever be the same,
> less they're meaningless?
>

I'm not saying that all deployments should be the same.

What I am saying is that cookbooks are code and if you are not deploying to
production in the same manner that you deploy to test, eventually there
will be some assumption made in test that doesn't come along for the ride
to production.

In my utopian vision of the future, the process to set up to deploy to test
is also implicitly testing the deployment deployment and management of the
service.


> > You might think I'm being zealot, and relatively speaking, you might be
> > right, but from my experience the extra work to do this upfront pays
> itself
> > back many times over. Also, I can show you where the real zealots live.
> :)
>
> I actually agree with you. My comments were specific to the proposed
> swift- cookbook. It do mostly what the online instruction provide (and
> used much of the bash shell code verbatim), with chef/ruby code added
> to attempt and handle idempotency.
>
> My point was that for SIAO, you don't really need config
> management.... and an "installer" as you call it is probably
> sufficient.
> Sorry if I wasn't clear.


My problem is that it is kind of a slippery slope and if people aren't
familiar with the tools and configuration management in theory and
practice, they take what they see as an example of chef code and it bleeds
over into other recipes.

Then they have a bit of a mess and everyone questions why they are using
chef instead of bash.

So if you want installers, just use bash, then no one gets confused.

For the children...

Though my preference would be insisting on idempotent recipes, if you write
installers with configuration management tools, put big red flashing lights
with sirens and PSA warning not to run with scissors.

>
> I'll get to your example in a bit (next comment below). But the
> example I used was trying to get at a different issue.
> In both cases i described, the information is "discovered" in your
> terminology. Chef reports the IP address (or crowbar - which just
> moves it per interface),
> The issue is - when a given piece of information can be retrieved from
> multiple (apparently equivalent) places - how do you choose which one
> to use? And what does it take to modify the choice?
> The eval approach allows modifying an attribute (rather than a recipe)
> to change the source of the discovered data (the IP address to use).
> This could help reduce the changes a potential user of a cookbook
> might need to make.


It's all ruby code. You can make anything come from anywhere.

I think this is one of the hardest questions to answer about chef code and
it's an art not a science.

Given the choice between options, I try to look for the most semantically
relevant intent revealing option and minimize surprise.

There is obviously subjectivity of what is surprising.


> funny.. there are cookbooks out there for swift that match your
> spectrum to a tee ;)
> The Crowbar cookbooks discover the disks (with a bit of filtering)
> The Voxel ones have a prescribed data bag, which determines the ring's
> content.


yep, thus the motivation for this thread.

we can't really hope to collapse the variation of the cookbooks unless we
as a community adopt guiding principles and conventions that are widely
applicable.

> Operational scenarios start begging the question of what should be managed
> > with chef at all. (they also beg the question of whether there should be
> > some more automated ring management in Swift itself)
> >
>
> that could be a bit dangerous - a brief failure in e.g. connectivity
> to a node holding 30TB of data should probably not trigger an
> automatic removal of the node...


Is it more dangerous than people manually editing attributes?

This is not a problem unique to Swift. There are more sophisticated and
tunable ways to handle a brief partition than remove the node. I'm not
saying it is the highest priority, but there are some interesting
approaches that could reduce operational overhead.