fuel-dev team mailing list archive

[Dmitry Borodaenko <dborodaenko@xxxxxxxxxxxx>]

+100500, decoupling our Puppet modules would simplify a lot of things.
A few comments inline:

On Thu, Jan 16, 2014 at 3:43 AM, Dmitry Ilyin <dilyin@xxxxxxxxxxxx> wrote:
> So we have an idea to separate this manifest to many small ones. Each of
> them would do only one task and orchestration engine would apply them in
> required order
> to deploy the node user have requested. With such architecture each 'stage'
> would be more or less isolated and it would be possible for a single
> developer
> to add a new 'stage' that would install his 'plugin'.

Considering that "stage" has a very specific meaning in Puppet,
reusing the same word here would introduce unnecessary confusion.
Unless you actually plan to use Puppet stages mechanism to implement
this approach (afaiu you intend to use orchestrator for that instead)
I think we should use a different term, e.g. "component" or "task".

> * Even if we'll make each 'stage' a separate manifest there still we be a
> need to use our current puppet modules as library. Or we should make every
>   modules fully self-contained? Or we could use something like python's
> virtualenv to provide each 'stage' with correct set of library modules?

In the spirit of the primary principle of refactoring (making one
major change at a time), I'd start with keeping our current modules
library intact. First refactoring should be replacing cross-module
dependencies with hard separation of groups of modules into
stages/components/tasks. Once that is working reasonably well, then we
can consider rearranging the library (which is also a long-overdue
refactoring, but large enough to be done as a separate step).

> * What is the best way to install packages then? Keep our current repository
> on master node or invent somethin else?

I think current repository should be fine. The only piece missing is
tools to add more packages into the RPM and DEB repositories on the
master node. A cleaner but more complex option would be to create
additional repositories on master nodes, and updating apt/yum configs
on target nodes to use all such repositories during deployment. The
latter would also make it easier to add a mechanism to distribute
security updates at some later point.

> * How and where to store information on what 'stages' should be done for
> which roles? There should be some metadata with fixed scenarios or there
> should be dynamic graph?

I would combine this problem with the need for an "empty" role. For
each role, define a full ordered list of stages (components/tasks) to
be applied. Then extract all common tasks shared between all roles
into the empty role and make that role a dependency of all other
roles.

> * Some 'plugins' and 'stages' may require to add options to settings panel.
> If we are going to have external plugins there should be a way to hook int
> setting and add your options.
>   This data will the be available in astute.yaml.

I think we should implement ability to override parts of astute.yaml
in plugins: allow plugin developers to provide yaml files that replace
parts of the astute.yaml structure or add new sections without
redefining the rest of the data. Here's how I implemented that in one
of my personal projects:

https://github.com/angdraug/samizdat/blob/master/lib/samizdat/engine/site.rb#L111

> * Our orchestation engine will have to become much more complex then what we
> have now. There should be queues of stages for each deploying node. And
> there should be support
>   for inter-node dependencies of stages.

Is Heat going to make this problem easier to solve?

-- 
Dmitry Borodaenko