cf-charmers team mailing list archive

[Benjamin Saller <benjamin.saller@xxxxxxxxxxxxx>]

Hi All,

[TL;DR we are going to start generating the charms from metadata in the
bundle]

We've had some interesting learning during the 1st day of the sprint here.
The most salient point being that volatility of their topology is higher
and more constant than we originally knew. They intend to add/remove
services from the runtime going forward in ways that don't encourage
encapsulation into our existing charms. An example of this might be the
addition of new -clock services which are singletons associated with
services like cloud-controller which manage a cron-like service. These were
recently added. To upgrade from one version of  cloudfoundry to another
will quite often then include the mutation, not just of the charm and their
runtime code, but of the topology itself.

We've been in the process of converting the existing charms to be driven by
what we call 'service configuration blocks' which are semi-declarative
blocks living in hooks/config.py by convention. All of the charms important
behaviour live with the exception of the install hook which we're
redesigning currently.

The proposal for dealing with the additional volatility is composed of a
few parts::

- Roll charm metadata up into the bundle for central management
    - Charm Definitions
        - charm metadata
        - charm service block
- Add information to the bundle mapping between a range of CF releases (eg
153-172)
    - CF release jobs (templates/spec/monit files)
    - list Charm Definitions mentioned above
    - a topology/bundle used for all release versions in the range
    - a migration from the previous topology range
          (orchestrated adds/removes of new services for example)
 - The bundle itself is a charm, which we deploy into the environment
    - It has a service setting of which cf-release to run
    - It generates a local (inside the environment) charm repo of charms
      from the above definitions (they are all data-drive boiler plate at
this point)
    - If an existing version is/was deployed the bundle's charm instance
arranges
      whatever migrations scripts are needed (often via juju run, but also
as
      (de)provisioning decisions executed in the running bundle

This should solve a slew of issues

   - Generate rather than write boilerplate
   - A place to include topology altering code
   - The ability to generate charms with compiled assets or direct asset
acquisition
      to a central mirror
  - A way to leverage those parts of our charm level orchestration which
persist from
    topology range to range.

I hope to add more follow on to this as the sprint progresses, but its
already looking pretty good.

Thanks,
-Ben