← Back to team overview

cloud-init team mailing list archive

Re: (aix-)port and cloud-init defaults: where and how to get them as much as possible in a focused location.


On 20-Dec-16 22:57, Scott Moser wrote:
some inline comments below.

I really appreciate your interest, and want to help out. 2 things though
a.) i'll be out after friday for 2 weeks
b.) converstations lkke this would be good to have on cloud-init mailing

You can subscribe at https://launchpad.net/~cloud-init

So for further discussion would you mind sending there?
You can introduce yourself and your goals.
a) I know a lot about AIX: I host two portals: www.rootvg.net and www.aixtools.net. I am on github as "aixtools". b) I try to not talk about my employer, however, I do not hide it either (IBM Lab Services) c) what I do here, and all in OSS is my own time, on my own personal hardware (so it is older model, like me ;)) d) I am a maverick (others might say "independent thinker", but arrogant works too) e) also new to python - so I may ask things you would/could/might consider self-evident.

a) healthy discussions (see below)
b) a port of cloud-init for AIX - and suitable for inclusion in the main branch (unlike the current "source" now available based on 0.7.5)
c) learn more python
d) find more interest aka backing for a patch I wrote for (c)python for AIX (Lib/ctypes).

FYI: I am still typing, but my mail program says I have received it. If this comes in double - please forgive the new guy.

There wont be a ton of traffic, but community involvement like this will
help others to see that they can help out.


On Tue, 20 Dec 2016, Michael Felt wrote:

On 20-Dec-16 20:14, Scott Moser wrote:
Looks great.
I pulled, I made some minor changes to the commit message, but just some
that aren't worth discussing.

Two questions:

A - regarding a more "portable", i.e., getting defaults into a single file,
rather than having to look everywhere for them

B- my "initial" approach to override (forgot the proper OO term for sub-class

==== A ====

Something - "smallish", but goes beyond just a change in ssh_util.py, e.g.,
move DEF_SSHD_CFG to settings.py

A grep two deep shows, for someone like myself - new to cloud-init - confusing
naming schemes:

e.g., I assume CFG_BUILTIN is the default(s) for config.cfg - so what is
BUILTIN_CFG - new name for same thing, or (imho) just a poor choice.
well, the BUILTIN_CFG that you see here is per-config module "BUILTIN".  I
guess your're right in that consistency would be nice, but they're
different things.
there is a cloudinit "global" builtin config and then each of the config
modules kind of have their own.
I suspected as much, but for the casual reader - CFG_BUILTIN and BUILTIN_CFG may not be clearly remembered.

I see your point about wanting a centralized default, but it makes sense
from a different perspective for each config module to have its own.. that
way the config modules are more "modules".

It seems like probably we want to get the 'distro' object involved in more
things.  Also, there is the Paths object which is somewhat relevant here.

michael@x071:[/data/prj/aixtools/cloud-init/cloudinit]grep CFG *.py */*.py
helpers.py:                                CFG_ENV_NAME)
helpers.py:        if CFG_ENV_NAME in os.environ:
helpers.py:            e_fn = os.environ[CFG_ENV_NAME]
settings.py:CFG_ENV_NAME = "CLOUD_CFG"
settings.py:    from cloudinit.distros.aix.settings import CFG_BUILTIN,
settings.py:    CFG_BUILTIN = {
ssh_util.py:DEF_SSHD_CFG = "/etc/ssh/sshd_config"
ssh_util.py:            ssh_cfg = parse_ssh_config_map(DEF_SSHD_CFG)
ssh_util.py:                        "%r instead", DEF_SSHD_CFG, auth_key_fn)
util.py:from cloudinit.settings import (CFG_BUILTIN)
util.py:    return obj_copy.deepcopy(CFG_BUILTIN)
config/cc_emit_upstart.py:        cmd = ['initctl', 'emit', str(n),
'CLOUD_CFG=%s' % cfgpath]
config/cc_fan.py:BUILTIN_CFG = {
config/cc_fan.py:    mycfg = util.mergemanydict([cfgin, BUILTIN_CFG])
config/cc_landscape.py:LSC_CLIENT_CFG_FILE = "/etc/landscape/client.conf"
config/cc_landscape.py:LSC_BUILTIN_CFG = {
config/cc_landscape.py:        LSC_BUILTIN_CFG,
config/cc_landscape.py:        LSC_CLIENT_CFG_FILE,
config/cc_landscape.py: util.ensure_dir(os.path.dirname(LSC_CLIENT_CFG_FILE))
config/cc_landscape.py:    util.write_file(LSC_CLIENT_CFG_FILE,
config/cc_landscape.py:    log.debug("Wrote landscape config file to %s",
config/cc_mcollective.py:SERVER_CFG = '/etc/mcollective/server.cfg'
config/cc_mcollective.py:def configure(config, server_cfg=SERVER_CFG,
config/cc_set_passwords.py:        old_lines =
config/cc_set_passwords.py: util.write_file(ssh_util.DEF_SSHD_CFG,
config/cc_snappy.py:BUILTIN_CFG = {
config/cc_snappy.py:    mycfg = util.mergemanydict([cfgin, BUILTIN_CFG])
sources/DataSourceAzure.py:DS_CFG_PATH = ['datasource', DS_NAME]
sources/DataSourceAzure.py: util.get_cfg_by_path(sys_cfg, DS_CFG_PATH, {}),
sources/DataSourceAzure.py:        user_ds_cfg =
util.get_cfg_by_path(self.cfg, DS_CFG_PATH, {})
sources/DataSourceSmartOS.py:DS_CFG_PATH = ['datasource', DS_NAME]
sources/DataSourceSmartOS.py: util.get_cfg_by_path(sys_cfg, DS_CFG_PATH, {}),

==== B ====

As much as possible I want to maintain "defaults", so I was thinking something
such as: (in settings.py)

# This is expected to be a yaml formatted file
import sys
if sys.platform[:3] == "aix":
     from cloudinit.distros.aix.settings import CFG_BUILTIN, CLOUD_CONFIG
     CLOUD_CONFIG = '/etc/cloud/cloud.cfg'

     # What u get if no config is provided
     CFG_BUILTIN = {
I think that we probably want to get away (as much as possible) from
static configuration here, but rather have things "do the right thing".
With the ability to statically define them via config if that doesnt work.
There will always be defaults - and there will always be differences in distributions. UNIX/POSIX has those differences (Solaris, HPUX, Tru-64, AIX, etc) as Linux also has differences (both coming from distribution (debian, ubuntu, redhat, suse (my first was slackware after I left minix). However, having them in a "single" place, or at least in a well-defined location would be much better.

New thought:
One of my problems, as I work though the code is knowing what something is supossed to be doing. One thing I miss - a lot - is __doc__ aka DocString for each of the files in cloud-init. Back to "default" definitions - I would look to something like __init__.py for definitions that are "global" to a module, and one of those items, imho, should be a DocString.

I know I am new to python - cloudinit.config - for example, looks to be a lot of files that are concerned with the config phase (hence the name), but I suspect that there are some very big differences in the files that are using some CFG variable: michael@x071:[/data/prj/aixtools/cloud-init/cloudinit/config]grep -c CFG *.py | grep -v :0

iow - I can imagine how cc_set_passwords is a "generic" config operation, but landscape, upstart, fan (no idea what that is), snappy, mcollective (again, hmm, what is that) are more closely related to a systems management approach for a (part of) a distro. I have to read, and guess what they do, or intend to do (just looked into all of them, and I am eating a few of my words - they have a nice DocString all -

so I will just come back to my "problem" - since "defaults" are not in a common location, nor is there a high-level description of the code structure (that I could find to date - I may just be blind in one eye and cannot see out the other - I have to read, line by line (class by class) what is in each file.

Please do not think I am trying to flame cloud-init. It is much better than anything I could even imagine! If anything, see me, as someone struggling to see what is in front of me. Maybe the problem is all (certainly largely) mine - I just need to study more. I assume you know the code well and maybe have (already expressed) ideas on how it can be improved.

So, two sub-goals in this mail:
a) to see and understand cloud-init better (think out-loud, elicit feedback/hints from you) b) to help you know where it is confusing (one person) aka FYI and I hope it helps!


The reverse could be the defaults are defined, and then "from ... import ..."
could be used to override selective bits,
or perhaps just overwrite just small bits. - One that is likely to frequently
be an issue is:

         'distro': 'ubuntu',

Again, I am still learning the "art of python" - it is very very different to
how I used C 30 years ago (I feel very "old dogish"),
and, if I knew how - I would replace the static text "ubuntu" with a variable
I could establish via sys.platform and/or

Your feedback on what you see as the correct solution path is appreciated.



Follow ups