nova team mailing list archive
Mailing list archive
Re: Gflags, Settings, Dependency Injection
So I know I haven't convinced everyone to love bzr yet ... but as they
are a large python project with command line and config file options -
and plugins - perhaps looking at the infrastructure/design they use
might be a good idea?
Also, the work derks did with cement might be of help.
I believe both are designed to do things similar to how you are
discussing them below (although different, of course - we're all python
devs, there's no way we're going to actually do things the same. :) )
(what Eric is saying makes sense to me - but I don't have a whole bunch
of stake either way here- I am a fan of reusing solutions that exist
where possible though of course)
On 07/28/2010 01:24 PM, Eric Day wrote:
> Hi Vish,
> If we want to keep things modular and have runtime module selection
> like you mention, we probably need to rethink flags. Using gflags
> may not be an option unless we can somehow make 'undefok=' a global
> option. In other project (that was not in Python, so no code to help),
> the flow is:
> * Enforce the use of module names in the options. For example, for
> generic queue module options use --queue.*, for libvirt module
> options, use --libvirt.*. If we want to make this seamless, we
> would probably need to use something else instead gflags or create
> a wrapper to enforce the required behavior.
> * Import the core option manager, first thing that happens when
> starting a binary.
> * Parse all options, separating each out into the modules they belong
> to. We don't know what is valid yet, but we can at least group by module.
> * Load any required modules via normal 'import' lines. They can verify
> options for their module space.
> * Have some core flags that specify which modules to load, for example,
> use rabbit vs fakerabbit. Then 'import' the selected optional modules.
> * As optional modules load, let them verify the module namespace
> options just like the required modules did.
> * Any options for modules that were not loaded are just ignored.
> Thoughts on this? It has worked out quite well in the other C++ project
> for me, and with Python it would be even easier to put together. :)
> On Wed, Jul 28, 2010 at 11:13:40AM -0700, Vishvananda Ishaya wrote:
>> I'm having some annoyances with gflags which I'd like to air out here.
>> Maybe we can come to a consensus about how to move forward with them. I
>> find gflags annoying in the following ways:
>> a) flags are irritating for global settings. Settings that apply to the
>> project as a whole have to be set in multiple places so that the binaries
>> all get them properly. This can be fixed somewhat by a shared flagfile.
>> For example:
>> --flagfile=/etc/nova/nova-common.conf # shared settings
>> --otherflag=true #manage specific settings
>> The problem here is that the shared settings can only include settings
>> that are imported by EVERY binary, or one of the binaries will choke. So
>> if you have a flag that 4 of 5 binaries use, you either have to set it in
>> four flagfiles or put it in common with an ugly undefok= line. This all
>> seems nasty to me. Other possibilities include moving truly
>> common/settings related flags into the common flags.py so that they are
>> available to all binaries. It all seems a bit hackish.
>> b) including files for flags only
>> There are places where we need access to a flag, but we aren't actually
>> making calls in the file. Pyflakes and pylint complain about unused
>> imports. Perhaps we fix this by moving these flags into common flagfile?
>> c) dependency injection
>> This is related to the issue above. If we are dynamically loading
>> specific drivers (for example the auth driver or a datastore backend) as
>> specified by a flag, the import is often done later than the parent file
>> is imported. Therefore using flags to configure settings for the driver
>> will fail, because the binary recognizing the flags is dependent on the
>> file that contains the flags being imported. Workarounds here include
>> finding a different method for dependency injection, hacking flags to
>> search for flags in injected dependencies somehow, or configuring drivers
>> differently than the rest of the system.
>> So I see 3 options for moving forward
>> 1) ditch gflags completely and use a different method for specifying
>> 2) use a combination of some kind of settings file for general
>> configuration, and flags for specific runtime settings/hacks
>> 3) find good standard practices/workarounds for the above issues
>> Mailing list: https://launchpad.net/~nova
>> Post to : nova@xxxxxxxxxxxxxxxxxxx
>> Unsubscribe : https://launchpad.net/~nova
>> More help : https://help.launchpad.net/ListHelp
> Mailing list: https://launchpad.net/~nova
> Post to : nova@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~nova
> More help : https://help.launchpad.net/ListHelp