nova team mailing list archive

[Andy Smith <andyster@xxxxxxxxx>]

Let's try this again, apparently my messages have not been going through to
the mailing list and I have not been getting rejection emails. I also am not
receiving emails despite being in a 'subscribed' state.

Wow! A whole mailing list I wasn't seeing!

A quick rundown of what it seems we want out of the configuration system:

 1. Command-line parsing that doesn't choke as easily on flags that are
not currently defined.
 2. Ability to refer to flags that are specific to another service for
service interoperability.
 3. Ability to deal with new flags at runtime for things such as
pluggable backends.

I'm a little sad to see #1 go just in case people typo a flag, but perhaps
that
can be fixed at a later date by doing a check to see whether a flag is very
similarly named to another flag that we do know about.

Anyway, I had a good idea in mind for how to solve all this so I went ahead
and made a branch:

https://code.launchpad.net/~termie/nova/mega_flags

The DECLARE syntax is still perhaps a little verbose in that you have to
declare
all the flags you intend to use, but I think that it is probably better that
you
say upfront everything you are expecting to use so that people reading the
file
know what you depend on.

--andy

On Thu, Jul 29, 2010 at 7:19 PM, Vishvananda Ishaya
<vishvananda@xxxxxxxxx>wrote:

> +termie since he wasn't subscribed
>
> On Wed, Jul 28, 2010 at 2:44 PM, Chris Behrens <
> chris.behrens@xxxxxxxxxxxxx> wrote:
>
>> ++
>>
>> We should list requirements and look for what we need.
>>
>>
>> On Jul 28, 2010, at 2:41 PM, Monty Taylor wrote:
>>
>> > On 07/28/2010 02:28 PM, Eric Day wrote:
>> >> ++
>> >>
>> >> I'm all for using an existing solution if one exists. I've not looked
>> >> enough to make calls either way though. I want to figure out *what*
>> >> we are looking for in features to make those decisions.
>> >
>> > ++
>> >
>> >> On Wed, Jul 28, 2010 at 01:37:18PM -0700, Monty Taylor wrote:
>> >>> So I know I haven't convinced everyone to love bzr yet ... but as they
>> >>> are a large python project with command line and config file options -
>> >>> and plugins - perhaps looking at the infrastructure/design they use
>> >>> might be a good idea?
>> >>>
>> >>> Also, the work derks did with cement might be of help.
>> >>>
>> >>> I believe both are designed to do things similar to how you are
>> >>> discussing them below (although different, of course - we're all
>> python
>> >>> devs, there's no way we're going to actually do things the same. :) )
>> >>>
>> >>> Monty
>> >>>
>> >>> (what Eric is saying makes sense to me - but I don't have a whole
>> bunch
>> >>> of stake either way here- I am a fan of reusing solutions that exist
>> >>> where possible though of course)
>> >>>
>> >>> On 07/28/2010 01:24 PM, Eric Day wrote:
>> >>>> Hi Vish,
>> >>>>
>> >>>> If we want to keep things modular and have runtime module selection
>> >>>> like you mention, we probably need to rethink flags. Using gflags
>> >>>> may not be an option unless we can somehow make 'undefok=' a global
>> >>>> option. In other project (that was not in Python, so no code to
>> help),
>> >>>> the flow is:
>> >>>>
>> >>>> * Enforce the use of module names in the options. For example, for
>> >>>>  generic queue module options use --queue.*, for libvirt module
>> >>>>  options, use --libvirt.*. If we want to make this seamless, we
>> >>>>  would probably need to use something else instead gflags or create
>> >>>>  a wrapper to enforce the required behavior.
>> >>>>
>> >>>> * Import the core option manager, first thing that happens when
>> >>>>  starting a binary.
>> >>>>
>> >>>> * Parse all options, separating each out into the modules they belong
>> >>>>  to. We don't know what is valid yet, but we can at least group by
>> module.
>> >>>>
>> >>>> * Load any required modules via normal 'import' lines. They can
>> verify
>> >>>>  options for their module space.
>> >>>>
>> >>>> * Have some core flags that specify which modules to load, for
>> example,
>> >>>>  use rabbit vs fakerabbit. Then 'import' the selected optional
>> modules.
>> >>>>
>> >>>> * As optional modules load, let them verify the module namespace
>> >>>>  options just like the required modules did.
>> >>>>
>> >>>> * Any options for modules that were not loaded are just ignored.
>> >>>>
>> >>>> Thoughts on this? It has worked out quite well in the other C++
>> project
>> >>>> for me, and with Python it would be even easier to put together. :)
>> >>>>
>> >>>> -Eric
>> >>>>
>> >>>> On Wed, Jul 28, 2010 at 11:13:40AM -0700, Vishvananda Ishaya wrote:
>> >>>>>   I'm having some annoyances with gflags which I'd like to air out
>> here.
>> >>>>>    Maybe we can come to a consensus about how to move forward with
>> them.  I
>> >>>>>   find gflags annoying in the following ways:
>> >>>>>   a) flags are irritating for global settings.  Settings that apply
>> to the
>> >>>>>   project as a whole have to be set in multiple places so that the
>> binaries
>> >>>>>   all get them properly.  This can be fixed somewhat by a shared
>> flagfile.
>> >>>>>    For example:
>> >>>>>   /etc/nova/nova-manage.conf:
>> >>>>>     --flagfile=/etc/nova/nova-common.conf # shared settings
>> >>>>>     --otherflag=true #manage specific settings
>> >>>>>   The problem here is that the shared settings can only include
>> settings
>> >>>>>   that are imported by EVERY binary, or one of the binaries will
>> choke.  So
>> >>>>>   if you have a flag that 4 of 5 binaries use, you either have to
>> set it in
>> >>>>>   four flagfiles or put it in common with an ugly undefok= line.
>>  This all
>> >>>>>   seems nasty to me.  Other possibilities include moving truly
>> >>>>>   common/settings related flags into the common flags.py so that
>> they are
>> >>>>>   available to all binaries.  It all seems a bit hackish.
>> >>>>>   b) including files for flags only
>> >>>>>   There are places where we need access to a flag, but we aren't
>> actually
>> >>>>>   making calls in the file.  Pyflakes and pylint complain about
>> unused
>> >>>>>   imports.  Perhaps we fix this by moving these flags into common
>> flagfile?
>> >>>>>   c) dependency injection
>> >>>>>   This is related to the issue above.  If we are dynamically loading
>> >>>>>   specific drivers (for example the auth driver or a datastore
>> backend) as
>> >>>>>   specified by a flag, the import is often done later than the
>> parent file
>> >>>>>   is imported.  Therefore using flags to configure settings for the
>> driver
>> >>>>>   will fail, because the binary recognizing the flags is dependent
>> on the
>> >>>>>   file that contains the flags being imported.  Workarounds here
>> include
>> >>>>>   finding a different method for dependency injection, hacking flags
>> to
>> >>>>>   search for flags in injected dependencies somehow, or configuring
>> drivers
>> >>>>>   differently than the rest of the system.
>> >>>>>   So I see 3 options for moving forward
>> >>>>>   1) ditch gflags completely and use a different method for
>> specifying
>> >>>>>   settings
>> >>>>>   2) use a combination of some kind of settings file for general
>> >>>>>   configuration, and flags for specific runtime settings/hacks
>> >>>>>   3) find good standard practices/workarounds for the above issues
>> >>>>>   Thoughts?
>> >>>>>   Vish
>> >>>>
>> >>>>> _______________________________________________
>> >>>>> Mailing list: https://launchpad.net/~nova
>> >>>>> Post to     : nova@xxxxxxxxxxxxxxxxxxx
>> >>>>> Unsubscribe : https://launchpad.net/~nova
>> >>>>> More help   : https://help.launchpad.net/ListHelp
>> >>>>
>> >>>>
>> >>>> _______________________________________________
>> >>>> Mailing list: https://launchpad.net/~nova
>> >>>> Post to     : nova@xxxxxxxxxxxxxxxxxxx
>> >>>> Unsubscribe : https://launchpad.net/~nova
>> >>>> More help   : https://help.launchpad.net/ListHelp
>> >>>>
>> >>
>> >
>> >
>> > _______________________________________________
>> > Mailing list: https://launchpad.net/~nova
>> > Post to     : nova@xxxxxxxxxxxxxxxxxxx
>> > Unsubscribe : https://launchpad.net/~nova
>> > More help   : https://help.launchpad.net/ListHelp
>>
>>
>> _______________________________________________
>> Mailing list: https://launchpad.net/~nova
>> Post to     : nova@xxxxxxxxxxxxxxxxxxx
>> Unsubscribe : https://launchpad.net/~nova
>> More help   : https://help.launchpad.net/ListHelp
>>
>
>