nova team mailing list archive

[Vishvananda Ishaya <vishvananda@xxxxxxxxx>]

This stuff all makes me happy. I think I can continue to deal with flags
with these changes. If we can create a best-practices for naming of flags,
where they should go, and how they fit into conf files, I think I'm content
to stick with gflags.


I approved the patch, but if anyone else disagrees/has additional
suggestions, speak up!


On Thu, Jul 29, 2010 at 10:58 AM, Andy Smith <andyster@xxxxxxxxx> wrote:

> Wow! A whole mailing list I wasn't seeing!
>
> A quick rundown of what it seems we want out of the configuration system:
>
>  1. Command-line parsing that doesn't choke as easily on flags that are
> not currently defined.
>  2. Ability to refer to flags that are specific to another service for
> service interoperability.
>  3. Ability to deal with new flags at runtime for things such as
> pluggable backends.
>
> I'm a little sad to see #1 go just in case people typo a flag, but perhaps
> that
> can be fixed at a later date by doing a check to see whether a flag is
> very
> similarly named to another flag that we do know about.
>
> Anyway, I had a good idea in mind for how to solve all this so I went ahead
> and made a branch:
>
> https://code.launchpad.net/~termie/nova/mega_flags
>
> The DECLARE syntax is still perhaps a little verbose in that you have to
> declare
> all the flags you intend to use, but I think that it is probably better
> that you
> say upfront everything you are expecting to use so that people reading the
> file
> know what you depend on.
>
>
>
> On Thu, Jul 29, 2010 at 7:19 PM, Vishvananda Ishaya <vishvananda@xxxxxxxxx
> > wrote:
>
>> +termie since he wasn't subscribed
>>
>> On Wed, Jul 28, 2010 at 2:44 PM, Chris Behrens <
>> chris.behrens@xxxxxxxxxxxxx> wrote:
>>
>>> ++
>>>
>>> We should list requirements and look for what we need.
>>>
>>>
>>> On Jul 28, 2010, at 2:41 PM, Monty Taylor wrote:
>>>
>>> > On 07/28/2010 02:28 PM, Eric Day wrote:
>>> >> ++
>>> >>
>>> >> I'm all for using an existing solution if one exists. I've not looked
>>> >> enough to make calls either way though. I want to figure out *what*
>>> >> we are looking for in features to make those decisions.
>>> >
>>> > ++
>>> >
>>> >> On Wed, Jul 28, 2010 at 01:37:18PM -0700, Monty Taylor wrote:
>>> >>> So I know I haven't convinced everyone to love bzr yet ... but as
>>> they
>>> >>> are a large python project with command line and config file options
>>> -
>>> >>> and plugins - perhaps looking at the infrastructure/design they use
>>> >>> might be a good idea?
>>> >>>
>>> >>> Also, the work derks did with cement might be of help.
>>> >>>
>>> >>> I believe both are designed to do things similar to how you are
>>> >>> discussing them below (although different, of course - we're all
>>> python
>>> >>> devs, there's no way we're going to actually do things the same. :) )
>>> >>>
>>> >>> Monty
>>> >>>
>>> >>> (what Eric is saying makes sense to me - but I don't have a whole
>>> bunch
>>> >>> of stake either way here- I am a fan of reusing solutions that exist
>>> >>> where possible though of course)
>>> >>>
>>> >>> On 07/28/2010 01:24 PM, Eric Day wrote:
>>> >>>> Hi Vish,
>>> >>>>
>>> >>>> If we want to keep things modular and have runtime module selection
>>> >>>> like you mention, we probably need to rethink flags. Using gflags
>>> >>>> may not be an option unless we can somehow make 'undefok=' a global
>>> >>>> option. In other project (that was not in Python, so no code to
>>> help),
>>> >>>> the flow is:
>>> >>>>
>>> >>>> * Enforce the use of module names in the options. For example, for
>>> >>>>  generic queue module options use --queue.*, for libvirt module
>>> >>>>  options, use --libvirt.*. If we want to make this seamless, we
>>> >>>>  would probably need to use something else instead gflags or create
>>> >>>>  a wrapper to enforce the required behavior.
>>> >>>>
>>> >>>> * Import the core option manager, first thing that happens when
>>> >>>>  starting a binary.
>>> >>>>
>>> >>>> * Parse all options, separating each out into the modules they
>>> belong
>>> >>>>  to. We don't know what is valid yet, but we can at least group by
>>> module.
>>> >>>>
>>> >>>> * Load any required modules via normal 'import' lines. They can
>>> verify
>>> >>>>  options for their module space.
>>> >>>>
>>> >>>> * Have some core flags that specify which modules to load, for
>>> example,
>>> >>>>  use rabbit vs fakerabbit. Then 'import' the selected optional
>>> modules.
>>> >>>>
>>> >>>> * As optional modules load, let them verify the module namespace
>>> >>>>  options just like the required modules did.
>>> >>>>
>>> >>>> * Any options for modules that were not loaded are just ignored.
>>> >>>>
>>> >>>> Thoughts on this? It has worked out quite well in the other C++
>>> project
>>> >>>> for me, and with Python it would be even easier to put together. :)
>>> >>>>
>>> >>>> -Eric
>>> >>>>
>>> >>>> On Wed, Jul 28, 2010 at 11:13:40AM -0700, Vishvananda Ishaya wrote:
>>> >>>>>   I'm having some annoyances with gflags which I'd like to air out
>>> here.
>>> >>>>>    Maybe we can come to a consensus about how to move forward with
>>> them.  I
>>> >>>>>   find gflags annoying in the following ways:
>>> >>>>>   a) flags are irritating for global settings.  Settings that apply
>>> to the
>>> >>>>>   project as a whole have to be set in multiple places so that the
>>> binaries
>>> >>>>>   all get them properly.  This can be fixed somewhat by a shared
>>> flagfile.
>>> >>>>>    For example:
>>> >>>>>   /etc/nova/nova-manage.conf:
>>> >>>>>     --flagfile=/etc/nova/nova-common.conf # shared settings
>>> >>>>>     --otherflag=true #manage specific settings
>>> >>>>>   The problem here is that the shared settings can only include
>>> settings
>>> >>>>>   that are imported by EVERY binary, or one of the binaries will
>>> choke.  So
>>> >>>>>   if you have a flag that 4 of 5 binaries use, you either have to
>>> set it in
>>> >>>>>   four flagfiles or put it in common with an ugly undefok= line.
>>>  This all
>>> >>>>>   seems nasty to me.  Other possibilities include moving truly
>>> >>>>>   common/settings related flags into the common flags.py so that
>>> they are
>>> >>>>>   available to all binaries.  It all seems a bit hackish.
>>> >>>>>   b) including files for flags only
>>> >>>>>   There are places where we need access to a flag, but we aren't
>>> actually
>>> >>>>>   making calls in the file.  Pyflakes and pylint complain about
>>> unused
>>> >>>>>   imports.  Perhaps we fix this by moving these flags into common
>>> flagfile?
>>> >>>>>   c) dependency injection
>>> >>>>>   This is related to the issue above.  If we are dynamically
>>> loading
>>> >>>>>   specific drivers (for example the auth driver or a datastore
>>> backend) as
>>> >>>>>   specified by a flag, the import is often done later than the
>>> parent file
>>> >>>>>   is imported.  Therefore using flags to configure settings for the
>>> driver
>>> >>>>>   will fail, because the binary recognizing the flags is dependent
>>> on the
>>> >>>>>   file that contains the flags being imported.  Workarounds here
>>> include
>>> >>>>>   finding a different method for dependency injection, hacking
>>> flags to
>>> >>>>>   search for flags in injected dependencies somehow, or configuring
>>> drivers
>>> >>>>>   differently than the rest of the system.
>>> >>>>>   So I see 3 options for moving forward
>>> >>>>>   1) ditch gflags completely and use a different method for
>>> specifying
>>> >>>>>   settings
>>> >>>>>   2) use a combination of some kind of settings file for general
>>> >>>>>   configuration, and flags for specific runtime settings/hacks
>>> >>>>>   3) find good standard practices/workarounds for the above issues
>>> >>>>>   Thoughts?
>>> >>>>>   Vish
>>> >>>>
>>> >>>>> _______________________________________________
>>> >>>>> Mailing list: https://launchpad.net/~nova
>>> >>>>> Post to     : nova@xxxxxxxxxxxxxxxxxxx
>>> >>>>> Unsubscribe : https://launchpad.net/~nova
>>> >>>>> More help   : https://help.launchpad.net/ListHelp
>>> >>>>
>>> >>>>
>>> >>>> _______________________________________________
>>> >>>> Mailing list: https://launchpad.net/~nova
>>> >>>> Post to     : nova@xxxxxxxxxxxxxxxxxxx
>>> >>>> Unsubscribe : https://launchpad.net/~nova
>>> >>>> More help   : https://help.launchpad.net/ListHelp
>>> >>>>
>>> >>
>>> >
>>> >
>>> > _______________________________________________
>>> > Mailing list: https://launchpad.net/~nova
>>> > Post to     : nova@xxxxxxxxxxxxxxxxxxx
>>> > Unsubscribe : https://launchpad.net/~nova
>>> > More help   : https://help.launchpad.net/ListHelp
>>>
>>>
>>> _______________________________________________
>>> Mailing list: https://launchpad.net/~nova
>>> Post to     : nova@xxxxxxxxxxxxxxxxxxx
>>> Unsubscribe : https://launchpad.net/~nova
>>> More help   : https://help.launchpad.net/ListHelp
>>>
>>
>>
>