openstack team mailing list archive

[Ewan Mellor <Ewan.Mellor@xxxxxxxxxxxxx>]

I wasn't proposing doing anything for Bexar -- it's too late for that.  I just want to come to some agreement on what we want direction we want to move.

Do we agree on the following?

1.  We _do_ want an openstack-common project.  Its aim will be to provide useful functionality (written in Python) that is common to more than one of the OpenStack projects.  It is not necessary for all projects to use what's in openstack-common, nor does it have to be useful to all projects in order for it to go in there.

2.  We want a lazy-loading implementation in openstack-common.

Assuming we agree on these things, Andy / Todd / Vish, could you between you agree on what form the lazy-loader should take, and put it in openstack-common?  I don't mind at all what form people want to use, but I would like a once-and-for-all decision made so that the next time we refactor the XenAPI code (which we're planning to do for Cactus), we can switch definitively.

Thanks,

Ewan.

> -----Original Message-----
> From: Jay Pipes [mailto:jaypipes@xxxxxxxxx]
> Sent: 14 January 2011 11:18
> To: Vishvananda Ishaya
> Cc: Andy Smith; Ewan Mellor; Todd Willey; openstack@xxxxxxxxxxxxxxxxxxx
> Subject: Re: [Openstack] Lazy import of modules
> 
> All,
> 
> To be clear, I wasn't stating that the LazyPluggable class and
> solution in Nova isn't good. Just that. *right now*, a few weeks
> before Bexar release, the best place to put this code is only in Nova.
>  Let it bake nicely during Cactus, and we can consider really
> utilising the openstack-common project at that point. :)
> 
> Hope that make sense,
> 
> jay
> 
> On Thu, Jan 13, 2011 at 8:43 PM, Vishvananda Ishaya
> <vishvananda@xxxxxxxxx> wrote:
> > The LazyPluggable in nova common was based on yours I think.  The
> class idea
> > is fine with me as long as those features are added.  You just have
> to make
> > sure that you don't eat the exception in the try, catch, because that
> is
> > what import class was doing, but it was catching underlying import
> errors
> > (i.e. a missing dependency) and hiding the underlying error.
> > Vish
> > On Jan 13, 2011, at 5:30 PM, Andy Smith wrote:
> >
> > While I agree largely with the statements so far, I think the main
> issue
> > with nova-common is that the swift project and the nova project have
> pretty
> > much no communication going on between them. I think that is okay for
> now,
> > the projects are two rather distinct codebases dropped wholesale next
> to
> > each other without any prior evidence that the developers have
> anything in
> > common besides python. As things progress and there are more reasons
> for
> > people to be using both projects at once I expect that will have to
> start
> > changing.
> > As for lazy import, my contribution to the discussion is the
> LazyPluggable
> > class in utils.py, it is currently different from the others because
> it
> > actually lazy (rather than just runtime) and because it limits the
> list of
> > importable modules but could easily have that removed. Between
> importing
> > objects vs modules that is a pretty easy thing to ignore with a try-
> catch
> > (try first to import it as a module if that fails split the tail off
> and try
> > as an object). LazyPluggable currently does that by allowing a tuple
> to be
> > set as the module, but it is easy enough to make it just try both.
> > Re: class vs instance, I think class is still the way to go i the
> majority
> > of cases but it doesn't matter in practice, if you know you are
> expecting a
> > class you'll just attempt to instantiate that when you get it, but if
> the
> > string is to a function rather than a class you can still use that
> function
> > as a factory without any change to your code.
> > Examples of the various use cases in the ideal system:
> > # Code expecting a class
> > virt_driver = LazyPluggable(FLAGS['virt_driver'])
> > class ComputeManager(object):
> >   def __init__(self):
> >     self.virt = virt_driver()
> > # Above works if virt_driver points to:
> > # nova.virt.foo.VirtDriver
> > class VirtDriver(object):
> >   pass
> > # or if it points to
> > # nova.virt.bar.VirtDriverFactory
> > def VirtDriverFactory():
> >   return VirtDriver()
> >
> > # Code expecting an instance
> > db_backend = LazyPluggable(FLAGS['db_backend'])
> > class ComputeManager(object):
> >   def __init__(self):
> >     self.db = db_backend
> >     self.db.some_func()
> > # Above works if db_backend points to:
> > # nova.db.foo
> > def some_func():
> >   pass
> > # or if it points to
> > # nova.db.bar.public_api
> > class PublicApi(object):
> >   def some_func(self):
> >     pass
> > public_api = PublicApi()
> > All that functionality can be integrated into LazyPluggable
> trivially.
> > Why LazyPluggable over import_*? It is declarative, you can place it
> at the
> > top of your file and people can see all the pluggable services this
> file
> > interacts with and which flags control them.
> > --andy
> > On Thu, Jan 13, 2011 at 3:43 PM, Devin Carlen
> <devin.carlen@xxxxxxxxx>
> > wrote:
> >>
> >> I see no problem with putting lazy loading in common.  Just because
> it's
> >> there doesn't mean swift has to use it. Common simply implies that
> they are
> >> modules used by several, but not necessarily all, projects.
> >>
> >>
> >> On Jan 13, 2011, at 3:24 PM, Vishvananda Ishaya wrote:
> >>
> >> > The lazy loading in common seems fine minus one small issue.  If I
> read
> >> > it correctly It looks like it is limited to a class or module.
> There
> >> > doesn't seem to be a way to proxy into an object itself.  I'd like
> to be
> >> > able to specify a class (or a method) and have it lazy loaded into
> an object
> >> > backend.  This is what import_object did automatically that makes
> it so
> >> > versatile.  Admittedly it had an annoying bug that ate underlying
> >> > exceptions, but imo, that is a bug that can be addressed.
> >> >
> >> > Vish
> >> >
> >> > On Jan 13, 2011, at 3:07 PM, Ewan Mellor wrote:
> >> >
> >> >> I can understand that the Swift guys don't want to destabilize
> their
> >> >> codebase, but Glance and Nova should have some common items, no?
>  Lazy
> >> >> loading, logging, and I18N support all spring to mind as likely
> to be common
> >> >> across Glance and Nova.
> >> >>
> >> >> Ewan.
> >> >>
> >> >>> -----Original Message-----
> >> >>> From: Jay Pipes [mailto:jaypipes@xxxxxxxxx]
> >> >>> Sent: 13 January 2011 11:50
> >> >>> To: Ewan Mellor
> >> >>> Cc: Todd Willey; openstack@xxxxxxxxxxxxxxxxxxx
> >> >>> Subject: Re: [Openstack] Lazy import of modules
> >> >>>
> >> >>> On Wed, Jan 12, 2011 at 6:43 PM, Ewan Mellor
> >> >>> <Ewan.Mellor@xxxxxxxxxxxxx> wrote:
> >> >>>> At the risk of starting to shave yaks: do we want to have an
> >> >>> openstack-common then?  It seems to be DOA at the moment.
> >> >>>
> >> >>> There's little to no agreement on common principles and code
> between
> >> >>> the projects, unfortunately. It would be best, IMHO, to just put
> this
> >> >>> in the Nova project alone. Swift coders don't have much interest
> in
> >> >>> using much of the code in Nova, and for Glance, we've used a bit
> of
> >> >>> code from Nova but will not likely be using much more, and may
> even
> >> >>> revert some of the Nova-centric stuff (like flags.py) to more
> >> >>> standards-based approaches used by Swift.
> >> >>>
> >> >>> -jay
> >> >> _______________________________________________
> >> >> Mailing list: https://launchpad.net/~openstack
> >> >> Post to     : openstack@xxxxxxxxxxxxxxxxxxx
> >> >> Unsubscribe : https://launchpad.net/~openstack
> >> >> More help   : https://help.launchpad.net/ListHelp
> >> >
> >> >
> >> > _______________________________________________
> >> > Mailing list: https://launchpad.net/~openstack
> >> > Post to     : openstack@xxxxxxxxxxxxxxxxxxx
> >> > Unsubscribe : https://launchpad.net/~openstack
> >> > More help   : https://help.launchpad.net/ListHelp
> >>
> >>
> >> _______________________________________________
> >> Mailing list: https://launchpad.net/~openstack
> >> Post to     : openstack@xxxxxxxxxxxxxxxxxxx
> >> Unsubscribe : https://launchpad.net/~openstack
> >> More help   : https://help.launchpad.net/ListHelp
> >
> >
> >
> > _______________________________________________
> > Mailing list: https://launchpad.net/~openstack
> > Post to     : openstack@xxxxxxxxxxxxxxxxxxx
> > Unsubscribe : https://launchpad.net/~openstack
> > More help   : https://help.launchpad.net/ListHelp
> >
> >