openstack team mailing list archive

[Eric Windisch <eric@xxxxxxxxxxxxxxxx>]

I'd much prefer that we move all the nova.rpc into common. This will facilitate the use of the code in other projects and perhaps even third-party applications. The fact this code isn't reused elsewhere is largely be because it isn't reusable in its current state.

For the ZeroMQ RPC driver, I'd rather that Glance/Nova can use the existing implementation, rather than having to develop and maintain a separate driver for the express purpose of supporting notifications. It is highly redundant.  I'm sure Russell must feel the same way about maintaining two Qpid drivers.

--  
Eric Windisch


On Tuesday, April 3, 2012 at 9:36 AM, Russell Bryant wrote:

> Thanks for sharing this information. For the future, I think this type
> of analysis and discussion is something that is great to have on the
> mailing list instead of just a private group. I wish I had seen it sooner.
>  
> The code in nova.rpc seems useful enough that it very well may be used
> elsewhere. I know it's more than what is needed for notifications, but
> it does support what is needed for notifications (and more). I like the
> idea of moving the whole thing instead of having separate messaging code
> for just notifications.
>  
> --  
> Russell Bryant
>  
> On 04/03/2012 09:20 AM, Swaminathan Venkataraman wrote:
> > Hi,
> > I'm actively working on the notification part. I did some analysis on
> > the code and dependencies and was planning to submit a blueprint by end
> > of the week. We can use that to finalize the interface for the
> > notification. The rpc implementation is rich (compared to just what we
> > need for notifications) because nova uses it for all rpc related
> > communications. The idea that I was working with was to just move what
> > we need for notifications. In that scenario we do not really need all of
> > rpc in openstack-common. If we do want a common implementation that all
> > openstack components can use to communicate the middleware, it might
> > make to sense to move the whole of rpc to openstack-common.  
> >  
> > Thoughts?
> >  
> >  
> > Anyways, here is the analysis and some of the comments I got...
> >  
> > Cheers,
> > Venkat
> >  
> > *
> >  
> >  
> > ---------- Forwarded message ----------
> > From: *Swaminathan Venkataraman* <venkat.db@xxxxxxxxx (mailto:venkat.db@xxxxxxxxx)
> > <mailto:venkat.db@xxxxxxxxx>>
> > Date: Mon, Mar 19, 2012 at 8:31 PM
> > Subject: Re: [Openstack] Notifiers
> > To: Monsyne Dragon <mdragon@xxxxxxxxxxxxx (mailto:mdragon@xxxxxxxxxxxxx)
> > <mailto:mdragon@xxxxxxxxxxxxx>>
> > Cc: Mark McLoughlin <markmc@xxxxxxxxxx <mailto:markmc@xxxxxxxxxx>>,
> > Jason Kölker <jason.koelker@xxxxxxxxxxxxx (mailto:jason.koelker@xxxxxxxxxxxxx)
> > <mailto:jason.koelker@xxxxxxxxxxxxx>>, Jay Pipes <jaypipes@xxxxxxxxx (mailto:jaypipes@xxxxxxxxx)
> > <mailto:jaypipes@xxxxxxxxx>>
> >  
> >  
> > I did some analysis on notifier and rpc in nova and there are a
> > bunch of dependencies that have to be sorted out before we can move
> > them to openstack-common. Here are some of the details.
> >  
> > o notifier and rpc use flags, utils, logging, context, db,
> > exception, from nova.
> > o
> > o The modules in notfier and rpc use FLAGS from flags.py which is
> > an instance of NovaConfigOpts. They mainly use it to register
> > the config options and access them. Given that, it seems like we
> > could use CommonConfigOpts directly to register the options.
> > This will eliminate the dependency on flags and flagfile.
> > o
> > o There are three functions that are used from utils - utcnow,
> > import_object, and to_primitive. There is a utils in
> > openstack-common which already contains utcnow and
> > import_object. The code also macthes line to line with the
> > implementation in nova. The to_primitive function is missing in
> > openstack-common. One option could be to move this function
> > alone to openstack-common which should eliminate the dependency
> > on the nova based utils.
> > o
> > o notifier and api use log from nova. In fact they work with an
> > instance of NovaContextAdapter which in turn is an instance of
> > LoggerAdapter. NovaContextAdapter is used to pass the context,
> > the instance uuid, and the nova version to the logger. The
> > modules in openstack-common are using the python logging module
> > directly. So, if we need notifier to be able to print contextual
> > information we will have to add this functionality to
> > openstack-common.
> > o
> > o Both nova and openstack-common have an implementation of
> > RequestContext. The one in Nova is richer and both notifier and
> > rpc use functionality from RequestContext in nova. The other
> > difference is that the RequestContext in nova uses a weak
> > refernce store to save the context information. I did see a
> > couple of instances where the context information was deleted
> > from the store, but I'm not sure whether it is being accessed.
> > So, should the context in openstack-common be enhanced?
> > o
> > o db from nova is used only by capacity_notifier. It looks like it
> > sends events that are only related to compute manager events.
> > So, should this be part of openstack-common?
> > I've not looked at exception. I'll also have to look at rpc in more
> > detail. Please do let me know if this is the right direction.
> >  
> > thanks,
> > Venkat
> >  
> > ---------- Forwarded message ----------
> > From: *Mark McLoughlin* <markmc@xxxxxxxxxx <mailto:markmc@xxxxxxxxxx>>
> > Date: Tue, Mar 20, 2012 at 8:05 PM
> > Subject: Re: [Openstack] Notifiers
> > To: Swaminathan Venkataraman <venkat.db@xxxxxxxxx (mailto:venkat.db@xxxxxxxxx)
> > <mailto:venkat.db@xxxxxxxxx>>
> > Cc: Monsyne Dragon <mdragon@xxxxxxxxxxxxx (mailto:mdragon@xxxxxxxxxxxxx)
> > <mailto:mdragon@xxxxxxxxxxxxx>>, Jason Kölker
> > <jason.koelker@xxxxxxxxxxxxx <mailto:jason.koelker@xxxxxxxxxxxxx>>, Jay
> > Pipes <jaypipes@xxxxxxxxx <mailto:jaypipes@xxxxxxxxx>>
> >  
> > ---------- Forwarded message ----------
> > From: *Mark McLoughlin* <markmc@xxxxxxxxxx <mailto:markmc@xxxxxxxxxx>>
> > Date: Tue, Mar 20, 2012 at 1:25 PM
> > Subject: Re: [Openstack] Notifiers
> > To: Swaminathan Venkataraman <venkat.db@xxxxxxxxx (mailto:venkat.db@xxxxxxxxx)
> > <mailto:venkat.db@xxxxxxxxx>>
> > Cc: Monsyne Dragon <mdragon@xxxxxxxxxxxxx (mailto:mdragon@xxxxxxxxxxxxx)
> > <mailto:mdragon@xxxxxxxxxxxxx>>, Jason Kölker
> > <jason.koelker@xxxxxxxxxxxxx <mailto:jason.koelker@xxxxxxxxxxxxx>>, Jay
> > Pipes <jaypipes@xxxxxxxxx <mailto:jaypipes@xxxxxxxxx>>
> >  
> >  
> > Hi Venkat,
> >  
> > Could you file a bug or blueprint against openstack-common with all this
> > great info?
> >  
> > Cheers,
> > Mark.
> >  
> > On Tue, 2012-03-20 at 19:37 +0530, Swaminathan Venkataraman wrote:
> > > Sure Mark, but here is a bit more of analysis that I did. I'll file a
> > > blueprint because I'm not sure if this is a bug.
> > >  
> > >  
> > > There is an exception module defined in openstack-common. This has a
> > > class named openstackException which is similar to NovaException and I
> > > guess is to be subclassed to define exceptions that go in
> > > openstack-common. openstack-common also defines a decorator for
> > > wrapping methods to catch exceptions, but it does not try to send the
> > > exception to the notification system like the one in nova.exception
> > > does. Based on this we could use exceptions in openstack-common in
> > > notifier and rpc. We'll still have to figure out what to do with
> > > exceptions which are common (like ClassNotFoundexception). We'll have
> > > to duplicate them or modules in nova will have to import two different
> > > exception modules with different hierarchies, if we move common
> > > exceptions to openstack-common. The ideal scenario is to have one
> > > single hierarchy for exceptions, but I guess that this is a bigger
> > > decision.
> > >  
> >  
> >  
> > Firstly, I wouldn't consider the openstack.common.exception as
> > particularly complete or final. I'd be happy for us to completely
> > re-think the approach to exceptions in openstack-common.
> >  
> > The wrap_exception() decorator could continue to live in nova for now.
> >  
> > exception.ClassNotFound is only raised by utils.import_object, but
> > openstack-common has its own version of that method and raises its own
> > exception.NotFound. So, no problem there.
> >  
> > I think its best to leave rabbit_notifier.py in nova for the moment.
> > Perhaps move it to the nova.rpc module.
> >  
> > > The driver for notification is decided based on
> > > FALGS.notification_driver. In the current scenario, since glance and
> > > nova use different methods to do notifications, they can end up with
> > > different drivers. If we do make the notifier common and if we want to
> > > provide the capability for each of the openstack components to use
> > > different drivers, we cannot just use the driver defined in the
> > > configuration. One way is to pass the notifier as a parameter or
> > > provide the configuration to be used to get the notifier and use
> > > different configuration parameters for different components.
> > >  
> >  
> >  
> > I think we'll want the log, list, no-op and test notifiers in
> > openstack-common.
> >  
> > The way to handle configuration is to do what glance does - have a
> > Notifier class which accepts a ConfigOpts as a constructor parameter.
> > The class defines a "strategy" or "driver" option and loads the
> > appropriate driver based on that option.
> >  
> > IMHO, the best way to approach moving stuff like this to
> > openstack-common is to start with an (almost) fresh slate - don't be too
> > worried about what Nova or Glance currently does, but instead try and
> > define the best API to meet their requirements.
> >  
> > Cheers,
> > Mark.
> >  
>  
>  
> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@xxxxxxxxxxxxxxxxxxx (mailto:openstack@xxxxxxxxxxxxxxxxxxx)
> Unsubscribe : https://launchpad.net/~openstack
> More help : https://help.launchpad.net/ListHelp
>  
>