openstack team mailing list archive

[Russell Bryant <rbryant@xxxxxxxxxx>]

Thanks for sharing this information.  For the future, I think this type
of analysis and discussion is something that is great to have on the
mailing list instead of just a private group.  I wish I had seen it sooner.

The code in nova.rpc seems useful enough that it very well may be used
elsewhere.  I know it's more than what is needed for notifications, but
it does support what is needed for notifications (and more).  I like the
idea of moving the whole thing instead of having separate messaging code
for just notifications.

-- 
Russell Bryant

On 04/03/2012 09:20 AM, Swaminathan Venkataraman wrote:
> Hi,
> I'm actively working on the notification part. I did some analysis on
> the code and dependencies and was planning to submit a blueprint by end
> of the week. We can use that to finalize the interface for the
> notification. The rpc implementation is rich (compared to just what we
> need for notifications) because nova uses it for all rpc related
> communications. The idea that I was working with was to just move what
> we need for notifications. In that scenario we do not really need all of
> rpc in openstack-common. If we do want a common implementation that all
> openstack components can use to communicate the middleware, it might
> make to sense to move the whole of rpc to openstack-common. 
> 
> Thoughts?
> 
> 
> Anyways, here is the analysis and some of the comments I got...
> 
> Cheers,
> Venkat
> 
>   *
> 
> 
>     ---------- Forwarded message ----------
>     From: *Swaminathan Venkataraman* <venkat.db@xxxxxxxxx
>     <mailto:venkat.db@xxxxxxxxx>>
>     Date: Mon, Mar 19, 2012 at 8:31 PM
>     Subject: Re: [Openstack] Notifiers
>     To: Monsyne Dragon <mdragon@xxxxxxxxxxxxx
>     <mailto:mdragon@xxxxxxxxxxxxx>>
>     Cc: Mark McLoughlin <markmc@xxxxxxxxxx <mailto:markmc@xxxxxxxxxx>>,
>     Jason Kölker <jason.koelker@xxxxxxxxxxxxx
>     <mailto:jason.koelker@xxxxxxxxxxxxx>>, Jay Pipes <jaypipes@xxxxxxxxx
>     <mailto:jaypipes@xxxxxxxxx>>
> 
> 
>     I did some analysis on notifier and rpc in nova and there are a
>     bunch of dependencies that have to be sorted out before we can move
>     them to openstack-common. Here are some of the details.
>      
>       o notifier and rpc use flags, utils, logging, context, db,
>         exception, from nova.
>       o
>       o The modules in notfier and rpc use FLAGS from flags.py which is
>         an instance of NovaConfigOpts. They mainly use it to register
>         the config options and access them. Given that, it seems like we
>         could use CommonConfigOpts directly to register the options.
>         This will eliminate the dependency on flags and flagfile.
>       o
>       o There are three functions that are used from utils - utcnow,
>         import_object, and to_primitive. There is a utils in
>         openstack-common which already contains utcnow and
>         import_object. The code also macthes line to line with the
>         implementation in nova. The to_primitive function is missing in
>         openstack-common. One option could be to move this function
>         alone to openstack-common which should eliminate the dependency
>         on the nova based utils.
>       o
>       o notifier and api use log from nova. In fact they work with an
>         instance of NovaContextAdapter which in turn is an instance of
>         LoggerAdapter. NovaContextAdapter is used to pass the context,
>         the instance uuid, and the nova version to the logger. The
>         modules in openstack-common are using the python logging module
>         directly. So, if we need notifier to be able to print contextual
>         information we will have to add this functionality to
>         openstack-common.
>       o
>       o Both nova and openstack-common have an implementation of
>         RequestContext. The one in Nova is richer and both notifier and
>         rpc use functionality from RequestContext in nova. The other
>         difference is that the RequestContext in nova uses a weak
>         refernce store to save the context information. I did see a
>         couple of instances where the context information was deleted
>         from the store, but I'm not sure whether it is being accessed.
>         So, should the context in openstack-common be enhanced?
>       o
>       o db from nova is used only by capacity_notifier. It looks like it
>         sends events that are only related to compute manager events.
>         So, should this be part of openstack-common?
>     I've not looked at exception. I'll also have to look at rpc in more
>     detail. Please do let me know if this is the right direction.
>      
>     thanks,
>     Venkat
> 
> ---------- Forwarded message ----------
> From: *Mark McLoughlin* <markmc@xxxxxxxxxx <mailto:markmc@xxxxxxxxxx>>
> Date: Tue, Mar 20, 2012 at 8:05 PM
> Subject: Re: [Openstack] Notifiers
> To: Swaminathan Venkataraman <venkat.db@xxxxxxxxx
> <mailto:venkat.db@xxxxxxxxx>>
> Cc: Monsyne Dragon <mdragon@xxxxxxxxxxxxx
> <mailto:mdragon@xxxxxxxxxxxxx>>, Jason Kölker
> <jason.koelker@xxxxxxxxxxxxx <mailto:jason.koelker@xxxxxxxxxxxxx>>, Jay
> Pipes <jaypipes@xxxxxxxxx <mailto:jaypipes@xxxxxxxxx>>
> 
> ---------- Forwarded message ----------
> From: *Mark McLoughlin* <markmc@xxxxxxxxxx <mailto:markmc@xxxxxxxxxx>>
> Date: Tue, Mar 20, 2012 at 1:25 PM
> Subject: Re: [Openstack] Notifiers
> To: Swaminathan Venkataraman <venkat.db@xxxxxxxxx
> <mailto:venkat.db@xxxxxxxxx>>
> Cc: Monsyne Dragon <mdragon@xxxxxxxxxxxxx
> <mailto:mdragon@xxxxxxxxxxxxx>>, Jason Kölker
> <jason.koelker@xxxxxxxxxxxxx <mailto:jason.koelker@xxxxxxxxxxxxx>>, Jay
> Pipes <jaypipes@xxxxxxxxx <mailto:jaypipes@xxxxxxxxx>>
> 
> 
> Hi Venkat,
> 
> Could you file a bug or blueprint against openstack-common with all this
> great info?
> 
> Cheers,
> Mark.
> 
> On Tue, 2012-03-20 at 19:37 +0530, Swaminathan Venkataraman wrote:
>> Sure Mark, but here is a bit more of analysis that I did. I'll file a
>> blueprint because I'm not sure if this is a bug.
>>
>>
>> There is an exception module defined in openstack-common. This has a
>> class named openstackException which is similar to NovaException and I
>> guess is to be subclassed to define exceptions that go in
>> openstack-common. openstack-common also defines a decorator for
>> wrapping  methods to catch exceptions, but it does not try to send the
>> exception to the notification system like the one in nova.exception
>> does. Based on this we could use exceptions in openstack-common in
>> notifier and rpc. We'll still have to figure out what to do with
>> exceptions which are common (like ClassNotFoundexception). We'll have
>> to duplicate them or modules in nova will have to import two different
>> exception modules with different hierarchies, if we move common
>> exceptions to openstack-common. The ideal scenario is to have one
>> single hierarchy for exceptions, but I guess that this is a bigger
>> decision.
> 
> Firstly, I wouldn't consider the openstack.common.exception as
> particularly complete or final. I'd be happy for us to completely
> re-think the approach to exceptions in openstack-common.
> 
> The wrap_exception() decorator could continue to live in nova for now.
> 
> exception.ClassNotFound is only raised by utils.import_object, but
> openstack-common has its own version of that method and raises its own
> exception.NotFound. So, no problem there.
> 
> I think its best to leave rabbit_notifier.py in nova for the moment.
> Perhaps move it to the nova.rpc module.
> 
>> The driver for notification is decided based on
>> FALGS.notification_driver. In the current scenario, since glance and
>> nova use different methods to do notifications, they can end up with
>> different drivers. If we do make the notifier common and if we want to
>> provide the capability for each of the openstack components to use
>> different drivers, we cannot just use the driver defined in the
>> configuration. One way is to pass the notifier as a parameter or
>> provide the configuration to be used to get the notifier and use
>> different configuration parameters for different components.
> 
> I think we'll want the log, list, no-op and test notifiers in
> openstack-common.
> 
> The way to handle configuration is to do what glance does - have a
> Notifier class which accepts a ConfigOpts as a constructor parameter.
> The class defines a "strategy" or "driver" option and loads the
> appropriate driver based on that option.
> 
> IMHO, the best way to approach moving stuff like this to
> openstack-common is to start with an (almost) fresh slate - don't be too
> worried about what Nova or Glance currently does, but instead try and
> define the best API to meet their requirements.
> 
> Cheers,
> Mark.
> 
> 
>