nova team mailing list archive

[Kapil Thangavelu <kapil.foss@xxxxxxxxx>]

On Tue, Aug 3, 2010 at 7:57 AM, Soren Hansen <soren.hansen@xxxxxxxxxxxxx>wrote:

> On Mon, Aug 02, 2010 at 03:19:26PM -0500, Justin Santa Barbara wrote:
> > A) Do I change the interface so that the method returns a deferred
> > even in the providers that are synchronous?  If so, we should just
> > have all interfaces return a deferred on all methods, so that the
> > interface doesn't expose the implementation.  However, this would make
> > everything messy...
>
> The fact that e.g. libvirt.get_info() is synchronous is, IMO, a bug. It
> calls out to libvirt and hangs the entire process until libvirt decides
> to respond. Until libvirt becomes async itself, we need make these calls
> in a separate thread or process so that we can carry on while waiting
> for libvirt to get back to us.
>
> > B) Should I type-check the result of the call and synchronously wait
> > on it if it is deferred?
>
> As Vish pointed out, that's what maybeDeferred is for.
>
> > (And can anyone point me at some code that can do a sync-wait?)
>
> Using standard Twisted, no. I doubt it's possible. You need threads or
> coroutines or something to pull that off and Twisted uses neither.
>

Well its possible, but its not useful, as it will still hang the process.



>
> > As an aside, and to borrow from the Wizard of Oz, "Twisted, I've a
> > feeling we're not in Python any more".  Every method is preceded by
> > @defer.inlineCallbacks, return x becomes returnValue(x), and there are
> > yields scattered throughout the code based on whether the called
> > method returns a deferred or not.
>
> Yup. inlineCallbacks is a bit of an acquired taste. I'm not entirely
> there yet myself.
>
> > Perhaps we can clarify the Twisted vs Eventlet discussion: would
> > Eventlet solve this particular problem?
>
> The inlineCallbacks thing? Yes. The libvirt thing? Not as far as I know.
> eventlet has this patching mechanism, but as I understand it, it relies
> on the particular module using Python's socket module to work, which
> python-libvirt doesn't, AFAIK.
>
>
making the libvirt sync call into async for the purposes of twisted should
be easy enough..

def threaded_invoke_sync_hang_api(on_result, args):
      result = libvirt.get_info(args)
      reactor.callFromThread(on_result, result)

def get_libvirt_info(*args):
      d = Deferred()
      reactor.deferToThread(threaded_invoke_sync_hang_api, args)
      return d

this is assuming of course the libvirt c bindings release the GIL. I'm sure
the same thing is possible using eventlet, but would probably require going
into the underlying greenlet facilities afaik instead of relying on a
generic monkey'd or socket api call.

cheers,

Kapil