nova team mailing list archive

[Monty Taylor <mordred@xxxxxxxxxxxx>]

++

On 08/04/2010 09:23 AM, Eric Day wrote:
> Hi Michael,
> 
> ++ On all points. Over the past few days while researching all this,
> I went from being in the middle to the Eventlet side. Another couple
> points to make:
> 
> * We are not actually using twisted for what it is powerful for -
>   non-blocking I/O handling. The AMQP and Redis libs are still using
>   blocking sockets and queues are being polled with a timer every 100ms
>   with sync operations. There is still some work to be done to use
>   it properly (replacing libs with twisted versions, restructure app).
> 
> * The places we are using twisted currently may not even need this,
>   using a multi-process or multi-thread model may make more sense. See
>   active thread discussing this right now.
> 
> * The application where we do need twisted/eventlet - nova-api - is
>   currently still on tornado, so we can start using eventlet there and
>   don't need to worry about converting twisted code already written
>   (no wasted resources).
> 
> * We have eventlet experts and a large scale app already on hand
>   with swift, so we can bug them with any questions. :)
> 
> -Eric
> 
> On Wed, Aug 04, 2010 at 12:03:49PM -0400, Michael Gundlach wrote:
>>    All,
>>    I propose that we go ahead and decide to use Eventlet.  We've all had a
>>    chance to talk, the decision isn't going to make itself, and it seems like
>>    we're coalescing around Eventlet:
>>      * In a long discussion about how to keep our code loosely coupled and
>>        easy to maintain (see Architecture For Shared Components on the
>>        openstack list), we concluded that WSGI would be a great solution, and
>>        eday has prototyped an Eventlet-powered WSGI app talking AMQP to
>>        Rabbit.  The Twisted WSGI approach apparently isn't stable or
>>        standard, but WSGI works just as you'd expect in Eventlet.
>>      * As hazmat points out in the "twisted v. eventlet" thread, you can't
>>        use pdb with Twisted, while pdb works just as you'd hope in Eventlet.
>>      * hazmat's experience during his lightning talk (see the same thread) is
>>        a good datapoint verifying that people unfamiliar with either project
>>        will probably prefer to work with Eventlet over Twisted: good for
>>        recruiting.
>>      * It's a weaker argument, but it's really hard to find any comparisons
>>        between Twisted and Eventlet on the web that say Twisted is preferable
>>        (has anyone found any?)
>>    And as always, there's the argument that Eventlet code is easier to read
>>    and understand, and therefore to maintain.
>>    I'm going to be working on adding the RS API to Nova, and I'm going to
>>    proceed in Eventlet.  If anyone feels strongly that we must use Twisted
>>    instead, speak up; otherwise, shall we call this decision made?
>>    trying-to-build-consensus-ly yrs,
>>    Michael
>>    On Tue, Aug 3, 2010 at 8:13 AM, Soren Hansen <soren.hansen@xxxxxxxxxxxxx>
>>    wrote:
>>
>>      On Mon, Aug 02, 2010 at 05:21:23PM -0500, Eric Day wrote:
>>      > I took a closer look at the inner workings of how Twisted is currently
>>      > being used, and I think there is still a good chunk of work to be done
>>      > to make things work properly. For example, both Redis and AMQP modules
>>      > are not using the Twisted I/O event loop. Both are still issuing sync
>>      > socket calls in their own libs which could block the entire
>>      > application.
>>
>>      Yup, this definitely needs fixing.
>>      > These things are very doable in Twisted, it will just require using
>>      > the txRedis and txAMQP modules. It will also introduce more deferreds
>>      > into the mix since we'll have non-blocking connections dependent on
>>      > each other. From this perspective, Eventlet may be easier since we
>>      > could still use the current libraries by patching
>>      > (http://eventlet.net/doc/patching.html).
>>
>>      Yes, both python-redis and python-amqplib use Python's standard socket
>>      module, so they can be patched.
>>
>>      Does anyone know how eventlet would deal with libvirt calls?
>>      python-libvirt just calls into libvirt obviously doesn't use Python's
>>      socket module.
>>      --
>>      Soren Hansen
>>      Systems Architect
>>      The Rackspace Cloud
>>
>>      _______________________________________________
>>      Mailing list: https://launchpad.net/~nova
>>      Post to     : nova@xxxxxxxxxxxxxxxxxxx
>>      Unsubscribe : https://launchpad.net/~nova
>>      More help   : https://help.launchpad.net/ListHelp
>>
>>  Confidentiality Notice: This e-mail message (including any attached or
>>  embedded documents) is intended for the exclusive and confidential use of the
>>  individual or entity to which this message is addressed, and unless otherwise
>>  expressly indicated, is confidential and privileged information of Rackspace.
>>  Any dissemination, distribution or copying of the enclosed material is prohibited.
>>  If you receive this transmission in error, please notify us immediately by e-mail
>>  at abuse@xxxxxxxxxxxxx, and delete the original message.
>>  Your cooperation is appreciated.
> 
>> _______________________________________________
>> Mailing list: https://launchpad.net/~nova
>> Post to     : nova@xxxxxxxxxxxxxxxxxxx
>> Unsubscribe : https://launchpad.net/~nova
>> More help   : https://help.launchpad.net/ListHelp
> 
> 
> _______________________________________________
> Mailing list: https://launchpad.net/~nova
> Post to     : nova@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~nova
> More help   : https://help.launchpad.net/ListHelp
>