nova team mailing list archive

[Justin Santa Barbara <justin@xxxxxxxxxxxx>]

I vote for simple python threading for the back-end services.

As someone that's been working in the back-end services, my code is less
elegant, harder to understand, and more buggy, by virtue of being
asynchronous.  I don't believe that is offset by any potential scalability
benefits when we're talking about the back-end services, which don't need to
scale to as many concurrent requests.

Indeed, the back-end services are dealing with systems that aren't
necessarily designed for highly concurrent operation (e.g. KVM, iSCSI or AoE
exporting, LVM). I'm betting we'll find lots of bugs if we try hitting those
systems with thousands of concurrent requests.

Also, it seems that eventlet is promising that one day we can simply drop in
some magic incantations and get async code for free, should we ever need it.


On Wed, Aug 4, 2010 at 9:03 AM, Michael Gundlach <
michael.gundlach@xxxxxxxxxxxxx> wrote:

> All,
>
> I propose that we go ahead and decide to use Eventlet.  We've all had a
> chance to talk, the decision isn't going to make itself, and it seems like
> we're coalescing around Eventlet:
>
>
>    - In a long discussion about how to keep our code loosely coupled and
>    easy to maintain (see Architecture For Shared Components on the openstack
>    list), we concluded that WSGI would be a great solution, and eday has
>    prototyped an Eventlet-powered WSGI app talking AMQP to Rabbit.  The Twisted
>    WSGI approach apparently isn't stable or standard, but WSGI works just as
>    you'd expect in Eventlet.
>
>
>
>    - As hazmat points out in the "twisted v. eventlet" thread, you can't
>    use pdb with Twisted, while pdb works just as you'd hope in Eventlet.
>
>
>
>    - hazmat's experience during his lightning talk (see the same thread)
>    is a good datapoint verifying that people unfamiliar with either project
>    will probably prefer to work with Eventlet over Twisted: good for
>    recruiting.
>
>
>
>    - It's a weaker argument, but it's really hard to find any comparisons
>    between Twisted and Eventlet on the web that say Twisted is preferable (has
>    anyone found any?)
>
>
> And as always, there's the argument that Eventlet code is easier to read
> and understand, and therefore to maintain.
>
> I'm going to be working on adding the RS API to Nova, and I'm going to
> proceed in Eventlet.  If anyone feels strongly that we must use Twisted
> instead, speak up; otherwise, shall we call this decision made?
>
> trying-to-build-consensus-ly yrs,
> Michael
>
>
> On Tue, Aug 3, 2010 at 8:13 AM, Soren Hansen <soren.hansen@xxxxxxxxxxxxx>wrote:
>
>> On Mon, Aug 02, 2010 at 05:21:23PM -0500, Eric Day wrote:
>> > I took a closer look at the inner workings of how Twisted is currently
>> > being used, and I think there is still a good chunk of work to be done
>> > to make things work properly. For example, both Redis and AMQP modules
>> > are not using the Twisted I/O event loop. Both are still issuing sync
>> > socket calls in their own libs which could block the entire
>> > application.
>>
>> Yup, this definitely needs fixing.
>>
>> > These things are very doable in Twisted, it will just require using
>> > the txRedis and txAMQP modules. It will also introduce more deferreds
>> > into the mix since we'll have non-blocking connections dependent on
>> > each other. From this perspective, Eventlet may be easier since we
>> > could still use the current libraries by patching
>> > (http://eventlet.net/doc/patching.html).
>>
>> Yes, both python-redis and python-amqplib use Python's standard socket
>> module, so they can be patched.
>>
>> Does anyone know how eventlet would deal with libvirt calls?
>> python-libvirt just calls into libvirt obviously doesn't use Python's
>> socket module.
>>
>> --
>> Soren Hansen
>> Systems Architect
>> The Rackspace Cloud
>>
>> _______________________________________________
>> Mailing list: https://launchpad.net/~nova
>> Post to     : nova@xxxxxxxxxxxxxxxxxxx
>> Unsubscribe : https://launchpad.net/~nova
>> More help   : https://help.launchpad.net/ListHelp
>>
>
> Confidentiality Notice: This e-mail message (including any attached or
> embedded documents) is intended for the exclusive and confidential use of the
> individual or entity to which this message is addressed, and unless otherwise
> expressly indicated, is confidential and privileged information of Rackspace.
> Any dissemination, distribution or copying of the enclosed material is prohibited.
> If you receive this transmission in error, please notify us immediately by e-mail
> at abuse@xxxxxxxxxxxxx, and delete the original message.
> Your cooperation is appreciated.
>
>
> _______________________________________________
> Mailing list: https://launchpad.net/~nova
> Post to     : nova@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~nova
> More help   : https://help.launchpad.net/ListHelp
>
>