nova team mailing list archive

[Eric Day <eday@xxxxxxxxxxxx>]

Hi Vish,

I completely understand not wanting to change things up again and
introduce new bugs, but we also need to think about the longevity
of the project beyond what may be a couple more weeks of headaches
getting everything running in a simpler framework. I'm not saying
Twisted is not the correct way to go, but I think at this point we
should be making decisions focused on the long term since we still
have some flexibility.

I took a closer look at the inner workings of how Twisted is currently
being used, and I think there is still a good chunk of work to be done
to make things work properly. For example, both Redis and AMQP modules
are not using the Twisted I/O event loop. Both are still issuing
sync socket calls in their own libs which could block the entire
application. Also, AMQP is polling queues on a timer every 100ms,
which is awfully chatty and introduces up to 100ms artificial delay
for requests (a big deal when there is a blocking HTTP request). This
really needs to be event driven from RabbitMQ (channel.wait()) rather
than polled.

These things are very doable in Twisted, it will just require
using the txRedis and txAMQP modules. It will also introduce more
deferreds into the mix since we'll have non-blocking connections
dependent on each other. From this perspective, Eventlet may be
easier since we could still use the current libraries by patching
(http://eventlet.net/doc/patching.html).

I could go either way still, but am leaning towards Eventlet to keep
the code base simpler. I'm sure Twisted will work just fine, but
we'll still need to switch in new modules for those two core services.

-Eric

On Fri, Jul 30, 2010 at 11:49:14AM -0700, Vishvananda Ishaya wrote:
>    One vote for sticking with Twisted here.
>    We've had more issues dealing with async processes, threading, and logging
>    then any other part of the Nova system. *After a lot of late debugging
>    nights we actually have a system that works and handles these things well.
>    *I think there is a hidden cost in switching is redoing much of this
>    effort and dealing with instability while we get the kinks worked out. *I
>    would prefer not to go through that process again :)
>    Debugging these issues is challenging and so is writing tests. *I would
>    suggest that if we are going to make a major switch like this, we spend
>    some time banging away at test coverage so that we can feel more confident
>    that we aren't breaking the sytem.
>    Vish
>    On Fri, Jul 30, 2010 at 8:42 AM, Eric Day <eday@xxxxxxxxxxxx> wrote:
> 
>      On Fri, Jul 30, 2010 at 10:51:06AM -0400, Jay Pipes wrote:
>      > FWIW, I'm perfectly happy to work with either. :)
> 
>      Same here, and I do see Soren's point about having a fairly large
>      protocol suite ready to go. On the other hand, Eventlet does
>      provide module patching to make existing modules non-blocking (see
>      http://eventlet.net/doc/basic_usage.html#patching-functions). Can
>      anyone speak to how well this works? My main concern, as
>      Michael pointed out, is the ease of use and ability to get others
>      involved. I've talked with other programmers in the past who had the
>      same reaction to Twisted. We need to think about the longevity of
>      the project and which choices will keep the barrier to entry as low
>      as possible while still meeting our needs.
>      -Eric
>      >
>      > -jay
>      >
>      > On Fri, Jul 30, 2010 at 8:25 AM, Michael Gundlach
>      > <michael.gundlach@xxxxxxxxxxxxx> wrote:
>      > > On Thu, Jul 29, 2010 at 5:42 PM, Soren Hansen
>      <soren.hansen@xxxxxxxxxxxxx>
>      > > wrote:
>      > >>
>      > >> On Thu, Jul 29, 2010 at 03:47:46PM -0500, Michael Gundlach wrote:
>      > >> > +1 for Eventlet for the reasons already stated. *Simpler code
>      often
>      > >> > translates to fewer bugs and faster development, and I expect
>      that
>      > >> > working with Eventlet will also mean more people who are
>      interested in
>      > >> > contributing to OpenStack.
>      > >>
>      > >> Really? You think there's more of a community around eventlet than
>      > >> Twisted?
>      > >
>      > > No info there. *I have not used Eventlet or Twisted in the past, and
>      am
>      > > giving the perspective of an outsider considering working with the
>      project.
>      > > *I had two offers earlier this year -- one to work at a company
>      solving a
>      > > large scaling problem on a Twisted system, and one to work solving a
>      large
>      > > scaling problem working in Eventlet. *They were pretty similar
>      offers so it
>      > > came down largely to choosing the technology I'd want to work with.
>      > > When I researched Twisted, I read or got the impression that:
>      > > - it had a "near-vertical learning curve"
>      > > - it was named Twisted for a reason -- that it's hard to think the
>      way
>      > > Twisted needs you to think
>      > > - its libraries were a morass (the homepage talks about having two
>      web
>      > > frameworks and it's not clear which one to choose)
>      > > When I researched Eventlet, I read or got the impression that:
>      > > - it is clean and intuitive to code in
>      > > - it was the thing that came along after Twisted to solve the
>      problem in a
>      > > better way
>      > > - momentum was gaining around it while Twisted was losing momentum
>      > > I took the Eventlet job with Rackspace (having mistakenly understood
>      during
>      > > interviews that Rackspace was for-sure migrating to Eventlet :) )
>      because
>      > > what I had been able to find out made Twisted look unappetizing and
>      Eventlet
>      > > look great.
>      > > I would imagine that my experience is typical -- a shallow review of
>      the two
>      > > technologies made Eventlet much more appealing to work with. *Thus
>      my
>      > > suggestion that we'll get more contributions if we go with Eventlet.
>      *That
>      > > said, if there's some technical reason that one system is better for
>      our
>      > > needs than the other, of course that trumps evangelism.
>      > > HTH,
>      > > Michael
>      > > _______________________________________________
>      > > Mailing list: https://launchpad.net/~nova
>      > > Post to * * : nova@xxxxxxxxxxxxxxxxxxx
>      > > Unsubscribe : https://launchpad.net/~nova
>      > > More help * : https://help.launchpad.net/ListHelp
>      > >
>      > >
>      >
>      > _______________________________________________
>      > Mailing list: https://launchpad.net/~nova
>      > Post to * * : nova@xxxxxxxxxxxxxxxxxxx
>      > Unsubscribe : https://launchpad.net/~nova
>      > More help * : https://help.launchpad.net/ListHelp
> 
>      _______________________________________________
>      Mailing list: https://launchpad.net/~nova
>      Post to * * : nova@xxxxxxxxxxxxxxxxxxx
>      Unsubscribe : https://launchpad.net/~nova
>      More help * : https://help.launchpad.net/ListHelp

> _______________________________________________
> Mailing list: https://launchpad.net/~nova
> Post to     : nova@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~nova
> More help   : https://help.launchpad.net/ListHelp