openstack team mailing list archive

[Johannes Erdfelt <johannes@xxxxxxxxxxx>]

On Fri, Mar 02, 2012, Duncan McGreggor <duncan@xxxxxxxxxxxxx> wrote:
> On Fri, Mar 2, 2012 at 2:40 PM, Johannes Erdfelt <johannes@xxxxxxxxxxx> wrote:
> > Twisted has a much harder programming model with the same blocking
> > problem that eventlet has.
> 
> Like so many things that are aesthetic in nature, the statement above
> is misleading. Using a callback, event-based, deferred/promise
> oriented system is hard for *some*. It is far, far easier for others
> (myself included).
> 
> It's a matter of perception and personal preference.
>
> It may be apropos to mention that Guido van Rossum himself has stated
> that he shares the same view of concurrent programming in Python as
> Glyph (the founder of Twisted):
>   https://plus.google.com/115212051037621986145/posts/a9SqS7faVWC
> 
> Glyph's post, if you can't see that G+ link:
>   http://glyph.twistedmatrix.com/2012/01/concurrency-spectrum-from-callbacks-to.html
> 
> One thing to keep in mind is that with Twisted, you always have the
> option of deferring to a thread for operations are not async-friendly.

It's a shame that post chooses to ignore eventlet-style concurrency. It
has all of the benefits of being almost as clear where concurrency can
occur without needing a macro key to constantly output 'yield'.

It also integrates with other python libraries better (but obviously not
perfectly).

Using coroutines for concurrency is anti-social programming. It excludes
a whole suite of libraries merely because they didn't conform to your
programming model.

However, this is the wrong discussion to be having. Concurrency isn't
the problem we should be worried about, it's isolation. If we can
sufficiently isolate the work that each daemon needs to do, then
concurrency is trivial. In the best case, they can be separate processes
and we don't need to worry about a programming model. If we're not being
too optimistic then threads with minimal locking is most likely.

JE