openstack team mailing list archive

["Brian Lamar" <brian.lamar@xxxxxxxxxxxxx>]

(There have been many more replies since I started writing this.)

Great stuff on the wiki. My $0.02:

+1 vote for Python/C modules for needed 'performance'.

I might be the only one thinking that Python is up to the task, but I wanted to throw that out there.

+0 for Erlang, although I *love* it and there are a ton of libraries out there, I'm really on the fence with how many people could actually contribute to the project. A lot of ramp-up and learning does not a stable project make (if the release schedule really is as tight as indicated on the Wiki). 

C++ and I have a history and I'd rather everything be written in C (yeah, I'm one of those people). For truly complex tasks C + Python work very well together and can produce some amazing results.

That being said, C++ is a solid choice.

Languages aside, it looks like you've got a decent start going, Eric, with Scale Stack. I do worry about reinventing the wheel a bit, although I'm not clear on the goals of your project so I could be wrong.

Could this service just be an aggregation of some other proven technologies? The first thing that came to mind when this service was suggested was using a proven scalable key/value store to provide a queue-like interface and then all we'd have to do is provide the interface on top of it.

HTTP + Authentication + Direct to customer's data store cluster = Queue Service?

Voldemort, Keyspace, Scalaris, and tons of other projects could be researched to provide a scalable back-end for the queuing system.

Maybe I don't quite understand which problem Scale Stack is solving? HTTP API + 

-----Original Message-----
From: "Eric Day" <eday@xxxxxxxxxxxx>
Sent: Thursday, February 17, 2011 5:21pm
To: openstack@xxxxxxxxxxxxxxxxxxx
Subject: [Openstack] Queue Service, next steps

Thanks to everyone who gave great feedback on the first queue service
thread. I've updated the wiki page to include the suggestions.

http://wiki.openstack.org/QueueService

With a decent vision of what we want to build, the next step is
figuring out how. In a previous thread it was suggested that the
preferred languages for OpenStack projects are Python, C, and
C++. Since there is an emphasis on speed and efficiency for the
queue service, I suggest we use C++. I expect this service to be
CPU bound and would benefit being able to leverage multiple cores
efficiently (within the same process), so I don't think Python
is a good fit. I think C++ is a better fit than C due to the need
for modular interfaces. While this can obviously be done in C, C++
APIs are more concise and much less error prone. The OO style will
also make it easier for Python developers who also want to learn and
assist with C++ projects.

Erlang is not on the preferred lists, but I would also put it out
there as an option. While it may be a great fit for a project like
this, I worry it won't attract the developer resources since Erlang
isn't really a first-class language yet.

If we decide to take the C++ path, I propose using a modular
application framework I've been working on over the past year (mostly
in my spare time). It provides a simple module programming interface
with dependency tracking (kind of like Linux kernel modules). It
already provides a multi-threaded event module (currently based on
libevent, but this is pluggable) with simple networking abstractions
built on top of it. We should be able to dive in an start writing the
HTTP protocol module and queue processing modules. You can check out
the current project at:

https://launchpad.net/scalestack
http://scalestack.org/

The intention of using a framework like this is so we can easily reuse
the other modules (auth, HTTP, logging, ...) for other OpenStack
services in the future. Much like we use Eventlet, WSGI, etc. (and
eventually openstack-common) for Python, we could prefer using the
modules in Scale Stack for lower level projects.

Thoughts?
-Eric

_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to     : openstack@xxxxxxxxxxxxxxxxxxx
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp