openstack team mailing list archive

[Eric Day <eday@xxxxxxxxxxxx>]

I checked out the CDMI spec, and while it covers the basics, it doesn't
seem to include features for visibility, concurrency control (multiple
workers grabbing messages via atomic operations), or long-polling. We
can still support it by having a CMDI protocol module if the protocol
gains more traction, but at this stage I wouldn't want to restrict
ourselves to just this protocol (and the features it describes)
this early on in a project.

Do you know what queue services use the CDMI spec?

As for Erlang, I'm currently looking into WebMachine and a couple
other HTTP/REST frameworks for relative performance comparison. The
Erlang core has quite a bit of HTTP protocol parsing built into it to
(written in C, passed in via socket reads), so for something simple
and lightweight we can use that. I'll be writing up more on this
later as I investigate.

-Eric

On Fri, Feb 18, 2011 at 09:23:28AM -0700, ksankar@xxxxxxxxxxxxxx wrote:
>      * On the topic of a REST API for the queue, I think implementing the
>        CDMI queue APIs might be a good choice.
>      * The spec is at
>        http://www.snia.org/tech_activities/publicreview/CDMI_Spec_v1.01f_DRAFT.pdf.
>        There are a few other things, but focus on Section 11 P.104, Queue
>        Object Resources.
>      * BTW, <cdmi-queue> is also an approved media-type
>        [http://www.iana.org/assignments/media-types/application/index.html]
>      * Would be happy to implement the CDMI REST interface, if all think this
>        is a good idea
>      * And on the topic of Erlang, it is viable. I have done some work with
>        it. If we also use WebMachine, REST interface would be easier.
>    Cheers
>    <k/>
> 
>      -------- Original Message --------
>      Subject: Re: [Openstack] Queue Service, next steps
>      From: Daniel Salinas <imsplitbit@xxxxxxxxx>
>      Date: Fri, February 18, 2011 8:20 am
>      To: "openstack@xxxxxxxxxxxxxxxxxxx" <openstack@xxxxxxxxxxxxxxxxxxx>
> 
>      I would totally be interested in an Erlang bootstrap. In SAT. Lets see
>      if
>      we can get a group together.
> 
>      On 2/18/11 4:03 AM, "Curtis Carter" <curtis.carter@xxxxxxxxxxxxx>;
>      wrote:
> 
>      >I'm in if it's done in Erlang.
>      >I am willing to give talks or help anyone wanting to learn Erlang in
>      the
>      >Austin/San Antonio area as well.
>      >
>      >On Feb 18, 2011, at 12:11 AM, Eric Day wrote:
>      >
>      >> Duck farm? :)
>      >>
>      >> Are you two concerned about building a developer community around a
>      >> project in Erlang? I'm all for going that route if other folks are
>      >> comfortable with it.
>      >>
>      >> I also have some concern around the speed of Erlang. It's great,
>      >> especially if you know what primitives can be expensive and how to
>      >> tame beam, but it will always be slower (sometimes significantly)
>      >> that C/C++ for some tasks.
>      >>
>      >> -Eric
>      >>
>      >> On Thu, Feb 17, 2011 at 03:26:09PM -0800, Joshua McKenty wrote:
>      >>> +1 for Erlang, as long as we have a duck farm.
>      >>>
>      >>> On Thu, Feb 17, 2011 at 2:25 PM, Devin Carlen
>      >>><devin.carlen@xxxxxxxxx>;
>      >>> wrote:
>      >>>
>      >>> I'll put in a +1 for Erlang as an OpenStack supported platform.
>      >>>We'd be
>      >>> able to write a stable queue with a much more concise code base,
>      >>>and
>      >>> this is project would be a great fit for Erlang.
>      >>> Devin
>      >>> On Feb 17, 2011, at 2:21 PM, Eric Day wrote:
>      >>>
>      >>>> Thanks to everyone who gave great feedback on the first queue
>      service
>      >>>> thread. I've updated the wiki page to include the suggestions.
>      >>>>
>      >>>> http://wiki.openstack.org/QueueService
>      >>>>
>      >>>> With a decent vision of what we want to build, the next step is
>      >>>> figuring out how. In a previous thread it was suggested that the
>      >>>> preferred languages for OpenStack projects are Python, C, and
>      >>>> C++. Since there is an emphasis on speed and efficiency for the
>      >>>> queue service, I suggest we use C++. I expect this service to be
>      >>>> CPU bound and would benefit being able to leverage multiple cores
>      >>>> efficiently (within the same process), so I don't think Python
>      >>>> is a good fit. I think C++ is a better fit than C due to the need
>      >>>> for modular interfaces. While this can obviously be done in C, C++
>      >>>> APIs are more concise and much less error prone. The OO style will
>      >>>> also make it easier for Python developers who also want to learn
>      and
>      >>>> assist with C++ projects.
>      >>>>
>      >>>> Erlang is not on the preferred lists, but I would also put it out
>      >>>> there as an option. While it may be a great fit for a project like
>      >>>> this, I worry it won't attract the developer resources since Erlang
>      >>>> isn't really a first-class language yet.
>      >>>>
>      >>>> If we decide to take the C++ path, I propose using a modular
>      >>>> application framework I've been working on over the past year
>      (mostly
>      >>>> in my spare time). It provides a simple module programming
>      interface
>      >>>> with dependency tracking (kind of like Linux kernel modules). It
>      >>>> already provides a multi-threaded event module (currently based on
>      >>>> libevent, but this is pluggable) with simple networking
>      abstractions
>      >>>> built on top of it. We should be able to dive in an start writing
>      the
>      >>>> HTTP protocol module and queue processing modules. You can check
>      out
>      >>>> the current project at:
>      >>>>
>      >>>> https://launchpad.net/scalestack
>      >>>> http://scalestack.org/
>      >>>>
>      >>>> The intention of using a framework like this is so we can easily
>      reuse
>      >>>> the other modules (auth, HTTP, logging, ...) for other OpenStack
>      >>>> services in the future. Much like we use Eventlet, WSGI, etc. (and
>      >>>> eventually openstack-common) for Python, we could prefer using the
>      >>>> modules in Scale Stack for lower level projects.
>      >>>>
>      >>>> Thoughts?
>      >>>> -Eric
>      >>>>
>      >>>> _______________________________________________
>      >>>> Mailing list: https://launchpad.net/~openstack
>      >>>> Post to : openstack@xxxxxxxxxxxxxxxxxxx
>      >>>> Unsubscribe : https://launchpad.net/~openstack
>      >>>> More help : https://help.launchpad.net/ListHelp
>      >>>
>      >>> _______________________________________________
>      >>> Mailing list: https://launchpad.net/~openstack
>      >>> Post to : openstack@xxxxxxxxxxxxxxxxxxx
>      >>> Unsubscribe : https://launchpad.net/~openstack
>      >>> More help : https://help.launchpad.net/ListHelp
>      >>
>      >> _______________________________________________
>      >> Mailing list: https://launchpad.net/~openstack
>      >> Post to : openstack@xxxxxxxxxxxxxxxxxxx
>      >> Unsubscribe : https://launchpad.net/~openstack
>      >> More help : https://help.launchpad.net/ListHelp
>      >
>      >
>      >
>      >Confidentiality Notice: This e-mail message (including any attached or
>      >embedded documents) is intended for the exclusive and confidential use
>      of
>      >the
>      >individual or entity to which this message is addressed, and unless
>      >otherwise
>      >expressly indicated, is confidential and privileged information of
>      >Rackspace.
>      >Any dissemination, distribution or copying of the enclosed material is
>      >prohibited.
>      >If you receive this transmission in error, please notify us immediately
>      >by e-mail
>      >at abuse@xxxxxxxxxxxxx, and delete the original message.
>      >Your cooperation is appreciated.
>      >
>      >
>      >_______________________________________________
>      >Mailing list: https://launchpad.net/~openstack
>      >Post to : openstack@xxxxxxxxxxxxxxxxxxx
>      >Unsubscribe : https://launchpad.net/~openstack
>      >More help : https://help.launchpad.net/ListHelp
> 
>      _______________________________________________
>      Mailing list: https://launchpad.net/~openstack
>      Post to : openstack@xxxxxxxxxxxxxxxxxxx
>      Unsubscribe : https://launchpad.net/~openstack
>      More help : https://help.launchpad.net/ListHelp

> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to     : openstack@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp