openstack team mailing list archive

[John Dickinson <me@xxxxxx>]

If you're looking for wsgi rate limiting, take a look at swift's rate limiting middleware. It's distributed, simple (uses memcached to keep track of what's going on), supports multiple tiers of rate limiting (with interpolation between them), and white/black lists.

While swift's rate limiting is tuned for swift, its technique could be pretty simply adapted to other applications. Swift's rate limiting is swift-specific in what it checks to rate limit on. Those checks should be able to be adapted to other use cases.

source: https://github.com/openstack/swift/blob/master/swift/common/middleware/ratelimit.py
docs: https://github.com/openstack/swift/blob/master/doc/source/ratelimit.rst
sample config: https://github.com/openstack/swift/blob/master/etc/proxy-server.conf-sample#L118

--John

On Mar 16, 2012, at 4:45 PM, Kevin L. Mitchell wrote:

> Howdy, folks.  I've been working on a replacement for nova's
> rate-limiting middleware that will handle the multiple-node case, and
> I've developed a fairly generic rate-limiting package, along with a
> second package that adapts it to nova.  (This means you could also use
> this rate-limiting setup with, say, glance, or with any other project
> that uses Python middleware.)  Here is some information:
> 
> * Turnstile
>        Turnstile is a piece of WSGI middleware that performs true distributed
>        rate-limiting.  System administrators can run an API on multiple
>        nodes, then place this middleware in the pipeline prior to the
>        application.  Turnstile uses a Redis database to track the rate at
>        which users are hitting the API, and can then apply configured rate
>        limits, even if each request was made against a different API node.
> 
>        - https://github.com/klmitch/turnstile
>        - http://pypi.python.org/pypi/turnstile
> 
> * nova_limits
>        This package provides the ``nova_limits`` Python module, which
>        contains the ``nova_preprocess()`` preprocessor, the
>        ``NovaClassLimit`` limit class, and the ``NovaTurnstileMiddleware``
>        replacement middleware class, all for use with Turnstile.  These
>        pieces work together to provide class-based rate limiting integration
>        with nova.
> 
>        - https://github.com/klmitch/nova_limits
>        - http://pypi.python.org/pypi/nova_limits
> 
> Both packages should be fairly well documented (start with README.rst),
> and please feel free to log issues or make pull requests.
> -- 
> Kevin L. Mitchell <kevin.mitchell@xxxxxxxxxxxxx>
> 
> 
> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to     : openstack@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp

Attachment: smime.p7s
Description: S/MIME cryptographic signature