openstack team mailing list archive

["Juan J." Martínez <juan@xxxxxxxxxx>]

On Thu, 2012-04-12 at 20:51 -0500, John Dickinson wrote:
> Swift keeps total bytes, container, and object count (eventually) up-to-date in the account metadata. There are also log processing tools (like slogging - http://github.com/notmyname/slogging) that can provide usage information (including bandwidth) based on swift logs.
> 
> While I think that it's appropriate for swift to generate the usage information (via internal processes or log processing), the appropriate place for quotas is in whatever system handles the concept of a user (normally the auth system). This way quotas are enforced by revoking or limiting access of the auth token.

Hello,

If it helps, that's the way we're accounting and managing quotas in our
swift deployment (Memstore).

We use a background process in each proxy server that runs through the
logs to account source IP, bytes in, bytes out and requests per account,
and another background process to account bytes, container and object
count using swift API.

We have a small middleware in the proxy that limits HTTP operations.

For example, if the disk is over quota we forbid PUT requests for that
account (GET, DELETE are still valid).

We have found quotas are little bit difficult to enforce though. Being
accurate has an important performance hit, so our quotas are
"eventually" enforced (blame memcache for that), and by now is close
enough for us.

Regards,

Juan

-- 
Juan J. Martinez
Development, MEMSET

mail: juan@xxxxxxxxxx
 web: http://www.memset.com/

Memset Ltd., registration number 4504980. 25 Frederick Sanger Road, Guildford, Surrey, GU2 7YD, UK.