openstack team mailing list archive

[Alex Yang <alex890714@xxxxxxxxx>]

In my view, the biggest problem of swift is not the new features but the
improvement of performance.

At first, we knew that container-updater, *-auditor, *-replicator will loop
all the files in the disk during the running interval. When the number of
files is very large, the process of replicator, updater and auditor will
spend much time. So, the time of eventual consistency is very long.

Our practice of  Sina Web Service Team https://launchpad.net/~sws:

total accounts:          121,961;

total containers:        160,703;

total objects:             14,291,519;

total storage usage:  1.3T

account replication time:    10 hours;

container replication time:  10 hours;

object replication time:       48 hours;

account audit time:               2 hours;

container audit time:             9 hours;

container update time:        19 hours;

This is terrible If we develop quota upon account db. There is long time
for eventual consistency.

Secondly,  there is a vicious circle during replication. The replicator
will query the account-server, container-server and object-server to
compare the metadata, and determine whether to sync. When the number of
files is very large, the frequent query make the account-server,
container-server and object-server become a bottle neck. This will
influence the process of proxy-server to work with back-end servers. There
a lot of Timeout(10s) ERROR in proxy-server's log and the load-average is
very high.
So, some PUT, POST operations failed, and the replicator to sync, and fail
more, and sync more...
.
In my opinion, we need to improve the process of replication and container
update by using event drive framework or something else..
My leader may discuss this topic at Design Summit,
http://openstackconferencespring2012.sched.org/speaker/huicheng

2012/4/13 John Dickinson <me@xxxxxx <javascript:_e({}, 'cvml',
'me@xxxxxx');>>

> Swift keeps total bytes, container, and object count (eventually)
> up-to-date in the account metadata. There are also log processing tools
> (like slogging - http://github.com/notmyname/slogging) that can provide
> usage information (including bandwidth) based on swift logs.
>
> While I think that it's appropriate for swift to generate the usage
> information (via internal processes or log processing), the appropriate
> place for quotas is in whatever system handles the concept of a user
> (normally the auth system). This way quotas are enforced by revoking or
> limiting access of the auth token.
>
> --John
>
>
> On Apr 12, 2012, at 11:53 AM, Frederik Van Hecke wrote:
>
> > Hi Kuo,
> >
> > One option would be to keep the usage information (num files, num bytes,
> etc) per container / account in an sqlite DB, just like it is done for
> account and container info.
> >
> > To avoid having to loop through all data at regular intervals (to update
> the info), additional logic could be added to the api methods to update the
> sqlite DB's when new files are added, files are deleted, etc. Such approach
> will require more lines of code, but will be far less stressful on
> performance.
> >
> > (the brute-force approach to loop through it at regular intervals will
> be hell on performance on large deployments..)
> >
> >
> > For data transfer billing based on download / upload amounts, a similar
> approach could be used.
> >
> > If no one else is looking into this, I would certainly be willing to
> help to help get this started.
> >
> >
> > Kind regards,
> > Frederik Van Hecke
> >
> > T:  +32487733713
> > E:  frederik@xxxxxxxxx <javascript:_e({}, 'cvml',
> 'frederik@xxxxxxxxx');>
> > W: www.cluttr.be
> >
> >
> >
> >
> >
> > This e-mail and any attachments thereto may contain information which is
> confidential and/or protected by intellectual property rights and are
> intended for the sole use of the recipient(s)named above. Any use of the
> information contained herein (including, but not limited to, total or
> partial reproduction, communication or distribution in any form) by persons
> other than the designated recipient(s) is prohibited. If you have received
> this e-mail in error, please notify the sender either by telephone or by
> e-mail and delete the material from any computer. Thank you for your
> cooperation.
> >
> >
> >
> > On Thu, Apr 12, 2012 at 17:45, Kuo Hugo <tonytkdk@xxxxxxxxx<javascript:_e({}, 'cvml', 'tonytkdk@xxxxxxxxx');>>
> wrote:
> > Hi folks ,
> >
> > I'm thinking about the better approach to manage "an user" or "an
> account" space usage quota in swift.
> > Is  there any related blueprint or sub-project even an idea around ?
> > Any suggestion of benefits to be an external service or to be a
> middle-ware in swift-proxy ?
> >
> > I'm concerning about such feature will reduce the performance of entire
> Swift environment.
> >
> > Appreciate :>
> >
> >
> >
> > --
> > +Hugo Kuo+
> > tonytkdk@xxxxxxxxx <javascript:_e({}, 'cvml', 'tonytkdk@xxxxxxxxx');>
> > +886 935004793
> >
> >
> > _______________________________________________
> > Mailing list: https://launchpad.net/~openstack
> > Post to     : openstack@xxxxxxxxxxxxxxxxxxx <javascript:_e({}, 'cvml',
> 'openstack@xxxxxxxxxxxxxxxxxxx');>
> > Unsubscribe : https://launchpad.net/~openstack
> > More help   : https://help.launchpad.net/ListHelp
> >
> >
> > _______________________________________________
> > Mailing list: https://launchpad.net/~openstack
> > Post to     : openstack@xxxxxxxxxxxxxxxxxxx <javascript:_e({}, 'cvml',
> 'openstack@xxxxxxxxxxxxxxxxxxx');>
> > Unsubscribe : https://launchpad.net/~openstack
> > More help   : https://help.launchpad.net/ListHelp
>
>
> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to     : openstack@xxxxxxxxxxxxxxxxxxx <javascript:_e({}, 'cvml',
> 'openstack@xxxxxxxxxxxxxxxxxxx');>
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
>
>


-- 
  Email:       alex890714@xxxxxxxxx
GitHub:       https://github.com/AlexYangYu
   Blog:        http://alexyang.sinaapp.com