← Back to team overview

openstack team mailing list archive

Re: [OpenStack][Swift] Fast way of uploading 200GB of 200KB files to Swift


Ok, thanks for all the tips/help.



On Mon, Jan 14, 2013 at 3:21 PM, Robert van Leeuwen <
Robert.vanLeeuwen@xxxxxxxxxxxxx> wrote:

>  > Allow me to rephrase.
> > I've read somewhere (can't remember where) that it would be faster to
> upload files if they would be uploaded to separate containeres.
> > This was suggested for a standard swift installation with a certain
> replication factor.
> > Since I'll be uploading the files with the replicators turned off, does
> it really matter if I insert a group of them in separate containeres?
> My guess is this concerns the SQLite database load distribution.
> So yes, it still matters.
> Just to be clear: turning replicators off does not matter at ALL when
> putting files in a healthy cluster.
> Files will be "replicated" / put on all required nodes at the moment the
> put request is done.
> The put request will only give an OK when there is quorum writing the file
> (the file is stored on more than half of the required object nodes)
> The replicator daemons do not have anything to do with this.
> Cheers,
> Robert
> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to     : openstack@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp

Follow ups