← Back to team overview

openstack team mailing list archive

Re: [OpenStack][Swift] Fast way of uploading 200GB of 200KB files to Swift

 

Well, I've fixed the node size and disabled the all the replicator and
auditor processes. However, it is even slower now than it was before :/.
Any suggestions?


On Mon, Jan 14, 2013 at 3:23 PM, Leander Bessa Beernaert <
leanderbb@xxxxxxxxx> wrote:

> Ok, thanks for all the tips/help.
>
> Regards,
>
> Leander
>
>
> On Mon, Jan 14, 2013 at 3:21 PM, Robert van Leeuwen <
> Robert.vanLeeuwen@xxxxxxxxxxxxx> wrote:
>
>>  > Allow me to rephrase.
>> > I've read somewhere (can't remember where) that it would be faster to
>> upload files if they would be uploaded to separate containeres.
>> > This was suggested for a standard swift installation with a certain
>> replication factor.
>> > Since I'll be uploading the files with the replicators turned off, does
>> it really matter if I insert a group of them in separate containeres?
>>
>> My guess is this concerns the SQLite database load distribution.
>> So yes, it still matters.
>>
>> Just to be clear: turning replicators off does not matter at ALL when
>> putting files in a healthy cluster.
>> Files will be "replicated" / put on all required nodes at the moment the
>> put request is done.
>> The put request will only give an OK when there is quorum writing the
>> file (the file is stored on more than half of the required object nodes)
>> The replicator daemons do not have anything to do with this.
>>
>> Cheers,
>> Robert
>>
>> _______________________________________________
>> Mailing list: https://launchpad.net/~openstack
>> Post to     : openstack@xxxxxxxxxxxxxxxxxxx
>> Unsubscribe : https://launchpad.net/~openstack
>> More help   : https://help.launchpad.net/ListHelp
>>
>>
>

Follow ups

References