openstack team mailing list archive

[Leander Bessa Beernaert <leanderbb@xxxxxxxxx>]

Allow me to rephrase. I've read somewhere (can't remember where) that it
would be faster to upload files if they would be uploaded to separate
containeres. This was suggested for a standard swift installation with a
certain replication factor. Since I'll be uploading the files with the
replicators turned off, does it really matter if I insert a group of them
in separate containeres?


On Mon, Jan 14, 2013 at 2:27 PM, Robert van Leeuwen <
Robert.vanLeeuwen@xxxxxxxxxxxxx> wrote:

>  > I see. With replication switched off during upload, does inserting
> into various containers speed up the process
> > or is it irrelevant?
>
> I'm not sure what's your question but maybe this helps:
> In short:
> The replication daemon is "walking" across your files to check if any
> files need to be replicated to other nodes
> (e.g. when a node was broken or a new node is added to the cluster).
> Because it is scanning your filesystem it eats up io's. That is why it
> speeds up the system if you turn this off.
>
> The number off objects in a container impacts the sqlite databases. Each
> container is a SQLite database.
> The SQLite database will get slower with more objects in them.
> I think the recommendations for spinning disks is max a few million for
> one container, when using SSD's for the databases you can go quite a bit
> higher though.
>
> Cheers,
> Robert
>
> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to     : openstack@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
>
>