← Back to team overview

openstack team mailing list archive

[OpenStack][Swift] Fast way of uploading 200GB of 200KB files to Swift


Hello all,

I'm trying to upload 200GB of 200KB files to Swift. I'm using 4 clients
(each hosted on a different machine) with 10 threads each uploading files
using the official python-swiftclient. Each thread is uploading to a
separate container.

I have 5 storage nodes and 1 proxy node. The nodes are all running with a
replication factor of 3. Each node has a quad-core i3 processor, 4GB of RAM
and a gigabit network interface.

Is there any way I can speed up this process? At the moment it takes about
20 seconds per file or more.



Follow ups