openstack team mailing list archive

[Pete Zaitcev <zaitcev@xxxxxxxxxx>]

On Mon, 8 Jul 2013 13:50:38 +0000
Klaus Schürmann <klaus.schuermann@xxxxxxxxxxxxx> wrote:

> I use a swift storage as a mail-store. Now I have about  1.000.000 objects stored in the cluster.

> Traffic Storagenode: http://www.schuermann.net/temp/storagenode.png
> Traffic Proxyserver: http://www.schuermann.net/temp/proxyserver.png
> 
> Can someone explain such behavior?

At a guess, raising number of objects makes number of partitions to
increase, and that increases the replication traffic, specifically
the number of sent MD5s for partitions.

It would be interesting to corellate the number of objects and
number of non-empty and empty partitions with the amounts of traffic.
If the increasing transfer of hashes is the reason, you could also
verify by graphing the traffic to port 873 separately. Swift never
replicates object bodies through its own HTTP interface, so this
splits control traffic from data traffic for you. Data traffic
should be driven by customer and node failures, not consistency
checking.

Be prepared to split up storage nodes, however. Even if there's no
bug with replication, its aggregate traffic increases with the
increases in object counts.

-- Pete