openstack team mailing list archive

Thread
Date

Re: [SWIFT] raising network traffic on the storage node

To: "openstack@xxxxxxxxxxxxxxxxxxx" <openstack@xxxxxxxxxxxxxxxxxxx>
From: Klaus Schürmann <klaus.schuermann@xxxxxxxxxxxxx>
Date: Tue, 9 Jul 2013 14:05:55 +0000
Accept-language: de-DE, en-US
Thread-index: Ac58rBEUuNV8LyArT9aKyUd497WSvA==
Thread-topic: [Openstack] [SWIFT] raising network traffic on the storage node

I measured the network traffic with darkstat:

server		In		Out		Total

storagenode1	699,118,562 	679,077,971 	1,378,196,533 	
storagenode2	168,636,360 	165,050,575 	333,686,935 	
storagenode3	166,583,442 	164,405,402 	330,988,844 	
storagenode4	164,282,250 	163,051,416 	327,333,666 	
storagenode5	164,000,162 	162,840,370 	326,840,532 	
proxynode1	7,339,629 	31,253,205 	38,592,834 	
proxynode2	8,236,128 	12,517,594 	20,753,722

This is a part of traffic to server storagenode3:

Port	In		Out		Total		Syns	
6000 	21,055,732 	347,350,916 	368,406,648 	47,388
6001 	19,717,608 	18,090,656 	37,808,264 	31,549
6002 	494,124 	316,830 	810,954 	883
36905 	39,660 		2,263 		41,923 		0
44687 	33,056 		1,944 		35,000 		0
47388 	31,691 		2,467 		34,158 		0
41999 	30,626 		1,788 		32,414 		0
34228 	26,552 		3,345 		29,897 		0

Is this correct configured?


-----Ursprüngliche Nachricht-----
Von: Openstack [mailto:openstack-bounces+klaus.schuermann=mediabeam.com@xxxxxxxxxxxxxxxxxxx] Im Auftrag von Robert van Leeuwen
Gesendet: Dienstag, 9. Juli 2013 09:09
An: openstack@xxxxxxxxxxxxxxxxxxx
Betreff: Re: [Openstack] [SWIFT] raising network traffic on the storage node

> If the replication traffic is responsible for this raising network traffic for only 1.200.000 objects, how much traffic I can 
> expect if I have 100.000.000 objects stored?
> The average size of my mailobjects are 120 kB. 
> It's planned to use all 12 hard drive slots of my DELL R720xd with 4 > TB drives
>. I have 5 storage nodes and 2 balanced proxy nodes. Will the replication traffic kill my system?

We are running with  > 400.000.000 objects  across 11 object storage nodes.
Total network traffic on any of those nodes is less then 10 MByte /second

However we have seen slowdowns with lots of small files and really big disks.
The issue is not related to the network but the local filesystem/disk.
When the inode cache gets insufficient you can see terrible slow-downs.
There have been a few threads about that in this list, having a lot of memory usually helps a bit.

Cheers,
Robert van Leeuwen
_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to     : openstack@xxxxxxxxxxxxxxxxxxx
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp