openstack team mailing list archive

[Sharif Islam <islamsh@xxxxxxxxxxx>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


When I run swift-bench with higher numbers (-n and -g 9999+) I sometime
see lots of failures with GET and DEL. I noticed these in the log:


> Oct 28 10:38:30 b001 object-replicator rsync: failed to connect to 172.29.202.13: No route to host (113)
> Oct 28 10:38:30 b001 object-replicator rsync error: error in socket IO (code 10) at clientserver.c(124) [sender=3.0.6]
> Oct 28 10:38:30 b001 object-replicator Bad rsync return code: ['rsync', '--recursive', '--whole-file', '--human-readable', '--xattrs', '--itemize-changes', '--ignore-existing', '--timeout=30', '--contimeout=30', '/srv/node/sda3/objects/104691/561', '/srv/node/sda3/objects/104691/274', '[172.29.202.13]::object/sda3/objects/104691'] -> 10
> Oct 28 10:38:30 b001 object-replicator rsync: failed to connect to 172.29.202.14: No route to host (113)
> Oct 28 10:38:30 b001 object-replicator rsync error: error in socket IO (code 10) at clientserver.c(124) [sender=3.0.6]
> Oct 28 10:38:30 b001 object-replicator Bad rsync return code: ['rsync', '--recursive', '--whole-file', '--human-readable', '--xattrs', '--itemize-changes', '--ignore-existing', '--timeout=30', '--contimeout=30', '/srv/node/sda3/objects/104691/561', '/srv/node/sda3/objects/104691/274', '[172.29.202.14]::object/sda3/objects/104691'] -> 10
> Oct 28 10:38:33 b001 object-replicator rsync: failed to connect to 172.29.202.13: No route to host (113)
> Oct 28 10:38:33 b001 object-replicator rsync error: error in socket IO (code 10) at clientserver.c(124) [sender=3.0.6]
> Oct 28 10:38:33 b001 object-replicator Bad rsync return code: ['rsync', '--recursive', '--whole-file', '--human-readable', '--xattrs', '--itemize-changes', '--ignore-existing', '--timeout=30', '--contimeout=30', '/srv/node/sda3/objects/245260/6b2', '[172.29.202.13]::object/sda3/objects/245260'] -> 10
> Oct 28 10:38:33 b001 object-replicator rsync: failed to connect to 172.29.202.13: No route to host (113)
> Oct 28 10:38:33 b001 object-replicator rsync error: error in socket IO (code 10) at clientserver.c(124) [sender=3.0.6]
> Oct 28 10:38:33 b001 object-replicator Bad rsync return code: ['rsync', '--recursive', '--whole-file', '--human-readable', '--xattrs', '--itemize-changes', '--ignore-existing', '--timeout=30', '--contimeout=30', '/srv/node/sda3/objects/113759/b74', '[172.29.202.13]::object/sda3/objects/113759'] -> 10
> Oct 28 10:38:33 b001 object-replicator rsync: failed to connect to 172.29.202.14: No route to host (113)
> Oct 28 10:38:33 b001 object-replicator rsync error: error in socket IO (code 10) at clientserver.c(124) [sender=3.0.6]
> Oct 28 10:38:33 b001 object-replicator Bad rsync return code: ['rsync', '--recursive', '--whole-file', '--human-readable', '--xattrs', '--itemize-changes', '--ignore-existing', '--timeout=30', '--contimeout=30', '/srv/node/sda3/objects/113759/b74', '[172.29.202.14]::object/sda3/objects/113759'] -> 10
> Oct 28 10:38:33 b001 object-replicator rsync: failed to connect to 172.29.202.13: No route to host (113)
> Oct 28 10:38:33 b001 object-replicator rsync error: error in socket IO (code 10) at clientserver.c(124) [sender=3.0.6]
> Oct 28 10:38:33 b001 object-replicator Bad rsync return code: ['rsync', '--recursive', '--whole-file', '--human-readable', '--xattrs', '--itemize-changes', '--ignore-existing', '--timeout=30', '--contimeout=30', '/srv/node/sda3/objects/30751/272', '[172.29.202.13]::object/sda3/objects/30751'] -> 10


I have some details here about my setup and numbers from swift-bench:

http://paste.openstack.org/show/2948/

I have one proxy node and 8 storage nodes, no SSL terminator/LB yet. I
ran the swift-bench from one the storages nodes.

The failures and errors are not consistent. For instance, yesterday I
noticed more failures, today it seems better. I would appreciate if
someone can suggest something to improve the performance. thanks.


- --sharif




- -- 
Sharif Islam
Senior Systems Analyst/Programmer
FutureGrid (http://futuregrid.org)
Pervasive Technology Institute, Indiana University Bloomington
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQEcBAEBAgAGBQJOqs5YAAoJEACffes9SivFskUH/126OjjMmKTkQHg4OpGJIfht
l5iOjsVZJCR3maIhENi2x3CiiY+EZ3yMKQWq+YReTWnui2bCN/laApt6VAT2385I
DrYztRVKj110/Wbc/rEY5mq3HWZwVZsqhqnCAMk2oW00Iaizlko3C5U+2JDLkBp3
mBbltk5j5mB8hf2Nen2Bv3onEvEgCUicI84FzIg5he1jg+Y5FpDfMNxyYBlVisyL
Ai/Lg/dBE8OYg8hWm9IdMckj/SQI4FQddwGD9HIeJlBgFG9ePgh7qf9Q8AfmXC3X
XqGbH3XVBgiDtWdm7ZBEyHalSVUoK2HYYmqzeq43efxozkChR74qLvMsusiy6lc=
=TC2I
-----END PGP SIGNATURE-----