openstack team mailing list archive

[Ywang225 <ywang225@xxxxxxx>]

Yes, the step to update object list in container should be the source, as it will wait for http connection timeout. From the error log, all connection to port 6001are failed, so I'd presume the container servers aren't running.

One straight testing is to comment out the call to container_update() in swift/obj/server.py:Put(),to see if better performance.

At disk drive level, you could check if NCQ is enabled by "cat /sys/block/sdb/device/queue_depth", if the output is a number bigger than 1, it's enabled, and normally, it should help at high concurrency.


--ywang

发自我的 iPhone

在 2011-12-20，12:41，Michael Barton <mike-launchpad@xxxxxxxxxxxxxxxx> 写道：

> On Mon, Dec 19, 2011 at 6:21 AM, Rustam Aliyev <rustam@xxxxxxx> wrote:
>> The only thing which looks suspicious to me are these errors:
>> 
>> Dec 18 04:01:28 ec01 object-server ERROR container update failed with
>> 10.0.1.3:6001/d01 (saving for async update later): Timeout (3s) (txn:
>> txdf95ad5a10844ee0b74d70d8a7638082)
>> Dec 18 04:01:28 ec01 object-server ERROR container update failed with
>> 10.0.1.2:6001/d01 (saving for async update later): Timeout (3s) (txn:
>> txee2545ba4610430fa3a6a166ca50c574)
>> Dec 18 04:01:28 ec01 object-server ERROR container update failed with
>> 10.0.1.8:6001/d01 (saving for async update later): Timeout (3s) (txn:
>> tx2546b29b15c643ec90a122a753dfddd3)
> 
> 
> Yeah, that is likely to be the culprit.  Each write is taking at least
> 3 seconds because it's timing out trying to update the container
> servers.
> 
> So you need to debug connectivity from this object server to those IP
> addresses on port 6001 -- that the IP addresses and port are correct,
> everything's on the same network, there aren't any firewall rules
> blocking those connections, that the container servers are running and
> accepting connections, etc.  I'll read through your paste in a bit and
> see if I notice anything.
> 
> -- Mike
> 
> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to     : openstack@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp