← Back to team overview

graphite-dev team mailing list archive

[Question #189746]: Timeout errors

 

New question #189746 on Graphite:
https://answers.launchpad.net/graphite/+question/189746

Recently my cache size was hitting the limit but by doing some tweaking and upping the MAX_UPDATES_PER_SECOND slightly the cache size has dropped significantly and stabilized.  However, now I am getting timeout errors: 

Mon Mar 05 10:54:25 2012 :: CarbonLink to ('127.0.0.1', 7002), exception while getting response
Traceback (most recent call last):
  File "/usr/local/rnt/webapp/graphite/render/datalib.py", line 148, in receiveResponse
    packet = connection.recv(remaining)
timeout: timed out

Load average is not terrible... but it does seem higher than it was before.
uptime
 10:56:17 up 137 days, 23:20,  1 user,  load average: 1.12, 1.58, 1.84

Memory usage:
free -m 
             total       used       free     shared    buffers     cached
Mem:         32168      30567       1600          0        761      26114
-/+ buffers/cache:       3691      28477
Swap:         2047          0       2047


iostat  -d /dev/sda6 -x 5 5
Device:         rrqm/s   wrqm/s     r/s     w/s   rsec/s   wsec/s avgrq-sz avgqu-sz   await  svctm  %util
sda6              0.24   116.51   27.12  365.89   399.03  3859.17    10.83     0.22    0.56   0.78  30.64

Device:         rrqm/s   wrqm/s     r/s     w/s   rsec/s   wsec/s avgrq-sz avgqu-sz   await  svctm  %util
sda6              0.00   138.00    9.60  655.40   112.00  6347.20     9.71    48.56   71.88   0.75  49.58

Device:         rrqm/s   wrqm/s     r/s     w/s   rsec/s   wsec/s avgrq-sz avgqu-sz   await  svctm  %util
sda6              0.00   109.20   17.20  529.00   198.40  5105.60     9.71    38.69   72.24   0.95  51.64

Device:         rrqm/s   wrqm/s     r/s     w/s   rsec/s   wsec/s avgrq-sz avgqu-sz   await  svctm  %util
sda6              0.00   127.35   10.58  581.84   135.73  5673.45     9.81    38.20   64.49   0.82  48.50

Device:         rrqm/s   wrqm/s     r/s     w/s   rsec/s   wsec/s avgrq-sz avgqu-sz   await  svctm  %util
sda6              0.00   165.40   16.40  424.40   169.60  4718.40    11.09    30.65   69.52   0.92  40.46


Cpu Usage ( according to carbon graphs ) is fluctuating between 40 and 45) 

What can I do to address these timeouts?  

-- 
You received this question notification because you are a member of
graphite-dev, which is an answer contact for Graphite.