graphite-dev team mailing list archive

Thread
Date

[Question #233013]: Graph only recording 2 minutes of data

To: graphite-dev@xxxxxxxxxxxxxxxxxxx
From: Paul O'Connor <question233013@xxxxxxxxxxxxxxxxxxxxx>
Date: Thu, 25 Jul 2013 15:31:20 -0000
Reply-to: question233013@xxxxxxxxxxxxxxxxxxxxx
Sender: bounces@xxxxxxxxxxxxx

New question #233013 on Graphite:
https://answers.launchpad.net/graphite/+question/233013

I have a single graphite node, running ceres and mega-carbon from Git. I'm processing about 100k metrics a minute and it's mostly working properly. However, there are a number of (seemingly) random metrics that from time to time will store 2 minutes of data and will start zeroing anything older than two minutes.

Using both ceres-read and slicecat produces the same output:
Ceres-read
 
Wed Jul 17 10:47:00 2013  0.0
Wed Jul 17 10:47:10 2013	0.0
Wed Jul 17 10:47:20 2013	0.0
Wed Jul 17 10:47:30 2013	0.0
Wed Jul 17 10:47:40 2013	0.0
Wed Jul 17 10:47:50 2013	0.0
Wed Jul 17 10:48:00 2013	23.4
Wed Jul 17 10:48:10 2013	19.9
Wed Jul 17 10:48:20 2013	26.4
Wed Jul 17 10:48:30 2013	21.7
Wed Jul 17 10:48:40 2013	22.6
Wed Jul 17 10:48:50 2013	26.7
Wed Jul 17 10:49:00 2013	24.1
Wed Jul 17 10:49:10 2013	23.4
Wed Jul 17 10:49:20 2013	18.0
Wed Jul 17 10:49:30 2013	29.3
Wed Jul 17 10:49:40 2013	29.6
Wed Jul 17 10:49:50 2013	21.6
 
 
Slicecat
 
[1374058020]  Wed Jul 17 10:47:00 2013	0.0
[1374058030]	Wed Jul 17 10:47:10 2013	0.0
[1374058040]	Wed Jul 17 10:47:20 2013	0.0
[1374058050]	Wed Jul 17 10:47:30 2013	0.0
[1374058060]	Wed Jul 17 10:47:40 2013	0.0
[1374058070]	Wed Jul 17 10:47:50 2013	0.0
[1374058080]	Wed Jul 17 10:48:00 2013	23.4
[1374058090]	Wed Jul 17 10:48:10 2013	19.9
[1374058100]	Wed Jul 17 10:48:20 2013	26.4
[1374058110]	Wed Jul 17 10:48:30 2013	21.7
[1374058120]	Wed Jul 17 10:48:40 2013	22.6
[1374058130]	Wed Jul 17 10:48:50 2013	26.7
[1374058140]	Wed Jul 17 10:49:00 2013	24.1
[1374058150]	Wed Jul 17 10:49:10 2013	23.4
[1374058160]	Wed Jul 17 10:49:20 2013	18.0
[1374058170]	Wed Jul 17 10:49:30 2013	29.3
[1374058180]	Wed Jul 17 10:49:40 2013	29.6
[1374058190]	Wed Jul 17 10:49:50 2013	21.6

Using ceres-write to store data at an old timestamp will store the data correctly and will not be over written. I'm not using any clustering, just carbon writer listening on port 2003 and 2004 which is forwarded from a load balancer (the load balancer was put in place when I started as the plan is to have multiple graphite nodes).

Any help with this would be greatly appreciated. 

-- 
You received this question notification because you are a member of
graphite-dev, which is an answer contact for Graphite.