graphite-dev team mailing list archive

[Esteban Dimitroff <question239496@xxxxxxxxxxxxxxxxxxxxx>]

New question #239496 on Graphite:
https://answers.launchpad.net/graphite/+question/239496

Hello,

I have a retentions line like this:

10s:180d;15m:2y

So I'm storing 10 seconds data for six months, and 15 minutes data for 2 years. 

The idea behind that amount of data points in the highest resolution retention is that we need to keep historical data with maximum precision available to be able of fine-grain comparing and because we are storing data that may not be looked at in a long time.

We are using the render API with raw data retrieval and some client side graphing tools: A customized version of timeserieswidget (https://github.com/vimeo/timeserieswidget) with Flot charts that allows us to zoom in and out.

So, when showing short time ranges we are fine with highest resolution retention, whether we look last hour or a single hour 3 months ago we want the highest resolution available.

But when comes to look at long range data (ie. a couple of months) we are having some performance issues. To avoid transferring high amount of data, I'm using summarize() function. But as far as I know it aggregates the information from the highest resolution retention, even if it is summarizing in buckets larger than the next retention ones. 

As a result, for some data intensive graphs (using around 10 variables and performing some calculations) , and long time ranges, the server is reading really big chunks of data and aggregating it, even when it is already aggregated in whisper database (in next retention), having some high response times in the browser.

So, in summary, is there a way to force whisper to use a specific retention when using the render API per request? Another solution would be to improve the retention selection algorithm to use the 'interval' parameter in summarize() function to choose the appropriate retention.

Thanks in advance, and sorry for the long description, I wanted to be as clear as possible.

Regards,

Esteban

-- 
You received this question notification because you are a member of
graphite-dev, which is an answer contact for Graphite.