graphite-dev team mailing list archive

[Gerrick W <question181335@xxxxxxxxxxxxxxxxxxxxx>]

Question #181335 on Graphite changed:
https://answers.launchpad.net/graphite/+question/181335

Description changed to:
Hi,

I thought I understood what aggregationMethod=sum storage-aggregation
setting did when reading readthedocs.com, although I am getting
different results than what I expect.  So I am curious if I am doing
something wrong, or if I need to prepare my data in a different way.

Currently I am recording transactions per second every second (no
gaps/no more than once a second) for datapoints being sent to carbon
(Not using carbon-aggregator.py or carbon-relay.py).  My storage-
aggregationMethod is set to Sum and xFilesFactor=0. My storage-schema
retention is 1s:7d,1m:30d,1h,360d.  I confirmed this is properly set by
doing an info on the wsp files.

As I can backfill data, I wiped all database files and preloaded 4 months of data. When I attempt to graph tps and view 4 months of data, I see...
* First 7 days - Huge spike upwards  
   * Example: 1000 Average
* -7 -> -30 days - Exponential spike downwards from the first 7 days.  
   * Example: 1000/60seconds = 16.7 Average
* -30 -> -4 months - Exponential spike downwards from the -30 day. 
   * Example: 16.7/60minutes = .28

A few notes:
* The graph shows the largest retention rate (every 1 hour in this case), although isn't showing the same consistent rate when two/three retentions are shown.  Saw this when adding &format=csv at the end.
* When doing a whisper-fetch on the .wsp file for (now -> -7 days) -  I see every 1 second set to 1000
* When doing a whisper-fetch on the same .wsp file (-7 days -> -30 days) - I see every 60 seconds is 1000
  * Thought this would be close to 1000x60seconds as it is set to sum
* When doing a whisper-fetch on the same .wsp file (-30 days -> -100 days) - I see every 3600 seconds is 1000
* Using the latest 9.9 graphite from src from this morning

Already mentioned these, but thought I'd summarize:
* When adding format=csv I can see all datapoints are the largest retention slice, as being 1 hour in this case.
* Double checked with whisper-info.py that aggregationMethod=sum, correct retentions, and xFileFactor=0 are set
* I preloaded all data, instead of letting the data flow from (now -> -7 day)  -> (-7 -> -30 day) retentions. 

For expected results of aggregationMethod=sum, I thought I would of saw
a steady graph between the retention rates, instead of the retention
rates affecting what the transaction per seconds look like.  Then if I
were for example do a summarize(tps, "1day") I was hoping I could see
the daily number of transaction over 6 months with the retention rates
summing the values so no data is lost.  Am I formating my graphs
incorrect, need to send rolled up data for different retention times, or
is there something I am forgetting?

Thanks for any help you can provide!

-- 
You received this question notification because you are a member of
graphite-dev, which is an answer contact for Graphite.