graphite-dev team mailing list archive

[Jens Rantil <question193502@xxxxxxxxxxxxxxxxxxxxx>]

New question #193502 on Graphite:
https://answers.launchpad.net/graphite/+question/193502

Hi,

I apologize for a quite long question. I tried to keep it short and concise.

Version of graphite: Latest from pip=0.9.9

I am trying to import a timeseries containing the value "1" when a certain event has happened. My idea is to visualize how many times this event happened for example per hour, day or week. Obviously doing this requires me to use an xFilesFactor=0 (since a lot of times will not have any values) and aggregationMethod=sum (since I want to sum the events when their aggregated).

However, my import does not seem to be giving me reasonable values; I am getting values of 1:s far back in time despite the fact that I know they should have happened more. My theory is that it has to do with the fact that aggregationMethod=sum is not fully kicking in. I am well aware of the fact that graphite-web averages things in its graphs, but since I've been trying out the summarize(..., ...) function, I don't think this is my issue.

Since I wasn't sure the expected behaviour when setting the same value over and over again (https://answers.launchpad.net/graphite/+question/193500), I tried different ways of importing my historical data such as importing into a "1m:2y" Whisper databas and then resizing it to "5m:2h, 1h:7d, 1d:730d" to get the aggregation right. Still the values don't seem to get right.

To try to figure out whether I am on the right track I made two test scripts:

Script 1:
=======================================
#!/usr/bin/bash

rm -f test.wsp
whisper-create.py --xFilesFactor=0 --aggregationMethod=sum test.wsp 1s:3s 5s:20s
CREATED=$(date +%s)
echo "Created: $CREATED"
whisper-update.py test.wsp $((CREATED)):1
whisper-update.py test.wsp $((CREATED-1)):1
whisper-update.py test.wsp $((CREATED-2)):1
whisper-update.py test.wsp $((CREATED-3)):1
whisper-update.py test.wsp $((CREATED-4)):1

echo
echo Using 1s resolution:
whisper-fetch.py --from=$((CREATED-4)) test.wsp
echo

echo Using 5s resolution:
whisper-fetch.py --from=$((CREATED-30)) test.wsp
=======================================

Output from script 1:
=======================================
$ time bash test.sh 
Created: test.wsp (124 bytes)
Created: 1334305281

Using 1s resolution:
1334305280	2.000000

Using 5s resolution:
1334305265	None
1334305270	None
1334305275	1.000000
1334305280	2.000000

real	0m0.164s
user	0m0.130s
sys	0m0.050s
=======================================

My question is; How come only 3 (or sometimes 2) values are only registered here? Is this a bug? Sure, I can understand this script might running across two seconds, but that wouldn't yield this result, right?

I also made a second test script 2:
=======================================
#!/usr/bin/bash

rm -f test.wsp
whisper-create.py --xFilesFactor=0 --aggregationMethod=sum test.wsp 1s:20s
CREATED=$(date +%s)
echo "Created: $CREATED"
whisper-update.py test.wsp $((CREATED)):1
whisper-update.py test.wsp $((CREATED-1)):1
whisper-update.py test.wsp $((CREATED-2)):1
whisper-update.py test.wsp $((CREATED-3)):1
whisper-update.py test.wsp $((CREATED-4)):1
whisper-update.py test.wsp $((CREATED-5)):1
whisper-update.py test.wsp $((CREATED-6)):1
whisper-update.py test.wsp $((CREATED-7)):1
whisper-update.py test.wsp $((CREATED-8)):1
whisper-update.py test.wsp $((CREATED-9)):1

echo "Before resizing:"
whisper-fetch.py --from=$((CREATED-30)) test.wsp
echo

whisper-resize.py --xFilesFactor=0 --aggregationMethod=sum test.wsp 1s:5s 60s:120s

echo "After resizing:"
whisper-fetch.py --from=$((CREATED-30)) test.wsp
=======================================

The output of script 2:
=======================================
$ time bash test2.sh 
Created: test.wsp (268 bytes)
Created: 1334305511
Before resizing:
1334305492	None
1334305493	None
1334305494	None
1334305495	None
1334305496	None
1334305497	None
1334305498	None
1334305499	None
1334305500	None
1334305501	None
1334305502	1.000000
1334305503	1.000000
1334305504	1.000000
1334305505	1.000000
1334305506	1.000000
1334305507	1.000000
1334305508	1.000000
1334305509	1.000000
1334305510	1.000000
1334305511	1.000000

Retrieving all data from the archives
Creating new whisper database: test.wsp.tmp
Created: test.wsp.tmp (124 bytes)
Migrating data...
Renaming old database to: test.wsp.bak
Renaming new database to: test.wsp
After resizing:
1334305500	1.000000

real	0m0.280s
user	0m0.200s
sys	0m0.090s
=======================================

My question is; Why doesn't "After resizing" value have a value of "10"? Am I getting something wrong here?

Thanks,
Jens

-- 
You received this question notification because you are a member of
graphite-dev, which is an answer contact for Graphite.