graphite-dev team mailing list archive

[Ats Kung <question239301@xxxxxxxxxxxxxxxxxxxxx>]

New question #239301 on Graphite:
https://answers.launchpad.net/graphite/+question/239301

Logstash config to send to Graphite:

input {
...
...
}
output {
stdout { debug => true debug_format => "json"}

if [type] == "TEST-LOG" {
graphite {
  host => "10.50.150.111"
  port => 2003
  debug => true
  metrics => [ "Test.%{service_name}.%{host_name}.metric1", "%{metric1}", "Test.%{service_name}.%{host_name}.metric2", "%{metric2}", "Test.%{service_name}.%{host_name}.metric3", "%{metric3}", "Test.%{service_name}.%{host_name}.metric4", "%{metric4}" ]
  }
 }
}

Result output from stdout -- Logstash filter look fine as expected.

{"message":"2013-11-14 00:00:00,host-01,serviceA,132,2256.2,2.1,205.439394\r","@timestamp":"2013-11-14T00:00:00.000Z","@version":"1","type":"Test-LOG","host":"host-x","path":"/shipper/raw/Test.csv","logdate":"2013-11-14 00:00:00","host_name":"host-01","service_name":"serviceA","metric1":"132","metric2":"2256.2","metric3":"2.1","metric4":"205.439394\r"}
{"message":"2013-11-14 00:00:00,host-02,serviceA,164,8123,0,620.006098\r","@timestamp":"2013-11-14T00:00:00.000Z","@version":"1","type":"Test-LOG","host":"host-x","path":"/shipper/raw/Test.csv","logdate":"2013-11-14 00:00:00","host_name":"host-01","service_name":"serviceA","metric1":"164","metric2":"8123","metric3":"0","metric4":"620.006098\r"}
{"message":"2013-11-14 00:00:00,host-03,serviceA,4622,0,27.412987\r","@timestamp":"2013-11-14T00:00:00.000Z","@version":"1","type":"Test-LOG","host":"host-x","path":"/shipper/raw/Test.csv","logdate":"2013-11-14 00:00:00","host_name":"host-01","service_name":"serviceA","metric1":"2310","metric2":"4622","metric3":"0","metric4":"27.412987\r"}
{"message":"2013-11-14 00:00:00,host-04,serviceA,296,2914,0,93.162162\r","@timestamp":"2013-11-14T00:00:00.000Z","@version":"1","type":"Test-LOG","host":"host-x","path":"/shipper/raw/Test.csv","logdate":"2013-11-14 00:00:00","host_name":"host-01","service_name":"serviceA","metric1":"296","metric2":"2914","metric3":"0","metric4":"93.162162\r"}

Result metric show in Whisper in Graphite -- A lot of data missing as random matric missing
Test.serviceA.host-01.metric1
Test.serviceA.host-01.metric2
Test.serviceA.host-02.metric1
Test.serviceA.host-02.metric2
Test.serviceA.host-02.metric3

**Metric data in Whisper random missing if re-run Logstash with the same output from stdout 

Expected  result
Test.serviceA.host-01.metric1
Test.serviceA.host-01.metric2
Test.serviceA.host-01.metric3
Test.serviceA.host-01.metric4
Test.serviceA.host-02.metric1
Test.serviceA.host-02.metric2
Test.serviceA.host-02.metric3
Test.serviceA.host-02.metric4
Test.serviceA.host-03.metric1
Test.serviceA.host-03.metric2
Test.serviceA.host-03.metric3
Test.serviceA.host-03.metric4
Test.serviceA.host-04.metric1
Test.serviceA.host-04.metric2
Test.serviceA.host-04.metric3
Test.serviceA.host-04.metric4

How to check where metric data lost is? 
How to fix these?
Could anyone help to advise? 
Thanks.

-- 
You received this question notification because you are a member of
graphite-dev, which is an answer contact for Graphite.