graphite-dev team mailing list archive

[Sumeet Grover <question242463@xxxxxxxxxxxxxxxxxxxxx>]

Question #242463 on Graphite changed:
https://answers.launchpad.net/graphite/+question/242463

Sumeet Grover posted a new comment:
Hi All,

Thanks for your comments, and especially the pointer to MTU. I've
checked the MTU for our ethernet sockets on the Graphite server is 1500
bytes, and the time the metrics got corrupted, there were more than
2,000,000 committed points in Carbon every second. However, our rmem_max
kernel parameter is around 220KiB, effectively around 110KiB.

We are running Carbon 0.9.10. I ran two simulation tests to find out if
it is Carbon, which corrupts the metrics. The answer was NO. Since then
we have reverted back to the theory that the problem is highly likely to
be with the processes that feed into Graphite. Below are my test
results, which proved that given a high volume of messages or messages
with invalid values, Carbon actually does not distort them:

SIMULATION TEST 1
Duration: 1h30m
Metrics Received by Carbon: Over 2,000,000
Committed Points: Over 150,000
Cache Queues: Over 45,000
RAM Usage: Over 200MiB

Test metrics were sent with valid values, i.e. INT/DECIMAL values.

SIMULATION TEST 2
Duration: 1h30m
Metrics Received by Carbon: Over 300,000
Committed Points: Over 100,000
Cache Queues: Over 45,000
RAM Usage: Over 200MiB

Test metrics were sent with random values which were invalid, i.e. alpha
numeric. Carbon ignored these values, proving our initial theory wrong -
i.e. an invalid values leads to distortion of metrics in Carbon.

-- 
You received this question notification because you are a member of
graphite-dev, which is an answer contact for Graphite.