graphite-dev team mailing list archive

[Jason Antman <question227319@xxxxxxxxxxxxxxxxxxxxx>]

Question #227319 on Graphite changed:
https://answers.launchpad.net/graphite/+question/227319

Jason Antman posted a new comment:
Are you seeing sufficient write performance on the NetApp? Granted we're
using older hardware (only able to push about 600 IOPS to our old 15k
SCSI disks) but our next cluster will likely be RAID10 of SSDs... the
testing that I did, plus a conversation with some guys on #graphite who
are running 2M+ metrics/minute, led me to the conclusion that only SSD
can keep up with the writes as fast as is needed for large amounts of
metrics.

The rendering side of things runs as a straightforward stateless HTTP
app, so about any VIP/pool configuration will work fine for that. I've
never tried it, but it should work fine with an empty CLUSTER_HOSTS on
the webapp, but all of the carbon-caches in CARBONLINK_HOSTS.

You can't have two active carbon-relays in round-robin. Well... off the
top of my head I guess you theoretically could, and it might or might
not work at all, but performance would go to hell once you start
queueing a few metrics in memory. If you're going to be using consistent
hashing, you should only have one active carbon-relay at a given level
at a time.

I don't see any reason for an iRule, straight out priority groups and a
working health check should be fine. I've (through an inadvertant QA
config setting change) thrown about 1000% of my cluster's metric
capacity at it, and I couldn't cause carbon-relay to hiccup, let alone
crash. I don't see thrashing very likely unless somehow you get both
relays to die and then somehow magically restart.

I know some shops are really big fans of NetApp or filers in general
(i've worked at a few of them, both NFS and iSCSI zealots) but IMHO if
you look at the size of the writes being done by carbon-cache to each
file, and the speed at which they happen, it seems like a LOT of
overhead. Carbon-cache seems to me to be designed with low-overhead
(i.e. local) storage in mind. If data integrity is the concern, carbon-
relay can be configured to ensure that each metric is delivered to at
least X caches.

-- 
You received this question notification because you are a member of
graphite-dev, which is an answer contact for Graphite.