← Back to team overview

graphite-dev team mailing list archive

[Question #170630]: Graphite frontend very slow with 5 machines

 

New question #170630 on Graphite:
https://answers.launchpad.net/graphite/+question/170630

I've got 5 machines setup with carbon-cache and web frontends. When data was on one machine, rendering and response time was excellent. Now it takes 10+ seconds (at least) to just grab the tree of stats, and lately has started timing out entirely.

I tried hooking in local memcached instances on each box (which have relatively low memory usage), but I'm not seeing any indication memcache is being used at all.

In my local_settings.py, I've got:
CLUSTER_SERVERS = ['graphite-storage-1', 'graphite-storage-2', 'graphite-storage-3', 'graphite-storage-4', 'graphite-storage-5']
and
CARBONLINK_HOSTS = ['graphite-storage-1:7002', 'graphite-storage-2:7002', 'graphite-storage-3:7002', 'graphite-storage-4:7002', 'graphite-storage-5:7002']

If I change CLUSTER_SERVERS to ['localhost'], things fly again. This makes sense, since it's not hitting the other boxes for data. But I need to be able to group data cross box. We're using consistent hashing for now (though the current setup can't re-shuffle data and we're growing, so I'll need to relay by hand going forward) and a one machine view isn't much use.

Am I using graphite in a way it wasn't intended? Is nobody else using this many machines? Do I need to do something more than add MEMCACHE_HOSTS = ['127.0.0.1:11211'] to get memcached support to work?


-- 
You received this question notification because you are a member of
graphite-dev, which is an answer contact for Graphite.