← Back to team overview

graphite-dev team mailing list archive

Re: [Question #182603]: Bulk loading of existing data

 

Question #182603 on Graphite changed:
https://answers.launchpad.net/graphite/+question/182603

    Status: Open => Answered

Nicholas Leskiw proposed the following answer:
Check MAX_CREATES_PER_MIN and the buffer size. Read the carbon
configuration file, there's other settings you may want to tweak.


Kurt Telep <question182603@xxxxxxxxxxxxxxxxxxxxx> wrote:

>New question #182603 on Graphite:
>https://answers.launchpad.net/graphite/+question/182603
>
>I have a huge amount of existing data to load into Graphite.   I've found that unless I have a .5 second pause between every 30 or so records I insert that records get dropped, and the whisper files don't get created for the data being added.  
>
>Is there any log that will actually show that this data is being dropped, as I don't see anything specifically in the logs.   Could it be that I'm pushing too much data to the socket and it's getting dropped?
>
>Is there a better way for me to bulk load besides pushing it through carbon?
>
>I've got about 2 billion rows to add initially, after that about 200 data points will be added every 2 minutes.
>
>-- 
>You received this question notification because you are a member of
>graphite-dev, which is an answer contact for Graphite.
>
>_______________________________________________
>Mailing list: https://launchpad.net/~graphite-dev
>Post to     : graphite-dev@xxxxxxxxxxxxxxxxxxx
>Unsubscribe : https://launchpad.net/~graphite-dev
>More help   : https://help.launchpad.net/ListHelp

-- 
You received this question notification because you are a member of
graphite-dev, which is an answer contact for Graphite.