graphite-dev team mailing list archive

[Nicholas Leskiw <question212303@xxxxxxxxxxxxxxxxxxxxx>]

Question #212303 on Graphite changed:
https://answers.launchpad.net/graphite/+question/212303

Nicholas Leskiw proposed the following answer:
You could pipe the find command to xargs like this:

find . -name '*.wsp' | xargs -P10 -n1 -I'{}'
/usr/local/bin/whisper-resize.py --nobackup {} 1m:1d

REPLACE 1m:1d with your retention rates and add any addition aggregation
method / xFilesFactor changes before the '{}'.

This will keep 10 threads running.


As far as shutting down carbon, it *shouldn't* cause an issue but it might
cause a few problems.  I've done small changes (~10000 files) without
shutting down carbon and ended up with 2-3 corrupt whisper files. Had to
delete them, they were non-recoverable. Not sure if that was issues with
leaving carbon up or some sort of bug / issues with whisper-resize.py
YMMV. If the cost of leaving carbon down > losing a few metrics, then don't
shut it down.


-Nick



On Mon, Nov 12, 2012 at 11:31 AM, Brett Riotto <
question212303@xxxxxxxxxxxxxxxxxxxxx> wrote:

> Question #212303 on Graphite changed:
> https://answers.launchpad.net/graphite/+question/212303
>
> Brett Riotto posted a new comment:
> Hey Guys,
> I really appreciate your quick responses and they both helped a ton! We
> were able to get the I/O wait down changing the MAX_UPDATES_PER_SECOND. So
> that got us to a useable state, but we would still like to reduce the
> granularity.
>
> Nick,
> I have been testing the resize and I am trying my best to speed it up. Is
> the best way, that you have found, to start the multiple find processes
> just to execute multiple commands from the prompt for different
> directories. Or is it beneficial to do some type of multi-threading,
> --threads=10. Also is there any reason to shut down carbon while we do the
> migration? Safer?
>
> Thanks
> Brett
>
> --
> You received this question notification because you are a member of
> graphite-dev, which is an answer contact for Graphite.
>
> _______________________________________________
> Mailing list: https://launchpad.net/~graphite-dev
> Post to     : graphite-dev@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~graphite-dev
> More help   : https://help.launchpad.net/ListHelp
>

-- 
You received this question notification because you are a member of
graphite-dev, which is an answer contact for Graphite.