graphite-dev team mailing list archive

[Chris Davis <chrismd@xxxxxxxxx>]

Hey everyone, so I've been busy working on some new features that are now
far enough along to merit testing. This will become the 0.9.5 release after
a couple weeks of testing. There are a lot of sizable changes coming at
once, more than I would like. I got carried away on starting new things
before other prior efforts were finished. I've learned my lesson, next
release will be much smaller and sooner. Here's the low-down on what's new
in trunk as of the past few days.

First is a sorely needed complete rewrite of carbon. Aside from performance
and maintainability concerns, as well as flat-out annoyances, I rewrote
carbon to facilitate a particular feature that my company needs. That is
federated storage. This means you can have any number of graphite servers
which are able to share data with each other transparently and
fault-tolerantly, and have it appear to be one big honking fast graphite
server to the user. More details on this later, I'm getting close to having
enough need to devote some serious time to writing documentation. Questions
welcome of course.

Next big change is moving to distutils. The install.py script has been a
total hack since day 1, good riddance. As part of moving to distutils I
decided to break graphite up into 3 separate packages: graphite (the
webapp), carbon, and whisper. I'll keep them all in the same repository and
they will share version numbers for now. All in all, it's a good change.
Here are a few gotchas: 1) you now build new graphite release tarballs by
running 'python setup.py sdist' to build a source distribution, you'd also
run this in carbon/ and whisper/ subdirs to build packages for them of
course. Modify setup.py to change the version. 2) graphite's default install
location is now /opt/graphite instead of /usr/local/graphite. I figure why
not save us all from carpal tunnel? If you wish to change this location at
install time just edit setup.cfg.

Another substantial change is in the Composer/Browser UI. There was this
horribly annoying bug where the browser tree would randomly not work and it
drove me insane. Turns out the cause was the fact that I was using
prototype.js in conjunction with Ext. They are supposed to work fine
together but apparently there are some bugs. I decided I really like Ext 3,
why not just go Ext all the way and ditch prototype. So that's what I did
and voila, the browser tree bug is gone. Note that prototype.js is still
used by the CLI, but if I ever get around to reworking that piece of junk
I'll definitely redo it as pure Ext. Ext is awesome.

I think that covers all the big changes but there are still lots of little
ones, here's a brief summary.

* fixed about a dozen open bugs from the launchpad tracker (which I will
close out after this email)
* lots of internal code cleanup
* refactored the lousy 'Targets' dialog into a slightly more robust 'Graph
Data' dialog
* added several handy rendering functions, easy to find with the new Graph
Data UI (look under 'Filter')
* implemented the storage of per-metric context data (ie. arbitrary
key=value pairs you can associate with a given metric). This is currently
unused but will definitely be useful in the future.
* rewrote the 'Auto-Completer' text field as a reusable Ext widget, which is
now used in a few other places in the UI
* added a 'rebuild_index.sh' script under misc/, set this up as a cron job
to maintain a proper 'index' file thus allowing the 'Search' UI tab to work.

That's all for now. The one thing I may add (other than bug fixes) before
the 0.9.5 release is mouseover documentation in the rendering functions
menu.

If you're willing to be brave or just sick of the old graphite version
please test this code. Let me know if you have any issues with it, and if
you've got any cool stuff you've implemented on a private branch that you'd
like to merge into trunk please let me know. Again I apologize for such a
big set of changes all at once, I plan on making a much smaller release
after 0.9.5, hopefully before the end of the year.

-Chris