graphite-dev team mailing list archive
-
graphite-dev team
-
Mailing list archive
-
Message #02420
[Merge] lp:~jblaine-kickflop/graphite/readme-install into lp:graphite
Jeff Blaine has proposed merging lp:~jblaine-kickflop/graphite/readme-install into lp:graphite.
Requested reviews:
graphite-dev (graphite-dev)
For more details, see:
https://code.launchpad.net/~jblaine-kickflop/graphite/readme-install/+merge/101793
Major overhaul of README and INSTALL files.
--
https://code.launchpad.net/~jblaine-kickflop/graphite/readme-install/+merge/101793
Your team graphite-dev is requested to review the proposed merge of lp:~jblaine-kickflop/graphite/readme-install into lp:graphite.
=== modified file 'INSTALL'
--- INSTALL 2011-04-01 07:28:09 +0000
+++ INSTALL 2012-04-12 18:43:27 +0000
@@ -1,150 +1,191 @@
- ExtJS
--------------------------------------------------------------------------------
-Graphite used to require an external Ext installation but now Ext is bundled
-with Graphite as allowed by the Ext FLOSS license exemption described here:
-
-http://extjs.com/products/floss-exception.php
-
-
- Run check-dependencies.py
--------------------------------------------------------------------------------
-This will check for graphite's runtime dependencies and let
-you know which you are missing. Download and install these modules and re-run
-check-dependencies.py until it detects all the required modules.
-
-Graphite requires:
- python2.4 or greater (python 2.5 or greater is required for AMQP support)
- whisper (available for download on the graphite project page)
- pycairo (with PNG backend support)
- django
- json (standard in python2.6) or simplejson
- mod_wsgi (optional, recommended) or mod_python (optional)
- python-ldap (optional - needed for ldap-based webapp authentication)
- python-memcached (optional - needed for webapp caching, big performance boost)
- python-sqlite2 (optional - a django-supported database module is required)
- Twisted (required only for carbon)
-
-
- Run setup.py
--------------------------------------------------------------------------------
-First take a look at the 'prefix' option in the setup.cfg file. This specifies
-where the graphite application will be installed. Note that if you plan on
-installing carbon on the same server you should use the same prefix for it.
-
-The prefix where graphite is installed under where be referred to as
-$GRAPHITE_ROOT from here on.
-
-Also, do NOT change the 'install-lib' option unless you really know what
-you are doing.
-
-
- Initialize the Database
-------------------------------------------------------------------------------
-Once you have installed graphite you must initialize its database like so:
-
-export PYTHONPATH=$PYTHONPATH:$GRAPHITE_ROOT/webapp
-cd $GRAPHITE_ROOT/webapp/graphite/
-sudo python manage.py syncdb
-
-Note that if you wish to use a custom database setup (like MySQL or Postgres)
-you will need to make those configuration changes first. See the "Custom
-Database Configuration" section below for details.
-
-
- Setup Filesystem Ownership
-------------------------------------------------------------------------------
-The user that apache runs as will need write access to the following directories:
-
-$GRAPHITE_ROOT/storage/
-$GRAPHITE_ROOT/storage/log/webapp/
-
-As well as write access to the following files:
-
-$GRAPHITE_ROOT/storage/graphite.db
-
-The apache user must also have read access to $GRAPHITE_ROOT/storage/whisper/
-
-
- Apache Configuration
--------------------------------------------------------------------------------
-When using apache there are two modules available to provide support for python
-web application.
-
-Graphite currently provides example configurations for both of these modules.
-Of the two mod_wsgi is the most up to date and actively maintained so it's
-highly recommended if you are starting afresh.
-
- Apache Configuration for Mod_wsgi (recommended)
--------------------------------------------------------------------------------
-
-If you are using mod_wsgi, Apache has to have mod_wsgi configured,
-this is usually done by including a line like the following in your
-httpd.conf:
-
-LoadModule wsgi_module modules/mod_wsgi.so
-
- Apache Configuration for Mod_python
--------------------------------------------------------------------------------
-
-If you choose to use mod_python, Apache has to have mod_python configured,
-this is usually done by including a line like the following in your
-httpd.conf:
-
-LoadModule python_module modules/mod_python.so
-
- General Apache Configuration
--------------------------------------------------------------------------------
-
-Once you have configured the apache module you are using you should configure
-a vhost for graphite (technically it doesn't have to be a vhost but its good
-practice). This can be done one of two ways.
-
-The first way (highly preferred) is to include independent vhost configs.
-
-# This goes in httpd.conf
-Include /usr/local/apache2/conf/vhosts.d/*.conf
-
-Then simply drop your graphite vhost conf file into the vhosts.d/ directory
-(or whatever directory your system uses) and apache is ready. You can use
-either of the example vhost confs as a starting point (See the examples
-directory of this package and choose the correct one based on whether you
-are using mod_wsgi or mod_python.)
-
-The second approach is to copy the contents of your chosen graphite vhost conf file
-and insert it down at the end of your httpd.conf.
-
-
- Custom Database Configuration
--------------------------------------------------------------------------------
-By default, the graphite webapp uses a sqlite database. As long as you have the
-appropriate sqlite module installed, you should not need to do any additional
-database configuration. But if you wish to use a different database such as
-MySQL or Postgres, read on.
-
-After you setup your database server, create a database and a user with full
-privileges on that database. Now we put that information into the graphite
-webapp local_settings.py file.
-
-Edit the $GRAPHITE_ROOT/webapp/graphite/local_settings.py
-
-In particular you will need to uncomment and modify all of the settings
-that begin with "DATABASE_". Once you have done this, we need to tell
-Django to create our tables.
-
-cd $GRAPHITE_ROOT/webapp/graphite/
-sudo python manage.py syncdb
-# you will be prompted to create an admin user, this is a good idea
-
-Restart apache and you should see the graphite webapp on the main page.
-If you encounter problems, you can increase the verbosity of error output
-by adding a "DEBUG = True" line to local_settings.py
-
-
- Next Steps - setup the backend (carbon)
--------------------------------------------------------------------------------
-Now you have finished installing the graphite webapp, the next thing to do is put some
-real data into it. This is accomplished by running the carbon-cache.py daemon
-and send it some data as demonstrated in the included
-./examples/example-client.py
-
-Instructions for doing this in the README file.
+Please read README before diving in.
+
+Dependencies
+============
+
+ Requirements
+ ------------
+
+ Python 2.4+ If you intend to use AMQP support, 2.5+ is required
+ whisper Available for download on the Graphite project page
+ carbon Available for download on the Graphite project page
+ pycairo ...with PNG backend support
+ Django Python module
+ django-tagging 0.3.1 or greater
+ json Python module (standard in Python 2.6), OR simplejson
+ Twisted Python module, for carbon
+ mod_wsgi OR mod_python, but mod_wsgi is recommended, and
+ installing one of the 2 is technically optional,
+ but you will be stuck with the built-in Django
+ development-class webserver without one of these
+
+ OPTIONAL
+ --------
+
+ python-ldap (needed for ldap-based webapp authentication)
+ python-memcached (needed for webapp caching, big performance boost)
+ python-sqlite2 (optional - a django-supported database module is required)
+
+ Dependency Checking!
+ --------------------
+
+ check-dependencies.py : This will check for graphite's runtime dependencies
+ and let you know which you are missing. Download and install the missing
+ pieces you care about and re-run check-dependencies.py until it detects
+ all the pieces you intend to use (obviously all of the required pieces).
+
+Graphite-Web Installation
+=========================
+
+ 0. Install all of the dependencies mentioned in the previous
+ section!
+
+ 1. Configure the 'prefix' option in the setup.cfg file. This specifies
+ where the graphite application will be installed. Note that if you
+ plan on installing carbon on the same server you should use the same
+ prefix for it.
+
+ The prefix where graphite is installed will be referred to as
+ $GRAPHITE_ROOT from here on in this document.
+
+ Do NOT change the 'install-lib' option unless you really
+ know what you are doing.
+
+ 2. Run: python setup.py install
+
+ 3. Once you have installed graphite you must initialize its
+ database like so:
+
+ export PYTHONPATH=$PYTHONPATH:$GRAPHITE_ROOT/webapp
+ cd $GRAPHITE_ROOT/webapp/graphite/
+ sudo python manage.py syncdb
+
+ Note that if you wish to use a custom database setup (like
+ MySQL or Postgres) you will need to make those configuration
+ changes first. See the "Custom Database Configuration" section
+ below for details.
+
+ 4. The user that Apache httpd runs as will need write access to
+ the following directories:
+
+ $GRAPHITE_ROOT/storage/
+ $GRAPHITE_ROOT/storage/log/webapp/
+
+ As well as write access to the following files:
+
+ $GRAPHITE_ROOT/storage/graphite.db
+
+ The apache user must also have read access to
+
+ $GRAPHITE_ROOT/storage/whisper/
+
+ 5. Continue to Apache Configuration AND Running carbon-cache
+ below.
+
+Apache Configuration
+====================
+
+ When using apache there are two modules available to provide
+ support for python web applications: mod_wsgi and mod_python
+
+ Graphite currently provides example configurations for both of
+ these modules. Of the two, mod_wsgi is the most up to date and
+ actively maintained so it's highly recommended if you are starting
+ afresh.
+
+ mod_wsgi (recommended, option 1)
+ --------------------------------
+
+ If you are using mod_wsgi, Apache has to have mod_wsgi configured,
+ this is usually done by including a line like the following in your
+ httpd.conf:
+
+ LoadModule wsgi_module modules/mod_wsgi.so
+
+ mod_python (option 2)
+ ---------------------
+
+ If you choose to use mod_python, Apache has to have mod_python
+ configured, this is usually done by including a line like the
+ following in your httpd.conf:
+
+ LoadModule python_module modules/mod_python.so
+
+ General Apache Configuration
+ ----------------------------
+
+ Now that you have decided on and configured mod_wsgi or
+ mod_python, there are some other items to address.
+
+ Configure a vhost for graphite (technically it doesn't have
+ to be a vhost but its good practice). This can be done one of
+ two ways.
+
+ The first way (highly preferred) is to include independent
+ vhost configs.
+
+ # This goes in httpd.conf
+ Include /your/apache/conf/dir/vhosts.d/*.conf
+
+ Then simply drop your graphite vhost conf file into the
+ vhosts.d/ directory (or whatever directory your system uses)
+ and apache is ready. You can use either of the example vhost
+ configurations as a starting point (See the examples
+ directory of this package and choose the correct one based on
+ whether you are using mod_wsgi or mod_python.)
+
+ The second approach is to copy the contents of your chosen
+ graphite vhost conf file and insert it down at the end of
+ your httpd.conf.
+
+Running carbon-cache.py
+=======================
+
+ Tell carbon-cache what user it should run as by editing
+ $GRAPHITE_ROOT/carbon/conf/carbon.conf
+
+ This must be a user with write privileges to
+
+ $GRAPHITE_ROOT/storage/whisper/
+ $GRAPHITE_ROOT/storage/log/carbon-cache/
+
+ Then run carbon-cache.py
+
+Appendix: Custom Database Configuration
+=======================================
+
+ By default, the graphite webapp uses a sqlite database. As long as you
+ have the appropriate sqlite module installed, you should not need to
+ do any additional database configuration. But if you wish to use a
+ different database such as MySQL or PostgreSQL, read on.
+
+ After you set up your database server, create a database and a
+ user with full privileges on that database. Now put that information
+ into the graphite webapp local_settings.py file:
+
+ $GRAPHITE_ROOT/webapp/graphite/local_settings.py
+
+ In particular you will need to uncomment and modify all of the settings
+ that begin with "DATABASE_". Once you have done this, we need to tell
+ Django to create our tables.
+
+ cd $GRAPHITE_ROOT/webapp/graphite/
+ sudo python manage.py syncdb
+ # you will be prompted to create an admin user, this is a good idea
+
+ Restart apache and you should see the graphite webapp on the main page.
+ If you encounter problems, you can increase the verbosity of error output
+ by adding a "DEBUG = True" line to local_settings.py
+
+Appendix: A Note about "ExtJS"
+==============================
+
+ This section is kept purely for historical purposes for now.
+ You may choose to ignore it.
+
+ Graphite used to require an external Ext installation but now
+ Ext is bundled with Graphite as allowed by the Ext FLOSS license
+ exemption described here:
+
+ http://extjs.com/products/floss-exception.php
+
=== modified file 'README'
--- README 2009-09-14 21:40:04 +0000
+++ README 2012-04-12 18:43:27 +0000
@@ -1,47 +1,63 @@
- Overview
--------------------------------------------------------------------------------
-Graphite consists of two major components
-
- 1) the frontend Django webapp that runs under mod_python Apache
- 2) the backend carbon-cache.py daemon
-
-Client applications connect to the running carbon-cache.py daemon on port 2003 and send it
-lines of text of the following format:
-
-"my.metric.name value unix_timestamp", for example:
-
-"performance.servers.www01.cpuUsage 42.5 1208815315"
-
-The metric name is like a filesystem path that uses . as a separator instead of /
-The value is some scalar integer or floating point value
-The unix_timestamp is unix epoch time, as an integer
-
-Each line like this corresponds to one data point for one metric.
-
-Once you've got some clients sending data to carbon-cache, you can view
-graphs of that data in the frontend webapp.
-
-
- Webapp Installation
--------------------------------------------------------------------------------
-Use the instructions in the INSTALL file.
-
-
- Running carbon-cache.py
--------------------------------------------------------------------------------
-First you must tell carbon-cache what user it should run as.
-This must be a user with write privileges to $GRAPHITE_ROOT/storage/whisper/
-Specify the user account in $GRAPHITE_ROOT/carbon/conf/carbon.conf
-
-This user must also have write privileges to $GRAPHITE_ROOT/storage/log/carbon-cache/
-
-
- Writing a client
--------------------------------------------------------------------------------
-First you obviously need to decide what data it is you want to graph with
-graphite. The script examples/example-client.py demonstrates a simple client
-that sends loadavg data for your local machine to carbon on a minutely basis.
-
-The default storage schema stores data in one-minute intervals for 2 hours.
-This is probably not what you want so you should create a custom storage schema
-according to the docs on the graphite wiki (http://graphite.wikidot.com)
+Overview
+========
+
+ Graphite is an enterprise-scale monitoring tool that runs well on cheap
+ hardware. It was originally designed and written by Chris Davis at
+ Orbitz in 2006 as side project that ultimately grew to be a foundational
+ monitoring tool. In 2008, Orbitz allowed Graphite to be released under
+ the open source Apache 2.0 license. Since then Chris has continued to
+ work on Graphite and has deployed it at other companies including Sears,
+ where it serves as a pillar of the e-commerce monitoring system. Today
+ many large companies use it.
+
+ Architecture in a Nutshell
+ --------------------------
+
+ Graphite consists of 3 main software components:
+
+ +------------------+<---- server0 metrics...
+ | carbon, manages |<------- server1 metrics...
+ | data writing |<---- server2 metrics...
+ +------------------+<---------- server-N metrics...
+ |
+ V
+ +---------+---------+ +--------------+
+ | whisper database +----------->| graphite-web |
+ +-------------------+ +--------------+
+
+ carbon
+
+ a daemon that listens for time-series data (Python, Twisted)
+
+ whisper
+
+ a simple database library for storing time-series data
+ (similar in design to RRD)
+
+ graphite webapp
+
+ A Django web application that renders graphs on-demand using Cairo.
+
+ Feeding in your data is pretty easy, typically most of the effort
+ is in collecting the data to begin with. As you send data points
+ to Carbon, they become immediately available for graphing in the
+ webapp. The webapp offers several ways to create and display
+ graphs including a simple URL API that makes it easy to embed
+ graphs in other webpages.
+
+Installation
+============
+
+ Please see the INSTALL file and the official documentation
+
+Resources
+=========
+
+ The official site
+
+ http://graphite.wikidot.com
+
+ The official documentation
+
+ http://graphite.readthedocs.org/
+
