yahoo-eng-team team mailing list archive

[Ramu Ramamurthy <ramu.ramamurthy@xxxxxxxxxx>]

Public bug reported:

Problem
--------

Currently, there is no application-metrics framework to measure neutron behavior,
and identify trends and problems in operation. RPC latencies, HW programming latency
(for security-groups, ovs), API latencies, DB latencies come to mind immediately as metrics
which need to be monitored, trended, and analyzed.

Logs are one piece of the puzzle but they do not capture tagged metrics data in a
format that can be easily analyzed.

This is filed as a RFE bug (following the blueprint protocol) and
this can be upgraded to a blueprint after discussion/inputs.

Fix
----

Provide a simple framework that has the following components:

    1) A metrics front end
          provide at the least the following apis
              @timeit(tag) - a decorator to time functions or code blocks.
              increment(tag)
              decrement(tag)
              set_value(tag, value)

    2) Metrics processor
          A class that can provide functionality like: sampling, throttling metrics data etc.

    3) Configurable Metrics output driver
          statsd-driver : output to statsd server
          graphite/carbon driver: output to carbon server
          logfile driver: output to logfile

 With the above library, neutron code is instrumented to add tagged
metrics.

I have some code, and can contribute it after community input/discussion on the rfe (following the
blueprint protocol).

Example Usecases
------------------

1) Time a function or code block
    @metrics.timeit
    def _apply(self):
        lock_name = 'iptables'
        if self.namespace:
            lock_name += '-' + self.namespace

Example output data (in graphite format)

neutron.agent.linux.iptables_manager._apply 319.076061249 1446328923
neutron.agent.linux.iptables_manager._apply 274.07002449 1446328925

Notes
------

1) Swift (and other projects ?) already have "statsd" based application metrics.
However, we want to design in a manner not entirely tied to statsd.

For example see.
https://review.openstack.org/#/c/6058/

2) Such a metrics library needs to ideally be an oslo project - as it can be used by all
openstack projects, but it can be proved in neutron first.

** Affects: neutron
     Importance: Undecided
     Assignee: Ramu Ramamurthy (ramu-ramamurthy)
         Status: New


** Tags: rfe

** Description changed:

  Problem
  --------
  
  Currently, there is no application-metrics framework to measure neutron behavior,
  and identify trends and problems in operation. RPC latencies, HW programming latency
- (for security-groups, ovs), API latencies, DB latencies come to mind immediately as metrics 
+ (for security-groups, ovs), API latencies, DB latencies come to mind immediately as metrics
  which need to be monitored, trended, and analyzed.
  
- Logs are one piece of the puzzle but they do not capture tagged metrics data in a 
+ Logs are one piece of the puzzle but they do not capture tagged metrics data in a
  format that can be easily analyzed.
  
  This is filed as a RFE bug (following the blueprint protocol) and
  this can be upgraded to a blueprint after discussion/inputs.
  
  Fix
  ----
  
  Provide a simple framework that has the following components:
  
-     1) A metrics front end
-           provide at the least the following apis
-               @timeit(tag) - a decorator to time functions or code blocks.
-               increment(tag)
-               decrement(tag)
-               set_value(tag, value)
-               
-     2) Metrics processor
-           A class that can provide functionality like: sampling, throttling metrics data etc.
-           
+     1) A metrics front end
+           provide at the least the following apis
+               @timeit(tag) - a decorator to time functions or code blocks.
+               increment(tag)
+               decrement(tag)
+               set_value(tag, value)
  
-     3) Configurable Metrics output driver
-           statsd-driver : output to statsd server
-           graphite/carbon driver: output to carbon server
-           logfile driver: output to logfile
+     2) Metrics processor
+           A class that can provide functionality like: sampling, throttling metrics data etc.
  
-  With the above library, neutron code is instrumented to add tagged
+     3) Configurable Metrics output driver
+           statsd-driver : output to statsd server
+           graphite/carbon driver: output to carbon server
+           logfile driver: output to logfile
+ 
+  With the above library, neutron code is instrumented to add tagged
  metrics.
  
- I have some code, and can contribute after community input/discussion on the rfe (following the 
+ I have some code, and can contribute it after community input/discussion on the rfe (following the
  blueprint protocol).
  
  Example Usecases
  ------------------
  
  1) Time a function or code block
-     @metrics.timeit
-     def _apply(self):
-         lock_name = 'iptables'
-         if self.namespace:
-             lock_name += '-' + self.namespace
+     @metrics.timeit
+     def _apply(self):
+         lock_name = 'iptables'
+         if self.namespace:
+             lock_name += '-' + self.namespace
  
  Example output data (in graphite format)
  
  neutron.agent.linux.iptables_manager._apply 319.076061249 1446328923
  neutron.agent.linux.iptables_manager._apply 274.07002449 1446328925
-     
+ 
  Notes
  ------
  
  1) Swift (and other projects ?) already have "statsd" based application metrics.
  However, we want to design in a manner not entirely tied to statsd.
  
  For example see.
  https://review.openstack.org/#/c/6058/
  
- 2) Such a metrics library needs to ideally be an oslo project - as it can be used by all 
+ 2) Such a metrics library needs to ideally be an oslo project - as it can be used by all
  openstack projects, but it can be proved in neutron first.

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1512864

Title:
  Application Metrics for Neutron

Status in neutron:
  New

Bug description:
  Problem
  --------

  Currently, there is no application-metrics framework to measure neutron behavior,
  and identify trends and problems in operation. RPC latencies, HW programming latency
  (for security-groups, ovs), API latencies, DB latencies come to mind immediately as metrics
  which need to be monitored, trended, and analyzed.

  Logs are one piece of the puzzle but they do not capture tagged metrics data in a
  format that can be easily analyzed.

  This is filed as a RFE bug (following the blueprint protocol) and
  this can be upgraded to a blueprint after discussion/inputs.

  Fix
  ----

  Provide a simple framework that has the following components:

      1) A metrics front end
            provide at the least the following apis
                @timeit(tag) - a decorator to time functions or code blocks.
                increment(tag)
                decrement(tag)
                set_value(tag, value)

      2) Metrics processor
            A class that can provide functionality like: sampling, throttling metrics data etc.

      3) Configurable Metrics output driver
            statsd-driver : output to statsd server
            graphite/carbon driver: output to carbon server
            logfile driver: output to logfile

   With the above library, neutron code is instrumented to add tagged
  metrics.

  I have some code, and can contribute it after community input/discussion on the rfe (following the
  blueprint protocol).

  Example Usecases
  ------------------

  1) Time a function or code block
      @metrics.timeit
      def _apply(self):
          lock_name = 'iptables'
          if self.namespace:
              lock_name += '-' + self.namespace

  Example output data (in graphite format)

  neutron.agent.linux.iptables_manager._apply 319.076061249 1446328923
  neutron.agent.linux.iptables_manager._apply 274.07002449 1446328925

  Notes
  ------

  1) Swift (and other projects ?) already have "statsd" based application metrics.
  However, we want to design in a manner not entirely tied to statsd.

  For example see.
  https://review.openstack.org/#/c/6058/

  2) Such a metrics library needs to ideally be an oslo project - as it can be used by all
  openstack projects, but it can be proved in neutron first.

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1512864/+subscriptions