Modules¶

livemetrics¶

This module implements a utility to collect (efficiently) business metrics showing the activity of an application.

For testing, we will reduce the tick interval

>>> import livemetrics.metrics
>>> livemetrics.metrics.Meter.TICK_INTERVAL = 0.1

First you need to create an object and to send an event:

>>> lm = LiveMetrics("version","about",lambda: True,lambda: True)
>>> lm.mark('start','ok')
>>> lm.mark('start','error')

This is enough to get a first set of metrics:

>>> import pprint
>>> pprint.pprint(lm.get_metrics()) #doctest: +ELLIPSIS
{'start': {'error': {'count': 1,
...
                     'rate1': 0.0,
                     'rate15': 0.0,
                     'rate5': 0.0},
           'ok': {'count': 1,
...
                  'rate1': 0.0,
                  'rate15': 0.0,
                  'rate5': 0.0}}}

We can now emit more events:

>>> import time
>>> time.sleep(0.02)
>>> lm.mark('start','ok')
>>> time.sleep(0.03)
>>> lm.mark('start','ok')
>>> lm.mark('start','error')
>>> time.sleep(0.2)
>>> round(lm.get_metrics()['start']['ok']['rate1'],2)
29.95
>>> round(lm.get_metrics()['start']['error']['rate1'],2)
19.97
>>> m = lm.get_metrics()
>>> m['start']['error']['rate1']<m['start']['error']['rate5']
True
>>> m['start']['error']['rate5']<m['start']['error']['rate15']
True
>>> m['start']['error']['rate1']<m['start']['ok']['rate1']
True
>>> m['start']['ok']['rate1']>29.0
True
>>> m['start']['error']['rate1']>19.0
True

For gauges, here is an example:

>>> lm = LiveMetrics("version","about",lambda: True)
>>> lm.gauge('G',10)
>>> lm.gauge('G',15)
>>> lm.gauge('G',20)
>>> lm.gauge('G',12)

>>> print(lm.get_gauges())
{'G': {'min': 10, 'max': 20, 'count': 12}}

>>> print(lm.get_gauges('G'))
{'min': 10, 'max': 20, 'count': 12}

>>> print(lm.get_gauges('G','max'))
20

This module also provides a decorator to encapsulate a meter and a histogram. Example:

>>> lm = LiveMetrics("version","about",lambda: True)
>>> @lm.timer('decorated','OK','EXC')
... def for_test(value,exc=False):
...     if exc: raise Exception("Exception")
...     time.sleep(0.1)
...     return value

Then, every time the function is called, new metrics are collected:

>>> for_test(1)
1
>>> for_test(2)
2
>>> for_test(3,True)
Traceback (most recent call last):
...
Exception: Exception

Metrics can be accessed using LiveMetrics.get_metrics():

>>> pprint.pprint(lm.get_metrics()) #doctest: +ELLIPSIS
{'decorated': {'EXC': {'count': 1,
...
                       'rate5': 0.0},
               'OK': {'count': 2,
...
                      'rate5': 10.0}}}

>>> pprint.pprint(lm.get_metrics('decorated')) #doctest: +ELLIPSIS
{'EXC': {'count': 1, 'mean': 0.0, 'rate1': 0.0, 'rate15': 0.0, 'rate5': 0.0},
 'OK': {'count': 2,
...
        'rate5': 10.0}}

>>> M = lm.get_metrics('decorated','OK')
>>> M['count']
2
>>> round(M['rate5'],1)
10.0

>>> lm.get_metrics('decorated','OK','count')
2

Event must be specified if result and metric are defined: >>> lm.get_metrics(None,’OK’,’count’) Traceback (most recent call last): … SyntaxError: if metric/result is specified, event must also be specified >>> lm.get_metrics(‘decorated’,None,’count’) Traceback (most recent call last): … SyntaxError: if metric is specified, result must also be specified

Or LiveMetrics.get_histograms():

>>> H = lm.get_histograms()
>>> H['decorated']['count']
3
>>> round(H['decorated']['quantiles'][0.05],1)
0.1
>>> round(H['decorated']['stddev'],2)
0.05

>>> H = lm.get_histograms('decorated')
>>> H['count']
3
>>> round(H['quantiles'][0.05],1)
0.1
>>> round(H['stddev'],2)
0.05

>>> pprint.pprint(lm.get_histograms('decorated','count')) #doctest: +ELLIPSIS
3

>>> math.isclose(lm.get_histograms('decorated','quantiles')[0.05],0.05,rel_tol=0.01)
True

>>> pprint.pprint(lm.get_histograms('decorated','distribution',scale=1)) #doctest: +ELLIPSIS
[3]

class livemetrics.LiveMetrics(version, about, is_healthy, is_ready=None)¶

Bases: object

An object to be used as a singleton to aggregate all live metrics of an application.

gauge(name, value)¶: Register a new gauge for this name. If value is a callable, it will be called everytime the gauge value is accessed.

get_gauges(name=None, metric=None)¶

Return a structure of dictionaries with the gauge metrics.

name: the name of the gauge

metric: the name of the metric, one of min, max, count

If metric is not provided, all metrics are returned.

If name is not provided, all gauges are returned.

get_histograms(event=None, metric=None, percentiles=[0.05, 0.25, 0.5, 0.75, 0.95], scale=10)¶

Return a structure of dictionaries with the histograms metrics.

event: the name of the event

metric: the name of the metric, one of count, min, max, mean, stddev, quantiles, distribution

percentiles: the list of percentiles for which the quantiles are returned

scale: the resolution (number of values) of the returned distribution

If metric is not provided, all metrics are returned.

If event is not provided, all histograms are returned.

get_metrics(event=None, result=None, metric=None)¶

Return a structure of dictionaries with the requested metrics.

event: the name of the event

result: the result of the event, as used in the call to mark()

metric: the name of the metric, one of mean, count, rate1, rate5, rate15.

If metric is not provided, all metrics are returned.

If result is not provided, all results are returned.

If event is not provided, all events are returned.

histogram(name, value)¶: Register a new value in a histogram.

mark(event, result)¶: Mark the execution of an event event with the result.

timer(event, ok, error)¶

Decorator to automate meter and histogram on one event.

event: the name of the event

ok: value when the function decorated terminates successfully

error: value when the function decorated terminates with an exception

For each call of the decorated function, a call to mark() is done for the event and result (ok or error), and a call to histogram() is made to measure the processing time.

ok and error can be callables that return a string compatible with json dictionary key

Warning

If applied on an async function decorated with aiohttp web.RouteTableDef(), it must be placed between the aiohttp annotation and the async function. Otherwise the aiohttp route will be decorated and not the actual function.

livemetrics.metrics¶

class livemetrics.metrics.EWMA(nb_minutes=1, interval=5)¶

Bases: object

An exponentially-weighted moving average. See http://en.wikipedia.org/wiki/Moving_average#Exponential_moving_average

Usage:

>>> e = EWMA(1)
>>> e.update()
>>> import time
>>> time.sleep(1)
>>> e.update()
>>> e._tick()
>>> e.rate  #doctest: +ELLIPSIS
0.39999...
>>> e._tick()
>>> e.rate  #doctest: +ELLIPSIS
0.368017...

update(n=1)¶: Indicate an event has happened. n specifies the number of events.

class livemetrics.metrics.Gauge(value=None)¶

Bases: object

A simple counter, registering a value and statistics (min and max) on this value.

>>> g = Gauge(10)
>>> g.mark(12)
>>> g.min,g.count,g.max
(10, 12, 12)

>>> g.mark(8)
>>> g.min,g.count,g.max
(8, 8, 12)

>>> g = Gauge()
>>> g.mark(12)
>>> g.min,g.count,g.max
(12, 12, 12)

>>> g.mark(8)
>>> g.min,g.count,g.max
(8, 8, 12)

>>> g.mark(20)
>>> g.min,g.count,g.max
(8, 20, 20)

>>> g.mark(15)
>>> g.min,g.count,g.max
(8, 15, 20)

The alias value can also be used:

>>> g.min,g.value,g.max
(8, 15, 20)

Any time, the value can be provided as a callable:

>>> import random
>>> g.mark(lambda: random.randrange(21,50))
>>> g.count>20
True

The value can also be a callable from the start

>>> g = Gauge(lambda: random.randrange(1,20))
>>> g.mark(10)
>>> g.min>=0,g.max<20
(True, True)

count¶: Return the current value of the gauge.

mark(value)¶: Register a new value in the gauge and update the statistics.

max¶: Return the maximum value ever registered in this gauge.

min¶: Return the minimum value ever registered in this gauge.

value¶: Alias on count().

class livemetrics.metrics.Histogram¶

Bases: object

An histogram represents the distribution of a set of values. Values are exponentially decayed so that more recent values have more weight than old values.

For testing, we will reduce the tick interval managing the decaying of values.

>>> Reservoir.TICK_INTERVAL = 0.1

Usage:

>>> his = Histogram()

A new object has no value yet:

>>> his.count
0

For the test, we Let’s add 100 sequential values:

>>> for i in range(100):
...    his.update(i+1)

The snapshot object gives access to some statistics:

>>> round(his.snapshot.mean,1)
50.5
>>> round(his.snapshot.stddev,1)
28.9

And to the value for a percentile:

>>> round(his.snapshot.get_value(0.25),1)
26.5

>>> his.snapshot.get_distribution()
[10, 10, 10, 10, 10, 10, 10, 10, 10, 10]

>>> his.snapshot.get_distribution(20)
[5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5]

If we wait a little and inject new values:

>>> import time
>>> time.sleep(1.0)
>>> for i in range(100):
...    his.update(i*2)

>>> his.snapshot.get_distribution(20)
[14, 15, 15, 15, 15, 15, 15, 15, 15, 15, 6, 5, 5, 5, 5, 5, 5, 5, 5, 5]
>>> round(his.snapshot.get_value(0.25),1)
34.0

>>> time.sleep(1.0)
>>> for i in range(100):
...    his.update(i*2)
>>> his.snapshot.get_distribution(20)
[19, 20, 20, 20, 20, 20, 20, 20, 20, 20, 11, 10, 10, 10, 10, 10, 10, 10, 10, 10]
>>> round(his.snapshot.get_value(0.25),1)
38.0

The size of the reservoir is by default 1024. When the number of values is over this limit, values are randomly removed, older values first.

>>> his50 = Histogram()
>>> his50._reservoir._size = 50
>>> for i in range(100):
...    his50.update(i+1)

>>> his.snapshot.mean!=his50.snapshot.mean
True
>>> his.snapshot.stddev!=his50.snapshot.stddev
True

count¶: Get the number of values recorded by this histogram.

snapshot¶: Get a WeightedSnapshot for this histogram. The snapshot will then give access to quantiles and to the distribution of values.

update(value)¶: Register a new value in the histogram.

class livemetrics.metrics.Meter¶

Bases: object

An object measuring the rates of occurence of an event. Each event is reported with a call to mark(). The rates are decayed as time goes by.

For testing, we will reduce the tick interval managing the decaying of values.

>>> Meter.TICK_INTERVAL = 0.1

Then we create a new object:

>>> mtr = Meter()

At start, it is empty:

>>> mtr.mean
0.0

Then if we mark an event:

>>> mtr.mark()

Mean is still 0 because there is no enough time difference to calculate it.

>>> mtr.mean
0.0

Let’s move forward in time:

>>> import time
>>> time.sleep(0.1)

and report a second event:

>>> mtr.mark()
>>> mtr.mean>17 and mtr.mean<25
True
>>> mtr.count
2
>>> mtr.mark()
>>> time.sleep(0.5)
>>> 10.0-mtr.rate1<0.1,10.0-mtr.rate5<0.1,10.0-mtr.rate15<0.1
(True, True, True)

Rates decreases as time goes:

>>> time.sleep(1.0)
>>> 9.8-mtr.rate1<0.1,9.9-mtr.rate5<0.1,10.0-mtr.rate15<0.1
(True, True, True)

We marked 3 times in 2.1 seconds

>>> round(mtr.mean,1)
1.9

count¶: Return the number of events reported to this object since it was created.

mark()¶: Indicate an event has just occurred.

mean¶: The number of events divided by the time the application is running. This gives a very rough estimation of the average number of events per seconds.

rate1¶: Return the 1-minute moving average rate.

rate15¶: Return the 15-minute moving average rate.

rate5¶: Return the 5-minute moving average rate.

class livemetrics.metrics.Reservoir¶

Bases: object

A reservoir holds the data of an Histogram. It records values and decays them as time passes. It keeps only a statistical representative set of values.

class livemetrics.metrics.WeightedSnapshot(values)¶

Bases: object

A (weighted) snapshot, i.e. a copy of a set of values that have a weight. The weight is used to calculate the quantiles.

This class is used by Histogram and returned by Histogram.snapshot

Example:

>>> snapshot = WeightedSnapshot( {i:WeightedSample(i,i) for i in range(10)})

Access the statistics of the values:

>>> print((snapshot.min,snapshot.max,round(snapshot.mean,1),round(snapshot.stddev,1)))
(0, 9, 6.3, 2.2)

>>> print(snapshot.get_value(0.5))
7.5

>>> print(snapshot.get_value(-1))
Traceback (most recent call last):
...
Exception: argument -1 is not in [0..1]

>>> print(snapshot.get_value(2))
Traceback (most recent call last):
...
Exception: argument 2 is not in [0..1]

>>> print(snapshot.get_distribution(5))
[2, 2, 2, 2, 2]

>>> print(snapshot.size)
10

The snapshot can also be empty:

>>> snapshot = WeightedSnapshot( {})
>>> print((snapshot.min,snapshot.max,round(snapshot.mean,1),round(snapshot.stddev,1)))
(0, 0, 0, 0)

>>> print(snapshot.get_value(0.5))
0.0

get_distribution(range_value=10)¶: Return the distribution with the request resolution (i.e. number of values)

get_value(quantile)¶

Access the value (the quantile) for a given percentile, i.e. the value that defines the boundary between the two segment [O,percentile] and [percentile,1].

The percentile must be between 0 and 1.

max¶: Maximum value in this snapshot.

mean¶: Weighted average of the values in this snapshot.

min¶: Minimum value in this snapshot.

size¶: Number of values in this snapshot.

stddev¶: Standard deviation of the weighted values in this snapshot.

livemetrics.dashboard¶

This module provides a (very) minimal application to demonstrate the usage of metrics.

Install with:

pip install livemetrics[dashboard]

Run with:

livemetrics-dashboard -i myconf.yaml

And then access http://localhost:9000/ to visualize the dashboard.

The configuration file describes:

The rows in the dashboard. One row corresponds to a set of meters/histograms from one server.
For each row, a list of meters and a list of hstograms.

For example:

---
- label: Sample
  id: server1
  server: http://localhost:7070/monitoring/v1
  meters:
  - label: Test OK
    id: t1
    event: test
    result: ok
    color: green
  histograms:
  - label: Exec Time
    id: h1
    events: [time]
    title: ms
    factor: 1000

The tag id is used to construct the id of the HTML objects. It must be unique.

Publishers¶

Publishers are classes that expose the live metrics through a REST interface.

livemetrics.publishers.http¶

This module provides a http.server.BaseHTTPRequestHandler subclass exposing a livemetrics.LiveMetrics object.

A minimal application would be:

import livemetrics
import livemetrics.publishers.http

import http.server

LM = livemetrics.LiveMetrics(version,about,is_healthy,is_ready)

class Sample(livemetrics.publishers.http.HTTPRequestHandler):

    @LM.timer("sample","ok","error")
    def do_GET(self):
        if self.path=='/sample':
            self.send_response(200)
            self.end_headers()
        else:
            return super().do_GET()

httpd = http.server.HTTPServer((host,port), lambda r,a,s: Sample(LM,r,a,s))
httpd.serve_forever()

class livemetrics.publishers.http.HTTPRequestHandler(LM, request, client_address, server)¶

Bases: http.server.BaseHTTPRequestHandler

Subclass this class in your application to inherit the handling of GET requests exposing a livemetrics.LiveMetrics object.

livemetrics.publishers.aiohttp¶

This module provides aiohttp routes exposing a livemetrics.LiveMetrics object.

A minimal application would be:

import livemetrics
import livemetrics.publishers.aiohttp

from aiohttp import web
routes = web.RouteTableDef()

LM = livemetrics.LiveMetrics(version,about,is_healthy,is_ready)

@routes.get("/sample")
@LM.timer('sample','ok','error')
async def sample(request):
    return web.Response(status=200)

app = web.Application()
app.add_routes(livemetrics.publishers.aiohttp.routes(LM))   # register the routes to publish the metrics
app.add_routes(routes)                                      # register the routes of the application
web.run_app(ap,host,port)                                   # run the application

livemetrics.publishers.aiohttp.routes(LM)¶

Return a list of routes to be registered in the aiohttp application.

LM: a livemetrics.LiveMetrics object

livemetrics.publishers.flask¶

This module provides a flask blueprint exposing a livemetrics.LiveMetrics object.

A minimal application would be:

import livemetrics
import livemetrics.publishers.flask

from flask import Flask
app = Flask(__name__)

LM = livemetrics.LiveMetrics(version,about,is_healthy,is_ready)

@app.route("/sample")
@LM.timer('sample','ok','error')
def sample():
    return "Sample!"

app.register_blueprint(livemetrics.publishers.flask.blueprint(LM))

livemetrics.publishers.flask.blueprint(LM)¶

Return a blueprint with the routes and view_func for the publication of the metrics.

LM: a livemetrics.LiveMetrics object

livemetrics.publishers.django¶

This module provides Django views exposing a livemetrics.LiveMetrics object.

A minimal application would be:

import livemetrics
import livemetrics.publishers.django

import django
from django.http import HttpResponse
from django.urls import path

# Declare a global LiveMetrics object in one of your modules
LM = livemetrics.LiveMetrics(version,about,is_healthy,is_ready)

@LM.timer('sample','ok','error')
def sample_view(request):
    return HttpResponse("sample")

In your urls.py, register your views with:

urlpatterns = [
    path('sample', sample_view, name='sample'),
]

urlpatterns += livemetrics.publishers.django.urlpatterns(LM)

livemetrics.publishers.django.urlpatterns(LM)¶

Return a list of Django paths to be registered in the application.

LM: a livemetrics.LiveMetrics object

Modules¶

livemetrics¶

livemetrics.metrics¶

livemetrics.dashboard¶

Publishers¶

livemetrics.publishers.http¶

livemetrics.publishers.aiohttp¶

livemetrics.publishers.flask¶

livemetrics.publishers.django¶

livemetrics

Navigation

Related Topics