runejuhl / grafana-timescaledb

A complete system for testing, experimentation and playing around with netdata, TimescaleDB and Grafana

NOTE!

This project is broken at the moment. I’ll try to get it fixed ASAP; have a look in https://github.com/runejuhl/grafana-timescaledb/tree/dev for a WIP development branch.

runejuhl

TimescaleDB, netdata and Grafana

A complete system for testing, experimentation and playing around with netdata, TimescaleDB and Grafana. Using netdata-tsrelay to relay metrics from netdata to TimescaleDB.

Provides a full system for testing and experimentation. To start it all, run:

make up logs

After the system has started, browse to http://localhost:3000. Here you need to set a new Grafana password, and add a data source.

The data source should have the following values:

host

timescaledb:5432

user

user1

password

password

database

metrics

A simple dashboard can be imported into Grafana.

Sample queries

SELECT time, host, metrics FROM netdata WHERE metrics ? 'system.load.load1' LIMIT 1 ;

SELECT time, host, metrics FROM netdata ORDER BY time DESC LIMIT 1 ;

SELECT time, host, d.metric, d.value::text::float FROM netdata, json_each(metrics::json) with ordinality as d(metric,value) WHERE metrics ?| array['system.load.load1'] LIMIT 10;

SELECT time, host, d.metric, d.value::text::float FROM netdata, json_each(metrics::json) with ordinality as d(metric,value) WHERE metrics ?| array['disk.mmcblk0.reads'] AND d.metric LIKE 'disk.%' LIMIT 10;

Cost of a JSON query

EXPLAIN ANALYZE SELECT time, host, d.metric, d.value::text::float FROM netdata, json_each(metrics::json) WITH ordinality as d(metric,value) WHERE d.metric LIKE 'disk.%' LIMIT 10;

Nested Loop  (cost=0.01..655.48 rows=466 width=61) (actual time=0.815..58.299 rows=1880 loops=1)
  ->  Append  (cost=0.00..63.65 rows=466 width=943) (actual time=0.027..0.509 rows=470 loops=1)
        ->  Seq Scan on netdata  (cost=0.00..0.00 rows=1 width=72) (actual time=0.012..0.012 rows=0 loops=1)
        ->  Seq Scan on _hyper_1_1_chunk  (cost=0.00..63.65 rows=465 width=945) (actual time=0.013..0.423 rows=470 loops=1)
  ->  Function Scan on json_each d  (cost=0.01..1.26 rows=1 width=64) (actual time=0.107..0.116 rows=4 loops=470)
        Filter: (metric ~~ 'disk.%'::text)
        Rows Removed by Filter: 46
Planning time: 0.543 ms
Execution time: 58.536 ms

Links

Using PostgreSQL in Grafana

http://docs.grafana.org/features/datasources/postgres/

TimescaleDB: Table management

https://docs.timescale.com/v0.11/using-timescaledb/schema-management

An attempt at literate programming with SQL

Doesn’t work yet, but it was a nice idea… https://orgmode.org/worg/org-contrib/babel/languages/ob-doc-sql.html

;; active Babel languages
(org-babel-do-load-languages
 'org-babel-load-languages
 '((sql . t)))
;; add additional languages with '((language . t)))

(let ((sql-port 55435)
      (sql-user "user1")
      (sql-server "localhost")
      (sql-database "metrics")
      (sql-password "password")
      (buffer-name  "*timescaledb sql*"))
  (sql-comint-postgres 'postgres '() buffer-name)
  (setq sql-buffer buffer-name)
  (run-hooks 'sql-set-sqli-hook))