DiegoDAF/pgMonitor.dba

All-inclusive PostgreSQL monitoring stack: Prometheus + Grafana + pgscv + YACE + node_exporter + cadvisor. Modular Docker Compose, SSH tunnel support, CloudWatch long-term retention.

★ 0Forks 0ShellGitHub ↗Compare

README

pgMonitor — all-inclusive PostgreSQL monitoring stack

Docker Compose stack that wires together Prometheus, Grafana, pgscv, YACE, node_exporter and cadvisor to give you a turn-key monitoring setup for fleets of PostgreSQL (self-managed and AWS RDS/Aurora).

This is the sanitized public version (pgMonitor.dba). A private version exists (pgMonitor.daf) with real infrastructure configs, but the architecture here is identical.

What you get

  • Prometheus — time-series store (default: 2000 days retention, ~22 GB/year for a ~30-DB fleet)
  • Grafana — dashboards (DBA health checks, pg_stat_statements top queries, RDS/Aurora CloudWatch)
  • pgscv — PostgreSQL metrics collector (fork with SSH tunnel support: DiegoDAF/pgscv.dba)
  • YACE — exports CloudWatch RDS/Aurora metrics to Prometheus for long-term retention (CloudWatch only retains 15 months)
  • node_exporter — host OS metrics
  • cadvisor — container-level metrics

Why

  • pgscv > postgres_exporter — collects pg_stat_statements, dead tuples, bloat, autovacuum, wait events, replication lag, sequence exhaustion, invalid indexes, tables without PK, TOAST bloat, and more (~775 metrics per DB).
  • SSH tunnel support — works for DBs that are only reachable through bastion hosts (common in Aurora/RDS setups).
  • CloudWatch backup — YACE preserves AWS metrics (CPU, IOPS, latency, connections, storage) in Prometheus, so you don't lose them after 15 months.

Architecture

Compose uses include: directives to wire modular YAML pieces:

pgmonitor.yaml              # main compose file with include: pointers
├── network.d/              # network config
├── volumes.d/              # prometheus-data + grafana-data bind mounts
├── services.d/             # one YAML per service
│   ├── prometheus.yaml
│   ├── grafana.yaml
│   ├── node_exporter.yaml
│   ├── cadvisor.yaml
│   ├── yace.yaml
│   └── pgscv-example-{rw,ro}.yaml    # duplicate per DB
├── config/
│   ├── prometheus.yaml     # scrape configs
│   ├── prometheus.d/       # file_sd_configs — one YAML per pgscv target
│   ├── pgscv-example-{rw,ro}.yaml    # pgscv per-DB config
│   ├── yace.yaml.example             # CloudWatch metric collection
│   └── grafana.conf.d/     # datasources + dashboards provisioning
└── secrets/
    └── example.conn.template         # template for .conn files (real .conn are gitignored)

Quick start

  1. Clone:

    git clone https://github.com/DiegoDAF/pgMonitor.dba.git
    cd pgMonitor.dba
  2. Config:

    cp .env.template .env                      # edit with your AWS keys + data path
    cp config/yace.yaml.example config/yace.yaml   # edit with your RDS instance IDs
  3. Data directories (bind mounts — Prometheus needs nobody, Grafana UID 472):

    sudo mkdir -p $PGMONITOR_DATA_PATH/{prometheus-data,grafana-data}
    sudo chown nobody:nogroup $PGMONITOR_DATA_PATH/prometheus-data
    sudo chown 472:root       $PGMONITOR_DATA_PATH/grafana-data
  4. Duplicate the pgscv examples for each DB you monitor:

    # For each database: pick a unique port (9890-9919 RW, 9920-9949 RO)
    cp services.d/pgscv-example-rw.yaml services.d/pgscv-mydb.yaml
    cp config/pgscv-example-rw.yaml config/pgscv-mydb.yaml
    cp config/prometheus.d/pgscv-example-rw.yaml config/prometheus.d/pgscv-mydb.yaml
    # Then edit ports, hostnames, and add an entry to the include: list in pgmonitor.yaml
  5. Create the .conn file with your PostgreSQL connection string (never commit):

    echo "postgres://user:pass@db-host:5432/postgres?sslmode=require" > secrets/mydb.conn
    chmod 600 secrets/mydb.conn
  6. Run:

    docker compose --file pgmonitor.yaml --project-name pgmonitor up -d

Default ports

Port Service
3000 Grafana UI
5000 YACE /metrics
8080 cadvisor /metrics
9090 Prometheus UI + /metrics
9100 node_exporter /metrics
9890-9919 pgscv writer endpoints
9920-9949 pgscv reader endpoints

All services use network_mode: host — firewall should expose only the ports you need (and ideally over a VPN interface like Tailscale, not the public internet).

Related projects

  • pgscv.dba — pgscv fork with SSH tunnel support (sanitized public version)
  • CHERTS/pgscv — upstream pgscv

License

See LICENSE.

Contributors

DiegoDAF

Issues