anas-aso / thanos_downloader

A helper application to use Prometheus TSDB data blocks uploaded by Thanos Sidecar component with a vanilla Prometheus setup.

Thanos Downloader

A helper application to use Prometheus TSDB data blocks uploaded by Thanos Sidecar component with a vanilla Prometheus setup.

The goal of this project is to allow for a simple use of Thanos Sidecar as a backup service for Prometheus data, and be able to use that data with a vanilla Prometheus setup without the need for the remaining Thanos components.

Configuration

Thanos Downloader uses the same configuration format used by Thanos Sidecar (e.g https://thanos.io/storage.md/#gcs). The available configuration flags can be found as below :

$ thanos_downloader --help
usage: thanos_downloader --interval.start=INTERVAL.START --interval.end=INTERVAL.END [<flags>]

Flags:
  --help                            Show context-sensitive help (also try --help-long and --help-man).
  --data.path="prom_data"           target data directory path.
  --interval.start=INTERVAL.START   start time of the requested interval in Unix time format (seconds).
  --interval.end=INTERVAL.END       end time of the requested interval in Unix time format (seconds).
  --config.path="config.yaml"       path to the Thanos format config file.
  --version                         Show application version.

How does it work

1. get list of blocks in the provided bucket
2. filter all blocks that satisfies the requested time range
3. get the block with the highest resolution and most samples count for each time interval. This is needed since the uploaded blocks might be overlapping due to Prometheus HA setups and/or Thanos downsampling.

Docker

The application is available as a docker image :

docker run --rm -it anasaso/thanos_downloader:latest --help

Examples

• How-To Docker
• How-To Kubernetes