onedr0p / exportarr

AIO Prometheus Exporter for Sabnzbd, Bazarr, Prowlarr, Lidarr, Readarr, Radarr, and Sonarr

Exportarr

AIO Prometheus Exporter for Sonarr, Radarr, Lidarr, Prowlarr, Readarr, Bazarr and Sabnzbd

Note: This exporter will not gather metrics from all apps at once. You will need an exportarr instance for each app. Be sure to see the examples below for more information.

Usage

Docker Compose

See examples in the examples/compose directory.

Kubernetes

See examples in the examples/kubernetes directory.

Docker CLI

Replace $app, $port and $apikey with one of the support apps, port and api key

# PORT must be unique across all Exportarr instances
docker run --name exportarr_$app \
  -e PORT=9707 \
  -e URL="http://x.x.x.x:$port" \
  -e APIKEY="$apikey" \
  --restart unless-stopped \
  -p 9707:9707 \
  -d ghcr.io/onedr0p/exportarr:latest $app

Visit http://127.0.0.1:9707/metrics to see the app metrics

CLI

Replace $app, $port and $apikey with one of the support apps, port and api key

./exportarr $app --help

# --port must be unique across all Exportarr instances
./exportarr $app \
  --port 9707 \
  --url "http://x.x.x.x:$port" \
  --api-key "$apikey"

Visit http://127.0.0.1:9707/metrics to see the app metrics

Configuration

Environment Variable	CLI Flag	Description	Default	Required
PORT	--port or -p	The port Exportarr will listen on		✅
URL	--url or -u	The full URL to Sonarr, Radarr, or Lidarr		✅
API_KEY	--api-key or -a	API Key for Sonarr, Radarr or Lidarr		❌
API_KEY_FILE	--api-key-file	API Key file location for Sonarr, Radarr or Lidarr		❌
CONFIG	--config or -c	Path to Sonarr, Radarr or Lidarr's config.xml (advanced)		❌
INTERFACE	--interface or -i	The interface IP Exportarr will listen on	0.0.0.0	❌
LOG_LEVEL	--log-level or -l	Set the default Log Level	INFO	❌
DISABLE_SSL_VERIFY	--disable-ssl-verify	Set to true to disable SSL verification	false	❌
AUTH_PASSWORD	--auth-password	Set to your basic or form auth password		❌
AUTH_USERNAME	--auth-username	Set to your basic or form auth username		❌
FORM_AUTH	--form-auth	Use Form Auth instead of basic auth	false	❌
ENABLE_ADDITIONAL_METRICS	--enable-additional-metrics	Set to true to enable gathering of additional metrics (slow)	false	❌
ENABLE_UNKNOWN_QUEUE_ITEMS	--enable-unknown-queue-items	Set to true to enable gathering unknown queue items	false	❌
PROWLARR__BACKFILL	--backfill	Set to true to enable backfill of historical metrics	false	❌
PROWLARR__BACKFILL_SINCE_DATE	--backfill-since-date	Set a date from which to start the backfill	1970-01-01 (epoch)	❌

Prowlarr Backfill

The Prowlarr collector is a little different than other collectors as it's hitting an actual "stats" endpoint, collecting counters of events that happened in a small time window, rather than getting all-time statistics like the other collectors. This means that by default, when you start the Prowlarr collector, collected stats will start from that moment (all counters will start from zero).

To backfill all Prowlarr Data, either use PROWLARR__BACKFILL or --backfill.

Note that the first request can be extremely slow, depending on how long your Prowlarr instance has been running. You can also specify a start date to limit the backfill if the backfill is timing out:

PROWLARR__BACKFILL_DATE_SINCE=2023-03-01 or --backfill-date-since=2023-03-01

exportarr's Issues

Integrate Bazarr

Hi, it's your biggest fan again. I like this new approach of centralizing everything :)

I think it would be nice to have also metrics for bazarr :)

exec user process caused "no such file or directory"

Since version 0.3.1 I'm getting the following error, also in master:

standard_init_linux.go:207: exec user process caused "no such file or directory"

0.3.0 works fine (but I have to specify the port :P). Maybe compiling to a different architecture or something? I'm running in a quite standard Ubuntu Server 18.04 LTS

Radarr exporter uses too many resources

Details

Radarr exporter uses too many resources compared to Sonarr and Lidarr exporters. My movie collection is not that big. Maybe you can take a look at the endpoints you are calling and optimize something.

• Radarr => 739 Movies
• Sonarr => 132 Series 6450 Episodes

exportarr: v1.1.0
ENABLE_ADDITIONAL_METRICS: false
ENABLE_UNKNOWN_QUEUE_ITEMS: false

My fix for now is to configure a huge scrape_interval in Prometheus.

  - job_name: 'radarr-exporter'
    scrape_interval: 600s
    static_configs:
      - targets: ['192.168.1.191:9708']

Update:
After I changed the scrape_interval, I can see Radarr is using less resources too:

Radarr - Logs reporting startup on Sonarr port 9708

Details

What steps did you take and what happened:

Followed the Readme to install Docker exportarr_radarr instance

sudo docker run --name exportarr_radarr \
  -e port=9708 \
  -e URL="http://192.168.nn.nn:7878" \
  -e APIKEY="0dxxxxxxxxxd7" \
  --restart unless-stopped \
  -p 9708:9708 \
  -d ghcr.io/onedr0p/exportarr:latest exportarr radarr

Viewed the logs with
**sudo docker logs --follow exportarr_radarr**
output was
**time="2021-03-21T21:32:17Z" level=info msg="Listening on 0.0.0.0:9707"**
What did you expect to happen:

Expected
time="2021-03-21T21:32:17Z" level=info msg="Listening on 0.0.0.0:**9708**"

Anything else you would like to add:

Issuing a curl command returned
curl -X GET http://127.0.0.1:9708/metrics

result was
curl: (56) Recv failure: Connection reset by peer

Additional Information:

So it appears the radarr metrics were started correctly.

Update flag `--enable-additional-metric` should be `--enable-additional-metrics`

Details

What steps did you take and what happened:

Looks like I missed the change on this, but we should update the flag to be --enable-additional-metrics instead of --enable-additional-metric

Update Grafana Dashboard to include Lidarr

Update Grafana Dashboard to include Sonarr and Lidarr.

Here is mine just in case it's useful for someone.
grafana_dashboard.txt

My config:

• Exportarr v1.1.0
• Sonarr 3.0.8.1507
• Radarr 4.1.0.6175
• Lidarr 1.0.2.2592

401 on metrics gathering

Details

What steps did you take and what happened:

I configured exportarr to gather metrics of sonarr and radarr. Both are getting a 401 when asking for metrics.
I have sonarr and radarr configurad with security.

and I have added the environment variables for basic auth.

What did you expect to happen:

Gathering metrics wo a problem.

lower request frequency

Hi,
It seems that exportarr it's polling the service (radarr or sonarr) every 10 seconds. Since the library status won't change that often, do you think it's possible to lower the requests frequency? Maybe using a variable to allow a custom setting?

I'm asking because for a huge library /radarr/api/v3/movie could take even more than 10 sec to be retrieved. Having that many calls it's slowly breaking the app.

Thanks

Instability causes service to die

Details

What steps did you take and what happened:

I have a couple exportarr docker images set up with sonarr and radarr in a docker compose file.
Sometimes, radarr or sonarr fail a request, and return an empty response, and exportarr just logs a fatal error and goes down. Radarr/sonarr recover, but exportarr is still dead. Trying to reload the whole lot with docker-compose down and docker-compose up -d doesn't restart exportarr containers.

What did you expect to happen:

It should log an error, maybe wait a moment, and then try again next time.
Or the docker image should restart it when it fails, but that then means you need a watcher and all sorts.

Anything else you would like to add:

This is the log line in question, but the fatal means it shuts down the exportarr service, which does not restart.

Additional Information:

It seems a bit silly for it to report a radarr_system_status/sonarr_system_status metric, if at the first sign of trouble, exportarr dies.

Provide default port

Any reason why you don't provide a default port? I'd be happy to stick to whatever you choose but I don't like having to write it twice on the k8s manifest, one for the envar and one for the ports section. What do you think?

Support reading config.xml to obtain api key and base url

In Kubernetes this container can run in the same pod as the *arr container.

It would be neat to allow exportarr to read from a path containing the config.xml to extract the API key and base url for configuration.

Automatically retrieve API key

Hi there! Love the project ❤️

It seems like Sonarr and Radarr provide their API keys in the response headers for any request as X-Api-Key. Is it possible to fetch the API key from the provided endpoint so we don't need to pass it as an environment variable? This would allow for unattended installations.

Handle Multiple API Keys

Details

Describe the solution you'd like:

I've got two instances of Radarr and one of Sonarr running in a Kubernetes cluster. It would be nice to be able to only have to run 1 deployment to monitor these. Right now I have to run 3.

Anything else you would like to add:

Additional Information:

I understand this would cause some issues with existing dashboards but I feel like that is out of this projects concern.

Create a latest image tag

I got some issues with the docker-compose

1. First issue with Enable Quality = True
   ERROR: The Compose file './docker-compose.yml' is invalid because:
   services.sonarr-exporter.environment.ENABLE_EPISODE_QUALITY_METRICS contains true, which is an invalid type, it should be a string, number, or a null

2. Second issue with Enable Quality = false
   ERROR: The Compose file './docker-compose.yml' is invalid because:
   services.sonarr-exporter.environment.ENABLE_EPISODE_QUALITY_METRICS contains false, which is an invalid type, it should be a string, number, or a null

3. The manifest was also not found
   Pulling sonarr-exporter (onedr0p/exportarr:latest)...
   ERROR: manifest for onedr0p/exportarr:latest not found

Add a Prowlarr integration

Details

Prowlarr aggregates metrics on the clients it uses and can report hit rates etc. It would be nice to be able to pipe these into prometheus and query with Grafana.

Anything else you would like to add:

For reference https://prowlarr.com

Additional Information:

Can't add Metrics to Grafana as a Data Source

Details

What steps did you take and what happened:

1. Selected Prometheus as a Data Source
2. Filled out my URL (http://pie.home:9707/metrics) directing to the Sonarr Exportarr Container
3. Pressed Save&Test
4. Error appeared:

Error reading Prometheus: bad_response: readObjectStart: expect { or n, but found <, error found in #1 byte of ...|<h1>Exporta|..., bigger context ...|<h1>Exportarr</h1><p><a href='/metrics'>metrics</a>|...)

What did you expect to happen:
That it can be added as a Data Source

Anything else you would like to add:
/

Additional Information:
Using Docker Containers

Make IncludeUnknownXItems in Queue metric an option

Would be nice to set this as a configuration option, default can be turned off.

Envar: ENABLE_UNKNOWN_QUEUE_ITEMS="false"
Flag: --enable-unknown-queue-items

Handle empty UrlBase when reading from config

Hi, I use a reverse proxy and handle all my k8s ingress as subdomains (e.g. sonarr.something.com). As such my UrlBase remains empty. I am not proficient in go but I think if we were able to check for a null/empty value here we could account for these scenarios.

Currently I see these errors in my logs which lead me to believe the url is simply malformed.

sonarr [Info] Auth: Auth-Unauthorized ip 127.0.0.1 url 'http://localhost:8989//api/v3/history'

exportarr/internal/client/client.go

Lines 47 to 65 in a2a6624

	// Use the values from config.xml if using the config flag
	if c.config.String("config") != "" {
	url = fmt.Sprintf("%s:%s%s/api/%s/%s",
	c.config.String("url"),
	c.configFile.Port,
	c.configFile.UrlBase,
	apiVersion,
	endpoint,
	)
	apiKey = c.configFile.ApiKey
	} else {
	// Otherwise use the value provided in the api-key flag
	url = fmt.Sprintf("%s/api/%s/%s",
	c.config.String("url"),
	apiVersion,
	endpoint,
	)
	apiKey = c.config.String("api-key")
	}

Unable to retrieve data into Grafana

Hi,

Thanks for the work on this exporter.
I have a issue when I try to use the dashboard Radarr V3 or Sonarr V3 in Grafana with Exportarr set up.
I can see the data on the url http://smetric:9707/metrics where smetric is the docker Exportarr configurated for my instance of Sonarr. I set the Data Source URL to http://smetric:9707/metrics but Grafana still return the HTML code of the homepage :

{
  "request": {
    "url": "api/datasources/proxy/3/api/v1/query?query=sonarr_history_total&time=1605645283",
    "method": "GET",
    "hideFromInspector": false
  },
  "response": "<h1>Radarr Exporter</h1><p><a href='/metrics'>metrics</a></p>"
}

The problems seems to be related to the homepage of Exportarr which is located on http://smetric:9707

Do you have a idea to resolve this issue ?
Thanks

Exportarr on docker swarm

I've been trying to get exportarr working within a stack in my swarm. This is the snippet of the docker-compose.yml

     ...
   radarr_exporter:
     image: onedr0p/exportarr:master
     command: ["exportarr", "radarr"]
     environment:
         PORT: 9811
         URL: "http://radarr:7878"
         APIKEY: ""
     depends_on:
         - radarr
     networks:
         - radarr
         - monitor
     deploy:
         replicas: 1
    ...

The container starts and dies and all i get in the log is:

No help topic for 'exportarr'

Could you give me a hint on whats going on? At first i thought that command is not taken into account while deploying the stack but i couldn't find that info on the docker reference.

Thanks

Sonarr additionla info

It will be great if you can grab also few additional details into the sonarr/ radarr grabber like - missing episodes / movies, downloaded today / last week ; downloading and etc. In this case I can (and beleave that this is valid not only for me) retire varken, as it's not support and brings a lot of issues (Radarr is not working with V3 of the API, using InfluxDB 1.8 and etc.).

Add prowlarr integration

Details

Describe the solution you'd like:

Prowlarr is slowly becoming the jackett and nzbhydra2 replacement but it's still in alpha.

Error retrieving statistics - status 303

Details

What steps did you take and what happened:
I simply installed via docker. When I go to the metrics page I get a fatal error when it tries to read the queue HTTP request.

2023-01-18 23:19:05 time="2023-01-19T05:19:05Z" level=info msg="Sending HTTP request to http://192.168.1.20:8989/api/v3/queue?page=1"
2023-01-18 23:19:06 time="2023-01-19T05:19:06Z" level=fatal msg="An error has occurred during retrieving statistics HTTP status 303 See Other (Location: /sonarr/api/v3/queue?page=1)"

What did you expect to happen:
My guess is the page=1 is appending ?APIKEY, where the ? is already used and it needs to append &APIKEY OR remove the page=1 entirely and keep the ?APIKEY. Just an assumption. Not sure how others aren't getting this.

Anything else you would like to add:
The result is the same with or without ENABLE_ADDITIONAL_METRICS and ENABLE_UNKNOWN_QUEUE_ITEMS and DISABLE_SSL_VERIFY set to true. I set the logs to debug and there isn't any more relevant info. I have no auth on my Sonarr. I have quadruple checked my api key is correct. My Sonarr version is 3.0.6.1196.

I've also tried v1.1 and master to account for my base url but both give me the same error.

Here's all I have:

2023-01-18 23:29:01 time="2023-01-19T05:29:01Z" level=info msg="Listening on 0.0.0.0:9707"
2023-01-18 23:29:09 time="2023-01-19T05:29:09Z" level=debug msg="172.17.0.1:60440 GET /\n"
2023-01-18 23:29:09 time="2023-01-19T05:29:09Z" level=debug msg="172.17.0.1:60440 GET /favicon.ico\n"
2023-01-18 23:29:10 time="2023-01-19T05:29:10Z" level=debug msg="172.17.0.1:60442 GET /metrics\n"
2023-01-18 23:29:10 time="2023-01-19T05:29:10Z" level=info msg="Sending HTTP request to http://192.168.1.20:8989/api/v3/rootfolder"
2023-01-18 23:29:10 time="2023-01-19T05:29:10Z" level=info msg="Sending HTTP request to http://192.168.1.20:8989/api/v3/health"
2023-01-18 23:29:10 time="2023-01-19T05:29:10Z" level=info msg="Sending HTTP request to http://192.168.1.20:8989/api/v3/series"
2023-01-18 23:29:10 time="2023-01-19T05:29:10Z" level=info msg="Sending HTTP request to http://192.168.1.20:8989/api/v3/system/status"
2023-01-18 23:29:10 time="2023-01-19T05:29:10Z" level=info msg="Sending HTTP request to http://192.168.1.20:8989/api/v3/queue?page=1&includeUnknownSeriesItems=true"
2023-01-18 23:29:10 time="2023-01-19T05:29:10Z" level=info msg="Sending HTTP request to http://192.168.1.20:8989/api/v3/history"
2023-01-18 23:29:10 time="2023-01-19T05:29:10Z" level=fatal msg="An error has occurred during retrieving statistics HTTP status 303 See Other (Location: /sonarr/api/v3/series)"
2023-01-18 23:29:13 time="2023-01-19T05:29:13Z" level=info msg="Listening on 0.0.0.0:9707"

support docker secrets

Details

I am running exportarr as a sidecart to sonarr/radarr/prowlarr in my home stack and I would love to be able to specify the APIKEY via Docker Secrets. ( I am using Docker Swarm to manage my nodes. ) Currently it is impossible to specify the APIKEY other than plaintext environment variable.

I have seen the example for k8s ( https://github.com/onedr0p/exportarr/blob/master/examples/kubernetes/sonarr-exporter.yaml#L77 ), but to my knowledge, Docker does not allow secrets to be loaded as environment variables. They are mounted in the container under the /run/secrets path. Which also causes a problem to be read, because the hardcoded path is under /etc ( https://github.com/onedr0p/exportarr/blob/master/cmd/exportarr/main.go#L261 ).

Describe the solution you'd like:

I would like the Docker Secret file location to be specified in an environment variable so exportarr can read it from there. Linuxserver uses the "FILE__" prefix for any environment variable in their framework. However, I assume the cli parser that is used in the project is not Docker specific ( https://cli.urfave.org/v1/examples/flags/#values-from-files ), so an alternative would be to specify these extra environment variables for sensitive parameters and introduce extra logic where they are used.

Anything else you would like to add:

Additional Information:

Prowler not working (No help topic for 'prowlarr')

Details

docker run --name exportarr_prowlarr \
  -e PORT=9710 \
  -e URL="http://100.84.114.64:9696" \
  -e APIKEY="xxxxxxx" \
  --restart unless-stopped \
  -p 9709:9709 \
  -d ghcr.io/onedr0p/exportarr:v1.0.0 prowlarr

When running this it doesn't work and the docker log just shows No help topic for 'prowlarr'. And the metrics are not accessible
Sonarr and Radar work fine

What did you expect to happen:

no error, metrics accessible

Anything else you would like to add:

Additional Information:
Ubuntu 22.04, Docker version 20.10.18, build b40c2f6

Add Readarr integration

Details

Describe the solution you'd like:

Integrate Readarr since its part of the "arr" apps but for reading.

Anything else you would like to add:

No API documentation but the API is very similar to Lidarr, see https://github.com/Readarr/Readarr/tree/develop/src/Readarr.Api.V1

Additional Information:

• https://readarr.com/

Refactor additional metrics into one setting

Details

Describe the solution you'd like:

I think we should flatten the following options into a global ENABLE_ADDITIONAL_METRICS flag and label it as (slow) in the docs.

• ENABLE_EPISODE_QUALITY_METRICS
• ENABLE_SONG_QUALITY_METRICS
• ENABLE_MONITORED_ALBUMS_METRICS

This will make the config a bit lighter and make it so we don't need different flags for different services, downside is the replacement envar/flag will enable all additional metrics if a service has more than one (like Lidarr)

UrlBase Usage Issue

Details

Including a UrlBase value is causing and issue with the URL generation missing a "/". (Tested with sonarr and radarr)

What steps did you take and what happened:

1. Include a URL base.

2. The "/" is stripped when being saved to config.xml
   <Config> <LogLevel>info</LogLevel> <UpdateMechanism>Docker</UpdateMechanism> <BindAddress>*</BindAddress> <EnableSsl>False</EnableSsl> <SslCertPath></SslCertPath> <Port>7878</Port> <UrlBase>radarr</UrlBase> <ApiKey>...</ApiKey> <AuthenticationMethod>None</AuthenticationMethod> <SslPort>9898</SslPort> <LaunchBrowser>True</LaunchBrowser> <Branch>master</Branch> <SslCertPassword></SslCertPassword> </Config>

3. The program crashes due to invalid url
   |   | 2022-03-30 10:07:54 | panic: runtime error: invalid memory address or nil pointer dereference   |   | 2022-03-30 10:07:54 | time="2022-03-30T00:07:54Z" level=info msg="Sending HTTP request to http://localhost:7878radarr/api/v3/movie"   |   | 2022-03-30 10:07:54 | time="2022-03-30T00:07:54Z" level=info msg="Sending HTTP request to http://localhost:7878radarr/api/v3/health"   |   | 2022-03-30 10:07:44 | time="2022-03-30T00:07:44Z" level=info msg="Listening on 0.0.0.0:9793"   |   | 2022-03-30 10:07:44 | time="2022-03-30T00:07:44Z" level=info msg="Getting Config from /config/config.xml"   |   | 2022-03-30 10:05:04 | /go/pkg/mod/github.com/prometheus/[email protected]/prometheus/registry.go:538 +0xb4d   |   | 2022-03-30 10:05:04 | created by github.com/prometheus/client_golang/prometheus.(*Registry).Gather   |   | 2022-03-30 10:05:04 | /go/pkg/mod/github.com/prometheus/[email protected]/prometheus/registry.go:446 +0x102   |   | 2022-03-30 10:05:04 | github.com/prometheus/client_golang/prometheus.(*Registry).Gather.func1()   |   | 2022-03-30 10:05:04 | /build/internal/collector/radarr/movie.go:93 +0x157   |   | 2022-03-30 10:05:04 | github.com/onedr0p/exportarr/internal/collector/radarr.(*radarrCollector).Collect(0xc0000c4d20, 0xc000045f60)   |   | 2022-03-30 10:05:04 | /build/internal/client/client.go:76 +0x6ed   |   | 2022-03-30 10:05:04 | github.com/onedr0p/exportarr/internal/client.(*Client).DoRequest(0xc0001f0440, {0x91039d, 0x5}, {0x85aea0, 0xc0000acf90})   |   | 2022-03-30 10:05:04 | goroutine 29 [running]:

What did you expect to happen:

It's unclear to me why this issue hasn't come up for anyone else. I'm assuming the UrlBase value is being saved with a "/" for others.

At some point before or at this line of code, a "/" should be checked for and added if necessary.

exportarr/internal/client/client.go

Lines 47 to 57 in a582d45

	// Use the values from config.xml if using the config flag
	if c.config.String("config") != "" {
	url = fmt.Sprintf("%s:%s%s/api/%s/%s",
	c.config.String("url"),
	c.configFile.Port,
	c.configFile.UrlBase,
	apiVersion,
	endpoint,
	)
	apiKey = c.configFile.ApiKey
	} else {

This will fix the URL to be http://localhost:7878/radarr/api/v3/health instead of http://localhost:7878radarr/api/v3/health

Sonarr configuration isn't rendering metrics

Details

What steps did you take and what happened:
Installed Docker Sonarr
docker ps linuxserver/sonarr "/init" 3 years ago Up 9 hours 0.0.0.0:8989->8989/tcp sonarr

Installed exportarr_sonarr (as per Readme)
ghcr.io/onedr0p/exportarr:master "/sbin/tini -- expor…" 8 seconds ago Up 7 seconds 0.0.0.0:9707->9707/tcp exportarr_sonarr

Tried both configurations

1. with API key specified

sudo docker run --name exportarr_sonarr \
  -e port=9707 \
  -e URL="http://192.168.nn.nn:8989" \
  -e APIKEY="53xxxxxxxxxxxxxxxxxxx2b" \
  -e LOG_LEVEL="DEBUG" \
  --restart unless-stopped \
  -p 9707:9707 \
  -d ghcr.io/onedr0p/exportarr:master exportarr sonarr

2. with config.xml referenced

sudo docker run --name exportarr_sonarr   -e port=9707   -e URL="http://192.168.nn.nn:8989" \
 -e CONFIG=/home/docker/sonarr/config/config.xml \
 -e LOG_LEVEL="DEBUG"   --restart unless-stopped   -p 9707:9707   \
-d ghcr.io/onedr0p/exportarr:master exportarr sonarr

All start okay,
time="2021-03-21T21:44:38Z" level=info msg="Listening on 0.0.0.0:9707"
the home page would render,

$> curl -X GET localhost:9707
<h1>Exportarr</h1><p><a href='/metrics'>metrics</a></p>

time="2021-03-21T21:46:37Z" level=debug msg="172.17.0.1:53210 GET /\n"

but the metrics page returns 404 in the logs

curl -X GET localhost:9707/metrics
curl: (52) Empty reply from server

but the logs report

time="2021-03-21T21:47:33Z" level=debug msg="172.17.0.1:53232 GET /metrics\n"
time="2021-03-21T21:47:33Z" level=info msg="Sending HTTP request to http://192.168.nn.nn:8989/api/v3/history"
time="2021-03-21T21:47:33Z" level=info msg="Sending HTTP request to http://192.168.nn.nn:8989/api/v3/rootfolder"
time="2021-03-21T21:47:33Z" level=info msg="Sending HTTP request to http://192.168.nn.nn:8989/api/v3/queue?page=1"
time="2021-03-21T21:47:33Z" level=info msg="Sending HTTP request to http://192.168.nn.nn:8989/api/v3/health"
time="2021-03-21T21:47:33Z" level=info msg="Sending HTTP request to http://192.168.nn.nn:8989/api/v3/system/status"
time="2021-03-21T21:47:33Z" level=info msg="Sending HTTP request to http://192.168.nn.nn:8989/api/v3/series"
time="2021-03-21T21:47:33Z" level=fatal msg="An error has occurred during retrieving statistics HTTP statuscode 404"

What did you expect to happen:

I expected the metrics page to render in the ui or return a valid response in the curl command

Anything else you would like to add:

The Sonarr instance is running for.3 years, no issues with it and the exportarr renders the home page so I suspect there is something between the connection of the two docker containers but I cannot see what it is

Additional Information:

Build and publish exportarr binaries to Github releases

Sorry, I'm new to Prometheus and node_exporter. I've downloaded your zip and also via the git clone, but nowhere in the resulting files is there any executable or make file.
I'm not using docker, just ubuntu 20.04.
The README.md says that to run it from CLI:

### Run from the CLI

```cmd
./exportarr --help
./exportarr sonarr --help
./exportarr radarr --help

Sonarr

./exportarr sonarr \
  --port 9707 \
  --url http://127.0.0.1:8989 \
  --api-key amlmndfb503rfqaa5ln5hj5qkmu3hy18 \
  --enable-episode-quality-metrics

But there's no "./exportarr" anywhere.

How do I use this with Prometheus?

WDYT? Refactoring to use `InvalidMetric` on errors, rather than `log.Fatal`

Details

Describe the solution you'd like:

I'd love to discuss making the following two changes (I'm glad to do the work, but want @onedr0p's blessing first, since I know I'm suggesting a refactor which will result in a slight change in behavior):

1. Add a <service>_scrape_duration_seconds metric to capture some of the timing information we're currently logging to debug
2. Move from log.Fatal to NewInvalidMetric for registering errors in scrapes

Anything else you would like to add:

I was thinking about #49, and while I think the docker hang-ups are likely not related to exportarr, it might be being exacerbated by log.Fatal calls (which cause the process to exit 1 and then be restarted by docker when errors are thrown). The prometheus docs recommend that when a collector hits an error collecting metrics from an endpoint. In the go prometheus library, we have InvalidMetric which can be passed back over the Collect channel to do this (there's a corresponding InvalidDesc which can be used for the Describe Channel).

I've been using this pattern on other exporters, and I've been playing around with it on a branch for exportarr. Any objection to me doing some refactoring to pass errors back to the collector so we can throw them with InvalidMetric?

Additional Information:

Docs for Metrics about Scrapes:
https://prometheus.io/docs/instrumenting/writing_exporters/#metrics-about-the-scrape-itself

Docs for Failed Scrapes:
https://prometheus.io/docs/instrumenting/writing_exporters/#failed-scrapes

Add statistics for indexers

Details

Describe the solution you'd like:
I would like a solution to compare indexers and check what indexer performs the best

RFC: Add support for sabnzbd

Details

Describe the solution you'd like:

@rtrox I saw you were working on an exporter for sabnzbd. I was wondering if you would want to include that in this project? I don't want to steal your thunder there and you've been excellent at helping out. Maybe this could open the door to supporting qBittorrent or other non-arr apps as well since a lot of these media-stack related applications will never have built-in exporters.

I am fine keeping the scope of this project just for the *arr apps but curious what you think moving forward.

Prowlarr Backfill: Add option to specify backfill duration

Details

Describe the solution you'd like:

With prowlarr activated option backfill it can happen, that the exporter times out or prowlarr crashes, because of too many items in history.
It would be nice to introduce an environment variable or cli flag to specify an amount of days to request, like you can do on the prowlarr stats page itself.

the added environment variable or cli flag should specify the amount of days to request.
so from that exportarr should request prowlarr with the backwards calculated amount of days from now, specified in the cli or environment variable

docker-compose examples are out of date

Details

What steps did you take and what happened:

Attempted to setup lidarr exporter using the docker-compose example.

What did you expect to happen:

Container should start and stay running. Metrics should be available for scraping. Instead the container exited immediately with the following:

lidarr_prom_exporter    | No help topic for 'exportarr'
lidarr_prom_exporter exited with code 3

Anything else you would like to add:

I believe the following change is needed.

- command: ["exportarr", "lidarr"]
+ command: ["lidarr"]

I'm guessing this change may be valid for the other *arr setups as well.

Lidarr exporter starts and runs as expected after above change.

api-version is required to not be empty

Details

What steps did you take and what happened:

After upgrading to exportarr v1.2, the following error is shown on startup:
ApiVersion:
required: api-version is required to not be empty

What did you expect to happen:

exportarr starts as normal.

Anything else you would like to add:

Checked config.xml for Sonarr and Radarr and no ApiVersion element exists
Radarr v4.4.2.6956
Sonarr v4.0.0.401

Additional Information:

Refactor: Switch to zap logging library

Details

Describe the solution you'd like:

logrus is currently in maintenance mode, and zap looks to be a good replacement.

Exportarr crashloops with a 404 trying to talk to radarr

My exportarr instance is crashlooping with these logs (in reverse chronological order, sorry, haven't figured out how to copy logs out of Graylog properly yet):

time="2020-04-17T16:20:23Z" level=fatal msg="An error has occurred during retrieving statistics HTTP statuscode 404"
time="2020-04-17T16:20:23Z" level=info msg="Sending HTTP request to http://radarr.multimedia:7878/api/v3/history"
time="2020-04-17T16:20:23Z" level=info msg="Sending HTTP request to http://radarr.multimedia:7878/api/v3/queue?page=1&includeUnknownMovieItems=true"
time="2020-04-17T16:20:23Z" level=info msg="Sending HTTP request to http://radarr.multimedia:7878/api/v3/health"
time="2020-04-17T16:20:23Z" level=info msg="Sending HTTP request to http://radarr.multimedia:7878/api/v3/rootfolder"
time="2020-04-17T16:20:23Z" level=info msg="Sending HTTP request to http://radarr.multimedia:7878/api/v3/system/status"
time="2020-04-17T16:20:23Z" level=info msg="Sending HTTP request to http://radarr.multimedia:7878/api/v3/movie"
time="2020-04-17T16:20:02Z" level=info msg="Listening on 0.0.0.0:9708"

I'm running this with https://github.com/billimek/billimek-charts/tree/master/charts/radarr, versions:

• linuxserver/radarr:v0.2.0.1480-ls58
• onedr0p/exportarr:v0.3.0

Rename the metric lidarr_songs_missing_total

The metric lidarr_songs_missing_total should be called lidarr_albums_missing_total

My config:

• Exportarr v1.1.0
• Sonarr 3.0.8.1507
• Radarr 4.1.0.6175
• Lidarr 1.0.2.2592

Add Lidarr integration

I currently don't run Lidarr, but plan to.

Include metric sonarr_episode_monitored_total

These metrics are available:

• sonarr_series_monitored_total
• sonarr_season_monitored_total

But this metric is missing:

• sonarr_episode_monitored_total

Refactor: Switch to koanf & cobra for config & CLI library

Details

Describe the solution you'd like:

https://github.com/knadh/koanf

Include queued metic for Sonarr and Radarr

Lidarr exposes this metric:

• lidarr_queue_total

I think it will be useful to have the same metrics in Sonarr and Radarr.

My config:

• Exportarr v1.1.0
• Sonarr 3.0.8.1507
• Radarr 4.1.0.6175
• Lidarr 1.0.2.2592

Grafana Dashboards

I lost my previous ones due to a data disaster, but I'll get new ones posted when I can

{
  "annotations": {
    "list": [
      {
        "builtIn": 1,
        "datasource": "-- Grafana --",
        "enable": true,
        "hide": true,
        "iconColor": "rgba(0, 211, 255, 1)",
        "name": "Annotations & Alerts",
        "type": "dashboard"
      }
    ]
  },
  "editable": true,
  "gnetId": null,
  "graphTooltip": 0,
  "id": 30,
  "links": [],
  "panels": [
    {
      "datasource": "Prometheus",
      "fieldConfig": {
        "defaults": {
          "custom": {},
          "displayName": "status",
          "mappings": [
            {
              "from": "",
              "id": 1,
              "operator": "",
              "text": "Online",
              "to": "",
              "type": 1,
              "value": "1"
            },
            {
              "from": "",
              "id": 2,
              "operator": "",
              "text": "Offline",
              "to": "",
              "type": 1,
              "value": "null"
            },
            {
              "from": "",
              "id": 3,
              "operator": "",
              "text": "Offline",
              "to": "",
              "type": 1,
              "value": "0"
            }
          ],
          "thresholds": {
            "mode": "absolute",
            "steps": [
              {
                "color": "rgb(255, 194, 48)",
                "value": null
              }
            ]
          }
        },
        "overrides": []
      },
      "gridPos": {
        "h": 5,
        "w": 7,
        "x": 0,
        "y": 0
      },
      "id": 2,
      "options": {
        "colorMode": "value",
        "graphMode": "none",
        "justifyMode": "center",
        "orientation": "auto",
        "reduceOptions": {
          "calcs": [
            "last"
          ],
          "fields": "",
          "values": false
        },
        "textMode": "auto"
      },
      "pluginVersion": "7.1.1",
      "targets": [
        {
          "expr": "radarr_system_status OR vector(0)",
          "format": "table",
          "instant": false,
          "legendFormat": "",
          "refId": "A"
        }
      ],
      "timeFrom": null,
      "timeShift": null,
      "title": "",
      "type": "stat"
    },
    {
      "datasource": "Prometheus",
      "description": "",
      "fieldConfig": {
        "defaults": {
          "custom": {},
          "displayName": "in queue",
          "mappings": [
            {
              "from": "",
              "id": 2,
              "operator": "",
              "text": "0",
              "to": "",
              "type": 1,
              "value": "null"
            }
          ],
          "thresholds": {
            "mode": "absolute",
            "steps": [
              {
                "color": "rgb(255, 194, 48)",
                "value": null
              }
            ]
          },
          "unit": "locale"
        },
        "overrides": []
      },
      "gridPos": {
        "h": 5,
        "w": 4,
        "x": 7,
        "y": 0
      },
      "id": 4,
      "options": {
        "colorMode": "value",
        "graphMode": "none",
        "justifyMode": "center",
        "orientation": "auto",
        "reduceOptions": {
          "calcs": [
            "lastNotNull"
          ],
          "fields": "",
          "values": false
        },
        "textMode": "auto"
      },
      "pluginVersion": "7.1.1",
      "targets": [
        {
          "expr": "sum(radarr_queue_total) OR vector(0)",
          "format": "table",
          "instant": false,
          "interval": "",
          "legendFormat": "",
          "refId": "A"
        }
      ],
      "timeFrom": null,
      "timeShift": null,
      "title": "",
      "type": "stat"
    },
    {
      "datasource": "Prometheus",
      "fieldConfig": {
        "defaults": {
          "custom": {},
          "displayName": "missing",
          "mappings": [],
          "thresholds": {
            "mode": "absolute",
            "steps": [
              {
                "color": "rgb(255, 194, 48)",
                "value": null
              }
            ]
          },
          "unit": "locale"
        },
        "overrides": []
      },
      "gridPos": {
        "h": 5,
        "w": 4,
        "x": 11,
        "y": 0
      },
      "id": 5,
      "options": {
        "colorMode": "value",
        "graphMode": "none",
        "justifyMode": "center",
        "orientation": "auto",
        "reduceOptions": {
          "calcs": [
            "lastNotNull"
          ],
          "fields": "",
          "values": false
        },
        "textMode": "auto"
      },
      "pluginVersion": "7.1.1",
      "targets": [
        {
          "expr": "radarr_movie_missing_total OR vector(0)",
          "format": "table",
          "instant": false,
          "interval": "",
          "legendFormat": "",
          "refId": "A"
        }
      ],
      "timeFrom": null,
      "timeShift": null,
      "title": "",
      "type": "stat"
    },
    {
      "datasource": "Prometheus",
      "fieldConfig": {
        "defaults": {
          "custom": {},
          "displayName": "wanted",
          "mappings": [],
          "thresholds": {
            "mode": "absolute",
            "steps": [
              {
                "color": "rgb(255, 194, 48)",
                "value": null
              }
            ]
          },
          "unit": "locale"
        },
        "overrides": []
      },
      "gridPos": {
        "h": 5,
        "w": 4,
        "x": 15,
        "y": 0
      },
      "id": 3,
      "options": {
        "colorMode": "value",
        "graphMode": "none",
        "justifyMode": "center",
        "orientation": "auto",
        "reduceOptions": {
          "calcs": [
            "last"
          ],
          "fields": "",
          "values": false
        },
        "textMode": "auto"
      },
      "pluginVersion": "7.1.1",
      "targets": [
        {
          "expr": "radarr_movie_wanted_total OR vector(0)",
          "format": "table",
          "instant": false,
          "legendFormat": "",
          "refId": "A"
        }
      ],
      "timeFrom": null,
      "timeShift": null,
      "title": "",
      "type": "stat"
    },
    {
      "datasource": "Prometheus",
      "fieldConfig": {
        "defaults": {
          "custom": {},
          "displayName": "movies",
          "mappings": [
            {
              "from": "",
              "id": 1,
              "operator": "",
              "text": "0",
              "to": "",
              "type": 1,
              "value": "null"
            }
          ],
          "thresholds": {
            "mode": "absolute",
            "steps": [
              {
                "color": "rgb(255, 194, 48)",
                "value": null
              }
            ]
          },
          "unit": "locale"
        },
        "overrides": []
      },
      "gridPos": {
        "h": 5,
        "w": 7,
        "x": 0,
        "y": 5
      },
      "id": 6,
      "options": {
        "colorMode": "value",
        "graphMode": "none",
        "justifyMode": "center",
        "orientation": "auto",
        "reduceOptions": {
          "calcs": [
            "lastNotNull"
          ],
          "fields": "",
          "values": false
        },
        "textMode": "auto"
      },
      "pluginVersion": "7.1.1",
      "targets": [
        {
          "expr": "radarr_movie_total OR vector(0)",
          "format": "table",
          "instant": false,
          "intervalFactor": 1,
          "legendFormat": "",
          "refId": "A"
        }
      ],
      "timeFrom": null,
      "timeShift": null,
      "title": "",
      "type": "stat"
    },
    {
      "datasource": "Prometheus",
      "fieldConfig": {
        "defaults": {
          "custom": {},
          "displayName": "monitored",
          "mappings": [
            {
              "from": "",
              "id": 1,
              "operator": "",
              "text": "0",
              "to": "",
              "type": 1,
              "value": "null"
            }
          ],
          "thresholds": {
            "mode": "absolute",
            "steps": [
              {
                "color": "rgb(255, 194, 48)",
                "value": null
              }
            ]
          },
          "unit": "locale"
        },
        "overrides": []
      },
      "gridPos": {
        "h": 5,
        "w": 4,
        "x": 7,
        "y": 5
      },
      "id": 10,
      "options": {
        "colorMode": "value",
        "graphMode": "none",
        "justifyMode": "center",
        "orientation": "auto",
        "reduceOptions": {
          "calcs": [
            "lastNotNull"
          ],
          "fields": "",
          "values": false
        },
        "textMode": "auto"
      },
      "pluginVersion": "7.1.1",
      "targets": [
        {
          "expr": "radarr_movie_monitored_total OR vector(0)",
          "format": "table",
          "instant": false,
          "intervalFactor": 1,
          "legendFormat": "",
          "refId": "A"
        }
      ],
      "timeFrom": null,
      "timeShift": null,
      "title": "",
      "type": "stat"
    },
    {
      "datasource": "Prometheus",
      "fieldConfig": {
        "defaults": {
          "custom": {},
          "displayName": "d/l past week",
          "mappings": [
            {
              "from": "",
              "id": 1,
              "operator": "",
              "text": "0",
              "to": "",
              "type": 1,
              "value": "null"
            }
          ],
          "thresholds": {
            "mode": "absolute",
            "steps": [
              {
                "color": "rgb(255, 194, 48)",
                "value": null
              }
            ]
          },
          "unit": "locale"
        },
        "overrides": []
      },
      "gridPos": {
        "h": 5,
        "w": 4,
        "x": 11,
        "y": 5
      },
      "id": 7,
      "options": {
        "colorMode": "value",
        "graphMode": "none",
        "justifyMode": "center",
        "orientation": "auto",
        "reduceOptions": {
          "calcs": [
            "lastNotNull"
          ],
          "fields": "",
          "values": false
        },
        "textMode": "auto"
      },
      "pluginVersion": "7.1.1",
      "targets": [
        {
          "expr": "sum(increase(radarr_movie_downloaded_total[7d])) OR vector(0)",
          "format": "table",
          "instant": false,
          "interval": "",
          "legendFormat": "",
          "refId": "A"
        }
      ],
      "timeFrom": null,
      "timeShift": null,
      "title": "",
      "type": "stat"
    },
    {
      "aliasColors": {},
      "bars": true,
      "dashLength": 10,
      "dashes": false,
      "datasource": "Prometheus",
      "fieldConfig": {
        "defaults": {
          "custom": {},
          "links": []
        },
        "overrides": []
      },
      "fill": 1,
      "fillGradient": 0,
      "gridPos": {
        "h": 10,
        "w": 19,
        "x": 0,
        "y": 10
      },
      "hiddenSeries": false,
      "id": 13,
      "legend": {
        "alignAsTable": false,
        "avg": false,
        "current": false,
        "hideEmpty": false,
        "hideZero": false,
        "max": false,
        "min": false,
        "rightSide": false,
        "show": false,
        "sideWidth": null,
        "total": true,
        "values": true
      },
      "lines": false,
      "linewidth": 1,
      "nullPointMode": "null",
      "percentage": false,
      "pluginVersion": "7.1.1",
      "pointradius": 2,
      "points": false,
      "renderer": "flot",
      "seriesOverrides": [],
      "spaceLength": 10,
      "stack": false,
      "steppedLine": false,
      "targets": [
        {
          "expr": "sort_desc(sum(radarr_movie_quality_total) by (quality))",
          "format": "time_series",
          "instant": true,
          "legendFormat": "{{quality}}",
          "refId": "A"
        }
      ],
      "thresholds": [],
      "timeFrom": null,
      "timeRegions": [],
      "timeShift": null,
      "title": "",
      "tooltip": {
        "shared": false,
        "sort": 0,
        "value_type": "individual"
      },
      "type": "graph",
      "xaxis": {
        "buckets": null,
        "mode": "series",
        "name": null,
        "show": true,
        "values": [
          "total"
        ]
      },
      "yaxes": [
        {
          "decimals": null,
          "format": "locale",
          "label": "movies by quality",
          "logBase": 1,
          "max": null,
          "min": null,
          "show": true
        },
        {
          "format": "short",
          "label": null,
          "logBase": 1,
          "max": null,
          "min": null,
          "show": false
        }
      ],
      "yaxis": {
        "align": false,
        "alignLevel": 3
      }
    },
    {
      "datasource": "loki",
      "fieldConfig": {
        "defaults": {
          "custom": {}
        },
        "overrides": []
      },
      "gridPos": {
        "h": 11,
        "w": 19,
        "x": 0,
        "y": 20
      },
      "id": 15,
      "options": {
        "showLabels": false,
        "showTime": false,
        "sortOrder": "Descending",
        "wrapLogMessage": true
      },
      "targets": [
        {
          "expr": "{app_kubernetes_io_name=\"radarr\"}",
          "refId": "A"
        }
      ],
      "timeFrom": null,
      "timeShift": null,
      "title": "",
      "type": "logs"
    }
  ],
  "refresh": "1m",
  "schemaVersion": 26,
  "style": "dark",
  "tags": [],
  "templating": {
    "list": []
  },
  "time": {
    "from": "now-2d",
    "to": "now"
  },
  "timepicker": {
    "refresh_intervals": [
      "10s",
      "30s",
      "1m",
      "5m",
      "15m",
      "30m",
      "1h",
      "2h",
      "1d"
    ]
  },
  "timezone": "",
  "title": "Radarr",
  "uid": "A8iPssjZk",
  "version": 28
}

Fix goreleaser builds

Details

The latest release had an issue building on Darwin/arm64

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

• fix(deps): update module github.com/knadh/koanf/providers/file to v1

Detected dependencies

dockerfile

Dockerfile

• golang 1.22.4-alpine

github-actions

.github/actions/docker-image/action.yaml

• docker/metadata-action v5
• docker/setup-qemu-action v3
• docker/setup-buildx-action v3
• docker/login-action v3
• docker/build-push-action v6

.github/actions/go-release/action.yaml

• actions/setup-go v5
• goreleaser/goreleaser-action v6

.github/actions/lint/action.yaml

• actions/setup-go v5
• golangci/golangci-lint-action v6

.github/workflows/on-merge.yml

• actions/checkout v4
• actions/checkout v4
• actions/checkout v4

.github/workflows/on-pr.yml

• actions/checkout v4
• actions/checkout v4

.github/workflows/on-release.yml

• actions/checkout v4
• actions/checkout v4
• actions/checkout v4
• actions/checkout v4

gomod

go.mod

• go 1.19
• github.com/gookit/validate v1.5.2
• github.com/knadh/koanf/providers/confmap v0.1.0
• github.com/knadh/koanf/providers/env v0.1.0
• github.com/knadh/koanf/providers/file v0.1.0
• github.com/knadh/koanf/providers/posflag v0.1.0
• github.com/knadh/koanf/v2 v2.1.1
• github.com/prometheus/client_golang v1.19.1
• github.com/spf13/cobra v1.8.1
• github.com/spf13/pflag v1.0.5
• github.com/stretchr/testify v1.9.0
• go.uber.org/zap v1.27.0
• golang.org/x/sync v0.7.0

---

• Check this box to trigger a request for Renovate to run again on this repository

sonarr_episode_quality_total missing in sonarr metrics

Details

What steps did you take and what happened:

I have checked the metrics endpoint for sonarr.
There was the sonarr_episode_quality_total missing

What did you expect to happen:

sonarr_episode_quality_total existing

sonarr_episode_quality_total - Missing from metrics

Details

What steps did you take and what happened:
Installed per the directions and scraped data. sonarr_episode_quality_total is not displayed at 127.0.0.1:9707/metrics. I do see radarr_episode_quality_total at 127.0.0.1:9708/metrics. I see code for it in series.go. Not sure why it is not being exported the metrics page.

What did you expect to happen:
Expected sonarr_episode_quality_total to be displayed at 127.0.0.1:9707/metrics.

Anything else you would like to add:
Running the latest docker image and Sonarr version 3.0.6.1471.

Additional Information:

Sonarr Episodes Qualiy

Details

I'm trying to get episodes quality, but even that I have line into the environment settings in docker_compose - episodes info is not retrived.

  ENABLE_EPISODE_QUALITY_METRICS: "true"

Add metric for missing episodes in Sonarr

Would be nice to have a metric for missing episode in Sonarr, like we already have for movies in Radarr: radarr_movie_missing_total.

Awesome work btw, thank you!