JMX metrics -> push gateway? about jmx_exporter

[brian-brazil]

Have you looked into the service discovery that was added in Prometheus 0.14.0? It'll let you dynamically configure what to monitor, and will produce overall better results than indirecting via a push gateway.

[beorn7]

In general, to do Prometheus monitoring "right", you want to have an HTTP server on the targets to monitor. Everything else is band aid at best. Or in other words: If you cannot have that HTTP server on most of your targets, I would simply not recommend Prometheus.

The JMX exporter will easily create a whole lot of metrics, and the Pushgateway is not meant to scale to a lot of metrics pushed from many different targets. For one, the one big scrape the Prometheus server has to perform, gets really big then. Also, you are creating another management chore: If one of your monitoring targets dies, what to do with the metrics it has pushed to the Pushgateway? They will just stick around forever, or you have to create another job that deletes metrics from dead targets...

[curtisallen]

@brian-brazil yeah I noticed consul was added with the release last week. May end up going that route. @beorn7 thanks for the comments, didn't consider push gateway metrics clutter before.
Thanks!