Graylog Metrics Reporter Plugins

Home Page: https://www.graylog.org/

License: GNU General Public License v3.0

Java 100.00%

graylog metrics datadog ganglia graphite librato influxdb prometheus statsd csv-export

graylog-plugin-metrics-reporter's Introduction

Graylog Metrics Reporter plugins

ATTENTION: This repository is no longer maintained and has been archived. Please check the metrics documentation for available metric exporters. Graylog server has a built-in Prometheus exporter.

A collection of plugins for reporting internal Graylog metrics to other systems.

Installation

Put the JAR file of the desired metrics reporter plugin into the Graylog plugin directory of each Graylog node and add the relevant configuration settings to their configuration files.

After installing the metrics reporter plugin and adding the configuration settings, Graylog needs to be restarted.

Specific settings for each metrics reporter plugin have been documented in the respective README files:

Development

This project is using Maven 3 and requires Java 8 or higher. The plugin will require Graylog 3.0.0 or higher.

Clone this repository.
Run mvn package to build JAR files of all plugins.
Optional: Run mvn jdeb:jdeb and mvn rpm:rpm to create a DEB and RPM package respectively.
Copy generated JAR file(s) from the "target" directory to your Graylog server plugin directory.
Restart the Graylog server.

License

This library is licensed under the GNU General Public License, Version 3.0.

See https://www.gnu.org/licenses/gpl-3.0.html or the LICENSE.txt file in this repository for the full license text.

graylog-plugin-metrics-reporter's People

Contributors

Stargazers

Watchers

Forkers

desdulianto hc4 annaken comradin gianrubio alcampos croiuki sherzberg saschaschuenemann xing xlogin datoma smartclip jamallmahmoudi mickymiek skilld-labs kveselis isabella232

graylog-plugin-metrics-reporter's Issues

Error 404 report to InfluxDB

I tried to use metrics-reporter-influxdb to export metrics data from graylog2 to influxdb. But the process failed with error:

2016-09-02T14:44:42.876+07:00 WARN [InfluxDbReporter] Unable to report to InfluxDB with error 'Server returned HTTP response code: 404 for URL: http://monit.waruna-group.co.id:8086/write?db
=%2Fshinken&precision=s with content :'Not Found''. Discarding data.

I use graylog-server-2.0.3-1 with metrics-reporter-influxdb-1.2.0.jar (i use the compiled jar file) on Debian Jessie with influxdb server influxdb-0.13.0-1 also on Debian Jessie.

graylog configuration for metrics-reporter-influxdb:

metrics_influxdb_enabled = true
metrics_influxdb_uri = http://shinken:<secret>@monit.example.com:8086/shinken
metrics_influxdb_group_gauges = true

The server.log shows that the database name is wrong (%2Fshinken) it should be shinken. Is my configuration wrong or the problem in on the metrics-reporter-influxdb-1.2.0.jar? I try to install the deb package but it failed with error:

root@logging:~# dpkg -i metrics-reporter-influxdb-1.2.0.deb 
dpkg: error processing archive metrics-reporter-influxdb-1.2.0.deb (--install):
parsing file '/var/lib/dpkg/tmp.ci/control' near line 1:
invalid package name (character ` ' not allowed (only letters, digits and characters `-+._'))
Errors were encountered while processing:
metrics-reporter-influxdb-1.2.0.deb

metrics-reporter-elasticsearch usage

I have added this plugin and restarted the graylog but I don't know how to use it and there is no description mentioned anywhere, as we have in Prometheus plugin,

Page not found for Prometheus metrics endpoint

I guess I'm missing something blindly obvious here, but I just can't get the Prometheus reporter to generate the metrics endpoint.

I've done the following:

Dropped the jar file into the plugin folder
Restarted Graylog server
Checked the plugin is listed in the list of loaded plugins
The graylog server logs reports the following when the plugin is loaded:

2019-02-05 11:31:08,905 INFO : org.graylog2.bootstrap.CmdLineTool - Loaded plugin: Internal Metrics Prometheus Reporter 2.4.0-beta.3 [org.graylog.plugins.metrics.prometheus.MetricsPrometheusReporterMetaData]
2019-02-05 11:31:14,830 INFO : org.graylog2.shared.initializers.PeriodicalsService - Not starting [org.graylog.plugins.metrics.prometheus.PushGatewayPeriodical] periodical. Not configured to run on this node.

Everything looks OK, but when I visit http://my-graylog.com:9000/api/plugins/org.graylog.plugins.metrics.prometheus/metrics all we see is Page not found message and a nice picture of a Gorilla with a banana.

What gives? Searching the Graylog server logs reveals nothing that indicates any problem.

Any suggestions on where to look next in order to diagnose further? We are using Graylog v2.4.0+2115a42, codename Wildwuchs.

Mike

Add docs about metrics-reporter-prometheus

Hello.
Please add a description of the metric values. For metric-reporter-prometheus.
It would also be nice to give a link to some sort of dashboard in grafana and examples of simple alerts. For example, an alert status of log entries.

Add spoof name configuration for ganglia export

Spoof name could be passed as last parameter of to GMetrcic constructor

ES Indexing errors with this plugin

I'm using Graylog 4.11 with version 3.0.0 of the metrics-reporter-gelf plugin running to log metrics back into Graylog. I've done no plugin configuration short of

metrics_gelf_enabled = true

in server.conf.

Most metrics are being logged every 15 seconds as expected but there are obviously some that are being dumped as I have 100k of indexing failures. I've narrowed it down to this plugin by routing all messages from my gelf input into a separate index. The only thing I have generating gelf messages into that input is this plugin. The input only listens on localhost so it isn't outside interference.

Every 5 minutes I get these indexer failures:

Timestamp	Index	Letter ID	Error message
a few seconds ago	gelf_0	0786ab1e-f535-11eb-8a1b-00155d366e62	ElasticsearchException[Elasticsearch exception [type=mapper_parsing_exception, reason=failed to parse field [value] of type [long] in document with id '0786ab1e-f535-11eb-8a1b-00155d366e62'. Preview of field's value: 'Wed Aug 04 15:01:52 UTC 2021']]; nested: ElasticsearchException[Elasticsearch exception [type=illegal_argument_exception, reason=For input string: "Wed Aug 04 15:01:52 UTC 2021"]];
a few seconds ago	gelf_0	0785c096-f535-11eb-8a1b-00155d366e62	ElasticsearchException[Elasticsearch exception [type=mapper_parsing_exception, reason=failed to parse field [value] of type [long] in document with id '0785c096-f535-11eb-8a1b-00155d366e62'. Preview of field's value: '[]']]; nested: ElasticsearchException[Elasticsearch exception [type=illegal_argument_exception, reason=For input string: "[]"]];
a few seconds ago	gelf_0	fe953d41-f534-11eb-8a1b-00155d366e62	ElasticsearchException[Elasticsearch exception [type=mapper_parsing_exception, reason=failed to parse field [value] of type [long] in document with id 'fe953d41-f534-11eb-8a1b-00155d366e62'. Preview of field's value: '[]']]; nested: ElasticsearchException[Elasticsearch exception [type=illegal_argument_exception, reason=For input string: "[]"]];
a few seconds ago	gelf_0	fe95b270-f534-11eb-8a1b-00155d366e62	ElasticsearchException[Elasticsearch exception [type=mapper_parsing_exception, reason=failed to parse field [value] of type [long] in document with id 'fe95b270-f534-11eb-8a1b-00155d366e62'. Preview of field's value: 'Wed Aug 04 15:01:52 UTC 2021']]; nested: ElasticsearchException[Elasticsearch exception [type=illegal_argument_exception, reason=For input string: "Wed Aug 04 15:01:52 UTC 2021"]];
a few seconds ago	gelf_0	f5aafb66-f534-11eb-8a1b-00155d366e62	ElasticsearchException[Elasticsearch exception [type=mapper_parsing_exception, reason=failed to parse field [value] of type [long] in document with id 'f5aafb66-f534-11eb-8a1b-00155d366e62'. Preview of field's value: 'Wed Aug 04 15:01:52 UTC 2021']]; nested: ElasticsearchException[Elasticsearch exception [type=illegal_argument_exception, reason=For input string: "Wed Aug 04 15:01:52 UTC 2021"]];
a few seconds ago	gelf_0	f5aa8648-f534-11eb-8a1b-00155d366e62	ElasticsearchException[Elasticsearch exception [type=mapper_parsing_exception, reason=failed to parse field [value] of type [long] in document with id 'f5aa8648-f534-11eb-8a1b-00155d366e62'. Preview of field's value: '[]']]; nested: ElasticsearchException[Elasticsearch exception [type=illegal_argument_exception, reason=For input string: "[]"]];
a minute ago	gelf_0	ecb82e12-f534-11eb-8a1b-00155d366e62	ElasticsearchException[Elasticsearch exception [type=mapper_parsing_exception, reason=failed to parse field [value] of type [long] in document with id 'ecb82e12-f534-11eb-8a1b-00155d366e62'. Preview of field's value: '[]']]; nested: ElasticsearchException[Elasticsearch exception [type=illegal_argument_exception, reason=For input string: "[]"]];
a minute ago	gelf_0	ecb87c4e-f534-11eb-8a1b-00155d366e62	ElasticsearchException[Elasticsearch exception [type=mapper_parsing_exception, reason=failed to parse field [value] of type [long] in document with id 'ecb87c4e-f534-11eb-8a1b-00155d366e62'. Preview of field's value: 'Wed Aug 04 15:01:52 UTC 2021']]; nested: ElasticsearchException[Elasticsearch exception [type=illegal_argument_exception, reason=For input string: "Wed Aug 04 15:01:52 UTC 2021"]];

My understanding is that GL will have calculated the field types for this input based on the message content and set that as the index's template in ES. Field refresh on this index is set to 5 seconds. I assume that something is being logged with the timestamp in a field which the ES indexer has determined should be a long, and again with something which is [] into a field defined as a long. So I think this could be resolved with a static ES template for this index?

Any suggestions as to how to resolve this gratefully received.

Graylog Internal Metrics should be ingestable into Graylog

Internal graylog metrics should be made available to graylog so that graylog can ingest and process these metrics.

The scenario I am aiming this at is to report on key metrics that graylog generates internally and have them available in a stream for alerting, pipelining and other types of lovely processing that I can do with graylog.

As a "belt and braces" approach to monitoring, I'd measure the value of "Uncommitted Journal Message Count" and action accordingly, however, I'd like to be able to also initiate an alert from within graylog.

While it is obviously "dangerous" to only rely on monitoring on a platform from within the platform that you are measuring, there is still value in doing so. Multiple views of failure allow a robust view of overall environment health to be made.

2.5 support

Can we sill use the 2.4 plugin for 2.5 installations?

Support Tags/Labels

To make the handling of metrics easier it would be nice to have the option to use tags/lables in the metrics. This way it would be easier to work with Grafana and the data given by Graylog.

Current Metrics

org_graylog2_inputs_extractors_GrokExtractor_grok_0e0ac0c0_e7c8_11e6_846a_0050569e5d86_executionTime{quantile="0.5",} 0.0
org_graylog2_inputs_extractors_GrokExtractor_grok_2834e930_1c26_11e6_90bc_0050569e5d86_executionTime{quantile="0.5",} 0.0

Requested Metrics

org_graylog2_inputs_extractors_GrokExtractor_grok_executionTime{quantile="0.5",id="0e0ac0c0_e7c8_11e6_846a_0050569e5d86"} 0.0
org_graylog2_inputs_extractors_GrokExtractor_grok_executionTime{quantile="0.5",id="2834e930_1c26_11e6_90bc_0050569e5d86"} 0.0

Issue with influxdb http output

I'm attempting to use the inflxudb plugin with the http output, but having an issue where the / in the uri is being included in the post url:

2016-05-26 08:01:33,638 WARN : com.izettle.metrics.influxdb.InfluxDbReporter - Unable to report to InfluxDB with error 'Server returned HTTP response code: 404 for URL: http://127.0.0.1:8086/write?db=%2Fgraylog&precision=s with content :'Not Found''. Discarding data.

`metrics_datadog_enabled` filter seems to be ignored

Re-posting this forum post as an issue here:

We’re currently using the Datadog metrics reporter. It’s working fine but we wanted to filter the number of metrics it sends to Graylog. I tried doing that by adding the following line to Graylog config:

metrics_datadog_include_metrics = org.graylog2.shared.journal.KafkaJournal.uncommittedMessages

However, when I added this line and restarted the Graylog server, the filtering did not kick in: all Graylog metrics were sent to Datadog, while I expected only uncommittedMessages to be sent.

I tried this on one more host with the same result. Also, tried with a wildcard pattern like org.graylog2.shared.* -- still all metrics were being sent.

I’ve only 2 other metrics related configuration:

metrics_datadog_enabled = true
metrics_datadog_transport = UDP

I am using the following JAR which does seem to contain the code required for metric filtering: https://github.com/graylog-labs/graylog-plugin-metrics-reporter/releases/download/2.4.0-beta.3/metrics-reporter-datadog-2.4.0-beta.3.jar 1

Add support for AWS Cloudwatch

https://github.com/blacklocus/metrics-cloudwatch

Add support for New Relic

https://github.com/palominolabs/metrics-new-relic

Add support version 2.5.0

Hello,
Could you may to add support version 2.5 Graylog.

metrics-reporter-librato-1.2.1-SNAPSHOT.jar Configuration

Compiled via Maven and got the Jar
metrics-reporter-librato-1.2.1-SNAPSHOT.jar

However - when graylog2 loads we are seeing the following message

2016-05-24T19:22:27.834+08:00 INFO [CmdLineTool] Loaded plugins: [Anonymous Usage Statistics 2.0.0 [org.graylog.plugins.usagestatistics.UsageStatsPlugin], Pipeline Processor Plugin 1.0.0-beta.2 [org.graylog.plugins.pipelineprocessor.ProcessorPlugin], Collector 1.0.0 [org.graylog.plugins.collector.CollectorPlugin], MapWidgetPlugin 1.0.0 [org.graylog.plugins.map.MapWidgetPlugin], Enterprise Integration Plugin 1.0.0 [org.graylog.plugins.enterprise_integration.EnterpriseIntegrationPlugin], Internal Metrics Librato Reporter 1.2.0 [org.graylog.plugins.metrics.librato.MetricsLibratoReporterPlugin]]
2016-05-24T19:22:27.855+08:00 ERROR [CmdLineTool] Invalid configuration
com.github.joschi.jadconfig.ParameterException: Required parameter "metrics_librato_username" not found.
at com.github.joschi.jadconfig.JadConfig.processClassFields(JadConfig.java:127) ~[graylog.jar:?]
at com.github.joschi.jadconfig.JadConfig.process(JadConfig.java:99) ~[graylog.jar:?]
at org.graylog2.bootstrap.CmdLineTool.processConfiguration(CmdLineTool.java:351) [graylog.jar:?]
at org.graylog2.bootstrap.CmdLineTool.readConfiguration(CmdLineTool.java:344) [graylog.jar:?]
at org.graylog2.bootstrap.CmdLineTool.run(CmdLineTool.java:177) [graylog.jar:?]
at org.graylog2.bootstrap.Main.main(Main.java:44) [graylog.jar:?]

I Understand there should be a configuration (file?) - where do we configure it ? as after this error the graylog will not load unless deleting the plugin from the plugins dir

Thanks

add support for kairosDB

it would be great if we implement KairosDB as target for the metrics.

at given time it is only possible to use the graphite output and the graphite input from kairosdb.

Add support for Apache Spark

https://github.com/ippontech/metrics-spark-reporter

metrics-reporter-csv-3.0.1-SNAPSHOT.jar Configuration Issue

Compiled via Maven and got the Jar
metrics-reporter-csv-3.0.1-SNAPSHOT.jar

even added the configuration in /etc/graylog/server/server.conf

metrics_csv_enabled = true
metrics_csv_report_interval = 15s
metrics_csv_directory = /tmp/graymetrics/
metrics_csv_locale =
metrics_csv_unit_rates = seconds
metrics_csv_unit_durations = milliseconds
metrics_csv_include_metrics = .*

I restarted the graylog-server but ends up with graylog-server.service failed.

Thanks.

Could not build JAR files via mvn package command

OS: Ubuntu 20.04.2 LTS
openjdk version "11.0.11" 2021-04-20
OpenJDK Runtime Environment (build 11.0.11+9-Ubuntu-0ubuntu2.20.04)
OpenJDK 64-Bit Server VM (build 11.0.11+9-Ubuntu-0ubuntu2.20.04, mixed mode, sharing)
Apache Maven 3.6.3
Maven home: /usr/share/maven
Java version: 11.0.11, vendor: Ubuntu, runtime: /usr/lib/jvm/java-11-openjdk-amd64

Logs:

[INFO] Compiling 4 source files to /home/ubuntu/graylog-plugin-metrics-reporter/metrics-reporter-core/target/classes
An exception has occurred in the compiler (9-internal). Please file a bug against the Java compiler via the Java bug reporting page (http://bugreport.java.com) after checking the Bug Database (http://bugs.java.com) for duplicates. Include your program and the following diagnostic in your report. Thank you.
java.io.UncheckedIOException: java.nio.file.NotDirectoryException: /usr/lib/jvm/java-11-openjdk-amd64/lib/modules
        at com.sun.tools.javac.file.Locations$BootClassPathLocationHandler.lazy(Locations.java:712)
        at com.sun.tools.javac.file.Locations$BootClassPathLocationHandler.isDefault(Locations.java:546)
        at com.sun.tools.javac.file.Locations.isDefaultBootClassPath(Locations.java:126)
        at com.sun.tools.javac.file.BaseFileManager.isDefaultBootClassPath(BaseFileManager.java:204)
        at com.sun.tools.javac.main.Arguments.validate(Arguments.java:479)
        at com.sun.tools.javac.api.JavacTaskImpl.prepareCompiler(JavacTaskImpl.java:168)
        at com.sun.tools.javac.api.JavacTaskImpl.access$000(JavacTaskImpl.java:69)
        at com.sun.tools.javac.api.JavacTaskImpl$1.call(JavacTaskImpl.java:96)
        at com.sun.tools.javac.api.JavacTaskImpl$1.call(JavacTaskImpl.java:93)
        at com.sun.tools.javac.api.JavacTaskImpl.handleExceptions(JavacTaskImpl.java:128)
        at com.sun.tools.javac.api.JavacTaskImpl.doCall(JavacTaskImpl.java:93)
        at com.sun.tools.javac.api.JavacTaskImpl.call(JavacTaskImpl.java:87)
        at com.google.errorprone.BaseErrorProneJavaCompiler$1.call(BaseErrorProneJavaCompiler.java:99)
        at com.google.errorprone.BaseErrorProneCompiler.run(BaseErrorProneCompiler.java:137)
        at com.google.errorprone.BaseErrorProneCompiler.run(BaseErrorProneCompiler.java:108)
        at com.google.errorprone.ErrorProneCompiler.run(ErrorProneCompiler.java:119)
        at com.google.errorprone.ErrorProneCompiler.compile(ErrorProneCompiler.java:66)
        at org.codehaus.plexus.compiler.javac.errorprone.JavacCompilerWithErrorProne$CompilerInvoker.compile(JavacCompilerWithErrorProne.java:219)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at org.codehaus.plexus.compiler.javac.errorprone.JavacCompilerWithErrorProne.performCompile(JavacCompilerWithErrorProne.java:91)
        at org.apache.maven.plugin.compiler.AbstractCompilerMojo.execute(AbstractCompilerMojo.java:943)
        at org.apache.maven.plugin.compiler.CompilerMojo.execute(CompilerMojo.java:137)
        at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:137)
        at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:210)
        at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:156)
        at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:148)
        at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:117)
        at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:81)
        at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:56)
        at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
        at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:305)
        at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:192)
        at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:105)
        at org.apache.maven.cli.MavenCli.execute(MavenCli.java:957)
        at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:289)
        at org.apache.maven.cli.MavenCli.main(MavenCli.java:193)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:282)
        at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:225)
        at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:406)
        at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:347)
Caused by: java.nio.file.NotDirectoryException: /usr/lib/jvm/java-11-openjdk-amd64/lib/modules
        at java.base/sun.nio.fs.UnixFileSystemProvider.newDirectoryStream(UnixFileSystemProvider.java:431)
        at java.base/java.nio.file.Files.newDirectoryStream(Files.java:472)
        at java.base/java.nio.file.Files.list(Files.java:3699)
        at com.sun.tools.javac.file.Locations$BootClassPathLocationHandler.systemClasses(Locations.java:665)
        at com.sun.tools.javac.file.Locations$BootClassPathLocationHandler.computePath(Locations.java:621)
        at com.sun.tools.javac.file.Locations$BootClassPathLocationHandler.lazy(Locations.java:709)
        ... 46 more
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for Graylog Metrics Reporter Plugin (Parent POM) 3.0.1-SNAPSHOT:
[INFO]
[INFO] Graylog Metrics Reporter Plugin (Parent POM) ....... SUCCESS [ 28.961 s]
[INFO] Graylog Metrics Reporter Core Classes .............. FAILURE [  2.772 s]
[INFO] Graylog Metrics Console Reporter Plugin ............ SKIPPED
[INFO] Graylog Metrics CSV Reporter Plugin ................ SKIPPED
[INFO] Graylog Metrics JMX Reporter Plugin ................ SKIPPED
[INFO] Graylog Metrics SLF4J Reporter Plugin .............. SKIPPED
[INFO] Graylog Metrics Ganglia Reporter Plugin ............ SKIPPED
[INFO] Graylog Metrics Graphite Reporter Plugin ........... SKIPPED
[INFO] Graylog Metrics Cassandra Reporter Plugin .......... SKIPPED
[INFO] Graylog Metrics Datadog Reporter Plugin ............ SKIPPED
[INFO] Graylog Metrics Elasticsearch Reporter Plugin ...... SKIPPED
[INFO] Graylog Metrics GELF Reporter Plugin ............... SKIPPED
[INFO] Graylog Metrics InfluxDB Reporter Plugin ........... SKIPPED
[INFO] Graylog Metrics Librato Reporter Plugin ............ SKIPPED
[INFO] Graylog Metrics MongoDB Reporter Plugin ............ SKIPPED
[INFO] Graylog Metrics OpenTSDB Reporter Plugin ........... SKIPPED
[INFO] Graylog Metrics Prometheus Reporter Plugin ......... SKIPPED
[INFO] Graylog Metrics statsd Reporter Plugin ............. SKIPPED
[INFO] Graylog Metrics CloudWatch Reporter Plugin ......... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  32.289 s
[INFO] Finished at: 2021-05-03T15:42:10Z
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.6.1:compile (default-compile) on project metrics-reporter-core: Compilation failure -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <args> -rf :metrics-reporter-core

Provide unauthenticated endpoint for Prometheus

Hi,

Would it be possible to provide an unauthenticated endpoint for the Prometheus exporter?

We are running graylog inside kubernetes, where the authentication doesn't play nice with Prometheus' kubernetes service discovery and would like to avoid having a custom configuration just for graylog.

Thanks

Generic metric names for prometheus

The prometheus plugin uses metric names like this:

org_graylog2_inputs_gelf_tcp_GELFTCPInput_5cc1bd20057c5200116dfdef_open_connections
org_graylog2_lookup_adapters_5cc1bc798e510600109570d9_refresh_count
org_graylog2_lookup_caches_5ca37df31210da0011143090_hits_total

It is really painful to write alarms for metrics like this. It would much better if the metrics would have names like this:

org_graylog2_inputs_open_connections {module: gelf_tcp, input_id: 5cc1bd20057c5200116dfde}
org_graylog2_lookup_adapters_refresh_count {adapter_id: 5cc1bc798e510600109570d9}
org_graylog2_lookup_caches_hits_total {cache_id: 5ca37df31210da0011143090}

This would allow us to specify generic prometheus alarms and generic grafana dashboards.

Host name 'repo.maven.apache.org' does not match the certificate subject provided by the peer

When doing a mvn package I get the following error:

[ERROR] Failed to execute goal on project metrics-reporter-parent: Could not resolve dependencies for project org.graylog.plugins:metrics-reporter-parent:pom:1.4.1-SNAPSHOT: Failed to collect dependencies at org.graylog2:graylog2-server:jar:2.0.0 -> org.glassfish.jersey.ext:jersey-bean-validation:jar:2.22.1 -> org.hibernate:hibernate-validator:jar:5.1.3.Final: Failed to read artifact descriptor for org.hibernate:hibernate-validator:jar:5.1.3.Final: Could not transfer artifact org.jboss.shrinkwrap:shrinkwrap-bom:pom:1.0.1 from/to central (https://repo.maven.apache.org/maven2): Host name 'repo.maven.apache.org' does not match the certificate subject provided by the peer (CN=repo1.maven.org, O="Sonatype, Inc", L=Fulton, ST=MD, C=US) -> [Help 1]

I found some other links and have tried their solutions to no avail. http://stackoverflow.com/questions/40682932/maven-certificate-broken-for-ubuntu-16-04

Any ideas?

Some buggy Prometheus metics?

Looks like the metric with name jvm_memory_non_heap_committed is constantly growing instead of being the correspond to real 'non-heap memory commited in bytes'.
Also the metric jvm_memory_non_heap_usage seems to be always grow to negative side instead of indidcating 'some real usage number between 0 and 1'.

Prometheus - not all provided information are exposed (pipeline rules)

Not all available metrics available in the UI are exposed via prometheus for the debugging of the processing pipeline rules.

Available

Exposed

0.5, 0.75, 0.95, 0.98, 0.99, 0.999 and count

org_graylog_plugins_pipelineprocessor_ast_Rule_58f74b8ecb08a070f6e8f144_58f74b20cb08a070f6e8f0ce_1_trace_evaluate_duration Generated from Dropwizard metric import 

(metric=org.graylog.plugins.pipelineprocessor.ast.Rule.58f74b8ecb08a070f6e8f144.58f74b20cb08a070f6e8f0ce.1.trace.evaluate.duration, type=com.codahale.metrics.Timer)

org_graylog_plugins_pipelineprocessor_ast_Rule_58f74b8ecb08a070f6e8f144_58f74b20cb08a070f6e8f0ce_1_trace_evaluate_duration summary org_graylog_plugins_pipelineprocessor_ast_Rule_58f74b8ecb08a070f6e8f144_58f74b20cb08a070f6e8f0ce_1_trace_evaluate_duration{quantile="0.5",} 6.294E-6 

org_graylog_plugins_pipelineprocessor_ast_Rule_58f74b8ecb08a070f6e8f144_58f74b20cb08a070f6e8f0ce_1_trace_evaluate_duration{quantile="0.75",} 9.741E-6 

org_graylog_plugins_pipelineprocessor_ast_Rule_58f74b8ecb08a070f6e8f144_58f74b20cb08a070f6e8f0ce_1_trace_evaluate_duration{quantile="0.95",} 1.1951000000000001E-5 

org_graylog_plugins_pipelineprocessor_ast_Rule_58f74b8ecb08a070f6e8f144_58f74b20cb08a070f6e8f0ce_1_trace_evaluate_duration{quantile="0.98",} 1.3784E-5 

org_graylog_plugins_pipelineprocessor_ast_Rule_58f74b8ecb08a070f6e8f144_58f74b20cb08a070f6e8f0ce_1_trace_evaluate_duration{quantile="0.99",} 1.4817E-5 

org_graylog_plugins_pipelineprocessor_ast_Rule_58f74b8ecb08a070f6e8f144_58f74b20cb08a070f6e8f0ce_1_trace_evaluate_duration{quantile="0.999",} 3.3044000000000004E-5 org_graylog_plugins_pipelineprocessor_ast_Rule_58f74b8ecb08a070f6e8f144_58f74b20cb08a070f6e8f0ce_1_trace_evaluate_duration_count 705.0

proper tagged metrics for datadog

today the datadog version send plain old statsd events to the datadog backend

Example 1

graylog_performance.org.graylog2.inputs.extractors.RegexExtractor.regex.cee156be_a210_40b4_8533_efead31f420b.completeExecutionTime.stddev

graylog_performance.org.graylog2.inputs.extractors.RegexExtractor.regex.completeExecutionTime.stddev

with tag

regex_id=cee156be_a210_40b4_8533_efead31f420b instead

Example 2

graylog_performance.org.graylog2.inputs.gelf.tcp.GELFTCPInput.5902685b9b54a94e0e4f76f6.total_connections

graylog_performance.org.graylog2.inputs.gelf.tcp.GELFTCPInput.total_connections

with tag

output_id=5902685b9b54a94e0e4f76f6

so one unique statsd key, and encode the variance in named tags - see datadog datagram format

Outputting things the way they are now, make them close to useless in the datadog UI, and will take up ~11k metrics (you pay per metric)

datadog plugin setup help??

I cant' seem to get the datadog metrics plugin to show up as a configurable input in the Graylog UI. Let me know if i've missed/misconfigured something.

Before building the .jar, I configured the "metrics-reporter-datadog" plugin by setting the following values in MetricsDatadogReporterConfiguration.java:

metrics_datadog_enabled = true
metrics_datadog_api_key = "myDatadogAPIkey"
metrics_datadog_hostname = "hostnameOfMyGraylogServer"

i left all other values to be default

Once i built the .jar with "mvn package", i placed it in /usr/share/graylog-server/plugin on my Graylog server and performed a "service graylog-server restart" , but i still do not see the ability to configure the plugin under "System/Inputs"

Other Info:

im running graylog-server 3.0.0
my version of maven is 3.6.2
i have Graylog in a cluster of 2 EC2 nodes
both nodes are sized r4.large

What am i missing here?

graylog v3.3.0 compatibility

Is this plugin compatible with graylog v3.3.0 ?

Add support for regex MetricFilter

The Dropwizard Metrics library supports filtering metrics for reporters using the MetricFilter interface.

The plugins provided by this project should support a metrics filter based on regular expressions against the metric name.

Add support for Zabbix

https://github.com/hengyunabc/metrics-zabbix

Plugin does not load without core jar deployed

A reporter plugin is supposed to load without the metrics-reporter-core-x.x.x.jar in the plugins/ folder but it does not. This is 100% reproducible on my setup and two customers had the same issue but not on @joschi's setup.

lennart@graylog-srv01:/usr/share/graylog-server/plugin$ unzip -l metrics-reporter-datadog-1.4.0.jar | grep org/graylog/plugins/metrics/core
        0  2016-10-28 15:25   org/graylog/plugins/metrics/core/
     1491  2016-10-28 15:25   org/graylog/plugins/metrics/core/BasePluginMetaData.class
        0  2016-10-28 15:25   org/graylog/plugins/metrics/core/jadconfig/
     2708  2016-10-28 15:25   org/graylog/plugins/metrics/core/jadconfig/StringMapConverter.class
     3594  2016-10-28 15:25   org/graylog/plugins/metrics/core/jadconfig/PatternListConverter.class
     1423  2016-10-28 15:25   org/graylog/plugins/metrics/core/RegexMetricFilter.class

Plugin version 1.4.0
Graylog 2.1.2+50e449a
openjdk version "1.8.0_111"

Graylog 3.0 support

will it be updated to work on 3.0 ?

ApiError HTTP 404 not found

I have configured latest version of metrics-reporter-prometheus[3.0.0] plugin for graylog server which is hosted on docker setup,the plugin is working fine for graylog version 3.0.0 but for version 3.0.1 it's throwing following exception when i try to access end points[/api/plugins/org.graylog.plugins.metrics.prometheus/metrics]
{
type: "ApiError",
message: "HTTP 404 Not Found"
}
Any work around to resolve this thing.

NoSuchMethodError: getHostText()

Hi, all metric reporters using the getHostText() are failing.

Exception in thread "main" java.lang.NoSuchMethodError: com.google.common.net.HostAndPort.getHostText()Ljava/lang/String;
at org.graylog.plugins.metrics.statsd.providers.StatsdProvider.get(StatsdProvider.java:39)

graylog: 2.4.3
java: 1.8.0_161

graylog-labs / graylog-plugin-metrics-reporter Goto Github PK