Comments (3)
Here's the dynamic template generated for this index (and therefore this plugin's messages because nothing else logs to that input)
$ curl -X GET "localhost:9200/_template/gelf-template?pretty=true"
{
"gelf-template" : {
"order" : -1,
"index_patterns" : [
"gelf_*"
],
"settings" : {
"index" : {
"analysis" : {
"analyzer" : {
"analyzer_keyword" : {
"filter" : "lowercase",
"tokenizer" : "keyword"
}
}
}
}
},
"mappings" : {
"_source" : {
"enabled" : true
},
"dynamic_templates" : [
{
"internal_fields" : {
"mapping" : {
"type" : "keyword"
},
"match_mapping_type" : "string",
"match" : "gl2_*"
}
},
{
"store_generic" : {
"mapping" : {
"type" : "keyword"
},
"match_mapping_type" : "string"
}
}
],
"properties" : {
"gl2_processing_timestamp" : {
"format" : "uuuu-MM-dd HH:mm:ss.SSS",
"type" : "date"
},
"gl2_accounted_message_size" : {
"type" : "long"
},
"gl2_receive_timestamp" : {
"format" : "uuuu-MM-dd HH:mm:ss.SSS",
"type" : "date"
},
"full_message" : {
"fielddata" : false,
"analyzer" : "standard",
"type" : "text"
},
"streams" : {
"type" : "keyword"
},
"source" : {
"fielddata" : true,
"analyzer" : "analyzer_keyword",
"type" : "text"
},
"message" : {
"fielddata" : false,
"analyzer" : "standard",
"type" : "text"
},
"timestamp" : {
"format" : "uuuu-MM-dd HH:mm:ss.SSS",
"type" : "date"
}
}
},
"aliases" : { }
}
}
The only field which is a [long] is gl2_accounted_message_size. So is this plugin causing that field to sometimes contain the timestamp or a null value?
from graylog-plugin-metrics-reporter.
I have resolved this myself, via https://community.graylog.org/t/graylog-metrics-plugin-feeding-data-via-gelf-to-graylog-causing-parsing-errors/16356/3
Most of the values for metrics are numbers so Graylog/ES correctly decide to store the "value" field as a [long]. However there are 2 metrics (at the time of writing):
org.graylog2.journal.oldest-segment
jvm.threads.deadlocks
where the value is either a string (timestamp) or a collection/array. Obviously this data will not go in a field with the type of long. The graylog community URL above provides a solution but only for 1 specific metric. I've put the GELF metrics input through a pipeline with the following rule, which has resolved the errors for me and should work as new metrics are added which are not numeric:
Rule "Cleanup: Non-numeric metrics value field"
when
has_field("value") AND
not is_long("value")
then
rename_field(
old_field: "value",
new_field: "value_string"
);
end
from graylog-plugin-metrics-reporter.
That didn't appear to be working either, but this does. Can't spend any more time on it right now, but if anyone else is having the same problem this will fix it.
Rule "Cleanup: Non-numeric metrics value field"
when
has_field("name") AND
has_field("value") AND
(to_string($message.name) == "org.graylog2.journal.oldest-segment" OR
to_string($message.name) == "jvm.threads.deadlocks")
then
let value_string = to_string($message.value);
set_field ("value_string",value_string);
remove_field("value");
end
from graylog-plugin-metrics-reporter.
Related Issues (20)
- proper tagged metrics for datadog HOT 2
- NoSuchMethodError: getHostText() HOT 2
- Support Tags/Labels HOT 3
- Provide unauthenticated endpoint for Prometheus HOT 4
- `metrics_datadog_enabled` filter seems to be ignored HOT 2
- 2.5 support HOT 1
- Page not found for Prometheus metrics endpoint HOT 3
- Add support version 2.5.0
- Graylog 3.0 support HOT 5
- Generic metric names for prometheus HOT 1
- metrics-reporter-elasticsearch usage HOT 1
- datadog plugin setup help?? HOT 4
- Some buggy Prometheus metics?
- Prometheus - not all provided information are exposed (pipeline rules) HOT 1
- graylog v3.3.0 compatibility HOT 2
- Add docs about metrics-reporter-prometheus
- metrics-reporter-csv-3.0.1-SNAPSHOT.jar Configuration Issue
- ApiError HTTP 404 not found
- Could not build JAR files via mvn package command
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from graylog-plugin-metrics-reporter.