hengyunabc / metrics-kafka Goto Github PK
View Code? Open in Web Editor NEWDropwizard Metrics reporter for kafka.
License: Apache License 2.0
Dropwizard Metrics reporter for kafka.
License: Apache License 2.0
Hi
I select your API for exporting metrics to kafka and I would like to know if it would be easy to add some tags to each message.
I think it will be convient to add a method withTags for example to the KafkaReporter (like opentsdb reporter api
https://github.com/sps/metrics-opentsdb
OpenTsdbReporter.forRegistry(environment.metrics())
.prefixedWith(environment.getName())
.withTags(ImmutableMap.of("other", "tags")) // static tags included with every metric
// .withBatchSize(10) // optional batching. unbounded by default. likely need to tune this.
.build(opentsdb)
.start(15L, TimeUnit.SECONDS); // tune your reporting interval
KafkaReporter kafkaReporter = KafkaReporter.forRegistry(metrics). .withTags(ImmutableMap.of("other", "tags")).config(config).topic(topic).hostName(hostName).prefix(prefix).build()
And json result to ::
{
"tags": {
"key1":"value1",
"key2":"value2"
},
"timers": {
"test.test-timer": {
"count": 43,
"max": 505.33599999999996,
"mean": 502.585391215306,
"min": 500.191,
"p50": 502.443,
"p75": 504.046,
"p95": 505.291,
"p98": 505.33599999999996,
"p99": 505.33599999999996,
"p999": 505.33599999999996,
"stddev": 1.6838970975560197,
"m15_rate": 0.8076284847453551,
"m1_rate": 0.8883929708459906,
"m5_rate": 0.8220236458023953,
"mean_rate": 0.9799289583409866,
"duration_units": "milliseconds",
"rate_units": "calls/second"
}
},
"durationUnit": "milliseconds",
"meters": {},
"clock": 1453287302764,
"hostName": "localhost",
"rateUnit": "second",
"histograms": {
"test.response-sizes": {
"count": 43,
"max": 142,
"mean": 123.29413148075862,
"min": 100,
"p50": 124,
"p75": 134,
"p95": 141,
"p98": 142,
"p99": 142,
"p999": 142,
"stddev": 12.28197980813012
}
},
I am newbie on kafka maybe tags can be set another way but it is a proposition
Hi,
In the line#69 of KafkaReporter.java, a single thread executor service is created to make the Kafka send operation asynchronously. While its working queue is unbounded.
But the ScheduleReporter of Dropwizard already had a scheduled executor to report the metrics periodically and asynchronously.
If the default metrics poll interval is 10 seconds so that the Dropwizard schedule reporter will call the report() method of Kafka reporter every 10 seconds.
If the Kafka send operation can complete in 10 seconds then there won't have any problem because there is no pending task in the working queue of the single thread executor service.
But if the Kafka send operation CANNOT complete in 10 seconds, then there will be more and more pending tasks queued up in the unbounded working queue of single thread executor service along with the time. Finally, the unbounded working queue will occupy all the heap memory.
If any problem, please let me know.
Thanks
Gary.Guo
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.