GithubHelp home page GithubHelp logo

metrics-kafka's People

Contributors

fungrim avatar hengyunabc avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

metrics-kafka's Issues

Is a easy way to add some tags/labels

Hi

I select your API for exporting metrics to kafka and I would like to know if it would be easy to add some tags to each message.
I think it will be convient to add a method withTags for example to the KafkaReporter (like opentsdb reporter api

https://github.com/sps/metrics-opentsdb

OpenTsdbReporter.forRegistry(environment.metrics())
                    .prefixedWith(environment.getName())
                    .withTags(ImmutableMap.of("other", "tags")) // static tags included with every metric
                    // .withBatchSize(10) // optional batching. unbounded by default. likely need to tune this.
                    .build(opentsdb)
                    .start(15L, TimeUnit.SECONDS); // tune your reporting interval
KafkaReporter kafkaReporter = KafkaReporter.forRegistry(metrics). .withTags(ImmutableMap.of("other", "tags")).config(config).topic(topic).hostName(hostName).prefix(prefix).build()

And json result to ::

{
   "tags": {
       "key1":"value1",
       "key2":"value2"
    },
    "timers": {
        "test.test-timer": {
            "count": 43,
            "max": 505.33599999999996,
            "mean": 502.585391215306,
            "min": 500.191,
            "p50": 502.443,
            "p75": 504.046,
            "p95": 505.291,
            "p98": 505.33599999999996,
            "p99": 505.33599999999996,
            "p999": 505.33599999999996,
            "stddev": 1.6838970975560197,
            "m15_rate": 0.8076284847453551,
            "m1_rate": 0.8883929708459906,
            "m5_rate": 0.8220236458023953,
            "mean_rate": 0.9799289583409866,
            "duration_units": "milliseconds",
            "rate_units": "calls/second"
        }
    },
    "durationUnit": "milliseconds",
    "meters": {},
    "clock": 1453287302764,
    "hostName": "localhost",
    "rateUnit": "second",
    "histograms": {
        "test.response-sizes": {
            "count": 43,
            "max": 142,
            "mean": 123.29413148075862,
            "min": 100,
            "p50": 124,
            "p75": 134,
            "p95": 141,
            "p98": 142,
            "p99": 142,
            "p999": 142,
            "stddev": 12.28197980813012
        }
    },

I am newbie on kafka maybe tags can be set another way but it is a proposition

memory issue in kafkareporter when backend kafka service is unavailable

Hi,
In the line#69 of KafkaReporter.java, a single thread executor service is created to make the Kafka send operation asynchronously. While its working queue is unbounded.
But the ScheduleReporter of Dropwizard already had a scheduled executor to report the metrics periodically and asynchronously.

If the default metrics poll interval is 10 seconds so that the Dropwizard schedule reporter will call the report() method of Kafka reporter every 10 seconds.
If the Kafka send operation can complete in 10 seconds then there won't have any problem because there is no pending task in the working queue of the single thread executor service.

But if the Kafka send operation CANNOT complete in 10 seconds, then there will be more and more pending tasks queued up in the unbounded working queue of single thread executor service along with the time. Finally, the unbounded working queue will occupy all the heap memory.

If any problem, please let me know.

Thanks
Gary.Guo

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.