Issue in running kafkamonitor on windows about kafka-monitor

[abhit011]

Guys any pointers ?

[smccauliff]

We don't have a build test system for Windows nor do we run this on Windows at LinkedIn (yes, I know we were acquired by MS) so there is probably something very simple that is wrong. I can look at this later today.

[smccauliff]

The classnames are changed from previous versions, but were never changed in the bat files. If you remove the ".core" from the class names in the bat file it should work. You may have troubles with the CLASSPATH I had to set it manually to get it work.

[lindong28]

@smccauliff I think we have actually change the script to load the correct class after we change the classnames in the commit d69ff73.

@abhit011 Can you tell us the git hash of the kafka-monitor you are running? Which command did you use that causes the class not found error?

[abhit011]

@lindong28
I have given git checkout from the main branch itself ... Though the hash is 5807f46
and I used .\bin\windows\kafka-monitor-start.bat .\config\kafka-monitor.properties this command to run the Kafka_monitor

[abhit011]

@smccauliff
You guys should start supporting windows :P Though I am also an ex MS employee ;)
Anyways I ram the command with removing .core from the bat files, but again it gave error like Error: Could not find or load main class com.linkedin.kmf.KafkaMonitor

[lindong28]

@abhit011 I see. An open source user contributed the script to run kafka-monitor in Windows. But I forgot to update this script when we changed the path of KafkaMonitor class. I just submitted the patch at https://github.com/linkedin/kafka-monitor/pull/67/files to fix this. @smccauliff will validate this on his Windows machine. Please feel free to download that patch and see if it fixes the problem.

[abhit011]

Yes i could able to run it on windows ... Thanks ...
Another thing, if I have to monitor just my Kafka topics traffic and my kafka producer / consumer ... How I am going to handle that ? I have just changed the broker and zoo keeper : addresses/port .

[lindong28]

@abhit011 Kafka Monitor can not monitor your producer/consumer/topics directly. It is used to approximate the experience of your producer/consumer/topics by monitoring your cluster, i.e. it is possible that Kafka Monitor reports drop in availability while your producer/consumer doesn't, and vise versa. You need to assign new/dedicated monitor topic to Kafka Monitor since it will produce fabricated traffic to the monitor topic. The only config you need to change is the broker and zookeeper urls. There are other configs that you can fine-tune for more advanced use-case.

[abhit011]

Thanx @lindong28 , i need to see what best i can do to monitor the live traffic of my kafka cluster..

[lindong28]

@abhit011 FYI, we use an auditing system at LinkedIn to check message loss. It is briefly described in https://engineering.linkedin.com/blog/2016/04/kafka-ecosystem-at-linkedin. The idea is for each producer to report the messages it has sent and we compare it with the actually number of messages that can be consumed. Also, since Kafka Producer and Kafka Consumer already reports many useful metrics (e.g. throughput) via JMX metrics, you have have them exported to an external system (e.g. graphite) so that you can monitor their performance.

[smccauliff]

Seems like this is fixed on master.