exabrial / graylog-plugin-openwire

Provides an input for Graylog using the ActiveMQ client library

Graylog Plugin Openwire

Bridges ActiveMQ->Graylog

Provides an input for Graylog using the ActiveMQ client library. Openwire is the native protocol for ActiveMQ, but all valid client ActiveMQ URLs can be used.

If you use Logback inside your Java [possibly EE] applications, I have two logback appenders you can use:

• https://github.com/exabrial/logback-jms-appender (Best for Java EE apps with a managed JMS Configuration)
• https://github.com/exabrial/logback-openwire-appender (Best for Spring Boot or non EE standalone apps)

Acknowledgements

The Graylog MQTT plugin lays the groundwork on how to accomplish writing your own inputs for Graylog. The code is well organized and very readable. https://github.com/graylog-labs/graylog-plugin-mqtt

License

Since Upsteam Graylog and the MQTT plugin are GPL, this project is GPL to conform with the terms of the license.

Installation

Download the plugin and place the .jar file in your Graylog plugin directory, which is usually called plugins.

cd ~
gpg --recv-keys 871638A21A7F2C38066471420306A354336B4F0D
wget https://github.com/exabrial/graylog-plugin-openwire/releases/download/graylog-plugin-openwire-2.0.1/graylog-plugin-openwire-2.0.1.jar.asc
wget https://github.com/exabrial/graylog-plugin-openwire/releases/download/graylog-plugin-openwire-2.0.1/graylog-plugin-openwire-2.0.1.jar
gpg --verify graylog-plugin-openwire-2.0.1.jar.asc graylog-plugin-openwire-2.0.1.jar
rm graylog-plugin-openwire-2.0.1.jar.asc
sudo mv graylog-plugin-openwire-2.0.1.jar /usr/share/graylog-server/plugin/
sudo chown root:root /usr/share/graylog-server/plugin/graylog-plugin-openwire-2.0.1.jar
sudo systemctl restart graylog-server

After that step is completed, restart your server and you should have the new input options available.

Configuration

Property Name	Example	Purpose
brokerUrl	failover:(ssl://activemq-1.example.com:61616,ssl://activemq-2.example.com:61616)?randomize=false&backup=true	The ActiveMQ client URL. Any valid ActiveMQ client URL can be used.
queueName	ch.qos.logback	The Queue name to read logs off of.
username	loguser	The username to use to establish the connection to ActiveMQ.
password	myPassw0rd	The password to use to establish the connection to ActiveMQ.
throttling_allowed	true	If the graylog server gets busy, slow down this input. Recommend checking this box.

graylog-plugin-openwire's Issues

missing event subscriber for an annotated method

Hi @exabrial

I am getting below error while trying to add this plugin in Graylog v3.2.4+a407287.

Error in GrayLog GUI:

An input has failed to start (triggered in a few seconds)

Input 5eac58f653c2382c37618f34 has failed to start on node 9ea8a85e-86f0-418c-8458-76d412556318 for this reason: »missing event subscriber for an annotated method. Is com.github.exabrial.graylog.OpenwireTransport@40ef29d0 registered?.«. This means that you are unable to receive any messages from this input. This is mostly an indication for a misconfiguration or an error.

Plugin doesn't ingest messages on Graylog 2.4.7

I've installed version 1.0.2 on Graylog 2.4.7. I'm intending to download messages from ActiveMQ 5. I've connected successfully to the queue, but messages don't get into graylog. I've found this messages on graylog server log:

019-06-21T11:27:52.425-03:00 ERROR [ActiveMQMessageConsumer] ID:srvhZZZZ-43728-1561066133943-13:1:1:1 Exception while processing message: ID:srvhXXXX-44195-1551968170890-5:7:-1:1:15229 java.lang.ClassCastException: org.apache.activemq.command.ActiveMQBytesMessage cannot be cast to javax.jms.TextMessage at com.github.exabrial.graylog.OpenwireConsumer$2.onMessage(OpenwireConsumer.java:61) ~[graylog-plugin-openwire-1.0.2.jar:?] at org.apache.activemq.ActiveMQMessageConsumer.dispatch(ActiveMQMessageConsumer.java:1404) [graylog-plugin-openwire-1.0.2.jar:?] at org.apache.activemq.ActiveMQMessageConsumer.iterate(ActiveMQMessageConsumer.java:1575) [graylog-plugin-openwire-1.0.2.jar:?] at org.apache.activemq.ActiveMQSessionExecutor.iterate(ActiveMQSessionExecutor.java:191) [graylog-plugin-openwire-1.0.2.jar:?] at org.apache.activemq.thread.PooledTaskRunner.runTask(PooledTaskRunner.java:133) [graylog-plugin-openwire-1.0.2.jar:?] at org.apache.activemq.thread.PooledTaskRunner$1.run(PooledTaskRunner.java:48) [graylog-plugin-openwire-1.0.2.jar:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_212] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_212] at java.lang.Thread.run(Thread.java:748) [?:1.8.0_212]

OpenWire Plugin not loading on Graylog 2.3.2

I've did my POC on VM where I´ve runned graylog 2.4.7. Now I want to reproduce this on my production server that run graylog 2.3.2

I´ve copied the jar file to the plugin dir and restarted the graylog service. When I go the INPUTs screen I don't see the opewire options available.

No version number properties

This is in the startup log:

^[[A^[[B2019-02-16T08:54:08.580-06:00 ERROR [Version] Unable to read com.github.exabrial.graylog-plugin-openwire/graylog-plugin.properties, this build has no version number.
com.github.zafarkhaja.semver.UnexpectedCharacterException: null
	at com.github.zafarkhaja.semver.VersionParser.consumeNextCharacter(VersionParser.java:516) ~[graylog.jar:?]
	at com.github.zafarkhaja.semver.VersionParser.digits(VersionParser.java:448) ~[graylog.jar:?]
	at com.github.zafarkhaja.semver.VersionParser.numericIdentifier(VersionParser.java:408) ~[graylog.jar:?]
	at com.github.zafarkhaja.semver.VersionParser.parseVersionCore(VersionParser.java:285) ~[graylog.jar:?]
	at com.github.zafarkhaja.semver.VersionParser.parseValidSemVer(VersionParser.java:255) ~[graylog.jar:?]
	at com.github.zafarkhaja.semver.VersionParser.parseValidSemVer(VersionParser.java:195) ~[graylog.jar:?]
	at com.github.zafarkhaja.semver.Version.valueOf(Version.java:265) ~[graylog.jar:?]
	at org.graylog2.plugin.Version.fromClasspathProperties(Version.java:219) [graylog.jar:?]
	at org.graylog2.plugin.Version.fromPluginProperties(Version.java:163) [graylog.jar:?]
	at com.github.exabrial.graylog.OpenwireInputMetadata.getRequiredVersion(OpenwireInputMetadata.java:46) [graylog-plugin-openwire-1.0.1.jar:?]
	at org.graylog2.bootstrap.CmdLineTool.loadPlugins(CmdLineTool.java:297) [graylog.jar:?]
	at org.graylog2.bootstrap.CmdLineTool.installPluginConfigAndBindings(CmdLineTool.java:270) [graylog.jar:?]
	at org.graylog2.bootstrap.CmdLineTool.run(CmdLineTool.java:172) [graylog.jar:?]
	at org.graylog2.bootstrap.Main.main(Main.java:50) [graylog.jar:?]
2019-02-16T08:54:08.588-06:00 INFO  [CmdLineTool] Loaded plugin: ActiveMQ Openwire Input Plugin 1.0.1 [com.github.exabrial.graylog.OpenwireGELFInput]

graylog-plugin-openwire should use java version 17 and needs graylog2-server dependencies uplifted.

uprade apache activemq lib to 5.16.7

OpenWire Plugin automatically reconnect to ActiveMQ after downtime

I've been using version 1.1.0 of the plugin without incidents for a few weeks.

We noticed after downtime of the queue, or any interruption in communication between plugin and ActiveMQ, the INPUT must be stoped and re started in order to continue ingesting messages from the queue.

Does the plugin support any kind of automatic reconnection? I'm investigating how to monitor the Queue and restart the INPUT from Graylog API. But I think that automatic reconnection will be interesteng.