fmtn / a Goto Github PK

Hello,

First, thanks for such a cool tool! I'm using it for random sorts of automation on our ActiveMQ servers.

When I try to git clone and then mvn clean install, it fails due to:

Tests in error:
testPutTopic(co.nordlander.a.AOpenWireTest)

I'm not sure exactly where to find the specific error -- this is the only thing I can see around the start of the tests:

Running co.nordlander.a.AOpenWireTest

0 messages cleared from queue://TARGET.QUEUE
0 messages cleared from queue://TEST.QUEUE
0 messages cleared from queue://SOURCE.QUEUE
Testing cmd: -b tcp://localhost:61916 -p "test-PLACEHOLDER" -W /tmp/junit7298892113214240993/junit661030776037202266/batch.txt -S "msg.body=msg.body.replace('PLACEHOLDER',entry);" TEST.QUEUE
3 messages sent
Message sent
Operation completed in 230ms (excluding connect)
0 messages cleared from queue://TARGET.QUEUE
0 messages cleared from queue://TEST.QUEUE
0 messages cleared from queue://SOURCE.QUEUE
4 msgs moved from SOURCE.QUEUE to TARGET.QUEUE
Operation completed in 52ms (excluding connect)
0 messages cleared from queue://TARGET.QUEUE
0 messages cleared from queue://TEST.QUEUE
0 messages cleared from queue://SOURCE.QUEUE
Writing 1 messages to dump file /tmp/junit7725678142798527935/junit998083138520055583/dump.json
Failed to write all messages to dump file. Reason: ReferenceError: "dummy" is not defined in at line number 1
Rolling back JMS transaction

docker image 1.6.0 is not yet present in docker hub

hdjd

In the usage description it says:

Set to disable transactions if not supported by platform. I.e. Azure Service Bus. When set to false, the Move option is NOT atomic.

To be more precise, the "Azure ServiceBus Basic" does not support transactions.
"Azure ServiceBus Standard" does support transactions (and topics).
I did not use "Azure ServiceBus Premium" yet, but that should at least be the same as "Standard".

In the read-me file, Azure ServiceBus is mentioned with:

A word of warning! There are some features not working with AMQP 1.0 in Service Bus. Some of which are mandatory to support the JMS API fully. This means some of the features of A will not work - or behave strangely.

Do you have more details on what "some features" includes?

Hi,

At datacenters behind firewall it is impossible to download maven and .jar package would be appreciated.

Please, provide a way to download the a.jar package, if possible.

Thanks.

Hi. Thanks for this utility!
I try to convert message with command parameter:
-S @transform.js
transform.js contains single row:
msg.stringProperties.set('foo', 'bar');
But result message hasn't this property in web-console. I tried set jms ttl property and many other, but all time cann't see some changes.
Can you describe what I should do for insert properties or jms ttl by using transformation with -S?

Messages with object properties, i.e. java.util.UUID from Azure Service Bus, fails somehow. This should not stop the reading process and simply be ignored by A

version :
Latest

JVM version (java -version):
openjdk version "11.0.6" 2020-01-14
OpenJDK Runtime Environment AdoptOpenJDK (build 11.0.6+10)
OpenJDK 64-Bit Server VM AdoptOpenJDK (build 11.0.6+10, mixed mode)

OS version (uname -a if on a Unix-like system):
Linux segotl2618 3.10.0-957.21.3.el7.x86_64 #1 SMP Fri Jun 14 02:54:29 EDT 2019 x86_64 x86_64 x86_64 GNU/Linux

Description of the problem including expected versus actual behavior:
I am sending a batch of messages.
In documentation, -r should set the JMSMessage replyTo attribute in the outgoing messages. But it doesn't. It is always null when entering the queue. The class MessageDump that is used for copying one object to a JMSMessage does not have a member for replyTo.
(Another thing is that using --reply-to instead of -r will not work at all)

Steps to reproduce:

1. Create a message file "m.txt" containing one or more messages.
2. send in the file with
   -p "PLACEHOLDER" -S "msg.body=msg.body.replace('PLACEHOLDER',entry);" -W ./m.txt
   --correlation-id anyID42 -r DEVICE.NOTIFY.RESPONSE
3. Check that the AMQ got the message anyID42 and verify that the replyTo is null instead of DEVICE.NOTIFY.RESPONSE.

Java 8 is needed to allow updates of third party libraries. Some with security vulnerabilities.

java -jar a-1.4.9-SNAPSHOT-jar-with-dependencies.jar -b tcp://10.70.20.124:24152 -P admin -U admin -l

avax.jms.JMSException: Cannot send, channel has already failed: tcp://10.70.20.124:24152
	at org.apache.activemq.util.JMSExceptionSupport.create(JMSExceptionSupport.java:72)
	at org.apache.activemq.ActiveMQConnection.syncSendPacket(ActiveMQConnection.java:1413)
	at org.apache.activemq.ActiveMQConnection.ensureConnectionInfoSent(ActiveMQConnection.java:1478)
	at org.apache.activemq.ActiveMQConnection.createSession(ActiveMQConnection.java:329)
	at co.nordlander.a.A.connect(A.java:378)
	at co.nordlander.a.A.run(A.java:189)
	at co.nordlander.a.A.main(A.java:157)
Caused by: org.apache.activemq.transport.InactivityIOException: Cannot send, channel has already failed: tcp://10.70.20.124:24152
	at org.apache.activemq.transport.AbstractInactivityMonitor.doOnewaySend(AbstractInactivityMonitor.java:328)
	at org.apache.activemq.transport.AbstractInactivityMonitor.oneway(AbstractInactivityMonitor.java:317)
	at org.apache.activemq.transport.TransportFilter.oneway(TransportFilter.java:94)
	at org.apache.activemq.transport.WireFormatNegotiator.oneway(WireFormatNegotiator.java:116)
	at org.apache.activemq.transport.MutexTransport.oneway(MutexTransport.java:68)
	at org.apache.activemq.transport.ResponseCorrelator.asyncRequest(ResponseCorrelator.java:81)
	at org.apache.activemq.transport.ResponseCorrelator.request(ResponseCorrelator.java:86)
	at org.apache.activemq.ActiveMQConnection.syncSendPacket(ActiveMQConnection.java:1388)

It would be useful to have a Docker container to run A from.

Tits

Really excited to try out an amq command line tool but running into a problem. We have ActiveMQ 5.4.x in production. Is there a minimum version of ActiveMQ that is supported? Here's the error:

[root@ewhserver1512 amqclient]# ./a -b tcp://localhost:61616
javax.jms.JMSException: Disposed due to prior exception
	at org.apache.activemq.util.JMSExceptionSupport.create(JMSExceptionSupport.java:72)
	at org.apache.activemq.ActiveMQConnection.syncSendPacket(ActiveMQConnection.java:1413)
	at org.apache.activemq.ActiveMQConnection.ensureConnectionInfoSent(ActiveMQConnection.java:1478)
	at org.apache.activemq.ActiveMQConnection.createSession(ActiveMQConnection.java:329)
	at co.nordlander.a.A.connect(A.java:376)
	at co.nordlander.a.A.run(A.java:184)
	at co.nordlander.a.A.main(A.java:155)
Caused by: org.apache.activemq.transport.TransportDisposedIOException: Disposed due to prior exception
	at org.apache.activemq.transport.ResponseCorrelator.onException(ResponseCorrelator.java:125)
	at org.apache.activemq.transport.TransportFilter.onException(TransportFilter.java:114)
	at org.apache.activemq.transport.TransportFilter.onException(TransportFilter.java:114)
	at org.apache.activemq.transport.WireFormatNegotiator.onException(WireFormatNegotiator.java:173)
	at org.apache.activemq.transport.WireFormatNegotiator.negociate(WireFormatNegotiator.java:161)
	at org.apache.activemq.transport.WireFormatNegotiator.onCommand(WireFormatNegotiator.java:123)
	at org.apache.activemq.transport.AbstractInactivityMonitor.onCommand(AbstractInactivityMonitor.java:301)
	at org.apache.activemq.transport.TransportSupport.doConsume(TransportSupport.java:83)
	at org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:233)
	at org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:215)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.IOException: Invalid version: 6, could not load org.apache.activemq.openwire.v6.MarshallerFactory
	at org.apache.activemq.util.IOExceptionSupport.create(IOExceptionSupport.java:46)
	... 7 more
Caused by: java.lang.IllegalArgumentException: Invalid version: 6, could not load org.apache.activemq.openwire.v6.MarshallerFactory
	at org.apache.activemq.openwire.OpenWireFormat.setVersion(OpenWireFormat.java:335)
	at org.apache.activemq.openwire.OpenWireFormat.renegotiateWireFormat(OpenWireFormat.java:614)
	at org.apache.activemq.transport.WireFormatNegotiator.negociate(WireFormatNegotiator.java:145)
	... 6 more
Caused by: java.lang.ClassNotFoundException: org.apache.activemq.openwire.v6.MarshallerFactory
	at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
	at java.lang.Class.forName0(Native Method)
	at java.lang.Class.forName(Class.java:348)
	at org.apache.activemq.openwire.OpenWireFormat.setVersion(OpenWireFormat.java:333)
	... 8 more

I'm on Java 8:

[root@ewhserver1512 amqclient]# /usr/bin/java -version
java version "1.8.0_121"
Java(TM) SE Runtime Environment (build 1.8.0_121-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.121-b13, mixed mode)

And ActiveMQ is definitely running - this is not a new install - as my app uses it successfully (with the localhost:61616 url).

Appreciate your help and happy to provide any additional data if necessary.

It would be convenient to push all files within a directory to a queue/topic.

Even wait for files using the -w flag

To cleanup a messy queue, I would like to consume (-g) all messages based on a specific selector.

Here is what I tried:

# Content of my queue:
########################

$ ./a  --broker tcp://test-broker-1:61616  test
-----------------
Message Properties
  event_category: trash
Payload:
{"logLevel":"INFO",
"messageId":"00004",
"serviceName":"auth",
"properties":{
        "inviteeName":"[email protected]",
        "initiatorName":"admin"},
"created":"2017-04-12T13:48:36.368+02:00"}

-----------------
Message Properties
  event_category: can
Payload:
{"logLevel":"INFO",
"messageId":"00004",
"serviceName":"auth",
"properties":{
        "inviteeName":"[email protected]",
        "initiatorName":"admin"},
"created":"2017-04-12T13:48:36.368+02:00"}

-----------------
Message Properties
  event_category: trash
Payload:
{"logLevel":"INFO",
"messageId":"00004",
"serviceName":"auth",
"properties":{
        "inviteeName":"[email protected]",
        "initiatorName":"admin"},
"created":"2017-04-12T13:48:36.368+02:00"}

Operation completed in 30067ms (excluding connect)

# Browse messages with selector
########################

$ ./a  --broker tcp://test-broker-1:61616 -s "event_category='trash'"  test
-----------------
Message Properties
  event_category: trash
Payload:
{"logLevel":"INFO",
"messageId":"00004",
"serviceName":"auth",
"properties":{
        "inviteeName":"[email protected]",
        "initiatorName":"admin"},
"created":"2017-04-12T13:48:36.368+02:00"}

-----------------
Message Properties
  event_category: trash
Payload:
{"logLevel":"INFO",
"messageId":"00004",
"serviceName":"auth",
"properties":{
        "inviteeName":"[email protected]",
        "initiatorName":"admin"},
"created":"2017-04-12T13:48:36.368+02:00"}

Operation completed in 30081ms (excluding connect)

# Trying to get messages with selector
########################

$ ./a  --broker tcp://test-broker-1:61616 -s "event_category='trash'" -g  test
No message received
Operation completed in 125ms (excluding connect)
$ ./a  --broker tcp://test-broker-1:61616 -s "event_category='trash'" -c 5 -g  test
No message received
Operation completed in 114ms (excluding connect)
$ ./a  --broker tcp://test-broker-1:61616 -s "event_category='trash'" -c 0 -g  test
No message received
Operation completed in 96ms (excluding connect)

Since the documentation says:

-s,--selector Browse or get with selector

So now it could be a bug 🐛 or PEBKAC 😊 . In any case, I hope you can help me out here.

Thanks in advance!

"a" supports priority and persistence settings when sending a message.
Can expiry (aka TTL) also be added?
i.e. apply setJMSExpiration to the message with a given timeout before it is sent.

Hi,

Thanks for the new release.

While playing around with it I noticed a few quirks, mainly in regard to the tests.

1. The tests don't run during the normal maven build because the test class ATestCases doesn't match the maven defaults (Test*, *Test or *TestCase). Maybe that was intentional on your part. If not, the quickest fix is just to change the class name to ATestCase (though that name is a little misleading).
2. When I first tried to run the tests I had some XML namespace-related errors referring the activemq.xml, specifically the broker tag. In the end, I found that including activemq-all as a test dependency in the pom instead of just activemq-broker made them go away. I'm still not entirely sure what was happening.
3. In the broker tag in activemq.xml, I guess you meant to specify dataDirectory="target/amqdata" and not dataDirectory="taget/amqdata"
4. You might choose to set a property for the encoding in the pom (of course, it's up to you):

    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>

The good news is that from the general, operational testing I've been able to do everything works!

Tracking issue for:

• https://github.com/fmtn/a/security/code-scanning/1

Hi Hello I am looking for open source projects to contribute graphical design, I have done several jobs for free in opensource projects, I would like to know if you are interested that I make a logo for this your project ?.

Greetings

Nuno Jesus

The fat-jar is pretty heavy by now. Most of the included jars are probably not needed.. Can it be refactored to include less junk?

How to generate JAR file which was mentioned in help docs.

The following does not work: ./a -p "Testing..." myQueueName. I need to do the following instead: ./a -p "Testing..." myQueueName q. No error message is presented. The command says the message was sent, but it was not.

The following commands result in java exceptions stating the parameter name is unknown:

./a -v
./a -l

But if I do the following they work:

./a -v q
./a -l q

ActiveMQ 5.17 requires java 11 so should.

Hi,

Since my upgrade to Fedora 33 I have this message : Warning: Nashorn engine is planned to be removed from a future JDK release using -x and -X options

are you aware of this issue for future version ?

It would be nice to be able to subscribe to advisory topics and display the result. For instance, to list the queues in the broker and whatnot.

Annoying "looping" print from MessageDumpWriter

There has been reports of -x operation hanging, consuming the message while not producing the output json. It's not yet known what causes the problem. A warning should be issues that a dump operation cannot rollback if an error occurrs.

Hi,

is there a method to count number of the messages in the queue via 'a' or this rather should be done with the JMX api or http layer on top of JMX api.

thanks

The command line tool looks great! However I'm facing an error, please refer to below for details.

• Platform: Windows 10
• JDK: 1.8.0_172
• Connecting to ActiveMQ server with SSL enabled.

I ran following command but got error as below.

a -b ssl://amq-dev.api.my.com.au:443 -U admin -p adminpass -l

I also modified a.bat to include the JVM args javax.net.ssl.keyStore and javax.net.ssl.keyStorePassword but the issue persists. In fact I don't need to do this because I have added the cert into default JRE lib\security\cacerts file.

javax.jms.JMSSecurityException: User name [null] or password is invalid.
        at org.apache.activemq.util.JMSExceptionSupport.create(JMSExceptionSupport.java:52)
        at org.apache.activemq.ActiveMQConnection.syncSendPacket(ActiveMQConnection.java:1399)
        at org.apache.activemq.ActiveMQConnection.ensureConnectionInfoSent(ActiveMQConnection.java:1478)
        at org.apache.activemq.ActiveMQConnection.createSession(ActiveMQConnection.java:329)
        at co.nordlander.a.A.connect(A.java:383)
        at co.nordlander.a.A.run(A.java:185)
        at co.nordlander.a.A.main(A.java:156)
Caused by: java.lang.SecurityException: User name [null] or password is invalid.
        at org.apache.activemq.security.JaasAuthenticationBroker.authenticate(JaasAuthenticationBroker.java:89)
        at org.apache.activemq.security.JaasAuthenticationBroker.addConnection(JaasAuthenticationBroker.java:68)
        at org.apache.activemq.broker.BrokerFilter.addConnection(BrokerFilter.java:97)
        at org.apache.activemq.broker.MutableBrokerFilter.addConnection(MutableBrokerFilter.java:102)
        at org.apache.activemq.broker.TransportConnection.processAddConnection(TransportConnection.java:842)
        at org.apache.activemq.broker.jmx.ManagedTransportConnection.processAddConnection(ManagedTransportConnection.java:77)
        at org.apache.activemq.command.ConnectionInfo.visit(ConnectionInfo.java:139)
        at org.apache.activemq.broker.TransportConnection.service(TransportConnection.java:326)
        at org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:190)
        at org.apache.activemq.transport.MutexTransport.onCommand(MutexTransport.java:50)
        at org.apache.activemq.transport.WireFormatNegotiator.onCommand(WireFormatNegotiator.java:113)
        at org.apache.activemq.transport.AbstractInactivityMonitor.onCommand(AbstractInactivityMonitor.java:300)
        at org.apache.activemq.transport.TransportSupport.doConsume(TransportSupport.java:83)
        at org.apache.activemq.transport.tcp.SslTransport.doConsume(SslTransport.java:102)
        at org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:214)
        at org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:196)
        at java.lang.Thread.run(Thread.java:748)
Caused by: javax.security.auth.login.FailedLoginException: user name is null
        at org.apache.activemq.jaas.PropertiesLoginModule.login(PropertiesLoginModule.java:84)
        at sun.reflect.GeneratedMethodAccessor119.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at javax.security.auth.login.LoginContext.invoke(LoginContext.java:755)
        at javax.security.auth.login.LoginContext.access$000(LoginContext.java:195)
        at javax.security.auth.login.LoginContext$4.run(LoginContext.java:682)
        at javax.security.auth.login.LoginContext$4.run(LoginContext.java:680)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:680)
        at javax.security.auth.login.LoginContext.login(LoginContext.java:587)
        at org.apache.activemq.security.JaasAuthenticationBroker.authenticate(JaasAuthenticationBroker.java:84)
        ... 16 more

I'm using Gboard to type in English (Canada) (QWERTY). You can try it at: https://gboard.app.goo.gl/C2m6h

Hi,

the readme states that SSL can be used by using "-Djavax.net.ssl.keyStore" and "-Djavax.net.ssl.trustStore":

java -Djavax.net.ssl.keyStore=/Users/petter/client.jks -Djavax.net.ssl.keyStorePassword=password -Djavax.net.ssl.trustStore=/Users/petter/truststore.jks -Djavax.net.ssl.trustStorePassword=password -jar a-1.6.0-jar-with-dependencies.jar -b ssl://example.org:61618 MY.QUEUE

As I'm working with self signed certs in my setup (2-way-SSL) I need both - a client keyStore and trustStore. While passing both arguments works perfectly fine for the trustStore, it does not work for the keyStore. It seems to me that -Djavax.net.ssl.keyStore is somehow 'ignored', so the keyStore is never used during the TLS/SSL handshake. The result is that the client certificate cannot be found and the handshake is not successful.

Any idea why this happens?

When message contains properties that have null values, NullPointerException is thrown when displaying properties:

-----------------
Message Properties
java.lang.NullPointerException
    at co.nordlander.a.A.outputProperties(A.java:807)
    at co.nordlander.a.A.outputMessage(A.java:611)
    at co.nordlander.a.A.executeGet(A.java:442)
    at co.nordlander.a.A.run(A.java:246)
    at co.nordlander.a.A.main(A.java:122)

Would be nice if the start script wouldn't depend on the JAR being in ~/bin
I.e. change the script to something like this:

BASEDIR=$(dirname "$0")
+java -jar $BASEDIR/a-*-jar-with-dependencies.jar "$@"

It would be nice to use System.exit with nonzero codes to indicate errors for better integration with shell scripts.

I recently discovered a fairly large dead letter queue and am trying too output (not dump; yet) the messages to files. When I run java -jar a-1.4.8-jar-with-dependencies.jar -b "ssl://localhost:61617" -U "procellaworkers" -P "dm9YDfSMKEKA0WB7qMcFkjoVKawOkuzw" ActiveMQ.DLQ -c 0 -j -o "messages/dlq" the output stops at 400.

I am concerned that if I do use dump in the future, that it will also be limited to 400.

Hi,

Great project!

However, move and copy didn't work as I expected and I think the problem is just a typo (repeated). Both the executeMove and executeCopy method of A.java begin with the 2 statements:

    Queue tq = tsess.createQueue(cmdLine.getArgs()[0]);
    Queue q =  tsess.createQueue(cmdLine.getOptionValue(CMD_MOVE_QUEUE)); // Source

But I think q and tq should be swapped:

    Queue q = tsess.createQueue(cmdLine.getArgs()[0]); // Source
    Queue tq =  tsess.createQueue(cmdLine.getOptionValue(CMD_MOVE_QUEUE)); // Target

Do you agree?

The command
a.bat -j -b tcp://localhost:61616 -p @message.xml ABC/XYZ
sends the message to the queue named 'ABCXYZ'. The '/' charachter was stripped out.

It would be nice to have the ability to get the count in a queue AND/OR know if a queue is empty.
A new commuters :
--get_count -N : return integer of count : -1 is queue does not exist
--is_empty -Z : return 1 for empty queue / 0 not empty : -1 is queue does not exist

thank U for your nice work

Any way to ignore SSL certificate issues?

Due to a bug in library org.apache.qpid:qpid-amqp-1-0-client-jms:0.32, the value 0 for --wait does not work when using AMQP.
The value 0 is interpreted as no-wait, completely opposite the expected behavior of waiting infinitely.

The bug is already clearly marked in the original source code as a "TODO" change.
see package org.apache.qpid.amqp_1_0.jms.impl, file MessageConsumerImpl.java, line 258.

I see at least 3 solutions for this:

1. instead, use function receive() (no parameters) when wait=0, that function does not have the above problem; or
2. fix the bug upstream and later update to the new version of the client library when dependabot warns for it; or
3. switch to another client library for amqp.

nr 1 is the quickest solution, nr 2 is the best solution.

what is your preference?