- 1. Overview
- 2. Demo Program Description
- 3. Pulsar Native Client API Demo
- 4. Starlight for JMS Demo
- 5. More Advanced JMS API Example
Starlight for JMS is a Java based API that allows existing JMS based applications connecting to Apache Pulsar as a JMS provider to publish and consume messages. Compared with an existing JMS provider like ActiveMQ, RabbitMQ, etc., Apache Pulsar is a modern, unified messaging and streaming platform that can deliver much higher throughput message publishing and consuming while maintaining consistent low latency. It also has other major benefits that are not available in any existing JMS providers, such as effective multi-tenancy support, native schema registry, out-of-the-box message geo-replication, and so on.
Starlight for JMS supports JMS 2.0 with backward-compatibility with JMS 1.1. It is a full-fledged API that supports the vast majority of the JMS functionalities as described JMS spec. It passes about 98% of the JMS compliance TCK test suite.
In this repo, we're going to do some demos about how to use Starlight for JMS to achieve some typical JMS message sending and receiving methods against an Apache Pulsar server.
- OS: Linux or MacOS
- Java: JDK 8 or 11
** latest release: 2.8.0
curl -L -O 'https://www.apache.org/dyn/mirrors/mirrors.cgi?action=download&filename=pulsar/pulsar-2.8.0/apache-pulsar-2.8.0-bin.tar.gz'
tar -zxvf apache-pulsar-2.8.0-bin.tar.gz
cd apache-pulsar-2.8.0/
export PULSAR_HOME=`pwd`
export PATH="$PULSAR_HOME/bin:$PATH"
which pulsar
Run the following command to start a standalone Pulsar server locally.
bin/pulsar standalone &
Pulsar has a set of built-in CLI command utilities:
- Pulsar Admin CLI: for the purpose of administrating Pulsar objects such as tenants, namespaces, topics, and etc.
- Pulsar Client CLI: for the purpose of quick simulation of a message sending and consuming client
- Pulsar Perf CLI: for Pulsar performance testing purposes
In this demo, we'll briefly explore some Pulsar Admin CLI commands.
In order to use these CLI utilities, we need to first configure the client connection property file.
- default: <PULSAR_HOME>/conf/client.conf
- https://pulsar.apache.org/docs/en/reference-configuration/#client
PULSAR_CLIENT_CONF=<custom_client_conf_path> pulsar-admin [sub-command ...]
- Verify Pulsar cluster
bin/pulsar-admin clusters list
- Manage Pulsar Tenant
bin/pulsar-admin tenants list
bin/pulsar-admin tenants create mytenant
bin/pulsar-admin tenants
- Manage Pulsar Namespace
bin/pulsar-admin namespaces create mytenant/ns0
bin/pulsar-admin namespaces list mytenant
bin/pulsar-admin namespaces
- Manage Pulsar Topics (Partitioned or not)
bin/pulsar-admin topics create mytenant/ns0/t0
bin/pulsar-admin topics list mytenant/ns0
bin/pulsar-admin topics create-partitioned-topic -p 5 mytenant/ns0/pt0
bin/pulsar-admin topics list-partitioned-topics mytenant/ns0
bin/pulsar-admin topics get-partitioned-topic-metadata mytenant/ns0/pt0
bin/pulsar-admin topics update-partitioned-topic mytenant/ns0/pt0 -p 6
In this repo, there are several sets of demo programs organized under different java packages (com.example.XYZ). These sets of programs are summarized as below:
| Demo Program Package | Description |
|---|---|
| com.example.pulsar.* | Simple message producer and consumer using native Pulsar client API |
| com.example.fastjms.* | Simple message producer and consumer using Starlight for JMS, including JMS topic related operations. |
| com.example.fastjms.queue_pattern.* | Specific JMS queue related operations using Starlight for JMS |
In order to run the demo programs against the Pulsar server, we need to first configure the connection properties. An example is as below:
## Pulsar Client connection configuration
webServiceUrl=http://localhost:8080
brokerServiceUrl=pulsar://localhost:6650
authPlugin=org.apache.pulsar.client.impl.auth.AuthenticationToken
authParams=token:<token_value>
## Other Pulsar client specific configuration
client.numIoThreads=2
## Pulsar producer specific configuration
producer.blockIfQueueFull=true
## Pulsar consumer specific configuration
consumer.receiverQueueSize=1000
## JMS specific configuration
jms.queueSubscriptionName=myjms-queue
jms.forceDeleteTemporaryDestinations=true
There are four major sections you can specify in the configuration property file:
-
Core Pulsar connection related settings
- service URLs - web and broker
- authentication
- TLS
- No prefix
- https://pulsar.apache.org/docs/en/reference-configuration/#client
-
Other Pulsar client connection related settings
- exclude settings from above
- prefix: client
- https://pulsar.apache.org/docs/en/client-libraries-java/#client
-
Pulsar Producer related settings
-
Pulsar Consumer related Settings
-
Pulsar JMS specific settings
% gradle clean build shadowJar
In order to use the Pulsar Native Client AP, includes the following dependency in your program: (gradle dependency)
implementation group: 'org.apache.pulsar', name: 'pulsar-client', version: '2.8.0'
The main demo program for demonstrating message publishing (producer) and message consuming (consumer) using Pulsar native client API is com.example.pulsar.PulsarSimpleDemo. Its usage is as below:
usage: PulsarSimpleDemo [-f <arg>] [-h] [-n <arg>] [-op <arg>] [-sn <arg>] [-st <arg>] [-t <arg>]
PulsarSimpleDemo Options:
-f <arg> Configuration properties file.
-h Displays this help message.
-n <arg> Number of messages
-op <arg> Operation type - "Producer" or "Consumer"
-sn <arg> Subscription name
-st <arg> Subscription type (default to "Exclusive")
-t <arg> Pulsar topic name
An example of publishing 10 messages to a Pulsar topic is as below:
% java -cp pulsar_exmples-1.0-SNAPSHOT-all.jar com.example.pulsar.PulsarSimpleDemo \
-f <path/to/conn.properties> \
-t persistent://mytenant/ns0/t0 \
-op producer \
-n 10
An example of receiving 10 messages from a Pulsar topic is as below:
% java -cp pulsar_exmples-1.0-SNAPSHOT-all.jar com.example.pulsar.PulsarSimpleDemo \
-f <path/to/conn.properties> \
-t persistent://mytenant/ns0/t0 \
-op consumer \
-n 10 \
-sn mysub \
-st Shared
In order to use Starlight for JMS, include the following dependency in your program: (gradle dependency)
implementation group: 'com.datastax.oss', name: 'pulsar-jms-all', version: '1.2.2'
The first JMS demo program, com.example.fastjms.JmsSimpleDemo, is similar to the Pulsar native client API demo program in the previous section. It follows JMS 2.0 spec and demonstrates how message sending/publishing and receiving/consuming works with Starlight for JMS.
Its usage is as below:
usage: JmsSimpleDemo [-dn <arg>] [-dt <arg>] [-f <arg>] [-h] [-n <arg>] [-op <arg>] [-sn <arg>]
JmsSimpleDemo Options:
-dn <arg> JMS destination name: pulsar topic name
-dt <arg> JMS destination type - "queue" or "topic"
-f <arg> Configuration properties file.
-h Displays this help message.
-n <arg> Number of messages
-op <arg> Operation type - "Producer, Consumer, SharedConsumer, DurableConsumer, SharedDurableConsumer"
-sn <arg> Subscription name
NOTE that
- For a "queue" destination, the valid operation type ("-op") can only be "Producer" (QueueSender) or "Consumer" (QueueReceiver)
- For a "topic" destination, the valid operation types can be all of them
- Producer (TopicPublisher)
- Consumer (TopicSubscriber)
- SharedConsumer
- DurableConsumer
- SharedDurableConsumer
An example of sending 10 messages to a JMS queue is as below:
% java -cp pulsar_exmples-1.0-SNAPSHOT-all.jar com.example.fastjms.JmsSimpleDemo \
-f <path/to/conn.properties> \
-dn persistent://mytenant/ns0/t1 \
-dt queue
-op producer \
-n 10
An example of receiving 10 messages from a JMS queue is as below:
% java -cp pulsar_exmples-1.0-SNAPSHOT-all.jar com.example.fastjms.JmsSimpleDemo \
-f <path/to/conn.properties> \
-dn persistent://mytenant/ns0/t1 \
-dt queue
-op consumer \
-n 10
An example of publishing 10 messages to a JMS topic is as below:
% java -cp pulsar_exmples-1.0-SNAPSHOT-all.jar com.example.fastjms.JmsSimpleDemo \
-f <path/to/conn.properties> \
-dn persistent://mytenant/ns0/t2 \
-dt topic
-op producer \
-n 10
An example of consuming 10 messages from a JMS topic is as below:
% java -cp pulsar_exmples-1.0-SNAPSHOT-all.jar com.example.fastjms.JmsSimpleDemo \
-f <path/to/conn.properties> \
-dn persistent://mytenant/ns0/t2 \
-dt topic
-op consumer \
-n 10
The second JMS demo program, com.example.fastjms.queue_pattern.QueuePatternDemo, is used to demonstrate JMS queue specific operations in JMS 1.1 spec. Its usage is as below:
usage: QueuePatternDemo [-dn <arg>] [-f <arg>] [-h] [-ms <arg>] [-pn <arg>]
QueuePatternDemo Options:
-dn <arg> JMS destination name: pulsar topic name
-f <arg> Configuration properties file.
-h Displays this help message.
-ms <arg> JMS message selector: "selector pattern"
-pn <arg> JMS Pattern Name - "QueueBrowser, QueueReceiver, QueueRequestor, QueueSender"
An example of sending 10 messages to a JMS queue using JMS QueueSender is as below:
% java -cp pulsar_exmples-1.0-SNAPSHOT-all.jar com.example.fastjms.QueuePatternDemo \
-f <path/to/conn.properties> \
-dn persistent://public/default/qpatn \
-pn QueueSender
The messages sent out have some properties including a sequence_id integer property. This will be used later with message selector for message filtering
- sent message: properties { sequence_id:0, jms_time:1634243810531 }; payload { pVMN9TKwXouGJGyu7CQ7 }
... ...
- sent message: properties { sequence_id:9, jms_time:1634243813925 }; payload { aJrY0Wz92mzjNwr1yUJJ }
An example of receiving 4 messages (via message selector) from a JMS queue using JMS QueueReceiver is as below:
% java -cp pulsar_exmples-1.0-SNAPSHOT-all.jar com.example.fastjms.QueuePatternDemo \
-f <path/to/conn.properties> \
-dn persistent://public/default/qpatn \
-pn QueueReceiver
-ms "sequence_id<=3"
An example of browsing messages from a JMS queue using JMS QueueBrowser is as below:
% java -cp pulsar_exmples-1.0-SNAPSHOT-all.jar com.example.fastjms.QueuePatternDemo \
-f <path/to/conn.properties> \
-dn persistent://public/default/qpatn \
-pn QueueBrowser
JMS QueueRequestor example requires 2 parts:
- one client application sends a messages to a JMS Queue and waits for a response
- one service application receives each message from the JMS Queue, processes it, and sends the response back
An example is as below:
% java -cp pulsar_exmples-1.0-SNAPSHOT-all.jar com.example.fastjms.QueuePatternDemo \
-f <path/to/conn.properties> \
-dn persistent://public/default/qpatn_req \
-pn QueueRequestor
This example starts a service application in the background listening to a JMS queue. It also acts as a client application sending messages whose payload value is a random integer. When the service application receives a message, it times the message value by 100 and sends the new value as the response back to the client. The log output of this example is as below:
2021-10-14 20:29:45.154 [main] - QueueRequestor:: sending 10 messages from queue: persistent://public/default/qpatn_req
2021-10-14 20:29:46.113 [main] - > sent message: {0}, response message: {0}
2021-10-14 20:29:46.230 [main] - > sent message: {9}, response message: {900}
2021-10-14 20:29:46.340 [main] - > sent message: {3}, response message: {300}
2021-10-14 20:29:46.451 [main] - > sent message: {2}, response message: {200}
2021-10-14 20:29:46.561 [main] - > sent message: {1}, response message: {100}
NOTE: Currently the source codes for these advanced JMS pattern demos are still under different GitHub repositories (please refer to each demo for more details). They will be consolidated under this repository in the future.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent