vert-x3 / vertx-mqtt

Vert.x MQTT

Vert.x MQTT

This project provides the following two different components :

• server : it's able to handle connections, communication and messages exchange with remote MQTT clients. Its API provides a bunch of events related to raw protocol messages received by clients and exposes some functionalities in order to send messages to them. It's not a fully featured MQTT broker but can be used for building something like that or for protocol translation (MQTT <--> ?).
• client : it's an MQTT client which is compliant with the 3.1.1 spec. Its API provides a bunch of methods for connecting/disconnecting to a broker, publishing messages (with all three different levels of QoS) and subscribing to topics.

See the in-source docs for more details:

• Website.
• Sources.

Some examples are available for getting started with the server under the vertx-examples project.

Running tests

Run all tests. The client is tested with the mqtt.eclipse.org server.

> mvn verify

Run all tests. The client is tested with a Docker container started by Maven on the 1884 port

> mvn verify -Plocal_test

Like above but with an manual Docker container start/stop.

> docker run -d -p 1884:1883 ansi/mosquitto
> mvn verify -Dmqtt.server.host=localhost -Dmqtt.server.port=1884

vertx-mqtt's Issues

After a Network Connection is established how server to determine the connection is active?

Hi, if a Network Connection is established,but the client do not send a CONNECT packet , according to MQTT 3.1.1 specification, the server should close the Network connection after a reasonable amount of time, In this case, How to solve this problem?
I think should edit in here,

@Override
  public MqttServer listen(int port, String host, Handler<AsyncResult<MqttServer>> listenHandler) {
    Handler<MqttEndpoint> h1 = endpointHandler;
    Handler<Throwable> h2 = exceptionHandler;
    server.connectHandler(so -> {
      NetSocketInternal soi = (NetSocketInternal) so;
      ChannelPipeline pipeline = soi.channelHandlerContext().pipeline();
      // here add a IdleStateHandler to close the inactive connection,and remove theIdleStateHandler  when CONNECT packet arrive.
      pipeline.addBefore("handler", "idle", new IdleStateHandler(0, 0, timeout));
      pipeline.addBefore("handler", "mqttEncoder", MqttEncoder.INSTANCE);
      if (this.options.getMaxMessageSize() > 0) {
        pipeline.addBefore("handler", "mqttDecoder", new MqttDecoder(this.options.getMaxMessageSize()));
      } else {
        // max message size not set, so the default from Netty MQTT codec is used
        pipeline.addBefore("handler", "mqttDecoder", new MqttDecoder());
      }

      MqttConnection conn = new MqttConnection(soi, options);

      soi.messageHandler(msg -> {
        conn.handleMessage(safeObject(msg, soi.channelHandlerContext().alloc()));
      });

      conn.init(h1, h2);

    });
    server.listen(port, host, ar -> listenHandler.handle(ar.map(this)));
    return this;
  }

what do yo think about this ?

Add test case for "Wrong message type"

In order to test the "Wrong message type" case in the MqttConnection class we need to send a poisoned MQTT packed with an unknown message type.

Respecting idleTimeout from MqttServerOptions (for disconnecting inactive clients)

We are implementing a server app based on vertx-mqtt-server.
We would like to have a possibility of defining global idle timeout for incoming connections.
There already is setIdleTimeout method inherited from TCPSSLOptions, but idleTimeout field is not used while IdleStateHandler is being set.
MQTT 3.1.1 spec says: "Note that a Server is permitted to disconnect a Client that it determines to be inactive or non-responsive at any time, regardless of the Keep Alive value provided by that Client."
According to the above statement in our opinion the idleTimeout value should have a precedence over keep alive value set by a mqtt client in CONNECT packet.
We can prepare a PR if you think that this change is appropriate. Please let us know in the comments.

Typo in the java docs: 'publishRelease' is used in place of 'publishReceived'

In java doc (index.adoc) in Handling client published message paragraph it is mentioned that publishReceived needs to be used for Qos of EXACTLY_ONCE but in the code block publishRelease is used instead.

NullPointer when message couldn't be parsed

There is a problem in MqttServerImpl when the incoming message couldn't be parsed.
In safeObject right after if (msg instanceof io.netty.handler.codec.mqtt.MqttMessage) there is the following statement: switch (mqttMessage.fixedHeader().messageType()) it accesses the message content without evaluating MqttMessage.decoderResult first. As all elements of a message are empty in case of a decoding error you will receive a NullPointerException.

When will this feature be realeased？

I find it has been now in tech preview for a long time. Thanks.

checking for memory leak when connect fails

Hi,

According to this issue, i have decided to check is there any memory leak in cases when an attempt at establishing a connection is getting failed.

Here is a reproducer:

       Vertx vertx = Vertx.vertx();
        CountDownLatch countDownLatch = new CountDownLatch(64);
        for (int j = 0; j < 64; j++) {
            new Thread(() -> {
                MqttClient client = MqttClient.create(vertx);
                CyclicBarrier cyclicBarrier = new CyclicBarrier(2);
                for (int i = 0; i < 10000; i++) {
                    client.connect(1883, "localhost", s -> {
                        try {
                            cyclicBarrier.await();
                        } catch (InterruptedException | BrokenBarrierException e) {
                            e.printStackTrace();
                        }
                    });
                    try {
                        cyclicBarrier.await();
                    } catch (InterruptedException | BrokenBarrierException e) {
                        e.printStackTrace();
                    }
                    cyclicBarrier.reset();
                }
                countDownLatch.countDown();
            }).start();
        }
        countDownLatch.await();
        vertx.close();

While running this code I was tracking a heap size. Here is how it was changing during the reproducer execution:

So, taking into account the screenshot I would say that there are no memory leaks.

• java version

java version "1.8.0_152"
Java(TM) SE Runtime Environment (build 1.8.0_152-b16)
Java HotSpot(TM) 64-Bit Server VM (build 25.152-b16, mixed mode)

• vertx-mqtt version: build based on e616d14

Vert.X MQTT Router

What is your opinion on adding a Router for handling MQTT messages from clients?

Something very similar to the Vert.X Web Router.

Potential usage:

MqttServer mqttServer = MqttServer.create(vertx, options);

MqttEndpoint mqttEndpoint;

Router router = Router.router(vertx);

mqttServer.endpointHandler(endpoint -> {

      endpoint
            .publishHandler(router::accept)
            .publishReleaseHandler(endpoint::publishComplete);
      mqttEndpoint = endpoint;
      endpoint.accept(true);

}).listen( ... );

Route tempsGroup1 = router.route().path("/groupOne/temperatureSensor/:deviceId");
Route lightGroup2 = router.route().path("/groupTwo/lightSensor/:deviceId");

tempsGroup1.handler(routingContext -> {

  String deviceId = routingContext.message().getParam("deviceId");
  Buffer payload = routingContext.message().payload();
  // handle temperature data
        
  if (routingContext.message.qosLevel() == MqttQoS.AT_LEAST_ONCE) {
    mqttEndpoint.publishAcknowledge(routingContext.message.messageId());
  } else if (routingContext.message.qosLevel() == MqttQoS.EXACTLY_ONCE) {
    mqttEndpoint.publishRelease(routingContext.message.messageId());
  }

});

I think it will add a ton of convenience instead of manually handling every published message.

My use case requires an API, but I prefer MQTT instead of HTTP since it is IoT based.

Default endpoint settings

By default when sending SUBSCRIBE server doesn't respond with a SUBACK packet. To enable responding we need to execute:

 endpoint.subscriptionAutoAck(true);

and the same situation with a PUBLISH packet.

But according to MQTT spec 3.1.1. Article 3.8.4 SUBSCRIBE Response: "when the Server receives a SUBSCRIBE Packet from a Client, the Server MUST respond with a SUBACK Packet". So, by default, it does not respond.

And my suggestion is that enable subscriptionAutoAck by default.

Add handling client with "zero length" client-id

In this case the server MUST check that clean session is true otherwise refuses connection.
If it can accept connection, a unique client-id must be generated and assigned to the endpoint.

MqttQos not there

hi,
Am trying to publish a message but looks like MqttQos class/enum is not in the netty library anymore hence code can't compile.i have imported all the library but looks like its missing in the entire vertx stack

Client Interoperability between Paho and Vertx: Multi level topics

Great to see MQTT additions to Vertx, Thank you.

We seem to have a issue with client side interoperability. For the server we do use https://github.com/GruppoFilippetti/vertx-mqtt-broker (latest)

• Paho Java client version 1.2.0
• Vertx Java client version 3.5.0

Multi level topic subscription with # wildcard (Example xxx/report/new/#)

Note: Message sent to xxx/report/new/123

1. Paho to Paho Subscription: works
2. Vertx to Vertx MQTT Client Subscription: works
3. Paho Sent Message is Received by Vertx client subscribing it: works
4. Vertx client sent message receiving by Paho Client: DOESN'T WORK

Just to add, in the case where there is a issue single level topic messages sent to wildcard still works.

Any idea what could be leading to this case? Thanks.

Add SSL/TLS support

MqttAuth and MqttWill as DataObject

is there a reason these classes couldn't be data object ?

they just contain data and don't have specific behavior.

"Uh oh! Event loop context executing with wrong thread!"

Under some conditions (connections breaking up with multiple clients) I do receive the following log messages:

10:21:41.572 [globalEventExecutor-1-7] WARN  i.n.util.concurrent.DefaultPromise - An exception was thrown by io.vertx.core.net.impl.ChannelProvider$$Lambda$152/1354547765.operationComplete()
java.lang.IllegalStateException: Uh oh! Event loop context executing with wrong thread! Expected null got Thread[globalEventExecutor-1-7,5,main]
	at io.vertx.core.impl.ContextImpl.lambda$wrapTask$2(ContextImpl.java:323)
	at io.vertx.core.impl.ContextImpl.executeFromIO(ContextImpl.java:200)
	at io.vertx.core.net.impl.NetClientImpl.failed(NetClientImpl.java:269)
	at io.vertx.core.net.impl.NetClientImpl.lambda$doConnect$7(NetClientImpl.java:230)
	at io.vertx.core.net.impl.ChannelProvider.lambda$connect$0(ChannelProvider.java:43)
	at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:507)
	at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:481)
	at io.netty.util.concurrent.DefaultPromise.access$000(DefaultPromise.java:34)
	at io.netty.util.concurrent.DefaultPromise$1.run(DefaultPromise.java:431)
	at io.netty.util.concurrent.GlobalEventExecutor$TaskRunner.run(GlobalEventExecutor.java:233)
	at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:138)
	at java.lang.Thread.run(Thread.java:748)

Sends multiple connection control packet throw error [MQTT-3.1.0-2]

The client sends multiple connection control packet, server through follow exception:
Unhandled exception java.lang.IllegalArgumentException: Duplicate handler name: idle

May be add the following code in MqttConnection.java

synchronized void handleMessage(Object msg) {
..........
switch (mqttMessage.fixedHeader().messageType()) {

    case CONNECT:
      
     if(endpoint == null || !endpointisConnected())
     {
            .......
     }

}

Username/Password Authentification [Question]

Simple question.

A bit confused :)
For what purpose does the client provide the username and password?
Does the client sign up at first login?
Who provides the username and password?
Or should I randomly generate 2 strings as the username and password for each client?

Changing client connect API

We should change the way used today about specifying server host and port to connect (and with latest changes even the SNI server name as well).
They should be just parameters of connect method overloads and should not live inside the MqttClientOptions.
My idea is to remove them from such options that should have only MQTT strictly related options.
Wdyt @vietj @Sammers21?

Expose peer certificates on MqttEndpoint

Just like HTTP connection, when SSL client authentication is enabled, that's useful to grab the peer (client) certificates.

That information is available on ConnectionBase, but not exposed through the MqttEndpoint interface.

Unsubscribe not working when there is more than 1 client

I have two MqttClient's working at the same time (each one in its own verticle). I do the following:

1. subscribe to topic X in client A
2. unsubscribe from topic X in client A
3. subscribe to topic X in client B

The result is that both client A and B now receive all the messages to topic X. I verified in WireShark that only one copy of the packets actually come in from the broker.

Add support for running the server in verticles

Building PR branches

I think that it is a very cool when you can see that all works(or not) directly on a PR page. So having something like that:

Is it possible to configure building of each PR and attaching of the build results?

Related discussion in Google Groups: https://groups.google.com/forum/#!topic/vertx-dev/e0Q11J6TSM4

The question of CONNECT packet check

All handleXXXX methods have do this kind of check except handleConnect method not, because the handleConnect method have a little difference with other handleXXXX methods, Consider this situation: if a client send CONNECT packet twice within one net connection, of course this is not permitted by mqtt 3.1.1 specification for client, but the handleConnect method in MqttConnection class will create another new MqttEndpointImpl Object, which is not conform to specification, While in top level when write a mqtt broker, there will be no method to do this kind of check

Cannot connect to Vertx MQTT Server #1

Hi,

i can not to connect to vertx mqqt server. See message:

java.lang.NoSuchMethodError: io.vertx.core.net.NetClient.connect(ILjava/lang/String;Ljava/lang/String;Lio/vertx/core/Handler;)Lio/vertx/core/net/NetClient;
[INFO] at io.vertx.mqtt.impl.MqttClientImpl.connect(MqttClientImpl.java:136)
[INFO] at io.vertx.book.message.HelloConsumerMicroserviceMqtt.start(HelloConsumerMicroserviceMqtt.java:34)
[INFO] at io.vertx.core.AbstractVerticle.start(AbstractVerticle.java:111)
[INFO] at io.vertx.core.impl.DeploymentManager.lambda$doDeploy$8(DeploymentManager.java:434)
[INFO] at io.vertx.core.impl.ContextImpl.lambda$wrapTask$2(ContextImpl.java:324)
[INFO] at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
[INFO] at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:403)
[INFO] at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:445)
[INFO] at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)
[INFO] at java.lang.Thread.run(Thread.java:748)
[INFO] jan 24, 2018 5:35:29 PM io.vertx.core.impl.launcher.commands.VertxIsolatedDeployer

how I kown the server publish message （use qos1） is success

when I publish message1 use method
MqttEndpoint publish(topic, Buffer payload, qosLevel, isDup, isRetain);

and i can get messageId by method
int lastMessageId()

and i kown one message is success when i get

client.publishAcknowledgeHandler(id->{
   String msg = getMailBox(client).getMsg(id);
   System.out.println("success:"+id+", msg:"+msg);
});

but in multi thread , the messageId maybe not the real messageId, when an other thread send message2 at same time

so if i want get real messageId for message1, i must use lock, is that?

Using `setIdleTimeout` causes duplicate filter exception

Setting the idle timeout for a client using:

final MqttClientOptions options = new MqttClientOptions();
options.setIdleTimeout(10);

Causes the following exception:

14:31:09.663 [vert.x-eventloop-thread-1] ERROR io.vertx.core.impl.ContextImpl - Unhandled exception
java.lang.IllegalArgumentException: Duplicate handler name: idle
	at io.netty.channel.DefaultChannelPipeline.checkDuplicateName(DefaultChannelPipeline.java:1071)
	at io.netty.channel.DefaultChannelPipeline.filterName(DefaultChannelPipeline.java:302)
	at io.netty.channel.DefaultChannelPipeline.addBefore(DefaultChannelPipeline.java:259)
	at io.netty.channel.DefaultChannelPipeline.addBefore(DefaultChannelPipeline.java:249)
	at io.vertx.mqtt.impl.MqttClientImpl.initChannel(MqttClientImpl.java:608)
	at io.vertx.mqtt.impl.MqttClientImpl.lambda$doConnect$2(MqttClientImpl.java:180)
	at io.vertx.core.net.impl.NetClientImpl.lambda$connect$2(NetClientImpl.java:118)
	at io.vertx.core.net.impl.NetClientImpl.lambda$null$9(NetClientImpl.java:256)
	at io.vertx.core.impl.ContextImpl.lambda$wrapTask$2(ContextImpl.java:342)
	at io.vertx.core.impl.ContextImpl.executeFromIO(ContextImpl.java:200)
	at io.vertx.core.net.impl.NetClientImpl.lambda$connected$10(NetClientImpl.java:252)
	at io.vertx.core.net.impl.VertxHandler.setConnection(VertxHandler.java:50)
	at io.vertx.core.net.impl.VertxNetHandler.handlerAdded(VertxNetHandler.java:44)
	at io.netty.channel.DefaultChannelPipeline.callHandlerAdded0(DefaultChannelPipeline.java:606)
	at io.netty.channel.DefaultChannelPipeline.addLast(DefaultChannelPipeline.java:235)
	at io.netty.channel.DefaultChannelPipeline.addLast(DefaultChannelPipeline.java:201)
	at io.vertx.core.net.impl.NetClientImpl.connected(NetClientImpl.java:262)
	at io.vertx.core.net.impl.NetClientImpl.lambda$doConnect$7(NetClientImpl.java:217)
	at io.vertx.core.net.impl.ChannelProvider.lambda$connect$0(ChannelProvider.java:41)
	at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:507)
	at io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:500)
	at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:479)
	at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:420)
	at io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:104)
	at io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:82)
	at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.fulfillConnectPromise(AbstractNioChannel.java:306)
	at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:341)
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:633)
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:580)
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:497)
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:459)
	at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)
	at java.base/java.lang.Thread.run(Thread.java:844)

Module name should be vertx-mqtt and not vertx-mqtt-server

Current we have

@ModuleGen(name = "vertx-mqtt-server", groupPackage = "io.vertx")

which generates code which an invalid package for javascript and ruby

Maximum msg payload is fixed to 8KB

MqttServerImpl instanciate MqttDecoder from netty-codec-mqtt library without parameters, which fixes the max possible message payload to 8092 (bytes) (seed https://github.com/netty/netty/blob/4.1/codec-mqtt/src/main/java/io/netty/handler/codec/mqtt/MqttDecoder.java#L64)

This might be a problematic limitation in some use-cases so mqtt-server should have an option to change this limit.

I'd propose to use NetworkOptions.receiveBufferSize to parameter this limit.
See https://github.com/Sqooba/vertx-mqtt-server/tree/msg-more-than-8k branch to somehow illustrate my proposal, while that's probably not the final version ;)

Keep alive mechanism doesn't work

If the client connects specifying a keep alive interval and it doesn't send new packets (at least the PINGREQ), the server doesn't disconnect the client. The idle mechanism provided by Netty isn't setup properly so no "idle" event on the socket is raised.

mqtt2kafka application run a deadlock for a period of time

• vertx 3.5.0
• netty 4.1.8.Final

I have a problem，the mqtt to Kafka application receives 51 messages every 2 seconds after 20 hours of deadlock,

MqttService

  Future<MqttClient> mqttClientFuture = Future.future();
        vertx.executeBlocking(future -> {
            client = MqttClient.create(vertx, options);
            future.complete(client);
        }, res -> {
            mqttClientFuture.complete((MqttClient) res.result());
        });
        mqttClientFuture.setHandler(result -> {
            client.rxConnect(port, host).flatMap(ack -> client.rxSubscribe(topic, 0)).subscribe(sr -> {
                log.debug("mqtt  connected  ,packetId: {}", sr);
            }, error -> {
                log.error("mqtt  connect error: {}", error);
            });
            log.debug("client: {},context: {}", client, vertx.getDelegate().getOrCreateContext());
            client.publishHandler(message -> {
              
                    kafkaService.sendMsg(kafkaTopic, message.payload().toString().getBytes());
                
            });

KafkaService

    @Override
    public void sendMsg(String topic, byte[] msg) {
        producer.rxWrite(KafkaProducerRecord.create(topic, msg)).subscribe(metadata -> {
        });
    }

vert.x-worker-thread-4 Mqtt subscribes to tiopic and accepts mqtt messages

vert.x-eventloop-thread-1 The message received by mqtt is forwarded to Kafka

Found one Java-level deadlock:
=============================
"vert.x-worker-thread-4":
  waiting to lock monitor 0x00007efef4020788 (object 0x0000000086964490, a io.vertx.mqtt.impl.MqttClientConnection),
  which is held by "vert.x-eventloop-thread-1"
"vert.x-eventloop-thread-1":
  waiting to lock monitor 0x00007efefc0287d8 (object 0x00000000869644b0, a io.vertx.core.net.impl.NetSocketImpl),
  which is held by "vert.x-worker-thread-4"

Java stack information for the threads listed above:
===================================================
"vert.x-worker-thread-4":
	at io.vertx.mqtt.impl.MqttClientConnection.handleMessage(MqttClientConnection.java:61)
	- waiting to lock <0x0000000086964490> (a io.vertx.mqtt.impl.MqttClientConnection)
	at io.vertx.mqtt.impl.MqttClientImpl.lambda$null$0(MqttClientImpl.java:183)
	at io.vertx.mqtt.impl.MqttClientImpl$$Lambda$118/252270494.handle(Unknown Source)
	at io.vertx.core.net.impl.NetSocketImpl.handleMessageReceived(NetSocketImpl.java:356)
	- locked <0x00000000869644b0> (a io.vertx.core.net.impl.NetSocketImpl)
	at io.vertx.core.net.impl.NetClientImpl$1.handleMessage(NetClientImpl.java:247)
	at io.vertx.core.net.impl.NetClientImpl$1.handleMessage(NetClientImpl.java:244)
	at io.vertx.core.net.impl.VertxHandler.lambda$channelRead$1(VertxHandler.java:150)
	at io.vertx.core.net.impl.VertxHandler$$Lambda$122/1742062813.run(Unknown Source)
	at io.vertx.core.impl.ContextImpl.lambda$wrapTask$2(ContextImpl.java:342)
	at io.vertx.core.impl.ContextImpl$$Lambda$31/1199262943.run(Unknown Source)
	at io.vertx.core.impl.TaskQueue.run(TaskQueue.java:80)
	at io.vertx.core.impl.TaskQueue$$Lambda$28/648680157.run(Unknown Source)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)
"vert.x-eventloop-thread-1":
	at io.vertx.core.net.impl.ConnectionBase.writeToChannel(ConnectionBase.java:109)
	- waiting to lock <0x00000000869644b0> (a io.vertx.core.net.impl.NetSocketImpl)
	at io.vertx.core.net.impl.ConnectionBase.writeToChannel(ConnectionBase.java:127)
	at io.vertx.core.net.impl.NetSocketImpl.writeMessage(NetSocketImpl.java:116)
	at io.vertx.mqtt.impl.MqttClientConnection.writeMessage(MqttClientConnection.java:240)
	at io.vertx.mqtt.impl.MqttClientImpl.write(MqttClientImpl.java:642)
	- locked <0x0000000086964490> (a io.vertx.mqtt.impl.MqttClientConnection)
	at io.vertx.mqtt.impl.MqttClientImpl.ping(MqttClientImpl.java:509)
	at io.vertx.mqtt.impl.MqttClientImpl$1.userEventTriggered(MqttClientImpl.java:618)
	at io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:329)
	at io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:315)
	at io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:307)
	at io.netty.handler.timeout.IdleStateHandler.channelIdle(IdleStateHandler.java:371)
	at io.netty.handler.timeout.IdleStateHandler$WriterIdleTimeoutTask.run(IdleStateHandler.java:529)
	at io.netty.handler.timeout.IdleStateHandler$AbstractIdleTask.run(IdleStateHandler.java:466)
	at io.netty.util.concurrent.PromiseTask$RunnableAdapter.call(PromiseTask.java:38)
	at io.netty.util.concurrent.ScheduledFutureTask.run(ScheduledFutureTask.java:120)
	at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
	at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:403)
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:463)
	at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)
	at java.lang.Thread.run(Thread.java:745)

Found 1 deadlock.

Client SNI support

currently it seems that the client does not have support for SNI server name when connecting to a server.

Problem with an example of code translation into ruby/js/kotlin/groovy

Take a look at the example of ruby/js/kotlin/groovy MQTT client usage code at documentation.

During the compile maven goal execution such error occurs when generating sources for ruby/js/kotlin/groovy:

Cannot generate VertxMqttClientExamples#example8 : It is not be possible to render an async result directly
Cannot generate VertxMqttClientExamples#example8 : It is not be possible to render an async result directly
Cannot generate VertxMqttClientExamples#example8 : It is not be possible to render an async result directly
Cannot generate VertxMqttClientExamples#example8 : It is not be possible to render an async result directly

How can it be fixed?

Message not reaching topic subscribers.

I know you are still doing active development but i was evaluating this module, but i am facing an issue when any publisher send a message to the subscribers. Subscribers are not getting any message.

Code used

package tech.pinhole.service.verticle;

import io.netty.handler.codec.mqtt.MqttQoS;
import io.vertx.core.AbstractVerticle;
import io.vertx.mqtt.MqttServer;
import io.vertx.mqtt.MqttTopicSubscription;

import java.nio.charset.Charset;
import java.util.ArrayList;
import java.util.List;

/**
 * @author tosheer.kalra
 */
public class MqttServerVerticle extends AbstractVerticle {


    private MqttServer mqttServer;

    @Override
    public void start() throws Exception {

        mqttServer = MqttServer.create(vertx);

        mqttServer.endpointHandler(endpoint -> {

            // shows main connect info
            System.out.println("MQTT client [" + endpoint.clientIdentifier() + "] request to connect, clean session = " + endpoint.isCleanSession());

            System.out.println("[keep alive timeout = " + endpoint.keepAliveTimeSeconds() + "]");

            // accept connection from the remote client
            endpoint.accept(true);

            // handling requests for subscriptions
            endpoint.subscribeHandler(subscribe -> {
                System.out.println("Subscription came. ");
                List<MqttQoS> grantedQosLevels = new ArrayList<>();
                for (MqttTopicSubscription s : subscribe.topicSubscriptions()) {
                    System.out.println("Subscription for " + s.topicName() + " with QoS " + s.qualityOfService());
                    grantedQosLevels.add(s.qualityOfService());
                }
                // ack the subscriptions request
                endpoint.subscribeAcknowledge(subscribe.messageId(), grantedQosLevels);

            });

            // handling requests for unsubscriptions
            endpoint.unsubscribeHandler(unsubscribe -> {

                for (String t : unsubscribe.topics()) {
                    System.out.println("Unsubscription for " + t);
                }
                // ack the subscriptions request
                endpoint.unsubscribeAcknowledge(unsubscribe.messageId());
            });

            // handling ping from client
            endpoint.pingHandler(v -> {

                System.out.println("Ping received from client");
            });

            // handling disconnect message
            endpoint.disconnectHandler(v -> {

                System.out.println("Received disconnect from client");
            });

            // handling closing connection
            endpoint.closeHandler(v -> {

                System.out.println("Connection closed");
            });

            // handling incoming published messages
            endpoint.publishHandler(message -> {

                System.out.println("Just received message on [" + message.topicName() + "] payload [" + message.payload().toString(Charset.defaultCharset()) + "] with QoS [" + message.qosLevel() + "]");

                endpoint.publish(message.topicName(),
                        message.payload(),
                        message.qosLevel(),
                        false,
                        false);

                if (message.qosLevel() == MqttQoS.AT_LEAST_ONCE) {
                    endpoint.publishAcknowledge(message.messageId());
                } else if (message.qosLevel() == MqttQoS.EXACTLY_ONCE) {
                    endpoint.publishReceived(message.messageId());
                }
            }).publishReleaseHandler(messageId -> {
                endpoint.publishComplete(messageId);
            });

        }).listen(ar -> {
            if (ar.succeeded()) {
                System.out.println("MQTT server is listening on port " + ar.result().actualPort());
            } else {
                System.out.println("Error on starting the server" + ar.cause());
            }
        });


    }

}

So When i am publishing any message from MQTT client to a topic. I can see the publish handler is getting invoked, but messages are not reaching the MQTT client which are subscriber to the same topic.

Add back-pressure support

MQTT doesn't have flow control but back-pressure support can be added at TCP level (as Vert.x websocket implementation)

Vertx mqtt server not work with MqttCalback

Hi,

I am trying to access the Vert.x mqtt server from client paho but method callback not work. I need help for to solution. Folowing the implementation of Vert.x Mqtt Server and client paho.

i have implementation of according with: https://vertx.io/docs/vertx-mqtt-server/java/.

@component(immediate = true)
@instantiate
public class VertxMqtt extends AbstractVerticle{

public static final String MQTT_SERVER_HOST = "localhost";
public static final int MQTT_SERVER_PORT = 1883;



private static final Logger LOG = LoggerFactory.getLogger(VertxMqtt.class);

public void init() {
    System.out.println("Inicializando Vertx");
}

private static void handleSubscription(MqttEndpoint endpoint) {
    endpoint.subscribeHandler(subscribe -> {

        List grantedQosLevels = new ArrayList<>();
        for (MqttTopicSubscription s : subscribe.topicSubscriptions()) {

            LOG.info("Subscription for " + s.topicName() + " with QoS " + s.qualityOfService());
            System.out.println("Subscription for " + s.topicName() + " with QoS " + s.qualityOfService());

            grantedQosLevels.add(s.qualityOfService());
        }

        endpoint.subscribeAcknowledge(subscribe.messageId(), grantedQosLevels);

    });
}

private static void handleUnsubscription(MqttEndpoint endpoint) {
    endpoint.unsubscribeHandler(unsubscribe -> {

        for (String t : unsubscribe.topics()) {
            LOG.info("Unsubscription for " + t);
        }

        endpoint.unsubscribeAcknowledge(unsubscribe.messageId());
    });
}


private static void publishHandler(MqttEndpoint endpoint) {

    System.out.println("MQTT client [{}] connected" + endpoint.clientIdentifier());

    LOG.info("MQTT client [{}] connected", endpoint.clientIdentifier());

    endpoint.publishHandler(message -> {

        LOG.info("Just received message [" + message.payload().toString(Charset.defaultCharset()) + "] with QoS [" + message.qosLevel() + "]");
        System.out.println("Just received message [" + message.payload().toString(Charset.defaultCharset()) + "] with QoS [" + message.qosLevel() + "]");

        if (message.qosLevel() == MqttQoS.AT_LEAST_ONCE) {
           
          endpoint.publishAcknowledge(message.messageId());
        } else if (message.qosLevel() == MqttQoS.EXACTLY_ONCE) {
            endpoint.publishRelease(message.messageId());
        }

    }).publishReleaseHandler(messageId -> {
        
        System.out.println("vertx " + "publishComplete"  );
           
        endpoint.publishComplete(messageId);
    });

}

private static void handleClientDisconnect(MqttEndpoint endpoint) {
    endpoint.disconnectHandler(h -> {
        System.out.println("The remote client has closed the connection.");
        LOG.info("The remote client has closed the connection.");
    });
}

@Validate
public void test() throws Exception {

    MqttServerOptions opts = new MqttServerOptions();
    opts.setHost(MQTT_SERVER_HOST);
    opts.setPort(MQTT_SERVER_PORT);

    Vertx vertx = executeWithTCCLSwitch(() -> Vertx.vertx());
    
    System.out.println("MqttServer" + vertx);

    MqttServer mqttServer = executeWithTCCLSwitch(() -> MqttServer.create(vertx, opts));

    System.out.println("star conectadno");
    mqttServer.endpointHandler(endpoint -> {

        System.out.println("star conectadno 2");
        // accept connection from the remote client
        endpoint.accept(false);

        handleSubscription(endpoint);

        System.out.println("star conectadno 3");
        handleUnsubscription(endpoint);
        publishHandler(endpoint);
        handleClientDisconnect(endpoint);
        // shows main connect info
        LOG.info("MQTT client [" + endpoint.clientIdentifier() + "] request to connect, clean session = " + endpoint.isCleanSession());

        if (endpoint.auth() != null) {
            System.out.println("[username = " + endpoint.auth().userName() + ", password = " + endpoint.auth().password() + "]");
        }
       
      
        if (endpoint.will() != null) {
            System.out.println("[will topic = " + endpoint.will().willTopic() + " msg = " + endpoint.will().willMessage()
                    + " QoS = " + endpoint.will().willQos() + " isRetain = " + endpoint.will().isWillRetain() + "]");
        }

        
        LOG.info("[keep alive timeout = " + endpoint.keepAliveTimeSeconds() + "]");

    })
            .listen(ar -> {

                if (ar.succeeded()) {

                    LOG.info("MQTT server is listening on port " + ar.result().actualPort());
                    System.out.println("MQTT server is listening on port " + ar.result().actualPort());

                } else {

                    LOG.error("Error on starting the server");
                    ar.cause().printStackTrace();
                }
            });

    //vertx.close();
   
   
   // Sharing mqtt server
   // DeploymentOptions options = new DeploymentOptions().setInstances(2);
   // vertx.deployVerticle("br.ufba.dcc.wiser.VertxMqtt", options);
}

}

# Implementation of client paho for access Vert.x MqttServer:

public class PublisherPaho implements MqttCallback {

private static int counter = 0;

public PublisherPaho() {

    Vertx vertx = Vertx.vertx();

    try {

        MemoryPersistence persistence = new MemoryPersistence();
        MqttClient client = new MqttClient(String.format("tcp://%s:%d", "localhost", 1883), "pub-0", persistence);

        MqttConnectOptions opt = new MqttConnectOptions();

        opt.setCleanSession(false);
        
        opt.setKeepAliveInterval(30);
    opt.setAutomaticReconnect(true);
    
        client.setCallback(this);

        client.connect(opt);
        
        client.subscribe("reactivetentando", 1);
	     
        
                client.publish("reactivetentando",
                      ("reactive microservices client paho " + counter++).getBytes(),
                       1,
                        false);
        
        System.out.println("Connected to the broker.. ");
        

        System.in.read();
    //    client.disconnect();

        vertx.close();

    } catch (IOException | MqttException e) {
        e.printStackTrace();
    }

}

public static void main(String[] args) throws MqttException {

    PublisherPaho p = new PublisherPaho();

}

@Override
public void connectionLost(Throwable thrwbl) {
    Vertx vertx = Vertx.vertx();

    try {

        MemoryPersistence persistence = new MemoryPersistence();
        MqttClient client = new MqttClient(String.format("tcp://%s:%d", "localhost", 1883), "pub-0", persistence);

        MqttConnectOptions opt = new MqttConnectOptions();

        opt.setUserName("karaf");
        opt.setPassword("karaf".toCharArray());

        client.setCallback(this);
        client.connect(opt);

// vertx.setPeriodic(1000, time -> {

            try {

                client.subscribe("REACTIVE");

                client.publish("REACTIVE",
                        ("reactive microservices client paho " + counter++).getBytes(),
                        1,
                        false);

            } catch (MqttException ex) {
                System.out.println("erro 2" + ex.getMessage());
                ex.printStackTrace();
            }

// });

        System.in.read();
        //client.disconnect();

        vertx.close();

    } catch (IOException | MqttException e) {
        e.printStackTrace();
    }
}

@Override
public void messageArrived(String string, MqttMessage mm) throws Exception {
   System.out.println("-------------------------------------------------");
	System.out.println("| Topic:" + string);
	System.out.println("| Message: " + new String(mm.getPayload()));
	System.out.println("-------------------------------------------------");
}

@Override
public void deliveryComplete(IMqttDeliveryToken imdt) {
    try {
        String message = new String(imdt.getMessage().getPayload());
        System.out.println("deliveryComplet " + message);
    } catch (MqttException e) {
        System.out.println(e.getMessage());
    }
}

}

Package as an OSGi bundle

vertx-mqtt-server is not packaged as an OSGi bundle yet.

It needs to export everything except internal packages.

MqttClient seems not to re-connect automatically

The MqttClientOptions do have a field reconnectAttempts, but it seems as if this field is not actually used and the client does not automatically re-connect.

If the server need a CONNECT packet repetitions check

does here need a CONNECT repetitions check like this?

 /**
   * Handle the MQTT message received by the remote MQTT client
   *
   * @param msg message to handle
   */
  synchronized void handleMessage(Object msg) {

    // handling directly native Netty MQTT messages, some of them are translated
    // to the related Vert.x ones for polyglotization
    if (msg instanceof io.netty.handler.codec.mqtt.MqttMessage) {

      io.netty.handler.codec.mqtt.MqttMessage mqttMessage = (io.netty.handler.codec.mqtt.MqttMessage) msg;

      DecoderResult result = mqttMessage.decoderResult();
      if (result.isFailure()) {
        chctx.pipeline().fireExceptionCaught(result.cause());
        return;
      }
      if (!result.isFinished()) {
        chctx.pipeline().fireExceptionCaught(new Exception("Unfinished message"));
        return;
      }

      // does here need a CONNECT repetitions check like this?
      if(endpoint == null && mqttMessage.fixedHeader().messageType() != MqttMessageType.CONNECT) {
        endpoint.close();
      }
      if(endpoint != null && mqttMessage.fixedHeader().messageType() == MqttMessageType.CONNECT) {
        endpoint.close();
      }
     ...

Overload client connect method with method that takes host and port parameter

So one can simply write:

client.create(port, host, ch -> { ... })

Add logging to the server

It's useful to have logging in the main parts of the server mainly printing messages which are sent and received by the server itself.

Add examples and documentation for SSL/TLS support

Add a closeHandler to the client

It could be useful to have the possibility to pass an handler to the client which is called when the connection is closed. In MQTT it can happen that the server closes the connection for different reasons so it's important to have the client being notified about that.

Consider usage of new MqttMessageBuilders from Netty

Rename client unsubscribeCompleteHandler to unsubscribeCompletionHandler

usually we use the terminology completionHandler , e.g MessageConsumer#completionHandler

Develop the persistence and recovery for "in flight" messages queues

The client should be able to persist the "in flight" messages queues (QoS 1 and 2) in order to recover them after coming back online (due to a previous disconnection).

Add an exception handler on MqttServer to be aware of error happening outside the MqttEndpoint

Mqtt Client doesn't close connection when there is no reply to CONNECT

Assuming the following happens:

• TCP connect is OK
• Client sends CONNECT
• Waits for CONACK

If the CONACK never is received by the client, but still the TCP connection remains open, then MQTT Client never reports back.

paho client in python won't work

when i connect my paho client (in python) to vertx mqtt server, it always raise the same error:

java.lang.Exception: Wrong message type
	at io.vertx.mqtt.impl.MqttServerConnection.handleMessage(MqttServerConnection.java:178)
	at io.vertx.mqtt.impl.MqttServerImpl.lambda$null$3(MqttServerImpl.java:89)
	at io.vertx.core.net.impl.NetSocketImpl.handleMessageReceived(NetSocketImpl.java:356)
	at io.vertx.core.net.impl.NetServerImpl$2.handleMessage(NetServerImpl.java:445)
	at io.vertx.core.net.impl.NetServerImpl$2.handleMessage(NetServerImpl.java:442)
	at io.vertx.core.net.impl.VertxHandler.lambda$channelRead$1(VertxHandler.java:150)
	at io.vertx.core.impl.ContextImpl.lambda$wrapTask$2(ContextImpl.java:342)
	at io.vertx.core.impl.TaskQueue.run(TaskQueue.java:80)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

WebSocket support

It would be great to have vertx-mqtt-server supporting WebSockets.
Are there plans to add WebSocket support in the near future?
Is it a difficult task?

exceptionHandler not used for lower level exceptions

I am running into the following issue:
I am using the following simple MQTT-Verticle

public class MqttVerticle extends AbstractVerticle{
  @Override
  public void start() throws Exception {
    Integer port = config().getInteger("mqttPort");

    System.out.println("STARTING "+port);
    MqttServer server = MqttServer.create(vertx);
    server.exceptionHandler(e -> {
      System.out.println("SERVER EXCEPTION " + port);
    });

    server.endpointHandler(endpoint -> {
      endpoint.exceptionHandler(e -> {
        System.out.println("ENDPOINT EXCEPTION "+e.toString());
      });
    });
    server.listen(port);
  }
}

I want to loadbalance using haproxy and I use the TCP-check for that. This causes logging of tons of java.io.IOException: Connection reset by peer.

None of the above exceptionHandlers is ever hit and the logs are caused by this block in ConnectionBase:

protected synchronized void handleException(Throwable t) {
    this.metrics().exceptionOccurred(this.metric(), this.remoteAddress(), t);
    if(this.exceptionHandler != null) {
      this.exceptionHandler.handle(t);
    } else {
      log.error(t);
    }

  }

My bet would be that we need to add exceptionHandler(rejectHandler); in MqttConnection.init.