liambindle / mqtt-c

A portable MQTT C client for embedded systems and PCs alike.

MQTT-C is an MQTT v3.1.1 client written in C. MQTT is a lightweight publisher-subscriber-based messaging protocol that is commonly used in IoT and networking applications where high-latency and low data-rate links are expected. The purpose of MQTT-C is to provide a portable MQTT client, written in C, for embedded systems and PC's alike. MQTT-C does this by providing a transparent Platform Abstraction Layer (PAL) which makes porting to new platforms easy. MQTT-C is completely thread-safe but can also run perfectly fine on single-threaded systems making MQTT-C well-suited for embedded systems and microcontrollers. Finally, MQTT-C is small; there are only two source files totalling less than 2000 lines.

A note from the author

It's been great to hear about all the places MQTT-C is being used! Please don't hesitate to get in touch with me or submit issues on GitHub!

Getting Started

To use MQTT-C you first instantiate a struct mqtt_client and initialize it by calling @ref mqtt_init.

    struct mqtt_client client; /* instantiate the client */
    mqtt_init(&client, ...);   /* initialize the client */

Once your client is initialized you need to connect to an MQTT broker.

    mqtt_connect(&client, ...); /* send a connection request to the broker. */

At this point the client is ready to use! For example, we can subscribe to a topic like so:

    /* subscribe to "toaster/temperature" with a max QoS level of 0 */
    mqtt_subscribe(&client, "toaster/temperature", 0);

And we can publish to a topic like so:

    /* publish coffee temperature with a QoS level of 1 */
    int temperature = 67;
    mqtt_publish(&client, "coffee/temperature", &temperature, sizeof(int), MQTT_PUBLISH_QOS_1);

Those are the basics! From here the examples and API documentation are good places to get started.

Building

There are only two source files that need to be built, mqtt.c and mqtt_pal.c. These files are ANSI C (C89) compatible, and should compile with any C compiler.

Then, simply #include <mqtt.h>.

Alternatively, you can build MQTT-C with CMake or the provided Makefile. These are provided for convenience.

Documentation

Pre-built documentation can be found here: https://liambindle.ca/MQTT-C. Be sure to check out the examples too.

The @ref api documentation contains all the documentation application programmers should need. The @ref pal documentation contains everything you should need to port MQTT-C to a new platform, and the other modules contain documentation for MQTT-C developers.

Testing and Building the Tests

The MQTT-C unit tests use the cmocka unit testing framework. Therefore, cmocka must be installed on your machine to build and run the unit tests. For convenience, a simple "makefile" is included to build the unit tests and examples on UNIX-like machines. The unit tests and examples can be built as follows:

    $ make all

The unit tests and examples will be built in the "bin/" directory. The unit tests can be run like so:

    $ ./bin/tests [address [port]]

Note that the \c address and \c port arguments are both optional to specify the location of the MQTT broker that is to be used for the tests. If no \c address is given then the Mosquitto MQTT Test Server will be used. If no \c port is given, port 1883 will be used.

Portability

MQTT-C provides a transparent platform abstraction layer (PAL) in mqtt_pal.h and mqtt_pal.c. These files declare and implement the types and calls that MQTT-C requires. Refer to @ref pal for the complete documentation of the PAL.

Contributing

Please feel free to submit issues and pull-requests here. When submitting a pull-request please ensure you have fully documented your changes and added the appropriate unit tests.

License

This project is licensed under the MIT License. See the "LICENSE" file for more details.

Authors

MQTT-C was initially developed as a CMPT 434 (Winter Term, 2018) final project at the University of Saskatchewan by:

• Liam Bindle
• Demilade Adeoye

mqtt-c's Issues

PINGRESP callback

Hi Liam,

Does this library provide a way to notify my application of a PINGRESP message? I can't find it in the docs, and I didn't immediately see anything regarding this in the source. If this functionality was intentionally left out of the library, just let me know. If not, I'm willing to take a shot at implementing it.

In my system, I have multiple brokers that could disappear silently at any moment, and I'm looking for a clean way to detect if they're still alive. Let me know if you know of a better way to accomplish this.

Thanks,
Josh

Automatic reconnect

Supporting the auto-reconnecting feature that some other MQTT libraries support would be a good enhancement. I'm not yet familiar with this feature, but it was requested by @jedidiahuang.

I'm not going to have time to add this right away, but I wanted to submit this issue to start a discussion/take note that its something that would be good to add.

If anyone has any comments/suggestion feel free to include them in this thread.

Todo list

• More testing - ?
• Documentation of testing - ~1.5 hours
• Examples - ~1 hour
• Finish README - ~30 min
• Finish documentation/editting of documentation - ~1 hour

Class submission specific (we can use a lot of the documentation for these):

• Design document for project submission. - ~30 min
• Report mesaurements/simulation results (we could talk about our testing). - ~30 min
• Evaluate success of the project. - ~30 min

How To Konwn NetWork Disconnect?

Hi, There is no callback,get me network disconnect? if i want to known how to do that?

simple_publisher tested on Ubuntu 16.04 and AIX 6.1

• Ubuntu 16.04

  • gcc -Wextra -Wall -std=gnu99 -Iinclude -Wno-unused-parameter -Wno-unused-variable -Wno-duplicate-decl-specifier examples/simple_publisher.c src/mqtt.c src/mqtt_pal.c -lpthread -o bin/simple_publisher
  • terminal 1: mosquitto_sub -h test.mosquitto.org -t "/datetime"
  • terminal 2: ./bin/simple_publisher test.mosquitto.org 1883 "/datetime"

• AIX 6.1

  • cc -Iinclude examples/simple_publisher.c src/mqtt.c src/mqtt_pal.c -lpthread -o bin/simple_publisher
  • terminal 1: mosquitto_sub -h test.mosquitto.org -t "/datetime"
  • terminal 2: ./bin/simple_publisher test.mosquitto.org 1883 "/datetime"

I have struggled with compilation and test which is why I'm leaving this note here. I hope you can put this to the example folder if you think this is helpful.

Thanks for your great work.
Lucas

HoW to run simplesubscriber and publisher?

I am Totally new to this,I want to run simple_subscriber and simple_publisher in Ubuntu 14.04 .I tried buy ./bin/simple_subscriber and same for publisher in other terminal.But data not received by subscriber .
SUBSCRIBER RUNNING**********
akbarsaleem@IM-PC-102:~/akbar/mqqt2/MQTT-C/bin$ ./simple_subscriber 127.0.0.1 1888
./simple_subscriber listening for 'datetime' messages.
Press CTRL-D to exit.

./simple_subscriber disconnecting from 127.0.0.1
akbarsaleem@IM-PC-102:~/akbar/mqqt2/MQTT-C/bin$**

---

PUBLISHER***
akbarsaleem@IM-PC-102:~/akbar/mqqt2/MQTT-C/bin$ ./simple_publisher 127.0.0.1 1883
./simple_publisher is ready to begin publishing the time.
Press ENTER to publish the current time.
Press CTRL-D (or any other key) to exit.

./simple_publisher published : "The time is 2019-07-30 17:58:35"
./simple_publisher published : "The time is 2019-07-30 17:58:41"akbarsaleem@IM-PC-102:~/akbar/mqqt2/MQTT-C/bin$

---

Please provide me details to run subscriber and publisher.

Thanks
Akbar

Is it possible to compile under Mac OS ?

I modified the pre-compile condition, then I had this issue.

Modification:

//..... no changes before 
#ifdef __APPLE__
    #include <limits.h>
    #include <string.h>
    #include <stdarg.h>
    #include <time.h>
    #include <arpa/inet.h>
    #include <pthread.h>
    #include <stdint.h>
//..... no changes after 

I receive error when I try to compile:

gcc -Wextra -Wall -std=gnu99 -Iinclude -Wno-unused-parameter -Wno-unused-variable -Wno-duplicate-decl-specifier examples/simple_publisher.c src/mqtt.c src/mqtt_pal.c -lpthread -o bin/simple_publisher
src/mqtt.c:1392:20: warning: passing an object that undergoes default argument
      promotion to 'va_start' has undefined behavior [-Wvarargs]
    va_start(args, packet_id);
                   ^
src/mqtt.c:1381:74: note: parameter of type 'uint16_t' (aka 'unsigned short') is
      declared here
ssize_t mqtt_pack_subscribe_request(uint8_t *buf, size_t bufsz, uint16_t packet_id, ...
                                                                         ^
src/mqtt.c:1472:20: warning: passing an object that undergoes default argument
      promotion to 'va_start' has undefined behavior [-Wvarargs]
    va_start(args, packet_id);
                   ^
src/mqtt.c:1462:76: note: parameter of type 'uint16_t' (aka 'unsigned short') is
      declared here
ssize_t mqtt_pack_unsubscribe_request(uint8_t *buf, size_t bufsz, uint16_t packet_id...
                                                                           ^
2 warnings generated.
Undefined symbols for architecture x86_64:
  "_mqtt_pal_recvall", referenced from:
      ___mqtt_recv in mqtt-7c5027.o
  "_mqtt_pal_sendall", referenced from:
      ___mqtt_send in mqtt-7c5027.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [bin/simple_publisher] Error 1

It's trying to fetch x86_64, but I think this is not the architecture for OS X? How can I fix it..

Question about possible bug in send logic

So I was reviewing the code and see something that confuses me:

It a message is partially sent:

   if(client->send_offset < msg->size) {
              /* partial sent. Await additional calls */
              break;
            } else {

But then a resent timeout occurs:

   /* check for timeout */
            if (MQTT_PAL_TIME() > msg->time_sent + client->response_timeout) {
                resend = 1;
                client->number_of_timeouts += 1;
                client->send_offset = 0;
            }

It seems to me you will be sending broken half packets since you reset send_offset halfway through a send of a packet.

It seems this bug doesn't happen in practice because there is a bug in the send calls for unix:

ssize_t mqtt_pal_sendall(mqtt_pal_socket_handle fd, const void* buf, size_t len, int flags) {
    size_t sent = 0;
    while(sent < len) {
        ssize_t tmp = send(fd, buf + sent, len - sent, flags);
        if (tmp < 1) {
            return MQTT_ERROR_SOCKET_ERROR;
        }
        sent += (size_t) tmp;
    }
    return sent;
}

You don't seem to be handling WOULD_BLOCK or EAGAIN for sending on a non blocking socket, this seems like a bug to me also, as the code seems to be attempting to handle blocking writes.

do these seem like bugs to you? or have I misunderstood?

Platform abstraction layer needs to be written.

OpenSSL subscribe

Hi Liam,
I am working on the function for OpenSSL Subscriber. I tried combining the OpenSSL_Publisher and Simple_Subscriber to produce the function. However, there is an error that I am facing, which is Subscriber subscribes all topics, regardless which topic I have set.
Then, I want to ask that do you have any example for OpenSSL subscriber?

Thanks in advance!

MQTT 5.0 support

First: Thanks for your great work!

I want to kindly ask about the plans / roadmap of supporting MQTT 5.0.

MQTT 5.0 standard is almost finished, it's very unlikely that there will be greater changes against the current draft.

Mosquitto announced commercial sponsoring for 5.0 protocol development: https://mosquitto.org/blog/2018/05/press-release/

There's public brokers available for testing: dotnet/MQTTnet#246 (comment)

So I think now is a good time to clarify whether MQTT 5.0 support is planned, and what's the timeframe (and whether contributions are welcome).

Or just close this issue with won't fix, to document that 5.0 support is not planned...

[Edit: I've been mislead by some press articles which stated that MQTT 5.0 is officially published - while it's almost finished, official release is more likely to happen in Q3 or Q4.]

test_client_subpub fails occasionally.

It looks like one of the publishes is getting delivered more than once (entirely possible for QOS==1) and this is causing the assert_true(state==5) to fail. I'll probably need to find a better mechanism for testing that the number of ingress publishes is correct.

Output looks like this:

$ ./tests 
[==========] Running 18 test(s).
[ RUN      ] test_mqtt_fixed_header
[       OK ] test_mqtt_fixed_header
[ RUN      ] test_mqtt_pack_connection_request
[       OK ] test_mqtt_pack_connection_request
[ RUN      ] test_mqtt_unpack_connection_response
[       OK ] test_mqtt_unpack_connection_response
[ RUN      ] test_mqtt_pack_disconnect
[       OK ] test_mqtt_pack_disconnect
[ RUN      ] test_mosquitto_connect_disconnect
[       OK ] test_mosquitto_connect_disconnect
[ RUN      ] test_mqtt_pack_publish
[       OK ] test_mqtt_pack_publish
[ RUN      ] test_mqtt_pubxxx
[       OK ] test_mqtt_pubxxx
[ RUN      ] test_mqtt_pack_subscribe
[       OK ] test_mqtt_pack_subscribe
[ RUN      ] test_mqtt_unpack_suback
[       OK ] test_mqtt_unpack_suback
[ RUN      ] test_mqtt_pack_unsubscribe
[       OK ] test_mqtt_pack_unsubscribe
[ RUN      ] test_mqtt_unpack_unsuback
[       OK ] test_mqtt_unpack_unsuback
[ RUN      ] test_mqtt_pack_ping
[       OK ] test_mqtt_pack_ping
[ RUN      ] test_mqtt_connect_and_ping
[       OK ] test_mqtt_connect_and_ping
[ RUN      ] test_message_queue
[       OK ] test_message_queue
[ RUN      ] test_packet_id_lfsr
[       OK ] test_packet_id_lfsr
[ RUN      ] test_client_simple
[       OK ] test_client_simple
[ RUN      ] test_client_simple_subpub
[       OK ] test_client_simple_subpub
[ RUN      ] test_client_subpub
state == 5
tests.c:874: error: Failure!

[  FAILED  ] test_client_subpub
[==========] 18 test(s) run.
[  PASSED  ] 17 test(s).
[  FAILED  ] 1 test(s), listed below:
[  FAILED  ] test_client_subpub

Websocket support

Any plans to support websockets?

Automatic recovery from MQTT_ERROR_SEND_BUFFER_IS_FULL

Hi

Thank you for MQTT-C contribution, it looks great and I am trying to use it in my linux embedded project.
I have experienced a problem of your Simple_Publisher.c:
When it is running and I press key to publish current time,
If I press key too quick, mqtt_publish() API return error: MQTT_ERROR_SEND_BUFFER_IS_FULL and no way to recover.

If I add mqtt_sync() after mqtt_publish(), it work very well. Since you use large buffer and a thread to do
mqtt_sync() , I'd like to know how to recover when MQTT_ERROR_SEND_BUFFER_IS_FULL error appears.

Thanks a lot

mqtt_subscribe and mqtt_publish is not ok when establishing encrypted channel based on TLS1.2

Hello,
the MQTT C code is run on MCU connects to mosquitto broker. using TLS1.2 to pass handshake, then mqtt_connect to broker and send pingres, all are ok. But when sending mqtt_publish and mqtt_subscribe, the broker will has an error. see attached file.

Cmoka new repo

Hello Liam,

this is just a little note to tell you that the CMOKA website linked by your page has broken/old versions, hard to install. According to this post:

clibs/cmocka#13

...the right repo where to find the new versions, without installation issues should be this:

https://git.cryptomilk.org/projects/cmocka.git/

I installed without problems the last version taken from there, and had troubles with the versions pointed by cmoka.org website. So, consider to update your README.
In addition, I finally included your excellent library in my project, and made a MQTT full compliant A/V recording/streaming system:

https://github.com/paolo-pr/laav/blob/master/mqtt-avsystem/README.md

Any feedback is appreciated!

The will message should be an application message.

Currently the will_message argument of mqtt_connect is a const char*. This needs to be changed to a void* with a size argument.

Support for SSL/TLS

Support for encryption would be good.

Prevent Client from receiving its own messages

Hi Liam,
Basically I am joining the simple_subscriber.c and simple_publisher.c into one script, then that can send and receive message at the same time. Now I am wondering that is any way that Client can prevent to receive its own published messages?
I know that we can do it by choosing different topic names, but I still want to find a better solution.

Thanks!

Malformed Connect Packet when Will is included

Found issue where the Connect Packet is malformed when will is included.

You aren't correctly including the will message length.

The following change to the mqtt_pack_connection_request function resolves the issue:

if (connect_flags & MQTT_CONNECT_WILL_FLAG) {
    buf += __mqtt_pack_str(buf, will_topic);
    /* will message length */
    *(uint16_t*) buf = (uint16_t) MQTT_PAL_HTONS(will_message_size);
    buf += 2;
    memcpy(buf, will_message, will_message_size);
    buf += will_message_size;
}

blocking socket support

Hi @LiamBindle,
Do you support blocking mode for receiving message?
I see a busy loop to call mqtt_sync(). It will be great if you have a blocking function
Thank you

How to set connect_flags?

Hi Liam,

I've applied your update. It works well now.
Would you mind to explain how to set connect_flags while calling mqtt_connect()?
For example, I want to inform broker to clean_session when connected
Thanks a lot

Best Regards,
Jedidiahuang

Preventing client ID collisions

Hi

I foud a strange situation, If at the same machine multiple clients was created using the same name they are interferint with each other and the conections stop working.

mqtt_connect(&client, 'CLIENTE NAME, NULL, NULL, 0, NULL, NULL, 0, 400);

to overcome this situations i created a function to generate a random name each time the conection will be created.

Searching on the web i found others thereads talking about this kind of situation, like this, mqttjs/MQTT.js#684

Maybe you could add some random number at the end of client's name before start the conection to the MQTT server.

Thanks for you work

Use of the session present flag

First of all thanks for the successful MQTT library for embedded systems.

I have the request to evaluate the session present flag from the CONNACK packet. If the flag is not set, my application must re-subscribe to the topics. If the flag is set my application may not subscribe to the topics again.

How would you implement that? I would suggest to specify another callback function during initialization. This is called when a new session has been established. The callback function can then ask the application to subscribe to the topics.

error:failed to load the certificate

I am having problems with running your openssl_publisher.
I get an error saying failed to load certificate.
My broker is on Amazon AWS EC2.
However if I try to publish with mosquitto command line with same parameters I am able to publish.

First two bytes incorrect for PUBLISH's with QoS 0

Dear Sir,

I run MQTT-C simple_publisher to publish a topic 'DAQ' with payload(application message) 'TEST123456789' and run MQTT-fx (free tool) to Subscribe 'DAQ' to receive this message.
MQTT-fx show the received message with extra 2 binary code in the beginning. '?ST123456789'
(? indicates a corrupted font)

Then I publish a topic 'datetime' from MQTT-fx with a message '2018/05/08 12:23:35'
While MQTT-C simple_subscriber subscribed topic 'datetime'.
in this call back:
void publish_callback(void** unused, struct mqtt_response_publish published)
{
...
topic_name[published->topic_name_size] = '\0';
printf("Received publish('%s'): %s\n", topic_name, (const char) published->application_message);
...
}
the content of published->application_message is '18/05/08 12:23:35' , missing first 2 bytes
Is it a bug? How to fix it?

Thanks a lot.

unaligned access exception

Hi,

First, thanks for your work!

I try to use the client on a cortex-m0+ SoC, and it works great with QOS0.
With QOS1 it throws an unaligned exception, in function

ssize_t mqtt_pack_pubxxx_request(uint8_t *buf, size_t bufsz,
enum MQTTControlPacketType control_type,
uint16_t packet_id)

when packed_id is casted at (uint16_t) buf = (uint16_t) MQTT_PAL_HTONS(packet_id);

It tries to write to address buf + 67 an uint16 packed_id variable, what is an unaligned exception on some arch.
So my proposal is to change ALL uint16_t casts to uint8_t buf to use memcpy, or use some kind of put_unaligned macros.

Regards,
Andras

MQTT_ERROR_SEND_BUFFER_IS_FULL error periodically

Hello,

I'm using MQTT-C in a project and I'm very happy with it. I based my multithreaded implementation on the reconnect_subscriber example and it works very good except for one thing. I'm publishing 1 message of 155 bytes every 2.5 seconds.

I originally had a send buffer size of 102400 bytes and I saw repeated error states containing the MQTT_ERROR_SEND_BUFFER_IS_FULL error. To test if this was a spurious error I have now made the send buffer smaller to 1550 bytes. I now get the error every 6 messages.

I have multiple threads in the program:

• A sync thread which calls mqtt_sync and mqtt_mq_clean every 10ms:

while(true) {
  mqtt_sync(&mqtt_client);
  mqtt_mq_clean(&(mqtt_client.mq));
  helper_sleep_ms(10);
}

• A transmit thread which reads a FiFo queue and calls the mqtt_publish routine:

if(message.data_size == 0) {
  mqtt_publish(&_dbtalk_network_grouptalk_client, message.topic, "", 0, MQTT_PUBLISH_QOS_0);
} else {
  mqtt_publish(&_dbtalk_network_grouptalk_client, message.topic, message.data, message.data_size, MQTT_PUBLISH_QOS_0);
}
  mqtt_sync(&_dbtalk_network_grouptalk_client);
  free(message.topic);
  free(message.data);

Following is my reconnect_handler function:

void _reconnect_handler(struct mqtt_client* client, void **args)
{
    while(true) {
        close(client->socketfd);
        client->socketfd = -1;

        if(client->error != MQTT_ERROR_INITIAL_RECONNECT) {
            log_error("MQTT", "Connection was in error state: %s\n", mqtt_error_str(client->error));
        }

        int fd = _open_client_socket(MQTT_ADDRESS, MQTT_PORT);
        if(fd == -1) {
            helper_sleep_ms(500);
            continue;
        }

        mqtt_reinit(client, fd, send_buffer, SENDBUF_SIZE, receive_buffer, RECVBUF_SIZE);

        enum MQTTErrors retvalue = mqtt_connect(client, "mqtt-client", NULL, NULL, 0, NULL, NULL, 0, 400);
        if(retvalue != MQTT_OK) {
            log_error("MQTT", "Could not connect to the broker (%d): %s\n", retvalue, mqtt_error_str(retvalue));
            helper_sleep_ms(500);
            continue;
        }
        
        for(int i = 0; i < MQTT_TOPIC_COUNT; ++i) {
            const char *topic = _get_topic(i);
            
            retvalue = mqtt_subscribe(client, topic, 0);            
            if(retvalue != MQTT_OK) {
                log_error("MQTT", "Could not subscribe to topic '%s' (%d): %s\n", topic, retvalue, mqtt_error_str(retvalue));
                continue;
            }
        }

        log_info("MQTT", "Connected to the broker and subscribed to MQTT topics\n");
        return;
    }
}

Could you please give me some pointer on what is going on?

Test failing

Hi
thanks for your work running your test I get:
[rainer@elfi MQTT-C]$ ./bin/tests
Staring MQTT-C unit-tests.
Using broker: "test.mosquitto.org:1883"

[MQTT Packet Serialization/Deserialization Tests]
[==========] Running 6 test(s).
[ RUN ] TEST__framing__fixed_header
[ OK ] TEST__framing__fixed_header
[ RUN ] UH��H���
Could not run test: Test failed with exception: Segmentation fault(11)Test teardown failed
[ ERROR ] UH��H���
[ RUN ] TEST__framing__subscribe
[ OK ] TEST__framing__subscribe
[ RUN ] UH��H��H�}��E���E��E� Could not run test: Test failed with exception: Segmentation fault(11)Test teardown failed [ ERROR ] UH��H��H�}��E���E��E�
[ RUN ] TEST__framing__disconnect
[ OK ] TEST__framing__disconnect
Segmentation fault (core dumped)

Running on fedore 29, kernel 4.19.6

Any Ideas.
Thx

Support other build systems

Hello,

Currently there is only a makefile to build with gcc,
would a PR to add a visual studio solution be accepted?

Or would a system like CMake be preferred in thatcase?

Regards,

Mark

Modify `simple_subscriber` example program to correct `topic` argument use

I believe I found a minor issue in the simple_subscriber example program - the topic passed in as a command line argument is not used (rather, currently, "datetime" is always used as the subscription topic). Diff for proposed correction:

diff --git a/examples/simple_subscriber.c b/examples/simple_subscriber.c
index 33097c9..332ecbd 100644
--- a/examples/simple_subscriber.c
+++ b/examples/simple_subscriber.c
@@ -87,7 +87,7 @@ int main(int argc, const char *argv[])
     }

     /* subscribe */
-    mqtt_subscribe(&client, "datetime", 0);
+    mqtt_subscribe(&client, topic, 0);

     /* start publishing the time */
     printf("%s listening for '%s' messages.\n", argv[0], topic);
@@ -133,4 +133,4 @@ void* client_refresher(void* client)
         usleep(100000U);
     }
     return NULL;

Ingress publish with QoS 2 could deliver twice.

In __mqtt_recv the MQTT_CONTROL_PUBLISH case of the big switch statement needs a mechanism for ensuring that QoS 2 publishes are not passed to the callback more than once.

This could be accomplished by looping through the message queue and checking for messages with the same packet ID.

Test unsubscribe.

Tests for unsubscribing still need to be written.

I Wants to implement MQTT applications as full duplex application

In this simplex communication implemented but I need to implement duplex communication.
I tried that also but getting same results on both sides, I modified the code as below.
/PUBLISHER**************************/
printf("Press CTRL-D (or any other key) to exit.\n\n");
start1: while(fgetc(stdin) == '\n') {

	time_t timer;
	time(&timer);
	struct tm* tm_info = localtime(&timer);
	char timebuf[26];
	bzero(timebuf,26);
	strftime(timebuf, 26, "%Y-%m-%d %H:%M:%S", tm_info);
	char application_message[256];
	bzero(application_message,256);
	topic ="datetime";
	printf("Press ENTER to publish the current time.\n");
	snprintf(application_message, sizeof(application_message), "The time is %s", tm_info);
	mqtt_publish(&client, topic, application_message, strlen(application_message) + 1, MQTT_PUBLISH_QOS_0);
	if (client.error != MQTT_OK) {
		fprintf(stderr, "error: %s\n", mqtt_error_str(client.error));
		exit_example(EXIT_FAILURE, sockfd, &client_daemon);
	}
	topic="temperature";
	/* subscribe */
	mqtt_subscribe(&client, topic, 0);
	goto start1;
}   

/**************************************************************************************************************/
/subscriber code/
start: while(fgetc(stdin) == '\n'){

	topic="datetime";
	/* subscribe */
	mqtt_subscribe(&client, topic, 0);
	/* start publishing the time */
	char application_message[256];
	bzero(application_message,256);
	topic ="temperature";
	char temperature[]="45";
	snprintf(application_message, sizeof(application_message), "The temperature is %s", temperature); 
	printf("Press ENTER to publish the current Temperature\n");   
	getchar(); 
	mqtt_publish(&client, topic, application_message, strlen(application_message) + 1, MQTT_PUBLISH_QOS_0);
	if (client.error != MQTT_OK) {
		fprintf(stderr, "error: %s\n", mqtt_error_str(client.error));
		exit_example(EXIT_FAILURE, sockfd, &client_daemon);
	}
	
	printf("Press CTRL-D to exit.\n\n");
	goto start;
}

/*********************************************************************************************************************/

Thankyou,
Akbar

How to know if all pending messages are read

after the connection was established I subscribe to "/foo/bar/#" to read all messages (I'am interested in the retained ones - all other I skip at this time). As soon all messages are read I unsubscribe (later on subscribe to a specific topic).

This first step is to build an inventory of all retained messages at any topic below "/foo/bar/#".

Mostly this is working but I have a timing issue, since I don't know how to detect the message consumption is ready ans all messages are read - means published to the callback.
I tried to call mqtt_sync but it did not help. The only way I can make shoure I didn't miss one is to subscribe, calling mqtt_sync and then sleep up to 5 seconds. In all other cases some messages where not received.

Is there any way to detect if I got all messages?

Best Regards

Cross Compilation issue

Brother, I could not find any solution about cross-compilation of this library. How to do it any solution?

How do you know if an operation is successful

Since all the API's (subscribe, publish, connect) all return with MQTT_OK once the packet is in the queue, how can you find out if the associated SubACK, PubACK or ConnACK has been received?

uninitialised value (pid_lfsr)

Hello,

Please have a look at this line:

uint16_t __mqtt_next_pid(struct mqtt_client *client) {
if (client->pid_lfsr == 0) {
client->pid_lfsr = 163u;
}

pid_lfsr is not initialised to a value, then valgrind outputs this error:

==13420== Conditional jump or move depends on uninitialised value(s)
==13420== at 0x20256A: __mqtt_next_pid (in /home/paolo/programmi/laav-master/mqtt-avsystem/MQTTLAAVSystem)
==13420== by 0x20316E: mqtt_subscribe (in /home/paolo/programmi/laav-master/mqtt-avsystem/MQTTLAAVSystem)

As a workaround, I set it to 0 in my test program:

client->pid_lfsr = 0;

but you should fix that.

Thanks for your library. It works very well with my project, in order to make an Audio/Video Streamer/Recorder controllable with MQTT (then it would be good for Home Assistant, for example):

https://github.com/paolo-pr/laav

Please, feel free to write me an-email, so I can show you more details about possible applications.

How to get number of writable bytes in send buffer

Hi Liam,

Is there a way to get the number of writable bytes remaining in a client's send buffer? I see that there is the curr_sz field in the client's message queue, but this doesn't appear to do exactly what I need it to. This field will remain unchanged (as far as I can tell) after a call to mqtt_sync.

Say my buffer size is 4096. Initially, curr_sz will be approximately 4096 (let's say 4000). After I publish a 150-byte message, curr_sz will decrease by just over 150 bytes. So now, curr_sz is somewhere around 3800. After I call mqtt_sync, curr_sz is still 3800. What I'm looking for is a field (or an expression, perhaps) that will give me 4000 after calling mqtt_sync, indicating that there is more room in the buffer because a message has been sent.

Basically, I'm trying to avoid the MQTT_ERROR_SEND_BUFFER_IS_FULL error. I am using the automatic recovery feature. But for my use case, it would be better for my application to not attempt to send a message that won't fit in the buffer.

Thanks for your help. Fantastic library, by the way.

Use of constants MQTT_PUBLISH_QOS_[1|2] in MQTT_CONTROL_PUBLISH

Liam,

I think I may have found a bug?

In the logic where MQTT_CONTROL_PUBLISH is handled (around line 586 of mqtt.c), the QOS level is used to determine the next logic state & what to send.

In this logic, I believe use of the constants MQTT_PUBLISH_QOS_1 and MQTT_PUBLISH_QOS_2 are incorrect. They are being compared against "qos_level", which has been decoded (bit shift applied), however, the constants themselves are still bit-shifted. This led to an unexpected sequence of handshaking when I was testing with QOS 2.

I believe the patch is something like:

diff --git a/src/mqtt.c b/src/mqtt.c
index 2fcf7e4..2fae709 100644
--- a/src/mqtt.c
+++ b/src/mqtt.c
@@ -586,14 +586,14 @@ ssize_t __mqtt_recv(struct mqtt_client *client)
                 break;
             case MQTT_CONTROL_PUBLISH:
                 /* stage response, none if qos==0, PUBACK if qos==1, PUBREC if qos==2 */
-                if (response.decoded.publish.qos_level == MQTT_PUBLISH_QOS_1) {
+                if (response.decoded.publish.qos_level == 1) {
                     rv = __mqtt_puback(client, response.decoded.publish.packet_id);
                     if (rv != MQTT_OK) {
                         client->error = rv;
                         MQTT_PAL_MUTEX_UNLOCK(&client->mutex);
                         return rv;
                     }
-                } else if (response.decoded.publish.qos_level == MQTT_PUBLISH_QOS_2) {
+                } else if (response.decoded.publish.qos_level == 2) {
                     /* check if this is a duplicate */
                     if (mqtt_mq_find(&client->mq, MQTT_CONTROL_PUBREC, &response.decoded.publish.packet_id) != NULL) {
                         break;
@@ -1593,4 +1593,4 @@ const char* mqtt_error_str(enum MQTTErrors error) {
     }
 }

Please let me know if this is an issue, or, if I am misunderstanding something.

Thanks!

Scott

BIO_do_connect will cause the application to exit if called after it has returned 0

In openssl_sockets.h BIO_do_connect is called repeatedly for 10 seconds or until it succeeds or fails. If the connection fails (i.e. there is no broker running), it will return 0. Calling it again after this will cause the process to exit from within the C runtime (in glibc on Linux, that is).

A possible solution is to exit the while loop if the BIO_do_connect returns 0, since is seem like the BIO will need to be re-initialized before it can be reused (or something). For the same reason, the extra call to BIO_do_connect after the while loop should not be made at it will also cause an exit.

Note that this does not seem to affect unencrypted BIO sockets (as in bio_sockets.h) the same way.

Suggested change that seems to fix the issue:

    int conres = 0;
    
    while ((int)time(NULL) - start_time < 10) {
        conres = BIO_do_connect(*bio);
        if (conres >= 0) {
            break;
        }
    }
    
    if (conres <= 0) {
        BIO_free_all(*bio);
        SSL_CTX_free(*ssl_ctx);
        *bio = NULL;
        *ssl_ctx=NULL;
        return;
    }

... to replace line 29->39 in https://github.com/LiamBindle/MQTT-C/blob/master/examples/templates/openssl_sockets.h

That way, the error can be handled upon return instead and reconnect attempts can be made.

Reconnecting subscriber client core dumps if the broker is not present

This happens when using BIO sockets (encrypted or not) and the broker is not present at startup (or is stopped):
In mqtt_sync(), __mqtt_recv() is called regardless of the state of the connection client->socketfd, causing a segmentation fault due to a call to BIO_should_retry() in mqtt_pal_recvall() with an illegal fd value.

This can be avoided by returning before the call to __mqtt_recv() in mqtt_sync() if the socketfd is 0, e.g:

   /* Added: */
    if (!client->socketfd) {
        client->error = MQTT_ERROR_SOCKET_ERROR;
        return MQTT_ERROR_SOCKET_ERROR;
    } 

    /* Call receive */
    err = __mqtt_recv(client);

This also allows the user to implement logic in the user space sync function that handles the error state, and e.g. changes to a longer sleep interval to avoid trying to reconnect at 100msec intervals (which may spam a log file that outputs connection status and attempts)

Split mqtt.h in detail and api headers

For an end-user none of the detail api are interesting,
so by moving them to a new header (mqtt_internal.h for example) this is less visible for the end user.

This leaves a cleaner user-facing header, and exposes less implementation details.

Error when creating a function

Hello,

First of all I appologize if I did something wrong nor a noob mistake. I'm not used to code.
Your example runs flawlessly and it is straight forward. I tried to create functions to perform opening socket task, publish and so on to get a much clearer main().
Then I get a segmentation error I can't understand.

Here is my modified code
`#include <unistd.h>
#include <stdlib.h>
#include <stdio.h>

#include <mqtt.h>
#include "examples/templates/posix_sockets.h"

int sockfd;
pthread_t client_daemon;

/**

• @brief The function that would be called whenever a PUBLISH is received.
• @note This function is not used in this example.
  /
  void publish_callback(void* unused, struct mqtt_response_publish *published);

/**

• @brief The client's refresher. This function triggers back-end routines to

•    handle ingress/egress traffic to the broker.
  

• @note All this function needs to do is call \ref __mqtt_recv and

•   \ref __mqtt_send every so often. I've picked 100 ms meaning that 
  

•   client ingress/egress traffic will be handled every 100 ms.
  

/
void client_refresher(void* client);

/**

• @brief Safelty closes the \p sockfd and cancels the \p client_daemon before \c exit.
  */
  void exit_example(int status, int sockfd, pthread_t *client_daemon);

/**

• A simple program to that publishes the current temperature whenever ENTER is pressed.
  */

void open_mqtt_socket(const char *, const char *, const char *, struct mqtt_client *);

int main(int argc, const char argv[])
{
const char addr;
const char* port;
const char* topic;

/* get address (argv[1] if present) */
if (argc > 1) {
    addr = argv[1];
} else {
    addr = "test.mosquitto.org";
}

/* get port number (argv[2] if present) */
if (argc > 2) {
    port = argv[2];
} else {
    port = "1883";
}

/* get the topic name to publish */
if (argc > 3) {
    topic = argv[3];
} else {
    topic = "mytopic";
}

///////////////////////////////////

struct mqtt_client client;
open_mqtt_socket(addr, port, topic, &client);

/* start a thread to refresh the client (handle egress and ingree client traffic) */
/*pthread_t client_daemon;
if(pthread_create(&client_daemon, NULL, client_refresher, &client)) {
    fprintf(stderr, "Failed to start client daemon.\n");
    exit_example(EXIT_FAILURE, sockfd, NULL);
}

/
/ start publishing the time */
printf("%s is ready to begin publishing the temperature.\n", argv[0]);
printf("Press ENTER to start .\n");
printf("Press CTRL-D (or any other key) to exit.\n\n");
float temp=20.0;
char application_message[6];
while(1) {

	/* print a message */
	snprintf(application_message, sizeof(application_message), "%f", temp);
	printf("%s published : \"%s\"\r\n", argv[0], application_message);
	
	/* publish the temperature */
    printf("Going to mqtt_publish\r\n");
    printf("longueur payload : %i\r\n", strlen(application_message));
    mqtt_publish(&client, topic, application_message, strlen(application_message) + 1, MQTT_PUBLISH_QOS_0);
    //mqtt_publish(&client, "garage", "toto", 4, MQTT_PUBLISH_QOS_0);

    /* check for errors */
    if (client.error != MQTT_OK) {
        fprintf(stderr, "error: %s\n", mqtt_error_str(client.error));
        exit_example(EXIT_FAILURE, sockfd, &client_daemon);
    }
    sleep(1);
    temp++;
}

/* disconnect */
printf("\n%s disconnecting from %s\n", argv[0], addr);
sleep(1);

/* exit */ 
exit_example(EXIT_SUCCESS, sockfd, &client_daemon);

}

void exit_example(int status, int sockfd, pthread_t *client_daemon)
{
if (sockfd != -1) close(sockfd);
if (client_daemon != NULL) pthread_cancel(*client_daemon);
exit(status);
}

void publish_callback(void** unused, struct mqtt_response_publish published)
{
/ not used in this example */
}

void* client_refresher(void* client)
{
while(1)
{
mqtt_sync((struct mqtt_client*) client);
usleep(100000U);
}
return NULL;
}

void open_mqtt_socket(const char *addr, const char *port, const char *topic, struct mqtt_client *client){

/* open the non-blocking TCP socket (connecting to the broker) */
int sockfd = open_nb_socket(addr, port);

if (sockfd == -1) {
    perror("Failed to open socket: ");
    exit_example(EXIT_FAILURE, sockfd, NULL);
}

/* setup a client */
//struct mqtt_client client;
uint8_t sendbuf[2048]; /* sendbuf should be large enough to hold multiple whole mqtt messages */
uint8_t recvbuf[1024]; /* recvbuf should be large enough any whole mqtt message expected to be received */
mqtt_init(/*&*/client, sockfd, sendbuf, sizeof(sendbuf), recvbuf, sizeof(recvbuf), publish_callback);
mqtt_connect(/*&*/client, "publishing_client", NULL, NULL, 0, NULL, NULL, 0, 400);

/* check that we don't have any errors */
if (client->error != MQTT_OK) {
    fprintf(stderr, "error: %s\n", mqtt_error_str(client->error));
    exit_example(EXIT_FAILURE, sockfd, NULL);
}

//pthread_t client_daemon;
if(pthread_create(&client_daemon, NULL, client_refresher, /*&*/client)) {
    fprintf(stderr, "Failed to start client daemon.\n");
    exit_example(EXIT_FAILURE, sockfd, NULL);
}

}`

So I basically moved all the mqtt_init and mqtt_connect stuff in open_mqtt_socket() function, created global variables for socket file descriptor and client_daemon pthread.
Then the failure occurs in mqtt_publish() when the macro MQTT_CLIENT_TRY_PACK calls mqtt_mq_register.

The line which makes the program hang is:
mq->queue_tail->size = nbytes;

What I don't understand is that MQTT_CLIENT_TRY_PACK is called one time before being called by mqtt_publish(). And the first time it is called, no segmentation fault is raised.

I hope I was clear enough to make you able to reproduce the problem

Thank you for your help
With my best regards

In order delivery not guaranteed for for QoS 2

Currently, MQTT-C does not guarantee to PUBLISH messages QoS 2 to the broker in order. This should be supported and should be pretty easy.

See this thread for a discussion delivery order.

In __mqtt_send we should add a flag, inflight_qos2, to indicate that a QoS 2 message is inflight to the broker. Then if inflight_qos2 is set, resend should be set to 0 for all subsequent QoS 2 PUBLISH messages. This can be implemented in the following section.

MQTT-C/src/mqtt.c

Lines 370 to 388 in 5e18fed

	int len = mqtt_mq_length(&client->mq);
	for(int i = 0; i < len; ++i) {
	struct mqtt_queued_message *msg = mqtt_mq_get(&client->mq, i);
	int resend = 0;
	if (msg->state == MQTT_QUEUED_UNSENT) {
	/* message has not been sent to lets send it */
	resend = 1;
	} else if (msg->state == MQTT_QUEUED_AWAITING_ACK) {
	/* check for timeout */
	if (MQTT_PAL_TIME() > msg->time_sent + client->response_timeout) {
	resend = 1;
	client->number_of_timeouts += 1;
	}
	}
	/* goto next message if we don't need to send */
	if (!resend) {
	continue;
	}

On line 370 we should initialize inflight_qos2 = 0. Then on line 384 add the following block:

384         if (msg->control_type == MQTT_CONTROL_PUBLISH &&
385             (msg->state == MQTT_QUEUED_UNSENT || 
386              msg->state == MQTT_QUEUED_AWAITING_ACK)) {
387             inspected = 0x03 & ((msg->start[0]) >> 1); /* qos */
388             if (inspected == 2) {
389                 if (inflight_qos2) resend = 0; /* stop a subsequent QoS 2 from sending */
390                 inflight_qos2 = 1; /* indicate that any subsequent QoS 2 should not be sent */
391             }
392         }

I will add this when I have some time this weekend (this, or something close to this should work in the mean time).

Example payload contains unwanted charactors

In your subscriber examples, directly printing (const char*) published->application_message occasionally gives extra unwanted charactors. This makes number-based payload unreliable and introduces errors into calculations.

To fix this, I've added usage of application_message_size in the printf statement. For example:
sprintf(payload, "%.*s", published->application_message_size, (const char*) published->application_message);

MQTT_ERROR_CONTROL_FORBIDDEN_TYPE error with QOS_1

I am using your library on an embedded device and its mostly working quite well, but I've run into an issue with receiving messages that were sent from Azure IOTHUB while the device is not connected to iothub.

The device connects only every so often to iothub to check for messages from the server and send messages to the server. It then disconnects the underlying socket and does a mqtt_reinit on the next connection to start again.

The issue I'm running into specifically is that when I set 1 in mqtt_subscribe in the max_qos argument to guarantee that I will get any messages from the server that were sent while I was offline, it gives me the MQTT_ERROR_CONTROL_FORBIDDEN_TYPE specifically in the following conditions

1. I send more than 1 message to the device while it is offline, it comes online and then receives the first message. After which it will (I assume) send an ack and whatever it receives next has a header with a control_type of 0x0 causing the MQTT_ERROR_CONTROL_FORBIDDEN_TYPE
   error.

After that error has occured I can no longer send anything to the server. When it reconnects again later it will receive the same message it already received and error again on the same thing. It doesn't even get to the second message.

Repeating this on every reconnect until the server stops sending the message.

2. The second scenario is when setting the mqtt_publish side to MQTT_PUBLISH_QOS_1 as well as the mqtt_subscribeside. Then I only ever get thatMQTT_ERROR_CONTROL_FORBIDDEN_TYPE` error and I cannot even receive one message.

I wondered if you had any insight as to why this might be happening?

Must modify mqtt_pal.h to use a custom socket handle

If the declaration of mqtt_pal_socket_handle were changed from a typedef to a #define then custom socket handles could be used without modifying mqtt_pal.h.

This would mean replacing lines 64-69 of mqtt_pal.h with the following.

    #ifndef mqtt_pal_socket_handle
        #ifdef MQTT_USE_BIO
            #include <openssl/bio.h>
            #define mqtt_pal_socket_handle BIO*
        #else
            #define mqtt_pal_socket_handle int
        #endif
    #endif

I noticed this when reviewing pull request #21.

Cant compile example: openssl_publisher

Hi,

Thank you for your work.
I am trying to use it to publish data. However I am unable to compile the example openssl_publisher.c with the following error:

me@mypc:~/Documents/myStuff/dev/iodevice$ make
gcc -Wextra -Wall -std=gnu99 -Iinclude -Wno-unused-parameter -Wno-unused-variable -Wno-duplicate-decl-specifier -lpthread -lcurl   -c -o openssl_publisher.o openssl_publisher.c
openssl_publisher.c: In function ‘main’:
openssl_publisher.c:93:24: warning: passing argument 2 of ‘mqtt_init’ makes integer from pointer without a cast [-Wint-conversion]
     mqtt_init(&client, sockfd, sendbuf, sizeof(sendbuf), recvbuf, sizeof(recvbuf), publish_callback);
                        ^~~~~~
In file included from openssl_publisher.c:9:
mqtt.h:1283:50: note: expected ‘mqtt_pal_socket_handle’ {aka ‘int’} but argument is of type ‘BIO *’ {aka ‘struct bio_st *’}
                           mqtt_pal_socket_handle sockfd,
                           ~~~~~~~~~~~~~~~~~~~~~~~^~~~~~
gcc -o iodevice openssl_publisher.o mqtt.o mqtt_pal.o -Wextra -Wall -std=gnu99 -Iinclude -Wno-unused-parameter -Wno-unused-variable -Wno-duplicate-decl-specifier -lpthread -lcurl
/usr/bin/ld: openssl_publisher.o: in function `open_nb_socket':
openssl_publisher.c:(.text+0x30): undefined reference to `TLS_client_method'
/usr/bin/ld: openssl_publisher.c:(.text+0x38): undefined reference to `SSL_CTX_new'
/usr/bin/ld: openssl_publisher.c:(.text+0x5c): undefined reference to `SSL_CTX_load_verify_locations'
/usr/bin/ld: openssl_publisher.c:(.text+0x85): undefined reference to `BIO_new_ssl_connect'
/usr/bin/ld: openssl_publisher.c:(.text+0xaf): undefined reference to `BIO_ctrl'
/usr/bin/ld: openssl_publisher.c:(.text+0xca): undefined reference to `SSL_ctrl'
/usr/bin/ld: openssl_publisher.c:(.text+0xea): undefined reference to `BIO_ctrl'
/usr/bin/ld: openssl_publisher.c:(.text+0x108): undefined reference to `BIO_ctrl'
/usr/bin/ld: openssl_publisher.c:(.text+0x128): undefined reference to `BIO_ctrl'
/usr/bin/ld: openssl_publisher.c:(.text+0x154): undefined reference to `BIO_ctrl'
/usr/bin/ld: openssl_publisher.c:(.text+0x189): undefined reference to `BIO_ctrl'
/usr/bin/ld: openssl_publisher.c:(.text+0x193): undefined reference to `ERR_get_error'
/usr/bin/ld: openssl_publisher.c:(.text+0x19b): undefined reference to `ERR_reason_error_string'
/usr/bin/ld: openssl_publisher.c:(.text+0x1be): undefined reference to `BIO_free_all'
/usr/bin/ld: openssl_publisher.c:(.text+0x1cd): undefined reference to `SSL_CTX_free'
/usr/bin/ld: openssl_publisher.c:(.text+0x1f1): undefined reference to `SSL_get_verify_result'
/usr/bin/ld: openssl_publisher.o: in function `main':
openssl_publisher.c:(.text+0x258): undefined reference to `OPENSSL_init_ssl'
/usr/bin/ld: openssl_publisher.c:(.text+0x25d): undefined reference to `ERR_load_BIO_strings'
/usr/bin/ld: openssl_publisher.c:(.text+0x26c): undefined reference to `OPENSSL_init_crypto'
/usr/bin/ld: openssl_publisher.c:(.text+0x27b): undefined reference to `OPENSSL_init_ssl'
/usr/bin/ld: openssl_publisher.o: in function `exit_example':
openssl_publisher.c:(.text+0x6ba): undefined reference to `BIO_free_all'
collect2: error: ld returned 1 exit status
make: *** [makefile:10: iodevice] Error 1
me@mypc:~/Documents/myStuff/dev/iodevice$ 

Am I doing something wrong, or missing something?

Thank you again for your work. :)

A mqtt_pal.c file for Windows?

This looked so nice and simple until it turned out that I have to create the mqtt_pal.c file for my platform. It would be great if someone has already made such a file for Windows. It looks like it demands a significant amount of knowledge about sockets in Windows and I do not.