MQTT_ERROR_CONTROL_FORBIDDEN_TYPE error with QOS_1 about mqtt-c

[LiamBindle]

The error MQTT_ERROR_CONTROL_FORBIDDEN_TYPE is set when the fixed header's control type is invalid/reserved value. According to the MQTT v3.1.1 spec, the control type values of 0 and 15 are reserved (see here).

The only place where MQTT_ERROR_CONTROL_FORBIDDEN_TYPE is actually ever set is

MQTT-C/src/mqtt.c

Lines 924 to 926 in 201a277

	if (!mqtt_fixed_header_rules.control_type_is_valid[control_type]) {
	return MQTT_ERROR_CONTROL_FORBIDDEN_TYPE;
	}

so I suspect your issue must be coming from here. Also for your reference, mqtt_fixed_header_rules::control_type_is_valid is initialized here

MQTT-C/src/mqtt.c

Lines 855 to 871 in 201a277

	{ /* boolean value, true if type is valid */
	0x00, /* MQTT_CONTROL_RESERVED */
	0x01, /* MQTT_CONTROL_CONNECT */
	0x01, /* MQTT_CONTROL_CONNACK */
	0x01, /* MQTT_CONTROL_PUBLISH */
	0x01, /* MQTT_CONTROL_PUBACK */
	0x01, /* MQTT_CONTROL_PUBREC */
	0x01, /* MQTT_CONTROL_PUBREL */
	0x01, /* MQTT_CONTROL_PUBCOMP */
	0x01, /* MQTT_CONTROL_SUBSCRIBE */
	0x01, /* MQTT_CONTROL_SUBACK */
	0x01, /* MQTT_CONTROL_UNSUBSCRIBE */
	0x01, /* MQTT_CONTROL_UNSUBACK */
	0x01, /* MQTT_CONTROL_PINGREQ */
	0x01, /* MQTT_CONTROL_PINGRESP */
	0x01, /* MQTT_CONTROL_DISCONNECT */
	0x00 /* MQTT_CONTROL_RESERVED */

which is just the implementation of the section of the spec that I referenced above.

So, somehow mqtt_pack_fixed_header must be getting called with fixed_header->control_type == 0. That means somwhere in here

MQTT-C/src/mqtt.c

Lines 1237 to 1259 in 201a277

	fixed_header.control_type = MQTT_CONTROL_PUBLISH;
	/* calculate remaining length */
	remaining_length = __mqtt_packed_cstrlen(topic_name);
	if (inspected_qos > 0) {
	remaining_length += 2;
	}
	remaining_length += application_message_size;
	fixed_header.remaining_length = remaining_length;
	/* force dup to 0 if qos is 0 */
	if (inspected_qos == 0) {
	publish_flags &= ~MQTT_PUBLISH_DUP;
	}
	/* make sure that qos is not 3 */
	if (inspected_qos == 3) {
	return MQTT_ERROR_PUBLISH_FORBIDDEN_QOS;
	}
	fixed_header.control_flags = publish_flags;
	/* pack fixed header */
	rv = mqtt_pack_fixed_header(buf, bufsz, &fixed_header);

the value of fixed_header.control_type must be changing from MQTT_CONTROL_PUBLISH to 0.

I haven't seen this before, but I'm struggling to see how fixed_header->control_type is changing after L1237. I'll revisit this sometime in the next couple days.

[LiamBindle]

Have you made any progress with tracking down what is going on?

So the messages with control_type of 0 are coming from Azure IOTHUB? If so, I believe the error would be on their end. It could also be a problem with how MQTT-C is parsing the fixed headers it receives (I haven't heard of this before though).

If the messages with control_type 0 are being generated by MQTT-C then you might have to step through lines I posted above and see at what point the control_type is being changed to 0.

Hope that helps, let me know how you're making out.

Liam

[TonyTheLion]

I've done some more testing, and it does look like we are receiving messages from Azure IOTHUB which have the wrong control_type.

I sent two messages from the Azure website to my device while the device is not connected. The messages had the following content:

test
test 1

I added some prints into the MQTT-C code, specifically where it unpacks the fixed header and the send/receive calls. Here is the output:

__mqtt_recv err: 1
mqtt_pal_recvall
fixed_header.control_type: 0x2
fixed_header.control_flags: 0x0
fixed_header.remaining_length: 0x2
mqtt_pal_recvall
fixed_header.control_type: 0x3
fixed_header.control_flags: 0x2
fixed_header.remaining_length: 0x77
[PRN] dispatch_mqtt.c:072 +publish_response_callback    :: publish_response_callback
[PRN] dispatch_mqtt.c:074 +publish_response_callback    :: test
mqtt_pal_recvall
fixed_header.control_type: 0x0
fixed_header.control_flags: 0x0
fixed_header.remaining_length: 0x79
header rule violation: -2147483646
__mqtt_recv err: -2147483646
[ERR] dispatch_mqtt.c:588 +mqtt_sync_callback   :: MQTT_ERROR_CONTROL_FORBIDDEN_TYPE
mqtt_pal_recvall

You can see we receive the first test correctly and then the second one it seems to receive the length of the packet (0x79 - two bytes longer than the first which makes sense) but the control_type is 0.

[LiamBindle]

Okay. Do you know if the packet sent by Azure IOTHUB has a control_type of 0, or if MQTT-C is parsing the packet incorrectly resulting in a control_type of 0? You could use Wireshark to inspect the ingress packets (format of the fixed_header is here). If you're having trouble interpreting the bytes of the packet, you could post a screenshot and I could give you a hand.

This would let us figure out it's an MQTT-C problem or an Azure IOTHUB problem.

[TonyTheLion]

I can definitely do a tcpdump but the connection is encrypted so you can't see the headers of the MQTT packets. Azure won't let me do unencrypted connections.

I can do a print of the raw bytes that come in on my end tho, if that is any help?

[LiamBindle]

Yeah, the encrypted bytes are no use, but if you could print the raw bytes that come in on your end we could take a look at that. Essentially, if you could dump buf here

MQTT-C/src/mqtt.c

Line 1644 in 201a277

ssize_t rv = mqtt_unpack_fixed_header(response, buf, bufsz);

right before you get the MQTT_ERROR_CONTROL_FORBIDDEN_TYPE error, we could take a look at that.

[TonyTheLion]

Sent the following while device is offline
test
test 1

so capturing that data gives the following:

received bytes:
0x20 0x2 0x1 0x0 0x32 0x77 0x0 0x6f 0x64 0x65 0x76 0x69 0x63 0x65 0x73 0x2f 0x6a 0x61 0x72 0x65 0x64 0x5f 0x62 0x65 0x61 0x67 0x6c 0x65 0x5f 0x62 0x6c 0x61 0x63 0x6b 0x2f 0x6d 0x65 0x73 0x73 0x61 0x67 0x65 0x73 0x2f 0x64 0x65 0x76 0x69 0x63 0x65 0x62 0x6f 0x75 0x6e 0x64 0x2f 0x25 0x32 0x34 0x2e 0x74 0x6f 0x3d 0x25 0x32 0x46 0x64 0x65 0x76 0x69 0x63 0x65 0x73 0x25 0x32 0x46 0x6a 0x61 0x72 0x65 0x64 0x5f 0x62 0x65 0x61 0x67 0x6c 0x65 0x5f 0x62 0x6c 0x61 0x63 0x6b 0x25 0x32 0x46 0x6d 0x65 0x73 0x73 0x61 0x67 0x65 0x73 0x25 0x32 0x46 0x64 0x65 0x76 0x69 0x63 0x65 0x42 0x6f 0x75 0x6e 0x64 0x0 0x75 0x74 0x65 0x73 0x74 0x32 0x79 0x0 0x6f 0x64 0x65 0x76 0x69 0x63 0x65 0x73 0x2f 0x6a 0x61 0x72 0x65 0x64 0x5f 0x62 0x65 0x61 0x67 0x6c 0x65 0x5f 0x62 0x6c 0x61 0x63 0x6b 0x2f 0x6d 0x65 0x73 0x73 0x61 0x67 0x65 0x73 0x2f 0x64 0x65 0x76 0x69 0x63 0x65 0x62 0x6f 0x75 0x6e 0x64 0x2f 0x25 0x32 0x34 0x2e 0x74 0x6f 0x3d 0x25 0x32 0x46 0x64 0x65 0x76 0x69 0x63 0x65 0x73 0x25 0x32 0x46 0x6a 0x61 0x72 0x65 0x64 0x5f 0x62 0x65 0x61 0x67 0x6c 0x65 0x5f 0x62 0x6c 0x61 0x63 0x6b 0x25 0x32 0x46 0x6d 0x65 0x73 0x73 0x61 0x67 0x65 0x73 0x25 0x32 0x46 0x64 0x65 0x76 0x69 0x63 0x65 0x42 0x6f 0x75 0x6e 0x64 0x0 0x76 0x74 0x65 0x73 0x74 0x20 0x32
fixed_header.control_type: 0x2
fixed_header.control_flags: 0x0
fixed_header.remaining_length: 0x2
mqtt_pal_recvall
received bytes:
0x32 0x77 0x0 0x6f 0x64 0x65 0x76 0x69 0x63 0x65 0x73 0x2f 0x6a 0x61 0x72 0x65 0x64 0x5f 0x62 0x65 0x61 0x67 0x6c 0x65 0x5f 0x62 0x6c 0x61 0x63 0x6b 0x2f 0x6d 0x65 0x73 0x73 0x61 0x67 0x65 0x73 0x2f 0x64 0x65 0x76 0x69 0x63 0x65 0x62 0x6f 0x75 0x6e 0x64 0x2f 0x25 0x32 0x34 0x2e 0x74 0x6f 0x3d 0x25 0x32 0x46 0x64 0x65 0x76 0x69 0x63 0x65 0x73 0x25 0x32 0x46 0x6a 0x61 0x72 0x65 0x64 0x5f 0x62 0x65 0x61 0x67 0x6c 0x65 0x5f 0x62 0x6c 0x61 0x63 0x6b 0x25 0x32 0x46 0x6d 0x65 0x73 0x73 0x61 0x67 0x65 0x73 0x25 0x32 0x46 0x64 0x65 0x76 0x69 0x63 0x65 0x42 0x6f 0x75 0x6e 0x64 0x0 0x75 0x74 0x65 0x73 0x74 0x32 0x79 0x0 0x6f 0x64 0x65 0x76 0x69 0x63 0x65 0x73 0x2f 0x6a 0x61 0x72 0x65 0x64 0x5f 0x62 0x65 0x61 0x67 0x6c 0x65 0x5f 0x62 0x6c 0x61 0x63 0x6b 0x2f 0x6d 0x65 0x73 0x73 0x61 0x67 0x65 0x73 0x2f 0x64 0x65 0x76 0x69 0x63 0x65 0x62 0x6f 0x75 0x6e 0x64 0x2f 0x25 0x32 0x34 0x2e 0x74 0x6f 0x3d 0x25 0x32 0x46 0x64 0x65 0x76 0x69 0x63 0x65 0x73 0x25 0x32 0x46 0x6a 0x61 0x72 0x65 0x64 0x5f 0x62 0x65 0x61 0x67 0x6c 0x65 0x5f 0x62 0x6c 0x61 0x63 0x6b 0x25 0x32 0x46 0x6d 0x65 0x73 0x73 0x61 0x67 0x65 0x73 0x25 0x32 0x46 0x64 0x65 0x76 0x69 0x63 0x65 0x42 0x6f 0x75 0x6e 0x64 0x0 0x76 0x74 0x65 0x73 0x74 0x20 0x32
fixed_header.control_type: 0x3
fixed_header.control_flags: 0x2
fixed_header.remaining_length: 0x77
[PRN] dispatch_mqtt.c:072 +publish_response_callback    :: publish_response_callback
[PRN] dispatch_mqtt.c:074 +publish_response_callback    :: test
mqtt_pal_recvall
received bytes:
0x0 0x79 0x0 0x6f 0x64 0x65 0x76 0x69 0x63 0x65 0x73 0x2f 0x6a 0x61 0x72 0x65 0x64 0x5f 0x62 0x65 0x61 0x67 0x6c 0x65 0x5f 0x62 0x6c 0x61 0x63 0x6b 0x2f 0x6d 0x65 0x73 0x73 0x61 0x67 0x65 0x73 0x2f 0x64 0x65 0x76 0x69 0x63 0x65 0x62 0x6f 0x75 0x6e 0x64 0x2f 0x25 0x32 0x34 0x2e 0x74 0x6f 0x3d 0x25 0x32 0x46 0x64 0x65 0x76 0x69 0x63 0x65 0x73 0x25 0x32 0x46 0x6a 0x61 0x72 0x65 0x64 0x5f 0x62 0x65 0x61 0x67 0x6c 0x65 0x5f 0x62 0x6c 0x61 0x63 0x6b 0x25 0x32 0x46 0x6d 0x65 0x73 0x73 0x61 0x67 0x65 0x73 0x25 0x32 0x46 0x64 0x65 0x76 0x69 0x63 0x65 0x42 0x6f 0x75 0x6e 0x64 0x0 0x76 0x74 0x65 0x73 0x74 0x20 0x32
fixed_header.control_type: 0x0
fixed_header.control_flags: 0x0
fixed_header.remaining_length: 0x79
header rule violation: -2147483646
__mqtt_recv err: -2147483646
[ERR] dispatch_mqtt.c:588 +mqtt_sync_callback   :: MQTT_ERROR_CONTROL_FORBIDDEN_TYPE

It does seem like they send us a null byte from what I can tell.

[LiamBindle]

Here's my interpretation of the packets:

Packet 1 (this is actually more than one packet):

0x20 0x2 0x1 0x0 0x32 0x77 0x0 0x6f 0x64 0x65 0x76 0x69 0x63 0x65 0x73 0x2f 0x6a 0x61 0x72 0x65 0x64 0x5f 0x62 0x65 0x61 0x67 0x6c 0x65 0x5f 0x62 0x6c 0x61 0x63 0x6b 0x2f 0x6d 0x65 0x73 0x73 0x61 0x67 0x65 0x73 0x2f 0x64 0x65 0x76 0x69 0x63 0x65 0x62 0x6f 0x75 0x6e 0x64 0x2f 0x25 0x32 0x34 0x2e 0x74 0x6f 0x3d 0x25 0x32 0x46 0x64 0x65 0x76 0x69 0x63 0x65 0x73 0x25 0x32 0x46 0x6a 0x61 0x72 0x65 0x64 0x5f 0x62 0x65 0x61 0x67 0x6c 0x65 0x5f 0x62 0x6c 0x61 0x63 0x6b 0x25 0x32 0x46 0x6d 0x65 0x73 0x73 0x61 0x67 0x65 0x73 0x25 0x32 0x46 0x64 0x65 0x76 0x69 0x63 0x65 0x42 0x6f 0x75 0x6e 0x64 0x0 0x75 0x74 0x65 0x73 0x74 0x32 0x79 0x0 0x6f 0x64 0x65 0x76 0x69 0x63 0x65 0x73 0x2f 0x6a 0x61 0x72 0x65 0x64 0x5f 0x62 0x65 0x61 0x67 0x6c 0x65 0x5f 0x62 0x6c 0x61 0x63 0x6b 0x2f 0x6d 0x65 0x73 0x73 0x61 0x67 0x65 0x73 0x2f 0x64 0x65 0x76 0x69 0x63 0x65 0x62 0x6f 0x75 0x6e 0x64 0x2f 0x25 0x32 0x34 0x2e 0x74 0x6f 0x3d 0x25 0x32 0x46 0x64 0x65 0x76 0x69 0x63 0x65 0x73 0x25 0x32 0x46 0x6a 0x61 0x72 0x65 0x64 0x5f 0x62 0x65 0x61 0x67 0x6c 0x65 0x5f 0x62 0x6c 0x61 0x63 0x6b 0x25 0x32 0x46 0x6d 0x65 0x73 0x73 0x61 0x67 0x65 0x73 0x25 0x32 0x46 0x64 0x65 0x76 0x69 0x63 0x65 0x42 0x6f 0x75 0x6e 0x64 0x0 0x76 0x74 0x65 0x73 0x74 0x20 0x32

• 0x20 0x2 0x1 0x0: this is the CONNACK
• 0x32 0x77: this is the fixed header of the received PUBLISH, and it says there are 119 bytes after the 0x77 byte in the packet.
• 0x0 0x6f: this says your topic name is 111 characters long
• 0x64 0x65 0x76 0x69 0x63 0x65 0x73 0x2f 0x6a 0x61 0x72 0x65 0x64 0x5f 0x62 0x65 0x61 0x67 0x6c 0x65 0x5f 0x62 0x6c 0x61 0x63 0x6b 0x2f 0x6d 0x65 0x73 0x73 0x61 0x67 0x65 0x73 0x2f 0x64 0x65 0x76 0x69 0x63 0x65 0x62 0x6f 0x75 0x6e 0x64 0x2f 0x25 0x32 0x34 0x2e 0x74 0x6f 0x3d 0x25 0x32 0x46 0x64 0x65 0x76 0x69 0x63 0x65 0x73 0x25 0x32 0x46 0x6a 0x61 0x72 0x65 0x64 0x5f 0x62 0x65 0x61 0x67 0x6c 0x65 0x5f 0x62 0x6c 0x61 0x63 0x6b 0x25 0x32 0x46 0x6d 0x65 0x73 0x73 0x61 0x67 0x65 0x73 0x25 0x32 0x46 0x64 0x65 0x76 0x69 0x63 0x65 0x42 0x6f 0x75 0x6e 0x64: this is your topic name
• 0x0 0x75: this is your packet ID
• 0x74 0x65 0x73 0x74: this is your application message (i.e. "test")
• So this packet appear okay.

But now I see something weird. The byte immediately following the last "t" changes from 0x32 (the correct first byte of the next PUBLISH packet) to 0x00. See here:

So somehow the buffer is changing. I don't really know how this could be happening though because buf is const in mqtt_unpack_response.

Oh, maybe...
When your callback gets the response struct, are you writing a null character at the end (to make it a c-string)? response->application_message points to the position in the buffer that contains the application message (see mqtt.c:1316), so if you were to modify this it would mess up the ingress buffer.

[TonyTheLion]

When your callback gets the response struct, are you writing a null character at the end (to make it a c-string)? response->application_message points to the position in the buffer that contains the application message (see mqtt.c:1316), so if you were to modify this it would mess up the ingress buffer.

Yes that was it. I was printing the publish->application_message in the callback and because it would print junk after the text, I wrote a \0 after it.

Removing that line fixed the issue with receive.

Thank you so much for your help in debugging this.

[LiamBindle]

Glad to help.

Let me know if you run into any more problems.