Introduction to MQTT Payload Format Indicator and Content Type | MQTT 5 Features

In this article, we will focus on the Payload Format Indicator and Content Type properties of MQTT 5.0, exploring how they make the parsing of messages more transparent and efficient.

What is Payload Format Indicator?

The Payload Format Indicator is a new property introduced in MQTT 5.0 to indicate the format of the payload in MQTT packets. However, the format of the payload in CONNECT, SUBSCRIBE, and UNSUBSCRIBE packets is fixed, so in practice, only PUBLISH and CONNECT packets need to declare the payload format.

If the value of the Payload Format Indicator is 0 or if this property is not specified, the current payload is an unspecified byte stream; if the value of this property is 1, the current payload is UTF-8 encoded character data.

This allows the receiver to check the format of the payload without having to parse the specific content. For example, the server can check if the payload is a valid UTF-8 string to avoid distributing incorrectly formatted application messages to subscribers. However, given the burden this operation imposes on the server and the benefits that can actually be achieved, this is usually an optional setting.

What is Content Type?

Content Type is also a new property introduced in MQTT 5.0, and similar to the Payload Format Indicator, it is also only available in PUBLISH and CONNECT packets.

The value of Content Type is a UTF-8 encoded string that describes the content of the application message, which helps the receiver understand how to parse the application message payload. For example, if the content of the message is a JSON object, then the Content Type could be set to "json".

The exact content of this string is entirely up to the sender and receiver, and throughout the transmission of the message, the server does not use this property to verify that the message content is formatted correctly; it is only responsible for forwarding this property to the subscriber as is.

So you can even use "cocktail" to describe the JSON type as long as the receiver understands it. However, in order to avoid unnecessary troubles, we usually recommend using known MIME types to describe the message content, such as application/json, application/xml, etc.

Content Type is useful in scenarios where multiple data types need to be supported. For example, when we send an image to the other party in a chat program, and the image may be in PNG, GIF, JPEG, etc., how do we indicate to the other party the format of the image that corresponds to the binary data we are sending?

Prior to 5.0, we might choose to include the image format in a theme, such as to/userA/image/png, but obviously, as the number of supported image formats increases, clients need to subscribe to more and more topics for various data formats. In 5.0, we simply set the Content Type property to image/png.

Do we have to use Payload Format Indicator and Content Type together?

Whether Payload Format Indicator and Content Type need to be used together depends on our application scenario.

For the subscriber, it can determine whether the content of the message should be a UTF-8 string or binary data based on the value of the Content Type, so the Payload Format Indicator is not very meaningful.

For the server, however, doesn't know the meaning of the Content Type value, so if we want the server to check whether the message payload conforms to the UTF-8 encoding specification, we must use the Payload Format Indicator property.

Demo

1. Access MQTTX Web on a Web browser.

2. Create an MQTT connection named pub for publishing messages, and connect it to the Free Public MQTT Server:

   

3. Create a client connection named sub in the same way and subscribe to the topic mqttx_89e3d55e/test using the Client ID as a prefix:

   

4. Then go back to the client pub, click the Meta button in the message bar, set the Payload Format Indicator to true, set the Content Type to application/json, publish a JSON-formatted message to the topic mqttx_89e3d55e/test, and then change it to application/x-www-form-urlencoded and then publish a form format message to the same topic:

   

   

5. The Content Type of the message will be forwarded to the subscriber as is, so the subscriber can know how to parse the content in the Payload based on the value of the Content Type:

   

6. Back on the publishing side, set the Payload Format Indicator to false and change the encoding format of the Payload to Hex, then enter FF as the content of the Payload and send it. 0xFF is a typical non-UTF-8 character:

   

7. Although it is displayed as garbled characters, the subscriber did receive the message with a Payload of 0xFF that we just sent. This is for performance reasons, EMQX currently does not check the Payload format:

   

In the terminal, we can also use the command line tool MQTTX CLI to accomplish the above, and we can subscribe to topics using the following command:

mqttx sub -h 'broker.emqx.io' -p 1883 -t 'random-string/demo' --output-mode clean

Then use the following commands to set the Payload Format Indicator and Content Type properties when you publish the message:

mqttx pub -h 'broker.emqx.io' -p 1883 -t 'random-string/demo' \
--payload-format-indicator \
--content-type 'application/json' \
-m '{"msg": "hello"}'

This is how the Payload Format Indicator and Content Type properties are used in MQTT 5.0, and you can get Python examples of them here.