MQTT Protocol: How It Works & Core Concepts Explained

This article will show readers how to get started with the MQTT protocol, with code examples. Beginners of the IoT and MQTT can use this article to understand MQTT-related concepts and quickly start developing MQTT services and applications.

What Is MQTT?

MQTT (Message Queuing Telemetry Transport) is a lightweight, publish-subscribe based messaging protocol designed for resource-constrained devices and low-bandwidth, high-latency, or unreliable networks. It is widely used in Internet of Things (IoT) applications, providing efficient communication between sensors, actuators, and other devices.

Why Is MQTT the Best Protocol for IoT?

MQTT has emerged as one of the best IoT protocols due to its unique features and capabilities tailored to the specific needs of IoT systems. Some of the key reasons include:

• Lightweight: IoT devices are often constrained in terms of processing power, memory, and energy consumption. MQTT's minimal overhead and small packet size make it ideal for these devices, as it consumes fewer resources, enabling efficient communication even with limited capabilities.
• Reliable: IoT networks can experience high latency or unstable connections. MQTT's support for different QoS levels, session awareness, and persistent connections ensures reliable message delivery even in challenging conditions, making it well-suited for IoT applications.
• Secure communications: Security is crucial in IoT networks as they often transmit sensitive data. MQTT supports Transport Layer Security (TLS) and Secure Sockets Layer (SSL) encryption, ensuring data confidentiality during transmission. Additionally, it provides authentication and authorization mechanisms through username/password credentials or client certificates, safeguarding access to the network and its resources.
• Bi-directionality: MQTT's publish-subscribe model allows for seamless bi-directional communication between devices. Clients can both publish messages to topics and subscribe to receive messages on specific topics, enabling effective data exchange in diverse IoT ecosystems without direct coupling between devices. This model also simplifies the integration of new devices, ensuring easy scalability.
• Continuous, stateful sessions: MQTT allows clients to maintain stateful sessions with the broker, enabling the system to remember subscriptions and undelivered messages even after disconnection. Clients can also specify a keep-alive interval during connection, which prompts the broker to periodically check the connection status. If the connection is lost, the broker stores undelivered messages (depending on the QoS level) and attempts to deliver them when the client reconnects. This feature ensures reliable communication and reduces the risk of data loss due to intermittent connectivity.
• Large-scale IoT device support: IoT systems often involve a large number of devices, requiring a protocol that can handle massive-scale deployments. MQTT's lightweight nature, low bandwidth consumption, and efficient use of resources make it well-suited for large-scale IoT applications. The publish-subscribe pattern allows MQTT to scale effectively, as it decouples sender and receiver, reducing network traffic and resource usage. Furthermore, the protocol's support for different QoS levels allows customization of message delivery based on the application's requirements, ensuring optimal performance in various scenarios.
• Language support: IoT systems often include devices and applications developed using various programming languages. MQTT's broad language support enables easy integration with multiple platforms and technologies, fostering seamless communication and interoperability in diverse IoT ecosystems. You can visit our MQTT Client Programming blog series to learn how to use MQTT in PHP, Node.js, Python, Golang, Node.js, and other programming languages.

Learn more in our article: What is MQTT and Why is it the Best Protocol for IoT?

How Does MQTT Work?

To understand how MQTT works, you need to first master the concepts of MQTT Client, MQTT Broker, Publish-Subscribe mode, Topic, and QoS:

MQTT Client

Any application or device running the MQTT client library is an MQTT client. For example, an instant messaging app that uses MQTT is a client, various sensors that use MQTT to report data are a client, and various MQTT testing tools are also a client.

MQTT Broker

The MQTT Broker handles client connection, disconnection, subscription, and unsubscription requests, and routing messages. A powerful MQTT broker can support massive connections and million-level message throughput, helping IoT service providers focus on business and quickly create a reliable MQTT application.

For more details on MQTT brokers, please check the blog The Ultimate Guide to MQTT Broker Comparison in 2023.

Publish–subscribe pattern

The publish-subscribe pattern differs from the client-server pattern in that it separates the client that sends messages (publisher) from the client that receives messages (subscriber). Publishers and subscribers do not need to establish a direct connection, and the MQTT Broker is responsible for routing and distributing all messages.

The following diagram shows the MQTT publish/subscribe process. The temperature sensor connects to the MQTT server as a client and publishes temperature data to a topic (e.g., Temperature), and the server receives the message and forwards it to the client subscribed to the Temperature topic.

Topic

The MQTT protocol routes messages based on topic. The topic distinguishes the hierarchy by slash /, which is similar to URL paths, for example:

chat/room/1

sensor/10/temperature

sensor/+/temperature

MQTT topic support the following wildcards: + and #.

• +: indicates a single level of wildcards, such as a/+ matching a/x or a/y.
• #: indicates multiple levels of wildcards, such as a/# matching a/x, a/b/c/d.

For more details on MQTT topics, please check the blog Understanding MQTT Topics & Wildcards by Case.

Quality of Service (QoS)

MQTT provides three kinds of Quality of Service and guarantees messaging reliability in different network environments.

• QoS 0: The message is delivered at most once. If the client is not available currently, it will lose this message.
• QoS 1: The message is delivered at least once.
• QoS 2: The message is delivered only once.

For more details on MQTT QoS, please check the blog Introduction to MQTT QoS (Quality of Service).

The MQTT Workflow

Now that we understand the basic components of MQTT, let’s see how the general workflow works:

1. Clients initiate a connection to the broker using TCP/IP, with optional TLS/SSL encryption for secure communication. Clients provide authentication credentials and specify a clean or persistent session.
2. Clients either publish messages to specific topics or subscribe to topics to receive messages. Publishing clients send messages to the broker, while subscribing clients express interest in receiving messages on particular topics.
3. The broker receives published messages and forwards them to all clients subscribed to the relevant topics. It ensures reliable message delivery according to the specified Quality of Service (QoS) level and manages message storage for disconnected clients based on session type.

Getting Started with MQTT: Quick Tutorial

Now we will show you how to start using MQTT with a few simple demos. Before we begin, you need to prepare an MQTT Broker and an MQTT Client.

Prepare an MQTT Broker

You can create an MQTT broker through private deployment or a fully managed cloud service. Or use a free public broker for testing.

• Private deployment

  EMQX is the most scalable open-source MQTT broker for IoT, IIoT, and connected vehicles. You can run the following Docker command to install EMQX.

  docker run -d --name emqx -p 1883:1883 -p 8083:8083 -p 8084:8084 -p 8883:8883 -p 18083:18083 emqx/emqx
  

• Fully managed cloud service

  The fully managed cloud service is the easiest way to start an MQTT service. As shown below, EMQX Cloud starts in minutes and runs in 17 regions across AWS, Google Cloud, and Microsoft Azure.

  

• Free public MQTT broker

  In this post, we will use the free public MQTT broker provided by EMQ, created based on the fully managed MQTT cloud service - EMQX Cloud. The server information is as follows.

  • Broker Address： broker.emqx.io
  • TCP Port： 1883
  • WebSocket Port： 8083

Prepare an MQTT Client

In this post, we will use the MQTT client tool provided by MQTTX that supports browser access: http://www.emqx.io/online-mqtt-client. MQTT X also provides a desktop client and a command line tool.

MQTTX is an elegant cross-platform MQTT 5.0 desktop client that runs on macOS, Linux, and Windows. Its user-friendly chat-style interface enables users to easily create multiple MQTT/MQTTS connections and subscribe/publish MQTT messages.

MQTTX Preview

Currently, there are mature open-source MQTT client libraries for all programming languages. We have selected popular MQTT client libraries & SDKs in various programming languages and provided code examples to help you quickly understand the use of MQTT clients.

Create an MQTT Connection

Before using the MQTT protocol to communicate, the client needs to create an MQTT connection to connect to the broker.

Go to http://www.emqx.io/online-mqtt-client with your browser and click on the New Connection button in the middle of the page and you will see the following page.

We enter Simple Demo in Name and click the Connect button in the upper right corner to create an MQTT connection. The following indicates that the connection is established successfully.

To learn more about MQTT connection parameters, please check out our blog post: How to Set Parameters When Establishing an MQTT Connection.

Subscribe to The Wildcard Topic

Next, we subscribe to the wildcard topic sensor/+/temperature in the Simple Demo connection created earlier, which will receive the temperature data reported by all sensors.

As shown below, click the New Subscription button and enter the topic sensor/+/temperature in the Topic field in the pop-up box, keeping the default QoS at 0.

Once the subscription is successful, you will see an additional record in the middle of the subscription list.

Publish MQTT Messages

Next, we click the + button on the left menu to create two connections, Sensor 1 and Sensor 2 respectively, to simulate two temperature sensors.

Once the connection is created, you will see three connections and the online status dots to the left of the connections will all be green.

Select the Sensor 1 connection, enter the publish topic sensor/1/temperature in the bottom left part of the page, enter the following JSON format message in the message box, and click the publish button at the bottom right to send the message.

{
  "msg": "17.2"
}

The message is sent successfully as follows.

Using the same steps, publish the following JSON message to the sensor/2/temperature topic in the Sensor 2 connection.

{
  "msg": "18.2"
}

You will see two new messages for the Simple Demo connection.

Click on the Simple Demo connection and you will see two messages sent by the two sensors.

MQTT Features Demonstration

Retained Message

When the MQTT client publishes a message to the server, Retained Message flag can be set. The Retained Message resides on the message server, and subsequent subscribers can still receive the message when they subscribe to the topic.

As shown below, we are sending two messages to the retained_message topic in the Sensor 1 connection with the Retain option checked.

Then, we subscribe to the retained_message topic in the Simple Demo connection. After the subscription is successful, the second retained message sent by Sensor 1 will be received, which shows that the server will only keep the last retained message for a topic.

For more details on Retained Message, please check the blog The Beginner's Guide to MQTT Retained Messages.

Clean Session

In general, an MQTT client can only receive messages published by other clients when it is online. If the client is offline and then online, it will not receive messages during the offline period.

However, if the client connects with Clean Session set to false and goes online again with the same Client ID, then the message server will keep a certain amount of offline messages for the client and send them to the client when it comes online again.

The public MQTT server used for this demonstration is set to keep offline messages for 5 minutes and the maximum number of messages is 1000 (no QoS 0 messages).

Next, we create an MQTT 3.1.1 connection and demonstrate the clean session with QoS 1.

MQTT 5 uses Clean Start and Session Expiry Interval to improve Clean Session. For details, please refer to the blog Clean Start and Session Expiry Interval.

Create a connection named MQTT V3, set Clean Session to false, and choose MQTT version 3.1.1.

Subscribe to clean_session_false topic after successful connection, and set QoS to 1.

After the successful subscription, click the Disconnect button in the upper right corner.

Next, create a connection named MQTT_V3_Publish, and the MQTT version is also set to 3.1.1. After the successful connection, publish three messages to the clean_session_false topic.

Then select the MQTT_V3 connection, click the connect button to connect to the server, and you will receive three offline messages.

For more details on Clean Session, please check the blog MQTT Persistent Session and Clean Session Explained.

Last Will

When the MQTT client makes a CONNECT request to the server, it can set whether to send the flag of Will Message , as well as the Topic and Payload.

When the MQTT client is abnormally offline (the DISCONNECT message is not sent to the server before the client disconnects), the MQTT server will publish a will message.

As follows, we create a connection named Last Will.

• To see the effect quickly, we set Keep Alive to 5 seconds.
• Set Last-Will Topic to last_will.
• Set Last-Will QoS to 1.
• Set Last-Will Retain to true.
• Set Last-Will Payload to offline.

After the connection is successful, we disconnect the computer network for more than 5 seconds (simulating an abnormal client disconnection), and then turn on the network again.

Then start the Simple Demo connection, and subscribe to the last_will topic. You will receive the will message set by the Last Will connection.

For more details on MQTT Will Message, please check the blog Use of MQTT Will Message.

Learn More About MQTT

At this point, we have finished explaining and demonstrating the basic concepts of MQTT and its usage process, so readers can try their hand at using the MQTT protocol based on what they have learned in this article.

Next, you can check out MQTT Guide 2023: Beginner to Advanced series of articles provided by EMQ to learn about MQTT Topics, Wildcards, Retained Messages, Last-Will, and other features. Explore more advanced applications of MQTT and get started with MQTT application and service development.

Try EMQX Cloud for Free

No credit card required

Get Started →