EMQX vs VerneMQ | 2023 MQTT Broker Comparison
Fan Wang
Table of Contents
Introduction
EMQX and VerneMQ are open-source, highly scalable distributed MQTT brokers written in Erlang/OTP, known for their robustness, fault tolerance, and scalability.
EMQX is now one of the most popular MQTT brokers in the world. While VerneMQ has not been actively developing and maintaining these years.
In the fourth blog of the “2023 MQTT Broker Comparison“ series, we will briefly compare these two brokers as a memo in the history of MQTT.
EMQX Overview
The EMQX project was launched on GitHub in 2012 and is licensed under Apache version 2.0. The idea for EMQX originated from the need for a massively scalable MQTT broker that could handle millions of concurrent connections.
EMQX is now the world's most scalable MQTT messaging server with a distributed architecture based on Mria + RLOG. EMQX 5.0, the latest version, scales to establish 100 million concurrent MQTT connections with a single cluster of 23 nodes.
See: Reaching 100M MQTT connections with EMQX 5.0
EMQX offers rich enterprise features, data integration, cloud hosting services, and commercial support from EMQ Technologies Inc. Over the years, EMQX has gained popularity among enterprises, startups, and individuals due to its performance, reliability, and scalability. EMQX is widely used in various industries, such as IoT, industrial IoT, connected cars, and telecommunications.
Pros:
- Supports large-scale deployments
- High availability
- Horizontal scalability
- High-performance and reliable
- Rich enterprise features
- Pioneering MQTT over QUIC
Cons:
- Complex to set up and configure
- Difficult to manage effectively
- Logs may be confusing
VerneMQ Overview
The VerneMQ project was launched in 2014 and initially developed by Erlio GmbH. As the second broker wrote in Erlang/OTP, the project is licensed under Apache Version 2.0 and borrowed some code from the EMQX project.
Regarding architectural design, VerneMQ supports MQTT message persistence in LevelDB and uses a clustering architecture based on the Plumtree library, which implements the Epidemic Broadcast Trees algorithm.
Unfortunately, this Plumtree cluster architecture has not proven to work, even though it seems perfect in theory. The VerneMQ team and community have spent many years trying to make it work, fixing problems such as network split, data inconsistency, and crash recovery.
Finally, the project has stopped being actively developed and maintained, with about 50 commits in the last 12 months.
Pros:
- High availability
- Horizontal scalability
- Message persistence
Cons:
- Not proofed clustering architecture
- Limited documentation
- Limited enterprise features
- Not actively developing
Community and Popularity
Both the EMQX and VerneMQ projects are hosted on GitHub. EMQX was launched in 2012 and is one of the earliest and now the highest-starred MQTT brokers with 11.4k stars. The VerneMQ project was created in 2014 and has 3k GitHub stars now.
| Community & Users | EMQX | VerneMQ | Notes and Links |
|---|---|---|---|
| GitHub Project | EMQX GitHub | VerneMQ GitHub | |
| Product Created | 2012 | 2014 | |
| License Mode | Apache License 2.0 | Apache License 2.0 | |
| Latest Release | v5.0.21 (March 2023) | v1.12.6.2 (Nov. 2022) | |
| GitHub Stars | 11.4k+ | 3k | |
| GitHub Forks | 2k | 300+ | EMQX GitHub Forks VerneMQ GitHub Forks |
| GitHub Commits | 14k+ | 2400+ | EMQX GitHub Commits VerneMQ GitHub Commits |
| GitHub Commits (Last 12 Months) | 3000+ | 50+ | |
| GitHub Issues | 3500+ | 1300+ | EMQX GitHub Issues VerneMQ GitHub Issues |
| GitHub PRs | 6000+ | 600 | EMQX GitHub PRs VerneMQ GitHub PRs |
| GitHub Releases | 260+ | 40 | EMQX GitHub Releases VerneMQ GitHub Releases |
| GitHub Contributors | 110+ | 50 | EMQX GitHub Contributors VerneMQ GitHub Contributors |
| Docker Pulls | 24M+ | 5M+ | EMQX Docker Pulls VerneMQ Docker Pulls |
Features and Capabilities
EMQX and VerneMQ fully implement the MQTT 3.1.1 and 5.0 specifications and support MQTT over WebSocket and SSL/TLS encryption. Both brokers offer various authentication mechanisms, including username-password, JWT, LDAP, and OAuth 2.0 authentication.
EMQX has multiple protocol gateways, including LwM2M/CoAP, MQTT-SN, and Stomp. EMQX 5.0 pioneers MQTT over QUIC, which has the potential to be the next-generation MQTT standard, with multiplexing and faster connection establishment and migration support.
In addition, EMQX provides rich enterprise features for management and integration, such as HTTP API, WebHook, and Rule Engine, which can integrate with Kafka, SQL, NoSQL databases, and cloud services via out-of-box data bridges.
| EMQX | VerneMQ | |
|---|---|---|
| MQTT 3.1.1 | ✅ | ✅ |
| MQTT 5.0 | ✅ | ✅ |
| MQTT over TLS | ✅ | ✅ |
| MQTT over WebSocket | ✅ | ✅ |
| MQTT over QUIC | ✅ | ❌ |
| LwM2M/CoAP | ✅ | ❌ |
| MQTT-SN | ✅ | ❌ |
| Stomp | ✅ | ❌ |
| MQTT Bridging | ✅ | ✅ |
| Authentication & ACL | ✅ | ✅ |
| Message Persistence | ✅ In RocksDB and external databases | ✅ In LevelDB |
| WebHook | ✅ | ✅ |
| Rule Engine | ✅ | ❌ |
| Data Integration | ✅ | ❌ |
| Cloud Service | Cloud Hosting Serverless MQTT BYOC |
❌ |
Scalability and Performance
EMQX and VerneMQ are designed for high performance, low latency, and scalability with a distributed architecture. Both can scale to handle millions of concurrent MQTT connections using a single cluster.
EMQX has over 30,000 clusters deployed in production with proven scalability and reliability. The latest EMQX 5.0 hits 100 million MQTT connections in the benchmark with a 23 nodes cluster.
For VerneMQ, few benchmark reports are available on the scalability and performance though it should work well in theory and design. You can benchmark it with the MQTT load testing tools like emqtt-bench, emqttb, or XMeter cloud service.
| EMQX | VerneMQ | Notes & Links | |
|---|---|---|---|
| Scalability | 4M MQTT connections per node 100M MQTT connections per cluster |
? | Reaching 100M MQTT connections with EMQX 5.0 |
| Performance | 4 million QoS0 MQTT msgs/sec per node 800k QoS1 msgs/sec 200k QoS2 msgs/sec |
? | |
| Latency | Single-digit millisecond latency in most scenarios | Up to seconds latency in some scenarios | |
| Reliability | Message Persistence in RocksDB and external Database | Message Persistence in LevelDB | Highly Reliable MQTT Data Persistence Based on RocksDB VerneMQ Storage |
| Clustering | 20+ nodes of cluster | ? | EMQX Cluster Scalability Vernemq - Cluster |
| Elastic and Resilient scaling at runtime | ✅ | ? | |
| Auto Clustering | ✅ | ? | EMQX Node Discovery and Autocluster |
| Zero Downtime/Hot Upgrade | ✅ | EMQX Release Upgrade |
? here means that we were unable to find any publicly available documentation or files that could serve as evidence regarding the item under discussion.
Data Integrations (Out of the box)
VerneMQ has limited support for MQTT data integration. It allows users to write plugins to ingest data into external databases or cloud services.
EMQX has a built-in SQL-based rule engine to help extract, filter, enrich, and transform MQTT messages in real-time within the broker.
The Enterprise Edition of EMQX can seamlessly integrate with Kafka, databases, and cloud services using the rule engine and out-of-the-box data bridges.
Extensibility
Both EMQX and VerneMQ can be easily extended with Hooks and Plugins. In addition, EMQX supports multiple protocol gateways, allowing users to develop new connection protocols.
| Extensibility | EMQX | VerneMQ | Notes and Links |
|---|---|---|---|
| Hooks | ✅ | ✅ | Hooks |
| Plugins | ✅ | ✅ | Plugins Plugin Development |
| Plugin Hot-loading | ✅ | ✅ | |
| Gateways | ✅ | ❌ | Gateway Introduction |
| ExHooks/gRPC | ✅ | ❌ | gRPC Hook Extension |
Operability and Observability
EMQX offers a user-friendly dashboard and extensive HTTP APIs. It supports monitoring with Prometheus, and Grafana. VerneMQ is easy to deploy and configure but lacks advanced management and monitoring features.
| EMQX | VerneMQ | Notes and Links | |
|---|---|---|---|
| Dashboard | ✅ | ❌ | Dashboard |
| Configuration | HOCON Format | Key-Value Fomat | |
| HTTP API | ✅ | ✅ | REST API |
| CLI | ✅ | ✅ | Command Line Interface |
| Config Hot update | ✅ | ❌ | Configuration Files |
| Metrics | ✅ | ✅ | Node metrics: Metrics $SYSTree Monitor - Metrics CLI |
| Grafana | ✅ | ✅ | EMQX | Grafana Labs VerneMQ Node Metrics | Grafana Labs |
| Cluster Metrics | ✅ | ✅ | Metrics Monitor - Metrics CLI |
| Alarm Alerts | ✅ | ❌ | System Topic |
| Slow Subscription Monitoring | ✅ | ❌ | Slow subscribers statistics |
| Prometheus | ✅ | ✅ | Integrate with Prometheus Prometheus |
Conclusion
In short, EMQX is one of the best choices for deploying MQTT brokers in production in 2023. If you want to dive into how to design a distributed MQTT broker and understand the challenges involved, you can read the source code of EMQX and VerneMQ on GitHub.
References
- VerneMQ - A MQTT broker that is scalable, enterprise ready, and open source
- VerneMQ Documentation
- EMQX: The World's #1 Open Source Distributed MQTT Broker
- EMQX 5.0 Documentation