Unpacking EMQX Enterprise 5.7: Durable Sessions, Message Schema Validation, and Advanced Rule Engine Capabilities

Morgan Hill, CA, May 28, 2024 – EMQ, a global leader in open-source MQTT messaging and the developer of the EMQX platform, is excited to announce the release of EMQX Enterprise 5.7.0. This latest version introduces several new features and improvements, including durable sessions, message schema validation, and enhanced debugging and tracing capabilities for the rule engine. Numerous enhancements and bug fixes have also been implemented to boost overall performance and stability.

Enhanced Persistence with Durable Sessions

The new Durable Session feature in EMQX provides robust persistence and high availability. This feature allows MQTT persistent sessions and their messages to be stored on disk, with session metadata and MQTT messages continuously replicated across multiple nodes within an EMQX cluster.

With flexible configuration options, users can customize the number of replicas per message or session by adjusting the replication factors and balancing persistence and performance.

Storing MQTT messages in shared, replicated persistent storage reduces memory usage for online and offline sessions. This supports larger sessions and greater message processing capacity compared to in-memory storage. Durable sessions enhance system reliability by enabling effective failover and recovery mechanisms, ensuring service continuity and high availability.

For more details on MQTT's durable sessions feature, click here.

Ensuring Data Integrity with Message Schema Validation

EMQX's Schema Validation feature ensures that MQTT messages meet the required structure and format. Non-compliant messages can be discarded or disconnected from the client, with logs generated and rule engine events triggered for further processing.

Schema validation supports formats like JSON Schema, Protobuf, and Avro, and can use built-in SQL statements to validate message formats from specific topics. By blocking non-compliant messages early, it ensures system stability and reliability.

Additionally, the same schema can be used for encoding/decoding and checking functions in the EMQX rule engine, as well as in external data systems and business processes, helping users achieve:

• Data Integrity: Ensures consistency and correctness by validating message structure and format.
• Data Quality: Maintains quality by checking for missing or invalid fields, data types, and formats.
• Uniform Data Model: Reduces inconsistencies and errors by enforcing a consistent data model across teams and projects.
• Reuse and Sharing: Enhances collaboration by allowing schema reuse and sharing, reducing duplication of effort and errors.

For further information on the message schema validation feature, click here.

Enhanced Rule Engine Debugging and Tracing

The rule engine now features advanced debugging and tracing capabilities. You can trigger a rule with simulated data or a real client, execute the rule's SQL and associated actions, and view execution results for each step.

When a rule's SQL or any action fails, the error log appears on the Dashboard, helping you quickly identify the issue and view structured error messages for troubleshooting. For instance, in the screenshot, the rule was triggered four times: the first three were successful, while the fourth failed due to an HTTP service action error, indicated by a 302 status code.

This enhanced feature ensures the entire rule functions correctly, facilitating quick issue resolution and preventing failures during actual execution.

To learn more about rule engine feature, click here.

Enhanced Flexibility with Client Attributes

Client attributes in EMQX allow you to assign additional key-value pairs to a client. These values can be derived from connection details (e.g., username, client ID, TLS certificate) or set based on authentication data. For example, you can configure EMQX to split the client ID at the colon (:) on connection and use the first segment as a VIN attribute:

mqtt.client_attrs_init = [
  {
    expression = "nth(1, tokens(clientid, ':'))"
    set_as_attr = "VIN"
  }
]

These attributes are versatile, supporting various EMQX features such as authentication & authorization, data integration, and MQTT extensions. For instance, in MySQL authorization, you can configure SQL queries to determine publish/subscribe permissions based on the VIN attribute:

SELECT
  permission, action, topic, qos, retain
FROM mqtt_acl
  WHERE VIN = ${client_attrs.VIN}

Unlike static attributes like client IDs, client attributes offer greater flexibility, simplifying development and enhancing adaptability and efficiency in diverse business scenarios.

For more information on the client attributes feature, click here.

Additionally, this release includes features such as enhanced rule action with streamlined variable input, log tracking feature enhancements, disconnecting clients when JWT authentication expires, plugin support for hot configuration, and a customizable UI.

For more information about EMQX Enterprise 5.7.0 and its comprehensive features, please visit our website.