MQTTX 1.10.0 Released: Advanced File Management and Configuration in CLI
MQTTX Team
Table of Contents
MQTTX 1.10.0 is now available!
This release significantly enhances file management and configuration capabilities, particularly in the CLI version. Key updates include support for reading and writing messages to files, advanced configuration options, text output mode, and improved logging. Additionally, the desktop version now supports database rebuilding to prevent issues with corrupted files and better handle large data for display. These aim to provide a more robust and user-friendly experience for all MQTTX users.
Download the latest version here: https://mqttx.app/downloads
File Management in CLI
MQTTX 1.10.0 introduces robust file read and write capabilities to the CLI. It seamlessly handles message payloads as file inputs and outputs over MQTT, facilitating integration and automation in data workflows.
File Reading
Note: Due to MQTT protocol constraints, payload size must not exceed 256MB. Verify your MQTT broker's payload size limit before transmitting.
Using the pub Command
Use the following command to read a message from a file:
mqttx pub -t topic --file-read path/to/file
The --file-read option allows you to read content directly from a file as the payload for publishing. This is convenient for using predefined test data for various publishing scenarios.
Using the bench pub Command
The command --file-read reads the message body from a file, similar to the simple pub command:
mqttx bench pub -c 10 -t topic --file-read path/to/file
Leveraging the capabilities of the bench command, you can split the input message into a single file using the --split option to send different data segments. By default, the character is \n.
For example, with a file containing:
hello world
You can use:
mqttx bench pub -c 10 -t topic --file-read path/to/file --split
If the file is comma-separated, change --split to ,:
mqttx bench pub -c 10 -t topic --file-read path/to/file --split ','
You can also set -im to define the interval for publishing messages.
File Writing
To write incoming messages to a file, use:
mqttx sub -t topic --file-write path/to/file
The --file-write option appends each message to a file, separated by a newline character \n by default, making it ideal for logging or accumulating text data. This feature is handy for applications that want to maintain a continuous log of received messages.
To change the delimiter, use the --delimiter option. For example:
mqttx sub -t topic --file-write path/to/file --delimiter ','
File Saving
For saving messages as separate files, use:
mqttx sub -t topic --file-save path/to/file
The --file-save option saves each incoming message as a separate file. Existing files will be automatically renumbered and saved to prevent overwrites. This feature is handy for applications that require storing individual messages for further processing or analysis.
For example, to save incoming messages to a specified directory:
mqttx sub -t topic --file-save /path/to/directory/message.txt
If /path/to/directory/message.txt already exists, new messages will be saved as message(1).txt, message(2).txt, and so on. This prevents any data loss due to overwriting existing files.
Do not use
--file-writeand--file-savetogether.
Type Format
Specify the file format for sending or saving. If not specified, it defaults to plaintext (UTF-8) format. The file content can also be formatted using the --format option to handle different data formats:
mqttx pub -t topic --file-read path/to/file --format type mqttx sub -t topic --file-save path/to/file --format type
Supported data formats for output to file include json, base64, hex, binary, and cbor.
This version also introduces a binary format to handle more common scenarios. MQTTX will generate the corresponding binary file based on the format by specifying binary.
If a file has the following extensions, it will automatically recognize the file format as binary:
- Image:
.png,.jpg,.jpeg,.gif,.bmp,.ico,.tif,.tiff - Video:
.mp4,.avi,.mov,.mkv,.flv,.wmv,.mpeg,.3gp - Audio:
.mp3,.wav,.flac,.aac,.ogg,.wma,.m4a,.m4p - Compressed:
.zip,.gz,.rar,.tar,.7z,.bz2,.xz,.jar - Binary:
.bin,.exe,.dll,.so,.dmg,.iso,.img - Document:
.pdf,.epub
To manually specify the binary format, use:
mqttx pub -t topic --file-read path/to/file --format binary
mqttx sub -t topic --file-save path/to/file --format binary
Configuration File in CLI
This version introduces a configuration file feature that stores default values for various settings, providing a simplified and customizable experience for MQTTX CLI. After initialization, the configuration file will be stored in the user's home directory at $HOME/.mqttx-cli/config.
Features
The configuration file includes settings for controlling the interface and functional parameters. These settings allow MQTTX CLI to use predefined values, improving efficiency and avoiding the need to enter information repeatedly.
Default:
- output
- text: Default mode provides concise output with crucial information.
- log: Displays detailed log output with date and time stamps.
MQTT:
- host: Default is localhost.
- port: Default is 1883.
- max_reconnect_times: Default is 10.
- username: Default is empty.
- password: Default is empty.
The output setting in the default section controls the CLI output display. Users can choose different modes according to their needs.
If the command line does not provide these parameters, MQTTX CLI will use the values from the configuration file in the mqtt section. host, port, username, and password
The max_reconnect_times controls the number of reconnection attempts. The connection will automatically close once this set number is reached to prevent infinite reconnections.
If configuration items like username and password are not required, they can be omitted from the configuration file.
Initializing Configuration
The configuration file is not provided by default. To create or update the configuration file, run the init command. This will prompt you to enter the desired values:
mqttx init
? Select MQTTX CLI output mode Text
? Enter the default MQTT broker host broker.emqx.io
? Enter the default MQTT port 1883
? Enter the maximum reconnect times for MQTT connection 5
? Enter the default username for MQTT connection authentication admin
? Enter the default password for MQTT connection authentication ******
Configuration file created/updated at /Users/.mqttx-cli/config
Example Configuration File
[default]
output = text
[mqtt]
host = broker.emqx.io
port = 1883
max_reconnect_times = 5
username = admin
password = public
Interface Improvements
The output setting in the configuration file offers two modes:
Text Mode: Provides a concise and clean output with only the critical information, making it easier to read and understand.
mqttx conn ✔ ConnectedLog Mode: This mode displays detailed log output with date and time stamps, which is helpful for recording and debugging.
mqttx conn [5/24/2024] [11:26:17 AM] › … Connecting... [5/24/2024] [11:26:17 AM] › ✔ Connected
Users can tailor the MQTTX CLI to better fit their workflow and preferences by selecting the appropriate output mode.
Desktop White Screen Issue
We have investigated reports of white screen issues in the Desktop version and identified two primary causes: corrupted database files and performance issues when rendering large message payloads. We have optimized MQTTX to address these problems.
Database Rebuild
Corrupted SQLite database files can result from several factors, particularly after a software upgrade:
- Incompatible Architectural Changes: Upgrades that alter the database structure without properly migrating old data.
- Interrupted Upgrades: Unexpected interruptions during software updates lead to incomplete database files.
- Race Conditions: Concurrent access is not managed correctly, causing write conflicts.
- Faulty Upgrade Scripts: Errors in database script execution during updates affect data integrity.
- Insufficient Disk Space: The lack of space during updates prevents complete data writes.
- File System or Hardware Issues: Underlying storage problems causing file corruption.
When such issues occur, users are unable to open MQTTX. The new version of MQTTX will display a database rebuild page when it detects a corrupted database. Users can click the rebuild button to fix the corrupted database file and reinitialize the data.
Note: After rebuilding the database, all local data will be lost.
Large Data Handling
Another issue is the performance hit caused by large message payloads. While typical MQTT messages are generally under 1MB, the maximum size can be 256 MB. When users send large files, rendering these messages in MQTTX can cause the UI to freeze or crash, resulting in a white screen.
In the new version, we have implemented a data threshold. When the payload size exceeds 512KB, MQTTX will display only a portion of the message content. Users can click "Show More" to view the full message. Additionally, users can save large messages to their local system using the "Save to Local" button for viewing with other applications.
These improvements ensure that MQTTX can handle large payloads more efficiently, preventing UI freezes and enhancing the overall user experience.
Web Updates
Added Support for BASE_URL Configuration via Env File
If you need to make personalized settings, such as modifying the default connection path, deployment path, or output path, you can make the corresponding modifications in the web/.env or web/.env.docker files. These two files correspond to different packaging requirements, and you can modify them according to your situation.
| Configuration Item | Description |
|---|---|
| VUE_APP_PAGE_TITLE | The title displayed in the browser's title bar |
| VUE_APP_PAGE_DESCRIPTION | A brief page description for SEO purposes |
| VUE_APP_DEFAULT_HOST | The default address of the MQTT broker server connections |
| BASE_URL | The root URL where the application is deployed. Helpful in constructing links and routing |
| VUE_APP_OUTPUT_DIR | The directory where the build files will be placed after compilation |
These updates aim to provide a more flexible and user-friendly experience, allowing you to customize your MQTTX Web setup to fit your needs.
Breaking Changes
| Old Command | New Command |
|---|---|
| mqttx conn -h broker.emqx.io -p 1883 --save | mqttx conn -h broker.emqx.io -p 1883 --save-options |
| mqttx conn --config /Users/mqttx-cli-config.json | mqttx conn --load-options /Users/mqttx-cli-options.json |
In this update, the --config parameter has been replaced with the --options parameter. This change is made to reflect better the purpose of these parameters, which is to save and load the frequently used command parameters.
- -so, --save-options: Save the parameters to the local configuration file, which supports JSON and YAML formats. The default path is
./mqttx-cli-options.json. - -lo, --load-options: Load the parameters from the local configuration file, which supports JSON and YAML formats. The default path is
./mqttx-cli-options.json.
Others
New Features and Improvements
- Auto Re-subscription Tips: The subscription dialog now has auto re-subscription tips. When subscribing, you can see whether the auto re-subscription feature is enabled.
- GPT-4o Support: Added support for GPT-4o in MQTTX Copilot, bringing advanced AI capabilities to your MQTTX experience.
Bug Fixes
- Version Update Dialog: The version update dialog was fixed to ensure it is properly adapted to dark mode and provides a consistent user experience across different themes.
- Topic Filtering: Resolved an issue with topic filtering, ensuring accurate and reliable filtering of topics.
- Logging Improvements: Enhanced the logging format and improved bench sub logs by graying out output meta information, making logs more readable and useful for debugging.
- Subscription Errors: Fixed the subscription error logic when dealing with multiple topics, ensuring smoother and more reliable subscriptions.
- CLI Pub Failure Handling: Improved the reconnection logic in the CLI for publishing failures, ensuring that the CLI handles pub failures more gracefully and attempts reconnections as needed.
These updates focus on improving user experience, enhancing functionality, and fixing critical bugs to ensure a smoother and more reliable operation of MQTTX.
Roadmap
- Payload Chart Visualization Enhancement - MQTTX Viewer:
- Topic Tree View: Enhance organization and visualization of topics.
- Diff View: Compare different messages or payloads easily.
- Dashboard View: Offer a customizable overview of MQTT activities for personalized insights.
- JSON View: Improve handling and display of JSON formatted data.
- System Topic View: Specialized view for system-related MQTT topics.
- Support for Configurable Disconnect Properties (MQTT 5.0): Enhance connection management with customizable disconnection settings.
- IoT Scenario Data Simulation: Bring this feature to the desktop client to ease IoT scenario testing.
- Sparkplug B Support: Extend MQTTX functionalities to include support for Sparkplug B.
- QoS 0 Message Storage Optimization: Configurable options to reduce storage space usage.
- MQTT GUI Debug Functionality: New features to aid in debugging MQTT communications.
- Plugin Functionality: Introduction of a plugin system supporting protocol extensions like CoAP and MQTT-SN.
- Avro Message Format Support: Encoding and decoding capabilities for Avro message format.
- Script Test Automation (Flow): Simplify the creation and management of automated testing workflows.