EMQX Cloud Rolls Out Shadow Service for Convenient IoT Data Caching

EMQ today announced a notable value-added service for its fully managed MQTT messaging platform, EMQX Cloud. The new Shadow service provides out-of-the-box platform-side data caching, helping dev teams to accelerate the development process for many IoT use cases.

Previously, developers needed to transfer IoT data to third-party services through the data integration component of EMQX Cloud before they could conduct further data processing, analysis, and application development. The Shadow service avoids reliance on third-party services, allowing developers to centralize device data caching, modification, and viewing directly within EMQX Cloud. They can quickly create object models, device shadows, and other resources related to data reporting and distribution that reduce build times, latency, and transmission costs.

As the global IoT industry continues to experience unprecedented growth, quickly connecting IoT devices and platform applications to support rapid implementation and market validation is becoming key to shaping core competitiveness and achieving business innovation. However, in the case of message interactions between IoT devices and applications, it is very common that the device side network is unstable, low-power devices are dormant, and mobile applications do not consume data regularly. To ensure reliable interactions, caching or persisting data, such as the latest reported messages from devices or configuration parameters issued by applications, in the MQTT message access layer can be an important requirement.

EMQX Cloud Shadow service solves the above-mentioned issues, empowering companies to speed up development while enhancing the reliability of data processing. After activating the service, users can configure shadow models, access data through a standard API, and monitor usage without requiring third-party integrations.

The long-awaited Shadow service on EMQX Cloud was informed by real customer needs across the automotive, manufacturing, finance, and other smart industries deployed in EMQX environments. Many IoT applications can benefit from Shadow service, such as the interactions between smart home devices, low-power smart meter data collection and configuration, and telematics message pushing.

With the data caching capability of the Shadow service, users can develop many applications without configuring external storage and network connectivity. It’s especially suitable for tracking device status.

Application requests to get device status are complicated by several conditions:

• Devices may go online and offline frequently because of network instability, so they cannot respond to application requests normally.
• Devices may need to respond to requests from multiple applications simultaneously, straining their limited processing capacity.
• Devices may transmit information even when there is no data consumer.
• Devices may transmit the same information in response to every request, even though different applications need to read different parts of the device information.

With the use of device shadowing, the device state change only needs to be synchronized to the device shadow once. Regardless of whether the application is online, the number of requests, and whether the device is online, the current state of the device can be obtained from the device shadow cache, enabling the decoupling of the application and the device.

Applications can send commands to devices, but when a device is offline or the device goes online and offline frequently because of an unstable network, the commands will fail to be sent. Using the device shadow mechanism, the commands issued by the application can be stored in the device shadow with a timestamp. When the device goes online again, it can obtain the commands from the device shadow and determine whether to execute them according to the timestamp.

Furthermore, Shadow service provides both MQTT and REST API interfaces to add, delete, and check cached documents for easy invocation by MQTT devices and application services. The MQTT interface provided by the Shadow service is fully compliant with the MQTT standard, and the client can invoke the Shadow service as long without the need for a customized SDK and without platform binding.

With the fully managed MQTT messaging service provided by EMQX Cloud combined with the Shadow service, users can easily integrate MQTT device access and message caching, greatly accelerating the speed of IoT application development. The flexible message caching data structure in the Shadow service can also help later business expansion, providing a guarantee for continuous business development.

For a deep dive into this new function, read EMQ’s recent blog post.

At present, the Shadow service provides a 7-day free trial with up to 1 GB of data. Users can log in to EMQX Cloud and provision Shadow service through the "Value-added Service" module on the top main menu or the "Shadow Service" module on the left menu of the deployment screen.

Any questions about the Shadow service should be directed to cloud-support@emqx.io.