EMQ Sponsors Open Source Summit Europe 2024 | Register Now →

Convenient Data Cache Service to Accelerate the Development of IoT Applications

EMQX Cloud Team
Oct 25, 2022
Convenient Data Cache Service to Accelerate the Development of IoT Applications

The fully hosted MQTT messaging cloud service EMQX Cloud can help users easily connect various IoT devices to the cloud and provide data integration with various third-party services to help users perform efficient data processing, storage and analysis.

To enable more convenient IoT data processing and further simplify the development process for users to build IoT applications, EMQX Cloud recently launched a new value-added service——Shadow Service.

Feature introduction

Shadow service is a device data caching service provided by EMQX Cloud, helping users quickly design and develop IoT applications through the definition and use of Topic and API.

Previously, users need to transfer IoT data to a third-party data service through the data integration service of EMQX Cloud before they conduct IoT data processing analysis and further development of IoT applications, eg. “first purchase third-party data integration resource cloud services, then create VPC peer-to-peer connections to connect EMQX Cloud and third-party cloud services, and finally create and develop IoT applications”.

The device data caching capability provided by the newly launched shadow service can save the steps of connecting the EMQX Cloud and third-party data services. Users can enable centralized device data caching, modification, and viewing within the EMQX Cloud, quickly create object models, device shadows, and other applications related to data reporting and distribution, greatly saving the time and cost of development.

Shadow service

Application scenarios

With the data caching capability provided by the shadow service, users can develop many applications without configuring external storage and network connectivity, very suitable for the following scenarios:

Application requests to get device status

  • The device goes online and offline frequently for unstable network, so it can not respond to the request of the application normally.
  • The device network is stable and responds to requests from multiple applications simultaneously. Even if the response results are the same, the device itself cannot load multiple requests for limited processing capacity.
  • The device transmits information, and there is no data consumer. Latest device information needs to be checked only after the application goes online.
  • The device transmits information, and different applications read different parts of the information.
  • The device transmits multiple sets of information, and the application comprehensively displays all the information.

With the use of device shadow, the device state change only needs to be synchronized to the device shadow once, and the application requests to obtain the device state. Regardless of whether the application is online, the number of requests, and whether the device is online, the current state of the device can be obtained from the device shadow to enable the decoupling of the application and the device.

The application sends commands to the device to change the device status

When the device is offline, or the device goes online and offline frequently for unstable network, the application sends control commands to the device. When the device is not online, the commands will fail to be sent.

Using the device shadow mechanism, the commands issued by the application can be stored in the device shadow with the timestamp. When the device goes online again, it obtains the commands from the device shadow and determines whether to execute them according to the timestamp.

User Guide

Provisioning and Billing Instructions

At present, the shadow service provides a 7-day free trial of 1G. You can log in to EMQX Cloud and provision shadow service through the "Value-added Service" module on the top menu or the "Shadow Service" module on the left menu.

Value-added Service

"Value Added Service" -> "Shadow Service" -> Activate Service on the top menu

"Shadow Service" -> Activate Service

"Shadow Service" ->Activate Service on the left menu

Note: Since the shadow service uses the cloud computing resources of AWS, the shadow service is currently available only for AWS professional version.

The cost of shadow service consists of three parts: memory size, number of requests, and outbound traffic.

You can select different memory sizes according to the estimated number of shadow models required. For the prices of different memory sizes and the number of shadow models created with customers, refer to the following table

Memory size Memory size price Estimated number of shadow models that can be created Requests cost Outbound traffic fee
1G $0.08 /hour 8000 $0.04 /10,000 times, less than 10,000 times will be charged as 10,000 times The cost of outbound traffic is calculated together with the current deployed traffic. Priority is given to the consumption of monthly free traffic, and the excess is calculated at $0.15 /GB.
2G $0.15 /hour 16000
4G $0.4 /hour 32000

Shadow Service

Specific billing standards and billing examples can be found at:

Feature Details

After the activation of service, you can find the three pages of "Usage Overview", "Shadow Models" and "API" in the navigation bar.

On the "Usage Overview" page, you can learn about the memory usage of the currently deployed shadow service in a timely manner through the storage usage, the monthly invocations, and the line chart of usage changes across time dimensions, so as to monitor and alert business usage.

Usage Overview

Note: The service system will take up about 10 MB of storage space by default.

On the “API” page, you can learn about the API definitions for creating, querying, updating, and deleting shadow models (information). Your IoT business can use these APIs to obtain information about shadow services to accelerate the development of IoT applications.

“API” page

At the same time, we provide you with several call examples for reference:

On the "Shadow Model List" page, you can add, edit, and modify shadow models, and import customized shadow models in batches through templates.

Shadow Model List


Click “Add”, fill in the relevant information, and click “Confirm” to create the shadow model.

shadow model

Field Description of Shadow Model:

Field Necessity Note
Name Required 3 to 50 characters, and can only contain letters, numbers, "-", "_", "."
ID Required ID is a globally unique identifier for the shadow model that will be used in Topic and API, with a minimum of 8 bits and a maximum of 64 bits, and can only contain letters, numbers, "_", and "-". If not filled in it will be automatically generated.
Note Conditional Put a note
Topic for publishing Generated Automatically generated, the topic is based on ID for message publishing.
Topic for subscription Generated Automatically generated, the topic is based on ID for message subscription.

Click the ID in the shadow model list to enter the shadow model details page. On this page, you can not only view and modify the name and remarks of the current model, but also see the latest data of JSON model, or even modify it.

shadow model details page

With the out-of-the-box shadow services, data caching requirements of different business scenarios in various industries can be met. The integration capabilities of MQTT device access and message caching provided by the shadow service will provide impetus for accelerating the development of IoT platforms and applications.

Try EMQX Cloud for Free
No credit card required
Get Started →

Related Posts