Save MQTT Data from EMQX Cloud to AWS DynamoDB through the public network

Amazon DynamoDB is a fully managed NoSQL database service that provides fast and predictable performance with seamless scalability. DynamoDB lets you offload the administrative burdens of operating and scaling a distributed database so that you don't have to worry about hardware provisioning, setup and configuration, replication, software patching, or cluster scaling. DynamoDB also offers encryption at rest, which eliminates the operational burden and complexity involved in protecting sensitive data.

In this article, we will simulate temperature and humidity data and report it to the EMQX Cloud via the MQTT protocol, after which we will use the EMQX Cloud data integration to enable NAT gateway and save the data to the AWS DynamoDB over the public network.

Introduction to EMQX Cloud

EMQX Cloud is the world's first fully managed MQTT 5.0 public cloud service for IoT from EMQ. EMQX Cloud provides a one-stop O&M colocation and a unique isolated environment for MQTT services. In the era of Internet of Everything, EMQX Cloud can help you quickly build industry applications and easily realize the collection, transmission, computation and persistence of IoT data.

MQTT Cloud

With the infrastructure provided by cloud providers, EMQX Cloud is available in dozens of countries and regions around the world, providing low-cost, secure, and reliable cloud services for 5G and Internet of Everything applications.

For more information, please go to the EMQX Cloud website or see the EMQX Cloud documentation.

Create Deployments

Create EMQX Cluster

Once logged in, click "Cloud Console" under the account menu and you will be able to see the green button to create a new deployment. EMQX Cloud offers a 14-day free trial of Standard and Professional plans. This tutorial uses the professional deployment as a demonstration.

Standard and Professional plans

Create an AWS deployment of EMQX Cloud, other options default.

Create an AWS deployment

When the status is Running, the creation of the deployment is complete.

status is Running

Create DynamoDB instances

If you are creating a DynamoDB instance for the first time, you can refer to the help document. First, go to the DynamoDB console and click Create Table.

Create DynamoDB instances

Fill in key information such as table name, partition key, and other defaults, which can be set according to your actual business needs.

Create DynamoDB Table

Until the status of the table is active, it means that you have successfully created the table 'temp_hum'.

the status of the table is active

Enable NAT Gateway

NAT gateways can provide network address translation services to provide Professional deployments with the ability to access public network resources without the need for VPC peering connections.

Enable NAT Gateway

Data Integrations

  1. Create DynamoDB resources

    Go to the Data Integrations page. On the data integration page, click DynamoDB resources.

    Go to the Data Integrations page

    Fill in the DynamoDB connection details, and then click test. Please check the DynamoDB service if the test fails. Click the New button after the test is passed and you will see the Create Resource successfully message.

    Fill in the DynamoDB connection details

  2. Create a new rule

    After the resource is successfully created, you can return to the data integration page and find the newly created resource, and click create rule. Our goal is that as long as the temp_hum/emqx topic has monitoring information, the engine will be triggered. Certain SQL processing is required here:

    • Only target the topic "temp_hum/emqx"
    • Get the three data we need: temperature, humidity

    According to the above principles, the SQL we finally get should be as follows:

    SELECT 
    id as msgid,
    topic, 
    payload 
    
    FROM "temp_hum/emqx"
    

    Create a new rule

    You can click SQL Test under the SQL input box to fill in the data:

    • topic: temp_hum/emqx
    • payload:
    {
      "temp": 26.3,
      "hum": 46.4
    }
    

    Click Test to view the obtained data results. If the settings are correct, the test output box should get the complete JSON data as follows:

    JSON data

    If the test fails, please check whether the SQL is compliant and whether the topic in the test is consistent with the SQL filled in.

  3. Add Action to Rule

    After completing the rule configuration, click Next to configure and create an action. Then enter the fields and tags as follows:

     DynamoDB Table:temp_hum
     Hash Key:msgid
    

    Add Action to Rule

Verification

  1. Use MQTTX to simulate data reporting

    We recommend you to use MQTT X, an elegant cross-platform MQTT 5.0 desktop client to subscribe/publish messages.

    Click on the add button and fill in the deployment information to connect to the deployment. You need to replace broker.emqx.io with the created deployment connection address, add client authentication information to the EMQX Cloud console. Enter the topic name and payload message to publish the message.

    MQTTX

  2. View rules monitoring

    Check the rule monitoring and add one to the number of success.

    View rules monitoring

  3. View results in NoSQL Workbench

    NoSQL Workbench for Amazon DynamoDB is a cross-platform client-side GUI application for modern database development and operations. You can reach out to it to connect to DynamoDB.

    connect to DynamoDB

    Go to the Operation Builder page. Select the table 'temp_hum'. Here you can see the results of the temperature and humidity data forwarding.

    Go to the Operation Builder page

Summary

So far, we have used EMQX Cloud data integration to save the entire process of data to the AWS DynamoDB over the public network. Then, you can integrate with AWS services to do more with your data, like using built-in tools to perform analytics, extract insights, and monitor traffic trends, which will save you more time to focus on building your IoT applications.

Try EMQX Cloud for Free
No credit card required
Get Started →

Related Posts

Bridge MQTT Data from EMQX Cloud to Confluent Cloud on GCP

In this article, we will share how to bridge MQTT data from EMQX Cloud to Confluent Cloud on GCP.

2022-09-07
Using EMQX Cloud Data Integrations to Save Data to ClickHouse Cloud

We will simulate temperature and humidity data and report these data to EMQX Cloud via the MQTT and then use the EMQX Cloud Data Integrations to save the data into ClickHouse Cloud.

2022-12-26
Save MQTT Data from EMQX Cloud on GCP to Timescale Cloud through the public network

We will simulate temperature and humidity data, report it to the EMQX cloud on the GCP platform via the MQTT protocol, and save the data to the Timescale Cloud over the public network.

2022-10-25