EMQX: connect and prepare IoT data
Connect MQTT devices at scale, filter and transform their telemetry, and route prepared data to Snowflake.
- Connect millions of devices
- Process millions of messages per second
- Prepare data with SQL rules
EMQX SNOWFLAKE INTEGRATION
Stream MQTT data into Snowflake
Connect millions of IoT devices through EMQX and stream their data to Snowflake for analytics and AI applications.
Overview
A foundation for your IoT data cloud
EMQX handles device connections and data preparation. Snowflake provides storage, queries, and analytics.
Snowflake: store and analyze IoT data
Store device telemetry alongside your business data for SQL analytics, dashboards, and AI workflows.
- Scale storage and compute
- IoT analytics and AI workflows
- Secure data sharing
Architecture
MQTT and Snowflake architecture
Devices publish to EMQX, which prepares MQTT messages and sends them to Snowflake for storage and analytics.
Sensors
Vehicles
Devices
Factories
Gateways
EMQX Enterprise
MQTT Broker
Rule Engine
Data Integration
Batch or stream
Snowflake
Dashboards
Analytics
Business Intelligence
Data Warehouse
Capabilities
Built for Snowflake ingestion
Use EMQX to prepare device data, choose a Snowflake upload mode, and monitor your ingestion pipeline.
Prepare data with SQL
Filter MQTT messages and transform payload fields with EMQX SQL rules before sending them to Snowflake.
Rule SQL referenceMap fields to your tables
Select device identifiers, timestamps, and measurements, then match rule output fields to your Snowflake table columns.
Table mapping guideBatch or stream to Snowflake
Choose aggregated file uploads through a Snowflake stage and pipe, or Snowpipe Streaming where supported by your EMQX version.
Upload mode guideConfigure in the Dashboard
Create the Snowflake connector, SQL rule, and sink in EMQX. Test the connection and verify sample messages in your target table.
Snowflake setup guideBuffering and retries
Handle temporary Snowflake interruptions with buffering and retries. Set buffer capacity and request expiry to suit your workload.
Buffer queue guideMonitor ingestion
Track successful, failed, and dropped requests in EMQX, and watch queue depth to identify problems in your Snowflake pipeline.
Delivery monitoring guideHow it works
From MQTT to Snowflake in three steps
Connect devices, prepare message fields, and configure a Snowflake sink in the EMQX Dashboard.
Before you start
- An EMQX version supporting your chosen ingestion mode.
- A Snowflake account, database, schema, and target table.
- Authentication and permissions required by your ingestion mode.
- 1
Connect devices
Connect devices to EMQX and publish telemetry to MQTT topics.
- 2
Prepare message fields
Use SQL rules to select measurements, transform values, and match the output to your Snowflake table columns.
- 3
Write to Snowflake
Configure a Snowflake sink with your chosen upload mode, then verify test messages in the target table.
Use cases
IoT analytics with EMQX and Snowflake
Bring device telemetry into Snowflake to analyze equipment health, vehicle operations, and energy demand.
Predictive maintenance
Send equipment telemetry through EMQX to Snowflake to analyze operating trends and build models that support predictive maintenance.
Explore industrial IoTConnected vehicles
Stream vehicle telemetry through EMQX to Snowflake for fleet analytics, driver behavior analysis, and route planning.
Explore connected vehiclesSmart energy grids
Collect smart-meter and grid telemetry with EMQX, then analyze it in Snowflake for demand forecasting, usage trends, and outage investigation.
Explore smart energyResources
Explore EMQX and Snowflake integration
FAQ
Frequently asked questions
Freshness depends on upload mode, batching, network conditions, and Snowflake processing. Aggregated uploads wait for a record limit or time interval; streaming uses the Snowpipe Streaming API. Measure end-to-end delay with representative messages.
EMQX can buffer requests and retry during temporary interruptions. Retention depends on the configured buffer mode, capacity, request expiry (TTL), and sink behavior. Monitor failed and dropped requests and test recovery with your workload.
Yes. SQL rules can filter messages, select fields, and transform values before the Snowflake sink receives them. Match the output fields to your table columns and test with representative telemetry.
EMQX applies SQL rules to MQTT messages, then forwards the results through a Snowflake sink. Depending on your version and configuration, the sink uses aggregated files loaded through a stage and pipe, or Snowpipe Streaming.
You need an EMQX version that supports your chosen upload mode, a Snowflake account, and the required authentication, permissions, and destination objects. Follow the mode-specific prerequisites in the documentation, then create the connector, rule, and sink and verify delivery.
Build your MQTT to Snowflake data pipeline
Plan your MQTT ingestion, data preparation, and Snowflake deployment with an EMQX expert.