Make telemetry queryable
Query device readings and events to investigate changes over time and build dashboards around your operations.
EMQX CLICKHOUSE INTEGRATION
Stream MQTT data into ClickHouse
Ingest, filter, and transform device telemetry with EMQX, then query it in ClickHouse for dashboards and analysis.
Overview
Turn MQTT telemetry into analytics
Query device history, compare equipment performance, and investigate changes with telemetry prepared by EMQX.
Keep only useful data
Filter messages and select the fields you need in EMQX, so ClickHouse receives data prepared for your queries.
Handle temporary interruptions
EMQX can queue requests during temporary ClickHouse interruptions and retry delivery within your configured buffer and expiry limits.
Architecture
From MQTT messages to ClickHouse queries
Devices publish to EMQX. SQL rules prepare telemetry for ClickHouse tables, where analytics tools query it.
Sensors
Vehicles
Devices
Factories
Gateways
EMQX Enterprise
MQTT Broker
Rule Engine
Data Integration
ClickHouse
Real-Time Dashboard
Real-Time Analytics
Business Intelligence
Data Warehouse
Capabilities
Build your ClickHouse ingestion pipeline
Prepare message fields, tune database writes, and monitor delivery from EMQX to ClickHouse.
Prepare data with SQL
Use SQL rules to select topics and fields, filter unwanted messages, and transform payload values before database writes.
Rule SQL referenceMap fields to your table
Map rule results to your ClickHouse columns with an INSERT template. Match values and data types to the target table.
ClickHouse Sink guideBatch database writes
Group multiple records into a batch to reduce database requests. Tune batch size with representative traffic and verify when records become available for queries.
Batch settingsChoose your query mode
Set EMQX’s query mode to synchronous or asynchronous. In async mode, MQTT publishing proceeds without waiting for the database response.
Query mode guideBuffering and retries
Configure buffer capacity and request expiry for your workload. Monitor delivery failures and test how the pipeline recovers after an interruption.
Buffer settingsMonitor ingestion
Track successful, failed, and dropped requests, along with queue depth, to identify delivery problems and tune your pipeline.
Monitoring guideHow it works
Connect your pipeline in three steps
Prepare a ClickHouse table and credentials that let EMQX insert data, then configure the pipeline in the EMQX Dashboard.
Before you start
- A ClickHouse endpoint reachable from EMQX.
- A target table for your telemetry.
- Credentials with write access to the table.
- 1
Connect devices
Connect devices or MQTT clients to EMQX and publish the telemetry you want to analyze.
- 2
Prepare messages
Create and test a SQL rule that selects messages and prepares fields for your ClickHouse table.
- 3
Write and verify
Configure a ClickHouse connector and sink with an INSERT template. Publish a test message, then query the table to confirm delivery.
Resources
Explore EMQX and ClickHouse integration
FAQ
Frequently asked questions
ClickHouse is a column-oriented SQL database for analytical queries. Use it to explore device history, aggregate telemetry, and power dashboards as your data grows.
EMQX can queue requests and retry after temporary connection failures. Requests can expire or be dropped when configured limits are reached. Set buffer capacity and request TTL for your workload, and monitor failed and dropped requests.
Yes. EMQX SQL rules can select topics and payload fields, filter messages, and transform values before the ClickHouse sink writes them to your table.
Throughput varies with message size, SQL rules, batch settings, network conditions, and ClickHouse capacity. Benchmark representative traffic through the complete pipeline, then tune batching and concurrency to meet your ingestion needs.
You need an EMQX deployment with ClickHouse integration, a reachable ClickHouse endpoint, a database name, credentials with write access, and a target table. Configure the connector, rule, and sink with an INSERT template, then verify delivery with a test message.
Build your MQTT to ClickHouse pipeline
Plan your MQTT ingestion, ClickHouse schema, and analytics goals with an EMQX expert.