Moving data from MySQL to ClickHouse sounds like a one-connector job. The real decision comes first: where does the copy live, and who else can read it? Most setups end in tables that only ClickHouse can open, so the next engine that wants the same orders needs a pipeline of its own.
These are the usual ways to do it, each with a real use:
- ClickPipes for MySQL: change data capture (CDC) from the binlog, run for you by ClickHouse Cloud, into
ReplacingMergeTreetables. Nothing to operate, but it is ClickHouse Cloud only, billed per ClickPipe, and ClickHouse is the only reader. - The MySQL table engine: nothing is copied at all. Every query goes to MySQL live and reads the current row, so the analytics load lands on the database that takes your writes.
- Debezium, Kafka and the ClickHouse Kafka engine: changes in seconds, paid for with Kafka, Kafka Connect, Debezium and a materialized view per table, plus merge logic you write for updates.
- Managed ELT (extract, load, transform) such as Fivetran, Estuary or Airbyte Cloud: nothing to host, priced per row, per GB or per connector, and the data lands as a second copy you pay to move and store.
This walkthrough takes a fifth route. OLake Go, an open source (Apache 2.0) replication tool, reads the MySQL binary log (binlog) and commits the changes as Apache Iceberg tables into an Apache Polaris catalog. ClickHouse attaches that catalog as one database, and DuckDB and Trino read the very same tables.
I ran every step on 2026-10-29 with OLake Go v1.1.0, MySQL 8.4, Polaris 1.8.0 on MinIO, ClickHouse 26.8.36 (the long-term support release), DuckDB 1.5.5 and Trino 484. The timings and row counts below are from that run.
What you'll build
- Source: MySQL with row-based binary logging and two tables,
customersandorders. - Catalog and storage: Polaris and MinIO in Docker, with the Parquet files in a bucket you own.
- Pipeline: OLake UI from Docker Compose. One job copies each table in full, then replays only the binlog.
- Readers: ClickHouse through its
DataLakeCatalogengine, then DuckDB and Trino.
You need: Docker, MySQL 8.0 or later, ClickHouse 26.1 or later (needed for DataLakeCatalog to read REST catalogs), and about 40 minutes.
Step 1: Turn on row-based binlog in MySQL
Put these lines in my.cnf and restart MySQL. On Amazon Relational Database Service (RDS), Aurora, Google Cloud SQL or Azure, set the same values in the parameter group or server flags instead.
server-id = 1 log_bin = mysql-bin binlog_format = ROW binlog_row_metadata = FULL binlog_expire_logs_seconds = 604800 # keep 7 days, so a paused job can catch up Then create the user OLake Go connects as, and the two sample tables:
CREATE USER IF NOT EXISTS 'olake'@'%' IDENTIFIED BY '<password>'; GRANT SELECT, REPLICATION SLAVE, REPLICATION CLIENT ON *.* TO 'olake'@'%'; FLUSH PRIVILEGES; CREATE TABLE customers ( id INT AUTO_INCREMENT PRIMARY KEY, name VARCHAR(100), email VARCHAR(200), city VARCHAR(100), tier VARCHAR(20), created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP, updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP); CREATE TABLE orders ( id INT AUTO_INCREMENT PRIMARY KEY, customer_id INT, product VARCHAR(100), amount DECIMAL(10,2), status VARCHAR(20), created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP, updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP, FOREIGN KEY (customer_id) REFERENCES customers(id)); Insert your own rows. My run started with 8 customers and 40 orders. The change script in step 7 updates two customers by name, Liam Carter and Zara Ali, so include those two if you want it to run unchanged.
Now the line most people skip, followed by two checks:
ANALYZE TABLE customers, orders; SHOW VARIABLES WHERE Variable_name IN ('log_bin', 'binlog_format', 'binlog_row_metadata', 'server_id'); SHOW GRANTS FOR 'olake'@'%'; OLake Go splits the first copy into parallel chunks using the row estimates in information_schema. A table that was just created or loaded has no estimates, and the first sync refuses to start until ANALYZE TABLE fills them in.
A few things that matter on a production server:
-
binlog_row_metadata = FULLputs column names and types in every event. That is how OLake Go maps changes correctly after a column is added. - Retention is your replay window. A job paused for longer than
binlog_expire_logs_secondscannot find its saved position and needs a fresh snapshot. Seven days of binlog on a busy server can be tens of gigabytes, so size the disk for it. - Tables without a primary key replicate inserts only. Updates and deletes have nothing to match on, so add a primary key or a unique index first.
Step 2: Run Polaris and MinIO, then create the catalog
Polaris is an open source Iceberg REST (Representational State Transfer) catalog. It stores no data, only pointers to table metadata, plus role-based access and short-lived storage credentials it hands to writers and readers. Save this as docker-compose.yml:
services: minio: image: minio/minio hostname: lake-minio command: server /data --console-address ":9001" environment: { MINIO_ROOT_USER: minioadmin, MINIO_ROOT_PASSWORD: minioadmin } ports: ["9000:9000", "9001:9001"] polaris: image: apache/polaris:latest depends_on: [minio] ports: ["8181:8181", "8182:8182"] environment: AWS_REGION: us-east-1 AWS_ACCESS_KEY_ID: minioadmin AWS_SECRET_ACCESS_KEY: minioadmin POLARIS_BOOTSTRAP_CREDENTIALS: POLARIS,root,<root-secret> polaris.realm-context.realms: POLARIS Start it with docker compose up -d. Polaris keeps its state in memory in this setup and starts empty each time, so run the next block once per start (a production deployment backs it with Postgres):
P=http://localhost:8181 # 1. A bucket for the tables aws --endpoint-url http://localhost:9000 s3 mb s3://lakehouse # 2. A root token TOKEN=$(curl -s $P/api/catalog/v1/oauth/tokens -H 'Polaris-Realm: POLARIS' \ -d grant_type=client_credentials -d client_id=root -d client_secret=<root-secret> \ -d scope=PRINCIPAL_ROLE:ALL | jq -r .access_token) H=(-H "Authorization: Bearer $TOKEN" -H 'Polaris-Realm: POLARIS' -H 'Content-Type: application/json') # 3. The catalog, pointing at the bucket curl -s "${H[@]}" $P/api/management/v1/catalogs -d '{"catalog": { "name": "olake", "type": "INTERNAL", "readOnly": false, "properties": {"default-base-location": "s3://lakehouse/olake"}, "storageConfigInfo": {"storageType": "S3", "allowedLocations": ["s3://lakehouse/olake"], "endpoint": "http://<minio-host>:9000/", "endpointInternal": "http://lake-minio:9000", "pathStyleAccess": true, "region": "us-east-1"}}}' # 4. A principal for OLake Go and the engines (save clientId and clientSecret from the response) curl -s "${H[@]}" $P/api/management/v1/principals -d '{"principal": {"name": "olake"}}' # 5. Roles, and the grant that lets the principal create namespaces and tables curl -s "${H[@]}" $P/api/management/v1/principal-roles -d '{"principalRole": {"name": "olake_role"}}' curl -s "${H[@]}" $P/api/management/v1/catalogs/olake/catalog-roles -d '{"catalogRole": {"name": "olake_catalog_role"}}' curl -s -X PUT "${H[@]}" $P/api/management/v1/principals/olake/principal-roles -d '{"principalRole": {"name": "olake_role"}}' curl -s -X PUT "${H[@]}" $P/api/management/v1/principal-roles/olake_role/catalog-roles/olake -d '{"catalogRole": {"name": "olake_catalog_role"}}' curl -s -X PUT "${H[@]}" $P/api/management/v1/catalogs/olake/catalog-roles/olake_catalog_role/grants -d '{"type": "catalog", "privilege": "CATALOG_MANAGE_CONTENT"}' Three details in that block decide whether the later steps work:
- The trailing slash on
endpoint. ClickHouse 26.8 and 26.10 build object paths from it. Without the slash, every ClickHouse read fails with a 404 even though the tables are listed. - No underscores in hostnames. Polaris runs on Java, whose URI parser rejects a host such as
lab_minio. That is why the container is calledlake-minio. - The principal's secret appears once, in the response to step 4. It goes into OLake Go and ClickHouse as
<client_id>:<client_secret>.
Check the principal before moving on. A good one returns JSON with an access_token:
curl -s http://localhost:8181/api/catalog/v1/oauth/tokens -H 'Polaris-Realm: POLARIS' \ -d grant_type=client_credentials -d client_id=<client_id> -d client_secret=<client_secret> \ -d scope=PRINCIPAL_ROLE:ALL Step 3: Start OLake UI and add the MySQL source
curl -sSL https://raw.githubusercontent.com/datazip-inc/olake-ui/master/docker-compose-v1.yml | docker compose -f - up -d Open http://localhost:8000 and sign in with admin / password. Go to Sources, click Create Source and pick MySQL.
Leave Enable CDC on and keep the default Initial Wait Time (10 in my run). For the connection, MySQL ran in another container, so the host was host.docker.internal, with database shop, user olake and SSL off.
Set up source tests the connection and then checks binlog_access, log_bin, binlog_format and binlog_row_metadata. If one fails, the dialog shows the current value next to the recommended one, so you know exactly what to change.
Step 4: Add the Polaris destination
Go to Destinations, click Create Destination, choose Apache Iceberg, catalog type Polaris and authentication type OAuth2. These are the values from my run:
| Field | Value |
|---|---|
| REST Catalog URI | http://host.docker.internal:8181/api/catalog |
| Polaris Catalog Name | olake |
| REST Auth URI | http://host.docker.internal:8181/api/catalog/v1/oauth/tokens |
| REST Credential | <client_id>:<client_secret> |
| REST scope | PRINCIPAL_ROLE:ALL |
| S3 Endpoint | http://host.docker.internal:9000 |
| AWS Region | us-east-1 |
| AWS Access Key, AWS Secret Key | the MinIO keys |
Click Create. OLake Go swaps the credential for a token, creates a namespace and a test table through Polaris, writes to the bucket and then drops the test table.
Step 5: Create the job and run the first sync
Go to Jobs and click Create Job. I named mine mysql_to_clickhouse and gave it a daily schedule; the schedule sets how often the binlog is drained, not how often a full copy runs.
Under Advanced Settings, set Target Query Engines to ClickHouse, DuckDB and Trino. That is the only format decision on the job: OLake Go works out a delete format all three can read. With ClickHouse in the list it picked equality deletes, which need no index kept between runs.
On the Streams page, tick customers and orders, keep Full Refresh + CDC and Upsert, and leave the Upsert Type as resolved. Then click Create Job and choose Sync Now from the job menu.
The first run copied all 48 records (8 customers and 40 orders, 2.10 KB) in 10 seconds, container start included. The destination namespace is named after the job and the source database, so it came out as mysql_to_clickhouse_shop. The Iceberg DB field at the top of the Streams page renames it if you change it before the first sync.
Step 6: Query the MySQL to ClickHouse tables
There is nothing to register. Run this in clickhouse-client:
SET allow_database_iceberg = 1; CREATE DATABASE polaris ENGINE = DataLakeCatalog('http://<polaris-host>:8181/api/catalog') SETTINGS catalog_type = 'rest', warehouse = 'olake', catalog_credential = '<client_id>:<client_secret>', oauth_server_uri = 'http://<polaris-host>:8181/api/catalog/v1/oauth/tokens', auth_scope = 'PRINCIPAL_ROLE:ALL', vended_credentials = true, region = 'us-east-1'; SHOW TABLES FROM polaris; SELECT id, name, city, tier FROM polaris.`mysql_to_clickhouse_shop.customers` ORDER BY id; Note the backticks. ClickHouse has a single level of namespace, so the Iceberg namespace and the table name travel together as one quoted identifier.
With vended_credentials = true, ClickHouse reads the files with per-table keys that Polaris issues, so no storage keys live in ClickHouse. The attached database is read-only: INSERT, ALTER and RENAME DATABASE return NOT_IMPLEMENTED.
Every table also carries four columns OLake Go adds: _olake_id, _op_type, _olake_timestamp and _cdc_timestamp. _op_type is r for a snapshot row, c for an insert, u for an update and d for a delete.
Step 7: Change rows in MySQL and sync again
This is where a nightly reload falls short. Run these 12 changes in MySQL:
INSERT INTO customers (name, email, city, tier) VALUES ('Maya Patel', 'maya@example.com', 'Pune', 'silver'), ('Lucas Silva', 'lucas@example.com', 'São Paulo', 'bronze'); INSERT INTO orders (customer_id, product, amount, status) VALUES (9, 'Laptop', 1299.00, 'placed'), (9, 'Mouse', 25.50, 'placed'), (10, 'Monitor', 349.99, 'placed'), (3, 'Keyboard', 89.00, 'placed'), (7, 'Webcam', 79.90, 'placed'); UPDATE customers SET tier = 'gold' WHERE name = 'Liam Carter'; UPDATE customers SET city = 'Abu Dhabi' WHERE name = 'Zara Ali'; UPDATE orders SET status = 'delivered' WHERE id IN (1, 4, 7); Click Sync Now again. The second run read 12 records (563 bytes) from the binlog and took 8 seconds.
The sync log shows what landed. For customers, one new data file holds 4 rows (2 inserts plus the new versions of 2 updated rows), and one small equality delete file retires the 2 old versions by key: addedDataFiles=1, addedRecords=4, addedEqualityDeleteFiles=1, addedEqualityDeletes=2. The orders table got the same shape, with 8 rows and 3 deletes.
Re-run the ClickHouse SELECT with no refresh step. It returned 10 rows, one per key: Liam Carter is gold, Zara Ali is in Abu Dhabi, and the two new customers are there.
Schema changes follow on their own. A column added in MySQL appears in the Iceberg table on the next sync, and widening changes such as int to bigint are applied. A narrowing or incompatible change fails the sync instead of corrupting the table.
Read the same tables from DuckDB and Trino
Nothing gets exported for this. DuckDB attaches the catalog with the principal's credentials:
INSTALL iceberg; LOAD iceberg; INSTALL httpfs; LOAD httpfs; CREATE SECRET polaris_secret ( TYPE iceberg, CLIENT_ID '<client_id>', CLIENT_SECRET '<client_secret>', OAUTH2_SERVER_URI 'http://<polaris-host>:8181/api/catalog/v1/oauth/tokens', OAUTH2_SCOPE 'PRINCIPAL_ROLE:ALL' ); CREATE SECRET store_secret ( TYPE s3, KEY_ID '<access key>', SECRET '<secret key>', ENDPOINT '<minio-host>:9000', URL_STYLE 'path', USE_SSL false, REGION 'us-east-1' ); ATTACH 'olake' AS polaris ( TYPE iceberg, ENDPOINT 'http://<polaris-host>:8181/api/catalog', SECRET polaris_secret, ACCESS_DELEGATION_MODE 'none' ); SELECT id, name, city, tier FROM polaris.mysql_to_clickhouse_shop.customers ORDER BY id; The S3 secret and ACCESS_DELEGATION_MODE 'none' are there because, on a laptop, the MinIO address Polaris hands out may not resolve. With Polaris on AWS S3, drop both and DuckDB uses the vended credentials.
For Trino, one file in etc/catalog/polaris.properties does it:
connector.name=iceberg iceberg.catalog.type=rest iceberg.rest-catalog.uri=http://<polaris-host>:8181/api/catalog iceberg.rest-catalog.warehouse=olake iceberg.rest-catalog.security=OAUTH2 iceberg.rest-catalog.oauth2.credential=<client_id>:<client_secret> iceberg.rest-catalog.oauth2.scope=PRINCIPAL_ROLE:ALL iceberg.rest-catalog.oauth2.server-uri=http://<polaris-host>:8181/api/catalog/v1/oauth/tokens iceberg.rest-catalog.vended-credentials-enabled=true fs.native-s3.enabled=true s3.region=us-east-1 s3.endpoint=http://<minio-host>:9000 s3.path-style-access=true DuckDB 1.5.5 and Trino 484 both returned the same 10 rows as ClickHouse. An orders roll-up by status matched MySQL to the cent in all three: 16 delivered for 8659.68, 18 placed for 10522.26 and 11 shipped for 6397.25.
Tips from the run
- Decimals become doubles.
decimal(p,s)is written as Icebergdoubleand shows up in ClickHouse asNullable(Float64). The sums still matched to the cent, but round at query time for exact decimal arithmetic. - Times are UTC, JSON is text.
datetimeandtimestamparrive asNullable(DateTime64(6, 'UTC')).json,enumandtimebecomeNullable(String); parse JSON withJSONExtract. - Hot dashboards still want MergeTree. A
DataLakeCatalogquery reads Parquet over the network with no primary index. Keep Iceberg as the shared source and load the hot slice into a MergeTree table on a schedule. - Plan compaction. Each sync adds a small data file and delete file per table, and nothing compacts them automatically. OLake Fusion does this for the catalogs OLake Go writes to, and Spark's
rewrite_data_filesworks through the same Polaris endpoint. - Deletion vectors are out on 26.8. That LTS release reads positional and equality deletes only, so keep the upsert type OLake Go picks.
How the MySQL to ClickHouse methods compare
| Method | Freshness | Updates and deletes | Where the data lands | What you run |
|---|---|---|---|---|
| ClickPipes for MySQL | CDC through the binlog | Yes, into ReplacingMergeTree | ClickHouse Cloud tables, ClickHouse only | Nothing; ClickHouse Cloud, billed per ClickPipe |
| MySQL table engine | Live, per query | Reads the current row | Nothing copied; each query hits MySQL | A MySQL login for ClickHouse |
| Debezium, Kafka, Kafka engine | Seconds | Yes, with your own merge logic | Kafka topics, then ClickHouse tables | Kafka, Connect, Debezium, a view per table |
| Managed ELT | Minutes, by plan | Varies by tool | A second copy in ClickHouse | Nothing; per row, per GB or per connector |
| OLake Go to Polaris (this post) | CDC on a schedule down to one minute | Yes, as Iceberg row-level deletes | Iceberg tables ClickHouse, DuckDB, Trino and Spark read | One Docker Compose stack, open source |
If you are already on ClickHouse Cloud, nothing else will read the data and every query needs MergeTree speed, ClickPipes is the shorter path. Writing Iceberg once earns its keep when a second engine needs the same rows, or when you want to own the files and the catalog.
FAQ
Can I write to the Polaris tables from ClickHouse?
No. A DataLakeCatalog database is read-only, and INSERT, ALTER and RENAME DATABASE all return NOT_IMPLEMENTED. OLake Go is the only writer here; copy rows into a MergeTree table when ClickHouse needs to own them.
Why does the ClickHouse query wrap the table name in backticks?
ClickHouse sees the attached catalog as one database with a flat list of tables. The Iceberg namespace mysql_to_clickhouse_shop and the table name are joined into a single identifier, and the backticks keep the dot inside it from being read as a separator.
Can ClickHouse use fixed storage keys instead of vended credentials?
Yes. Replace vended_credentials = true with storage_endpoint, aws_access_key_id and aws_secret_access_key. End storage_endpoint with a slash, or ClickHouse 26.8 builds a doubled object key and every read returns a 404.
How often should the job run?
As often as your dashboards need, down to one minute. Each run only drains the binlog since the last saved position, which in my run meant 8 seconds for 12 changes. Keep binlog retention longer than the longest gap you expect between runs.
Where to go from here
- Point the source at your production MySQL. On RDS the binlog is on once automated backups are enabled; set
binlog_formattoROWin the parameter group. - Swap MinIO for a real S3 bucket and an Identity and Access Management (IAM) role, and leave
endpointandendpointInternalout of the Polaris catalog. - Add more tables by ticking them on the job's Streams page, after an
ANALYZE TABLEon each. - The full guide, with the data type table and the troubleshooting list, is on olake.io.
If a step behaves differently on your setup, leave a comment and I will update the post.








