MySQL to Snowflake: Binlog CDC Without Kafka or Debezium, Step by Step

MySQL already records every insert, update and delete in its binary log. The simplest pipelines into Snowflake ignore that log and re-copy whole tables on a timer. That is why deleted rows linger in the warehouse and every dashboard runs one load behind.

The usual ways to move MySQL data into Snowflake:

• Manual export: mysqldump or SELECT ... INTO OUTFILE, then COPY INTO. You run a script and a scheduler, but it does full reloads only, so updates and deletes never arrive as changes.
• Debezium, Kafka and the Snowflake Kafka connector: changes land within seconds. To get there you operate Kafka, Kafka Connect, Debezium and a schema registry.
• Managed extract, load, transform (ELT) tools such as Fivetran, Estuary or Airbyte Cloud: nothing to host, and freshness in minutes depending on the plan. Pricing is per row, per GB or per connector, and the result is a second copy inside Snowflake.
• The Snowflake Openflow connector for MySQL: Snowflake's own binlog reader, writing native tables and billed in Snowflake credits. A sound choice when Snowflake is the only reader.

This tutorial takes a fifth route. OLake Go, an open source (Apache 2.0) replication tool, reads the binlog with change data capture (CDC) and commits the rows as Apache Iceberg tables into Horizon, the catalog built into every Snowflake account. Snowflake treats them as its own managed tables, and DuckDB or Spark can read the same files.

I ran this against MySQL 8.4 and a Snowflake account on AWS (ap-south-1) with OLake Go v1.1.0. The first sync copied 48 records in 34 seconds, and a second sync replayed 12 changes in 23 seconds. Every number below is from that run.

What you'll build

• Source: a MySQL database named shop with two tables, customers and orders, and row-based binary logging on
• Pipeline: OLake UI from its Docker Compose install, running one job that snapshots the tables and then replays the binlog
• Destination: Snowflake-managed tables in a database named OLAKE_DEMO, queried in Snowsight

You need: MySQL 5.7 or later, Docker, a Snowflake account where you can run a one-time setup as ACCOUNTADMIN, and about 30 minutes.

Step 1: Turn on the binlog and create a user

Four things must be true on the MySQL server. The binlog is on with a server_id, the format is ROW, row metadata is FULL, and one user can read both the tables and the log.

Put the settings in my.cnf and restart:

[mysqld]
server-id = 1
log_bin = mysql-bin
binlog_format = ROW
binlog_row_metadata = FULL
binlog_expire_logs_seconds = 604800

The last line keeps 7 days of binlog so a paused job can still catch up. On Amazon Relational Database Service (RDS), Aurora, Cloud SQL or Azure, the same settings go in a parameter group or server parameter instead of my.cnf. Aurora MySQL does not have the binlog enabled by default, so turn it on in the cluster parameter group.

Next, the two tables I used:

CREATE TABLE customers (
  id INT AUTO_INCREMENT PRIMARY KEY,
  name VARCHAR(100), email VARCHAR(200), city VARCHAR(100), tier VARCHAR(20),
  created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
  updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP);

CREATE TABLE orders (
  id INT AUTO_INCREMENT PRIMARY KEY,
  customer_id INT, product VARCHAR(100), amount DECIMAL(10,2), status VARCHAR(20),
  created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
  updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
  FOREIGN KEY (customer_id) REFERENCES customers(id));

Insert whatever rows you like, or point the pipeline at tables you already have. My run started with 8 customers and 40 orders.

Then create the user and refresh the table statistics:

CREATE USER IF NOT EXISTS 'olake'@'%' IDENTIFIED BY '<password>';
GRANT SELECT, REPLICATION SLAVE, REPLICATION CLIENT ON *.* TO 'olake'@'%';
FLUSH PRIVILEGES;

ANALYZE TABLE customers, orders;

⚠️ Keep the ANALYZE TABLE line. The snapshot is split into parallel chunks using table statistics, and a freshly loaded table has none. My first attempt stopped with stats not populated for table[shop.orders] until I ran it.

Confirm everything before moving on:

SHOW VARIABLES WHERE Variable_name IN ('log_bin', 'binlog_format', 'binlog_row_metadata', 'server_id');
SHOW GRANTS FOR 'olake'@'%';

Step 2: Prepare Snowflake

Run this once in a Snowsight worksheet as ACCOUNTADMIN. It creates a warehouse for your own queries, the database the tables land in, a role and a service user for the writer, a network policy, and the token.

USE ROLE ACCOUNTADMIN;

CREATE WAREHOUSE IF NOT EXISTS OLAKE_WH WAREHOUSE_SIZE = XSMALL AUTO_SUSPEND = 60 AUTO_RESUME = TRUE;
CREATE DATABASE IF NOT EXISTS OLAKE_DEMO;
ALTER DATABASE OLAKE_DEMO SET CATALOG = 'SNOWFLAKE';
ALTER DATABASE OLAKE_DEMO SET EXTERNAL_VOLUME = 'SNOWFLAKE_MANAGED';

CREATE ROLE IF NOT EXISTS OLAKE_ROLE;
GRANT USAGE, CREATE SCHEMA ON DATABASE OLAKE_DEMO TO ROLE OLAKE_ROLE;
GRANT USAGE ON WAREHOUSE OLAKE_WH TO ROLE OLAKE_ROLE;
GRANT ROLE OLAKE_ROLE TO ROLE SYSADMIN;

CREATE USER IF NOT EXISTS OLAKE_SVC TYPE = SERVICE DEFAULT_ROLE = OLAKE_ROLE;
GRANT ROLE OLAKE_ROLE TO USER OLAKE_SVC;

USE SCHEMA OLAKE_DEMO.PUBLIC;
CREATE NETWORK RULE IF NOT EXISTS olake_ip
  MODE = INGRESS TYPE = IPV4 VALUE_LIST = ('<OLake Go egress IP>/32');
CREATE NETWORK POLICY IF NOT EXISTS olake_network_policy
  ALLOWED_NETWORK_RULE_LIST = ('OLAKE_DEMO.PUBLIC.olake_ip');
ALTER USER OLAKE_SVC SET NETWORK_POLICY = olake_network_policy;

SELECT CURRENT_ORGANIZATION_NAME() || '-' || CURRENT_ACCOUNT_NAME() AS account_identifier,
       CURRENT_REGION() AS region;

ALTER USER OLAKE_SVC ADD PROGRAMMATIC ACCESS TOKEN OLAKE_PAT
  ROLE_RESTRICTION = 'OLAKE_ROLE' DAYS_TO_EXPIRY = 30;

Save three values: account_identifier, region and the token secret. Snowflake shows the secret once.

On a laptop the egress IP is your public IP, which curl -s https://checkip.amazonaws.com prints.

⚠️ The network policy is the step people miss. Without one on the user, the last statement fails with Network Policy is required when creating a programmatic access token for user OLAKE_SVC.

The CATALOG and EXTERNAL_VOLUME settings on the database are why no bucket, IAM role or external volume is needed on your side. Every table created in OLAKE_DEMO through Horizon lands on Snowflake's own storage as a managed table. ROLE_RESTRICTION pins the token to OLAKE_ROLE, so a leaked token cannot act as ACCOUNTADMIN.

You can test the token before touching the UI. This is the same exchange the writer makes on every sync:

curl -s -X POST "https://<account_identifier>.snowflakecomputing.com/polaris/api/catalog/v1/oauth/tokens" \
  -H 'Content-Type: application/x-www-form-urlencoded' \
  --data-urlencode grant_type=client_credentials \
  --data-urlencode scope=session:role:OLAKE_ROLE \
  --data-urlencode client_secret=<your PAT>

A working token returns a JSON object with an access_token.

Step 3: Start OLake UI and add the MySQL source

curl -sSL https://raw.githubusercontent.com/datazip-inc/olake-ui/master/docker-compose-v1.yml | docker compose -f - up -d

Open http://localhost:8000 and sign in with admin / password.

Open Sources, click Create Source and pick MySQL. Under Change data capture (CDC) leave Enable CDC on. My run had Initial Wait Time at 10.

Under Connection details enter the host, port, database shop, user olake and its password. My MySQL ran in another container, so the host was host.docker.internal, and SSL was disabled for the local test.

Click Set up source. It tests the connection and then checks binlog_access, log_bin, binlog_format and binlog_row_metadata before anything is saved. If one fails, the dialog shows the current value next to the recommended one.

Step 4: Add the Snowflake destination

Open Destinations and click Create Destination. Pick Apache Iceberg as the connector, Horizon as the catalog type and Programmatic Access Token (PAT) as the authentication type.

Field	Value
Horizon Iceberg REST Catalog URI	https://<account_identifier>.snowflakecomputing.com/polaris/api/catalog
Snowflake Database	OLAKE_DEMO
Programmatic Access Token	the secret from step 2
Session Role Scope	session:role:OLAKE_ROLE
Cloud Storage Region	the region value from step 2, AWS_AP_SOUTH_1 in my run

Click Create. The form calls the Horizon endpoint and exchanges the token, so a bad token or a blocked IP shows up here.

Step 5: Create the job and run the first sync

Open Jobs and click Create Job. I named mine mysql_to_snowflake. Give it a schedule, then select the source and destination.

Under Advanced Settings, set Target Query Engines to Snowflake, plus DuckDB and Apache Spark if they will read the tables too. This is the only format decision in the whole setup. OLake Go works out which Iceberg delete format every listed engine can read and applies it to each stream.

Click Next. On the Streams page tick customers and orders, set the sync mode to Full Refresh + CDC and the ingestion mode to Upsert. Leave Upsert Type as it was resolved, which was Positional Deletes in my run.

Click Create Job, open the job's menu and choose Sync Now.

The first sync copied 8 customers and 40 orders (48 records) in 34 seconds, container start included. It records the binlog position first and then copies each table in parallel chunks, so nothing written during the copy is lost.

Step 6: Query the tables in Snowsight

There is no CREATE ICEBERG TABLE and no refresh. The schema is named after the job and the source database, so mine is mysql_to_snowflake_shop.

SHOW ICEBERG TABLES IN SCHEMA OLAKE_DEMO."mysql_to_snowflake_shop";

SELECT "id", "name", "city", "tier"
FROM OLAKE_DEMO."mysql_to_snowflake_shop"."customers"
ORDER BY "id";

⚠️ Keep the double quotes. The schema and tables keep their lowercase MySQL names, and Snowflake upper-cases anything unquoted. Without quotes you get Object 'OLAKE_DEMO.MYSQL_TO_SNOWFLAKE_SHOP.CUSTOMERS' does not exist or not authorized.

Each table also gets four extra columns: _olake_id (a hash of the primary key), _op_type, _olake_timestamp and _cdc_timestamp.

Step 7: Change rows in MySQL and sync again

Now the part a full reload cannot do. Run 7 inserts and 5 updates in MySQL:

INSERT INTO customers (name, email, city, tier) VALUES
  ('Maya Patel', '[e-mail]', 'Pune', 'silver'),
  ('Lucas Silva', '[e-mail]', 'São Paulo', 'bronze');
INSERT INTO orders (customer_id, product, amount, status) VALUES
  (9, 'Laptop', 1299.00, 'placed'), (9, 'Mouse', 25.50, 'placed'),
  (10, 'Monitor', 349.99, 'placed'), (3, 'Keyboard', 89.00, 'placed'), (7, 'Webcam', 79.90, 'placed');
UPDATE customers SET tier = 'gold'      WHERE name = 'Liam Carter';
UPDATE customers SET city = 'Abu Dhabi' WHERE name = 'Zara Ali';
UPDATE orders    SET status = 'delivered' WHERE id IN (1, 4, 7);

Choose Sync Now again. The second sync read 12 records (563 bytes) from the binlog and finished in 23 seconds.

For customers it wrote one data file with 4 rows: the 2 inserts and the new versions of the 2 updated rows. One small delete file retires the 2 old versions, and orders has the same shape with 8 rows and 3 deletes.

Re-run the query with "_op_type" added to the column list. Liam Carter is gold, Zara Ali moved to Abu Dhabi, and the two new customers are there.

_op_type is r for a row from the first snapshot, c for an insert, u for an update and d for a delete.

Read the same tables from DuckDB

Because the tables are Iceberg, a second engine reads the same files through the same Horizon endpoint. DuckDB wants a bearer token instead of the raw PAT, so run the curl exchange from step 2 and paste the access_token:

INSTALL iceberg; LOAD iceberg;
INSTALL httpfs;  LOAD httpfs;

CREATE SECRET horizon_secret (TYPE iceberg, TOKEN '<access_token from /v1/oauth/tokens>');

ATTACH 'OLAKE_DEMO' AS horizon (
  TYPE iceberg,
  ENDPOINT 'https://<account_identifier>.snowflakecomputing.com/polaris/api/catalog',
  SECRET horizon_secret,
  ACCESS_DELEGATION_MODE 'vended_credentials'
);

SELECT id, name, city, tier FROM horizon.mysql_to_snowflake_shop.customers ORDER BY id;

DuckDB 1.5.5 returned the same 10 customers, and so did Spark 3.5.6 with Iceberg 1.10.0. An orders roll-up by status matched MySQL to the cent in all three engines: 16 delivered for 8659.68, 18 placed for 10522.26 and 11 shipped for 6397.25.

The session token lives about an hour. The network policy from step 2 covers every use of the token, so DuckDB must also connect from an allowed IP.

Data types worth knowing

MySQL	Snowflake shows	Note
int, mediumint, smallint, tinyint	NUMBER(10,0)
bigint	NUMBER(19,0)
decimal(p,s), float	FLOAT	decimals are stored as floats, so round at query time for exact arithmetic
datetime, timestamp	TIMESTAMP_LTZ(6)	UTC; datetime has no zone in MySQL and is read as UTC
json, enum, time, bit(1), text types	VARCHAR	parse json with PARSE_JSON()

The Snowflake side follows Snowflake's Iceberg data types page.

Tips and limits

• Always set the target query engines. The command-line default for updates is equality deletes, and Snowflake reads positional deletes and deletion vectors on Iceberg tables, not equality delete files.
• Binlog retention is your replay window. A job paused longer than binlog_expire_logs_seconds cannot resume and needs a fresh snapshot.
• Put token renewal on a calendar. The token above expires after 30 days, and 365 is the maximum. An expired token fails the next sync at the catalog handshake.
• Tables need a primary key. Without one, inserts replicate but updates and deletes are not applied.
• Freshness is a schedule, down to one minute. This is not sub-second streaming.
• DuckDB on Azure does not work here. DuckDB only attaches REST catalogs backed by S3, S3 Tables or Google Cloud Storage, so use Spark or Trino as the second engine on an Azure account.
• Schema changes follow along. A new MySQL column appears on the next sync, widening changes such as int to bigint are applied, and narrowing changes fail the sync instead of corrupting the table.

How the methods compare

Method	Freshness	Updates and deletes	Where the data lands	What you run
mysqldump or INTO OUTFILE, then COPY INTO	hours to daily	no, full reloads only	files you control	a script and a scheduler
Debezium, Kafka, Snowflake Kafka connector	seconds	yes, with tombstone handling	Kafka topics, then Snowflake tables	Kafka, Kafka Connect, Debezium, a schema registry
Fivetran, Estuary, Airbyte Cloud	minutes, by plan	yes	Snowflake tables, as a second copy	nothing; per row, per GB or per connector pricing
Snowflake Openflow connector for MySQL	CDC through the binlog	yes, as soft deletes plus journal tables	native Snowflake tables	an Openflow deployment, billed in Snowflake credits
OLake Go into Horizon (this post)	CDC on a schedule down to one minute	yes, as Iceberg row-level deletes	Iceberg tables that Snowflake, DuckDB, Spark and Trino read	one Docker Compose stack, open source

If Snowflake will be the only reader and you want Snowflake to run the pipeline, Openflow is the simpler choice. The route in this post earns its keep when a second engine needs the same rows or you want to own the files.

Where to go from here

• Point the source at a real database. RDS, Aurora, Cloud SQL and Azure differ only in where the binlog settings live.
• Want the files in your own S3 bucket? Use AWS Glue as the catalog, and Snowflake reads the tables through a read-only catalog integration that you refresh after each sync.
• A CDC table that receives a small file every minute will want data-file compaction from the writer side. Snowflake expires old snapshots and compacts manifests on managed tables itself.
• The Glue setup, the Spark connection and the full type table are in the MySQL to Snowflake guide.

If a step behaves differently on your setup, leave a comment with the error text and I'll update the post.