DEV Community: Fritz Larco

Sync PostgreSQL to MotherDuck with Sling

Fritz Larco — Mon, 01 Jun 2026 17:02:53 +0000

Introduction

MotherDuck is a serverless analytics service built on DuckDB. It hosts DuckDB databases in the cloud and keeps the same SQL surface you'd use locally. PostgreSQL is what most apps run on for transactional data.

So you usually want both: Postgres for the app, MotherDuck for analytics. The part in the middle that copies tables across is what Sling does.

This guide replicates a PostgreSQL schema into MotherDuck with Sling, in both full-refresh and incremental modes. The CLI output and row counts below come from an actual run, not a fabricated one.

Installing Sling

Sling is a single binary. Pick whichever install method fits your environment:

# macOS / Linux
curl -fsSL https://slingdata.io/install.sh | bash

# Windows
irm https://slingdata.io/install.ps1 | iex

# Python
pip install sling

Confirm the install:

sling --version

Full installation notes are in the Sling CLI Getting Started Guide.

Configuring the PostgreSQL Source

Sling reads connection details from ~/.sling/env.yaml, environment variables, or sling conns set. For PostgreSQL you'll need host, port, database, user, and password.

Using sling conns set:

sling conns set PG_SOURCE type=postgres host=host.ip user=myuser \
  database=mydb password=mypass port=5432

Or in ~/.sling/env.yaml:

connections:
  PG_SOURCE:
    type: postgres
    host: host.ip
    user: myuser
    password: mypass
    port: 5432
    database: mydb
    sslmode: require
    schema: public

Test it:

sling conns test PG_SOURCE

The PostgreSQL connection docs cover SSL, IAM auth, and other options.

Configuring the MotherDuck Target

A MotherDuck connection needs the database name and a service token. You can generate a token from the MotherDuck UI.

sling conns set MOTHERDUCK type=motherduck \
  database=my_db motherduck_token=eyJhbGciOi...

Or the URL form:

sling conns set MOTHERDUCK url="motherduck://my_db?motherduck_token=eyJhbGciOi..."

Or in ~/.sling/env.yaml:

connections:
  MOTHERDUCK:
    type: motherduck
    database: my_db
    motherduck_token: eyJhbGciOi...

Test it:

sling conns test MOTHERDUCK

Full options (attach modes, copy method, DuckDB version pinning) are in the MotherDuck connection docs.

A Full-Refresh Replication

For this run the PostgreSQL source has three tables in a demo_pg_motherduck schema:

customers — 5,000 rows
orders — 30,000 rows, with an updated_at timestamp
events — 60,000 rows, with an occurred_at timestamp

The replication file lives next to wherever you want to run Sling from:

# replication.yaml
source: PG_SOURCE
target: MOTHERDUCK

defaults:
  mode: full-refresh
  object: demo_pg_motherduck.{stream_table}

streams:
  demo_pg_motherduck.customers:
    primary_key: [customer_id]

  demo_pg_motherduck.orders:
    primary_key: [order_id]
    update_key: updated_at

  demo_pg_motherduck.events:
    primary_key: [event_id]
    update_key: occurred_at

A few things to point out:

object: demo_pg_motherduck.{stream_table} is a runtime variable. Sling substitutes the source table name into the target object, so you don't repeat yourself per stream.
primary_key and update_key are set even though the mode here is full-refresh. The next section flips to incremental without touching those declarations; only the mode changes.
The target schema gets created automatically by Sling on the first run. No manual CREATE SCHEMA needed.

Run it:

sling run -r replication.yaml

Real output, trimmed for readability:

INF Sling Replication [3 streams] | PG_SOURCE -> MOTHERDUCK

INF [1 / 3] running stream demo_pg_motherduck.customers
INF reading from source database
INF writing to target database [mode: full-refresh]
INF created table "demo_pg_motherduck"."customers"
INF inserted 5000 rows into "demo_pg_motherduck"."customers" in 11 secs [425 r/s] [390 kB]
INF execution succeeded

INF [2 / 3] running stream demo_pg_motherduck.orders
INF created table "demo_pg_motherduck"."orders"
INF inserted 30000 rows into "demo_pg_motherduck"."orders" in 14 secs [2,131 r/s] [2.6 MB]
INF execution succeeded

INF [3 / 3] running stream demo_pg_motherduck.events
INF created table "demo_pg_motherduck"."events"
INF inserted 60000 rows into "demo_pg_motherduck"."events" in 9 secs [6,036 r/s] [3.3 MB]
INF execution succeeded

INF Sling Replication Completed in 40s | PG_SOURCE -> MOTHERDUCK | 3 Successes | 0 Failures

95,000 rows across three tables, end to end, in 40 seconds. The _tmp tables that show up in the full log are Sling's staging step before it swaps the data into the final target. They get cleaned up automatically.

Verification

A count(*) from MotherDuck right after the run:

select 'customers' as t, count(*) c from demo_pg_motherduck.customers
union all select 'orders',    count(*)   from demo_pg_motherduck.orders
union all select 'events',    count(*)   from demo_pg_motherduck.events;

+-----------+-------+
| T         |     C |
+-----------+-------+
| customers |  5000 |
| orders    | 30000 |
| events    | 60000 |
+-----------+-------+

A small sample to confirm the data made the trip with types intact:

select event_id, customer_id, event_type, region, occurred_at
from demo_pg_motherduck.events
order by event_id limit 5;

+----------+-------------+------------+--------+-------------------------------+
| EVENT_ID | CUSTOMER_ID | EVENT_TYPE | REGION | OCCURRED_AT                   |
+----------+-------------+------------+--------+-------------------------------+
|        1 |           2 | click      | us-2   | 2025-01-01 00:00:01 +0000 UTC |
|        2 |           3 | signup     | us-3   | 2025-01-01 00:00:02 +0000 UTC |
|        3 |           4 | purchase   | us-4   | 2025-01-01 00:00:03 +0000 UTC |
|        4 |           5 | page_view  | us-5   | 2025-01-01 00:00:04 +0000 UTC |
|        5 |           6 | click      | us-6   | 2025-01-01 00:00:05 +0000 UTC |
+----------+-------------+------------+--------+-------------------------------+

Numeric, varchar, and timestamp columns round-tripped cleanly. Nullable columns (region is null on every seventh row in the source) are preserved as nulls, not as the string "NULL".

Switching to Incremental

Full-refreshing a 60,000-row table every day is fine. Full-refreshing a 600-million-row event table every day is not. Sling's incremental mode reads only the rows newer than the highest update_key already in the target.

Drop customers from the streams (it changes slowly enough to keep on full-refresh in a separate run, or rebuild weekly) and switch the mode:

# replication-incremental.yaml
source: PG_SOURCE
target: MOTHERDUCK

defaults:
  mode: incremental
  object: demo_pg_motherduck.{stream_table}

streams:
  demo_pg_motherduck.orders:
    primary_key: [order_id]
    update_key: updated_at

  demo_pg_motherduck.events:
    primary_key: [event_id]
    update_key: occurred_at

Insert 1,000 new orders and 2,500 new events on the source (this simulates a day's worth of data flowing in), then run again:

sling run -r replication-incremental.yaml

INF Sling Replication [2 streams] | PG_SOURCE -> MOTHERDUCK

INF [1 / 2] running stream demo_pg_motherduck.orders
INF getting checkpoint value (updated_at)
INF writing to target database [mode: incremental]
INF inserted 1000 rows into "demo_pg_motherduck"."orders" in 9 secs [104 r/s] [86 kB]
INF execution succeeded

INF [2 / 2] running stream demo_pg_motherduck.events
INF getting checkpoint value (occurred_at)
INF writing to target database [mode: incremental]
INF inserted 2500 rows into "demo_pg_motherduck"."events" in 6 secs [358 r/s] [137 kB]
INF execution succeeded

INF Sling Replication Completed in 20s | PG_SOURCE -> MOTHERDUCK | 2 Successes | 0 Failures

The getting checkpoint value line is where Sling looks at the target, finds the largest updated_at already present, and uses that as the lower bound on the source query. Only the new rows come across:

select 'orders' as t, count(*) c from demo_pg_motherduck.orders
union all select 'events',  count(*)   from demo_pg_motherduck.events;

+--------+-------+
| T      |     C |
+--------+-------+
| orders | 31000 |
| events | 62500 |
+--------+-------+

Orders went from 30,000 to 31,000. Events went from 60,000 to 62,500. Matches what was inserted on the source.

If you need updates as well as inserts (a row's updated_at changes and the existing row should be replaced rather than duplicated), keep mode: incremental and make sure primary_key is set. Sling will upsert against the primary key instead of appending. The replication modes docs cover the trade-offs.

Common Tweaks

A few options you'll reach for once the basics are in place:

Schema and column casing. MotherDuck (DuckDB) is case-sensitive, and Sling defaults to keeping the source casing. Add target_options: { column_casing: snake } under defaults if your Postgres source has mixed-case identifiers and you want a clean snake_case target.
Add new columns automatically. When the source schema changes, set target_options: { add_new_columns: true } so Sling alters the MotherDuck table on the next run. Without it, new source columns get dropped at the boundary.
Pick a copy method. The default for MotherDuck is csv_http. For very wide rows or large text values, switch to arrow_http via copy_method: arrow_http in the connection config. It's usually faster and avoids CSV escaping edge cases.
Filter at the source. Use a custom sql: block in a stream to project columns or filter rows before they leave Postgres. Cheaper than dragging unused columns to MotherDuck.

Where to Go Next

The same replication pattern works for any of Sling's 30+ database sources into MotherDuck: MySQL, SQL Server, Snowflake, BigQuery, and the rest. Swap the source connection and leave the target alone.

If you'd rather store flat files than warehouse tables, see PostgreSQL to S3 as Parquet, which uses the same replication file shape with a file-system target. For a local DuckDB setup instead of a managed MotherDuck one, see PostgreSQL to DuckDB. For team workflows with scheduling and alerting on top of the same CLI, look at the Sling Platform.

Questions go to Discord or GitHub Issues.

Replicate MySQL to ClickHouse with Sling

Fritz Larco — Tue, 26 May 2026 13:29:56 +0000

Introduction

ClickHouse is a columnar OLAP database. It runs aggregate queries across billions of rows in seconds. MySQL is what most apps run on for transactional reads and writes. Different jobs, different storage shapes, which is why people end up running them side by side: MySQL for the app, ClickHouse for analytics on top of the app's data.

The piece in the middle, the bit that copies tables from MySQL into ClickHouse and keeps them current, is what Sling does.

This guide replicates a MySQL schema into ClickHouse with Sling, in both full-refresh and incremental modes. The CLI output, row counts, and timings below all come from an actual run against a Docker MySQL on the source side and a self-hosted ClickHouse 25.4 on the target side. The same configuration works against ClickHouse Cloud; only the connection URL changes.

Installing Sling

Sling is a single binary. Pick whichever install method fits your environment:

# macOS / Linux
curl -fsSL https://slingdata.io/install.sh | bash

# Windows
irm https://slingdata.io/install.ps1 | iex

# Python
pip install sling

Confirm the install:

sling --version

Installation notes for every platform are in the Sling CLI Getting Started Guide.

Configuring the MySQL Source

Sling reads connection details from ~/.sling/env.yaml, environment variables, or sling conns set. For MySQL you need host, port, database, user, and password.

A read-only Sling user is the right shape for replication:

CREATE USER 'sling'@'%' IDENTIFIED BY '<password>';
GRANT SELECT ON <source_schema>.* TO 'sling'@'%';

Using sling conns set:

sling conns set MYSQL_SOURCE type=mysql host=host.ip user=sling \
  database=mydb password=mypass port=3306

Or in ~/.sling/env.yaml:

connections:
  MYSQL_SOURCE:
    type: mysql
    host: host.ip
    user: sling
    password: mypass
    port: 3306
    database: mydb

If your MySQL requires TLS, append ?tls=skip-verify to the URL form, or set tls: skip-verify in the YAML. Test it:

sling conns test MYSQL_SOURCE

The MySQL connection docs cover SSL, IAM auth, and the rest of the options.

Configuring the ClickHouse Target

ClickHouse speaks two protocols: native (port 9000) and HTTP (port 8123 / 8443 with TLS). Sling supports both. For self-hosted clusters the native protocol is usually the fastest path; for ClickHouse Cloud, the HTTPS endpoint is the supported one.

Self-hosted, native protocol:

sling conns set CLICKHOUSE type=clickhouse host=host.ip user=default \
  password=mypass port=9000 database=default

ClickHouse Cloud over HTTPS:

sling conns set CLICKHOUSE \
  url="https://default:mypass@xxxxxx.us-east-1.aws.clickhouse.cloud:8443/default"

Or in ~/.sling/env.yaml:

connections:
  CLICKHOUSE:
    type: clickhouse
    host: host.ip
    user: default
    password: mypass
    port: 9000
    database: default

Test it:

sling conns test CLICKHOUSE

The ClickHouse connection docs list every option, including the HTTP URL form and the export_stream_format setting for tuning the staging file format.

A Full-Refresh Replication

For this run the MySQL source has three tables in a demo_mysql_clickhouse database:

customers — 5,000 rows
orders — 30,000 rows, with an updated_at timestamp
events — 60,000 rows, with an occurred_at timestamp

The replication file lives next to wherever you run Sling from:

# replication.yaml
source: MYSQL_SOURCE
target: CLICKHOUSE

defaults:
  mode: full-refresh
  object: demo_mysql_clickhouse.{stream_table}

streams:
  demo_mysql_clickhouse.customers:
    primary_key: [customer_id]

  demo_mysql_clickhouse.orders:
    primary_key: [order_id]
    update_key: updated_at

  demo_mysql_clickhouse.events:
    primary_key: [event_id]
    update_key: occurred_at

A few things worth pointing out:

object: demo_mysql_clickhouse.{stream_table} is a runtime variable. Sling substitutes the source table name into the target object, so you don't repeat yourself per stream.
primary_key and update_key are set even though the mode is full-refresh. The next section flips to incremental without touching those declarations; only the mode changes.
The target database (demo_mysql_clickhouse on ClickHouse) gets created automatically by Sling on the first run. No manual CREATE DATABASE needed on the target side.

Run it:

sling run -r replication.yaml

Real output, trimmed for readability:

INF Sling Replication [3 streams] | MYSQL_SOURCE -> CLICKHOUSE

INF [1 / 3] running stream demo_mysql_clickhouse.customers
INF reading from source database
INF writing to target database [mode: full-refresh]
INF created table `demo_mysql_clickhouse`.`customers`
INF inserted 5000 rows into `demo_mysql_clickhouse`.`customers` in 0 secs [8,853 r/s] [396 kB]
INF execution succeeded

INF [2 / 3] running stream demo_mysql_clickhouse.orders
INF created table `demo_mysql_clickhouse`.`orders`
INF inserted 30000 rows into `demo_mysql_clickhouse`.`orders` in 1 secs [29,381 r/s] [2.8 MB]
INF execution succeeded

INF [3 / 3] running stream demo_mysql_clickhouse.events
INF created table `demo_mysql_clickhouse`.`events`
INF inserted 60000 rows into `demo_mysql_clickhouse`.`events` in 0 secs [81,559 r/s] [3.2 MB]
INF execution succeeded

INF Sling Replication Completed in 4s | MYSQL_SOURCE -> CLICKHOUSE | 3 Successes | 0 Failures

95,000 rows across three tables, end to end, in 4 seconds. The _tmp tables that show up in the full log are Sling's staging step before it swaps the data into the final target. They get cleaned up automatically.

When Sling creates the table, it asks for MergeTree with the primary key columns as the sorting key. That's a fine baseline for analytical queries. The "Common Tweaks" section below covers how to override it when you need partitioning, replication, or a different engine.

Verification

A count() from ClickHouse right after the run:

SELECT 'customers' AS t, count() AS c FROM demo_mysql_clickhouse.customers
UNION ALL SELECT 'orders',    count()   FROM demo_mysql_clickhouse.orders
UNION ALL SELECT 'events',    count()   FROM demo_mysql_clickhouse.events;

+-----------+-------+
| T         | C     |
+-----------+-------+
| customers |  5000 |
| orders    | 30000 |
| events    | 60000 |
+-----------+-------+

A small sample to confirm the data made the trip with types intact:

SELECT event_id, customer_id, event_type, region, occurred_at
FROM demo_mysql_clickhouse.events
ORDER BY event_id LIMIT 5;

+----------+-------------+------------+--------+-------------------------------+
| EVENT_ID | CUSTOMER_ID | EVENT_TYPE | REGION | OCCURRED_AT                   |
+----------+-------------+------------+--------+-------------------------------+
|        1 |           2 | signup     | us-2   | 2025-01-01 00:00:01 +0000 UTC |
|        2 |           3 | purchase   | us-3   | 2025-01-01 00:00:02 +0000 UTC |
|        3 |           4 | logout     | us-4   | 2025-01-01 00:00:03 +0000 UTC |
|        4 |           5 | page_view  | us-1   | 2025-01-01 00:00:04 +0000 UTC |
|        5 |           6 | click      | us-2   | 2025-01-01 00:00:05 +0000 UTC |
+----------+-------------+------------+--------+-------------------------------+

Numeric, varchar, and timestamp columns round-tripped cleanly. The nullable region column (every seventh row in the source is null) lands as ClickHouse Nullable(String) and preserves nulls as nulls, not as the literal string "NULL".

Switching to Incremental

Full-refreshing a 60,000-row event table every day is fine. Full-refreshing a 600-million-row event table every day is not. Sling's incremental mode reads only the rows newer than the highest update_key already in the target.

Drop customers from the streams (it changes slowly enough to keep on full-refresh in a separate run, or rebuild weekly) and switch the mode:

# replication-incremental.yaml
source: MYSQL_SOURCE
target: CLICKHOUSE

defaults:
  mode: incremental
  object: demo_mysql_clickhouse.{stream_table}

streams:
  demo_mysql_clickhouse.orders:
    primary_key: [order_id]
    update_key: updated_at

  demo_mysql_clickhouse.events:
    primary_key: [event_id]
    update_key: occurred_at

Insert 1,000 new orders and 2,500 new events on the source (a stand-in for a day of fresh data), then run again:

sling run -r replication-incremental.yaml

INF Sling Replication [2 streams] | MYSQL_SOURCE -> CLICKHOUSE

INF [1 / 2] running stream demo_mysql_clickhouse.orders
INF getting checkpoint value (updated_at)
INF writing to target database [mode: incremental]
INF inserted 1000 rows into `demo_mysql_clickhouse`.`orders` in 0 secs [1,926 r/s] [93 kB]
INF execution succeeded

INF [2 / 2] running stream demo_mysql_clickhouse.events
INF getting checkpoint value (occurred_at)
INF writing to target database [mode: incremental]
INF inserted 2500 rows into `demo_mysql_clickhouse`.`events` in 0 secs [4,040 r/s] [134 kB]
INF execution succeeded

INF Sling Replication Completed in 2s | MYSQL_SOURCE -> CLICKHOUSE | 2 Successes | 0 Failures

SELECT 'orders' AS t, count() c FROM demo_mysql_clickhouse.orders
UNION ALL SELECT 'events', count()   FROM demo_mysql_clickhouse.events;

+--------+-------+
| T      | C     |
+--------+-------+
| orders | 31000 |
| events | 62500 |
+--------+-------+

Orders went from 30,000 to 31,000. Events went from 60,000 to 62,500. Matches what was inserted on the source.

ClickHouse's MergeTree family is append-friendly. In incremental mode Sling inserts the new rows directly into the main table without rewriting partitions. If you also need updates (a row's updated_at changes and you want the existing target row replaced rather than duplicated), keep mode: incremental and make sure primary_key is set. Sling will use a ReplacingMergeTree-style upsert path against that key. The replication modes docs cover the trade-offs.

Common Tweaks

A few options worth reaching for once the basics are in place:

Pick a table engine. ClickHouse's default MergeTree is a fine baseline, but for high-write or replicated clusters you'll want ReplicatedMergeTree, partitioning by month, and a TTL. Set target_options.table_ddl per stream with the full CREATE TABLE you want; Sling will use it instead of generating its own. Example: engine = MergeTree() ORDER BY (customer_id, occurred_at) PARTITION BY toYYYYMM(occurred_at).
Add new columns automatically. When the source schema changes, set target_options: { add_new_columns: true } so Sling alters the ClickHouse table on the next run. Without it, new source columns get dropped at the boundary.
Tune the staging format. Sling stages data as a file before bulk-loading into ClickHouse. The default is CSVWithNames, which is robust but verbose. For wide rows or large text values, set export_stream_format: Parquet on the ClickHouse connection. Usually faster and more compact on the wire.
Filter at the source. Use a custom sql: block in a stream to project columns or filter rows before they leave MySQL. Cheaper than dragging unused columns to ClickHouse, and it keeps row payloads small for the network hop.

Where to Go Next

The same replication pattern works for any of Sling's 30+ database sources into ClickHouse: PostgreSQL, SQL Server, Snowflake, BigQuery, and the rest. Swap the source connection and leave the target alone. For the equivalent flow from a Postgres source, see PostgreSQL to ClickHouse.

If your downstream is more cloud-warehouse than columnar engine, MySQL to MotherDuck covers the same setup with DuckDB-on-the-cloud as the target. For team workflows with scheduling and alerting on top of the same CLI, look at the Sling Platform.

Questions go to Discord or GitHub Issues.

Load PostgreSQL into Apache Iceberg with Sling

Fritz Larco — Mon, 18 May 2026 13:31:11 +0000

Introduction

Apache Iceberg is the table format that turns a pile of Parquet files in object storage into something that behaves like a warehouse table. You get schema evolution, hidden partitioning, time travel, and consistent reads from whichever engine you point at the table. PostgreSQL is where most operational data starts. Moving it into Iceberg gives you an analytics copy that DuckDB, Spark, Trino, Snowflake, and Athena can all read without anyone needing to agree on a single warehouse vendor first.

Sling speaks the Iceberg REST catalog directly. From the configuration side an Iceberg target is just another database connection: point Sling at the catalog URL and the underlying object store, then declare your streams. No JVM, no Spark, no manual manifest writing.

This guide replicates a Postgres schema into Iceberg using Sling. The catalog is Cloudflare R2's managed Iceberg REST catalog and the storage layer underneath is R2. Every CLI line, row count, and timing below comes from an actual run against those endpoints.

Installing Sling

Sling is a single binary. Pick whichever install fits:

# macOS / Linux
curl -fsSL https://slingdata.io/install.sh | bash

# Windows
irm https://slingdata.io/install.ps1 | iex

# Python
pip install sling

Confirm:

sling --version

Full install notes are in the Sling CLI Getting Started Guide.

Configuring the Postgres Source

Sling reads connection details from ~/.sling/env.yaml, environment variables, or sling conns set. A read-only user is enough:

CREATE USER sling WITH PASSWORD '<password>';
GRANT CONNECT ON DATABASE mydb TO sling;
GRANT USAGE ON SCHEMA public TO sling;
GRANT SELECT ON ALL TABLES IN SCHEMA public TO sling;
ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT SELECT ON TABLES TO sling;

Then register the connection:

sling conns set POSTGRES type=postgres host=host.ip user=sling \
  database=mydb password=mypass port=5432

Or in ~/.sling/env.yaml:

connections:
  POSTGRES:
    type: postgres
    host: host.ip
    user: sling
    password: mypass
    port: 5432
    database: mydb

If your Postgres requires SSL, append sslmode: require. Test it:

sling conns test POSTGRES

The Postgres connection docs cover SSL, IAM, and the rest.

Configuring the Iceberg Target

Sling treats Iceberg as a database-class target. The connection captures two things: the catalog, which stores table metadata, and the warehouse, which stores the actual Parquet data files. Sling supports REST, AWS Glue, and SQL catalogs. This guide uses REST.

For Cloudflare R2's Iceberg catalog you need the catalog URL, an API token, the warehouse identifier (account-id + bucket name), and S3-compatible credentials for the R2 bucket underneath. All four come from the R2 dashboard.

connections:
  ICEBERG:
    type: iceberg
    catalog_type: rest
    rest_uri: https://catalog.cloudflarestorage.com/<accountid>/<bucket>
    rest_token: <r2_catalog_api_token>
    rest_warehouse: <accountid>_<bucket>
    s3_access_key_id: <r2_access_key_id>
    s3_secret_access_key: <r2_secret_access_key>

For a self-hosted Lakekeeper or Nessie catalog, the shape is the same; only the rest_uri and rest_warehouse change. For AWS Glue, set catalog_type: glue and glue_warehouse: s3://my-bucket/warehouse. The Iceberg connection docs walk through each catalog type.

Test it:

sling conns test ICEBERG

A Full-Refresh Replication

For this run the Postgres source has three tables in a demo_postgres_iceberg schema:

users — 8,000 rows
orders — 35,000 rows
events — 60,000 rows, with an occurred_at timestamp

The replication file:

# replication.yaml
source: POSTGRES
target: ICEBERG

defaults:
  mode: full-refresh
  object: demo_postgres_iceberg.{stream_table}

streams:
  demo_postgres_iceberg.users:
  demo_postgres_iceberg.orders:
  demo_postgres_iceberg.events:
    mode: incremental
    primary_key: [event_id]
    update_key: occurred_at

A few notes:

object: follows the usual <namespace>.<table> shape. Sling creates the Iceberg namespace if it doesn't already exist in the catalog.
{stream_table} is a runtime variable. Sling substitutes the source table name so you don't repeat yourself.
The third stream switches to mode: incremental with an update_key. That's the only diff between a one-shot bulk load and an ongoing append flow.

Run it:

sling run -r replication.yaml

Real output, trimmed:

INF Sling CLI | https://slingdata.io
WRN for mode 'incremental' with iceberg target, primary-key is ineffective,
    incremental merge is not yet supported (only appends)
INF Sling Replication [3 streams] | POSTGRES -> ICEBERG

INF [1 / 3] running stream demo_postgres_iceberg.users
INF created table "demo_postgres_iceberg"."users"
INF streaming data (direct insert)
INF inserted 8000 rows into "demo_postgres_iceberg"."users" in 11 secs [713 r/s] [519 kB]

INF [2 / 3] running stream demo_postgres_iceberg.orders
INF created table "demo_postgres_iceberg"."orders"
INF inserted 35000 rows into "demo_postgres_iceberg"."orders" in 9 secs [3,721 r/s] [2.1 MB]

INF [3 / 3] running stream demo_postgres_iceberg.events
INF getting checkpoint value (occurred_at)
INF writing to target database [mode: incremental]
INF created table "demo_postgres_iceberg"."events"
INF inserted 60000 rows into "demo_postgres_iceberg"."events" in 7 secs [8,190 r/s] [4.5 MB]

INF Sling Replication Completed in 29s | POSTGRES -> ICEBERG | 3 Successes | 0 Failures

103,000 rows across three tables, 29 seconds end-to-end. The warning at the top deserves a real answer; see the section on incremental modes further down.

Verification

Sling can query Iceberg tables directly through its DuckDB-backed reader. Tables are addressed as iceberg_catalog.<namespace>.<table>:

sling conns exec ICEBERG \
  "select 'users' as t, count(*) as c
     from iceberg_catalog.demo_postgres_iceberg.users
   union all
   select 'orders', count(*) from iceberg_catalog.demo_postgres_iceberg.orders
   union all
   select 'events', count(*) from iceberg_catalog.demo_postgres_iceberg.events"

+--------+-------+
| T      |     C |
+--------+-------+
| users  |  8000 |
| orders | 35000 |
| events | 60000 |
+--------+-------+

Row counts match the source. A sample of users confirms columns and types survived the trip:

sling conns exec ICEBERG \
  "select user_id, email, country, signup_at
     from iceberg_catalog.demo_postgres_iceberg.users
    order by user_id limit 5"

+---------+-------------------+---------+-------------------------------+
| USER_ID | EMAIL             | COUNTRY | SIGNUP_AT                     |
+---------+-------------------+---------+-------------------------------+
|       1 | user1@example.com | BR      | 2025-01-01 00:14:00 -0300 -03 |
|       2 | user2@example.com | DE      | 2025-01-01 00:28:00 -0300 -03 |
|       3 | user3@example.com | FR      | 2025-01-01 00:42:00 -0300 -03 |
|       4 | user4@example.com | JP      | 2025-01-01 00:56:00 -0300 -03 |
|       5 | user5@example.com | UK      | 2025-01-01 01:10:00 -0300 -03 |
+---------+-------------------+---------+-------------------------------+

Postgres jsonb lands as a structured column too. Sampling events:

+----------+---------+------------+----------------------+----------------------+
| EVENT_ID | USER_ID | EVENT_TYPE | PAYLOAD              | OCCURRED_AT          |
+----------+---------+------------+----------------------+----------------------+
|    60001 |       2 | click      | {"v": 1, "utm": "x"} | 2026-05-11 ...       |
|    60002 |       3 | signup     | {"v": 2, "utm": "x"} | 2026-05-11 ...       |
|    60003 |       4 | purchase   | {"v": 3, "utm": "x"} | 2026-05-11 ...       |
+----------+---------+------------+----------------------+----------------------+

Any other Iceberg reader sees the same data: DuckDB with the iceberg extension, Spark, Trino, Athena, Snowflake's catalog-linked databases. That portability is the reason for the catalog in the first place.

Running an Incremental Append

After the bulk load, the day-to-day shape is: every few minutes (or hours, or once a day), pick up the new rows since the last run and append them to the Iceberg table. Sling's incremental mode does this. The state (the last seen value of the update_key) is tracked by Sling itself, so you don't need to manage a state file the way you would for a file-based target.

Insert 2,500 new events on the source (a stand-in for fresh activity):

insert into demo_postgres_iceberg.events (event_id, user_id, event_type, payload, occurred_at)
select 60000 + n, 1 + (n % 8000), 'click',
       jsonb_build_object('utm','x','v', n % 100),
       now() - (n * interval '1 second')
  from generate_series(1, 2500) g(n);

Run a single-stream replication that touches only events:

# replication-incremental.yaml
source: POSTGRES
target: ICEBERG

defaults:
  object: demo_postgres_iceberg.{stream_table}

streams:
  demo_postgres_iceberg.events:
    mode: incremental
    update_key: occurred_at

sling run -r replication-incremental.yaml

INF Sling Replication | POSTGRES -> ICEBERG | demo_postgres_iceberg.events
INF getting checkpoint value (occurred_at)
INF reading from source database
INF writing to target database [mode: incremental]
INF streaming data (direct insert)
INF inserted 2500 rows into "demo_postgres_iceberg"."events" in 8 secs [294 r/s] [178 kB]
INF execution succeeded

Sling read the saved checkpoint, pulled only rows newer than the last occurred_at it saw, and appended exactly the 2,500 new rows. A readback confirms the new total:

sling conns exec ICEBERG \
  "select min(occurred_at), max(occurred_at), count(*)
     from iceberg_catalog.demo_postgres_iceberg.events"

+-------------------------------+--------------------------------------+--------+
| MIN_OCCURRED_AT               | MAX_OCCURRED_AT                      | COUNT  |
+-------------------------------+--------------------------------------+--------+
| 2025-03-01 00:00:40 -0300 -03 | 2026-05-11 08:42:59.533692 -0300 -03 |  62500 |
+-------------------------------+--------------------------------------+--------+

60,000 + 2,500 = 62,500. The new high-water mark on occurred_at is the timestamp of the freshest insert. The next scheduled run will start from there.

Append-incremental vs merge-incremental

That warning Sling printed on the first run matters:

WRN for mode 'incremental' with iceberg target, primary-key is ineffective,
    incremental merge is not yet supported (only appends)

For database targets like Postgres or Snowflake, Sling's incremental mode is a merge: a row whose primary_key already exists in the target gets updated in place. For an Iceberg target today, incremental means append only. New rows go in, existing rows stay as-is, and a primary_key declared on the stream is parsed but not enforced.

That is fine when your source is append-only: events, immutable transactions, log data. It is the wrong default if your source has mutable rows you need reflected on the lake side. Until merge lands, two patterns work:

Snapshot replays. Run mode: full-refresh on a cadence that matches your freshness budget. Iceberg's snapshot model means readers always see a consistent table; the old snapshot is replaced atomically. For tables in the low millions this is faster than it sounds.
CDC-style append plus downstream resolution. Append every Postgres change to Iceberg as-is (using a logical-replication tool or trigger-based capture) and resolve the latest-state view at read time with something like qualify row_number() over (partition by pk order by event_ts desc) = 1. A bit more work at query time, very cheap at write time.

Track the Iceberg connector docs for when full merge mode ships.

Common tweaks

Choose the right catalog. REST is the most portable: the same connection shape works for Cloudflare R2, Lakekeeper, Nessie, Polaris, and any other REST-compatible catalog. Glue is the simplest in AWS-native shops. SQL catalog is fine for local dev. Avoid wiring a different catalog per environment if you can help it; the table layout doesn't care, but the metadata location does.
Namespace organization. Treat namespaces (demo_postgres_iceberg.users) the way you treat warehouse schemas: one per source system, or one per data domain. Don't dump everything into default.
Filter at the source. Use a sql: block per stream to project columns or filter rows before they leave Postgres. Smaller Parquet files, smaller manifests, cheaper queries downstream.
Time travel for free. Every replication produces a new Iceberg snapshot. Readers can time-travel to a previous snapshot, which is useful for "what did this table look like before yesterday's run?" without storing your own backups.
Maintain the table. Like any Iceberg table, periodic compaction and snapshot expiration keep the file count and metadata size from growing without bound. Set this up on a separate schedule from the replication itself.

Where to go next

The same pattern works for any of Sling's 30+ database sources into Iceberg: MySQL, SQL Server, Snowflake, BigQuery, MongoDB, and the rest. Swap the source and leave the target alone.

If the underlying R2 storage is what brought you here, the Postgres → R2 as Parquet walkthrough shows the same source landing as raw Parquet files instead of an Iceberg table, which is useful when downstream readers don't need a catalog. For a deeper comparison of file-format targets, see Postgres → S3 as Parquet and Postgres → DuckDB.

For team workflows with scheduling, alerting, and audit trails on top of the same CLI, look at the Sling Platform.

Questions go to Discord or GitHub Issues.