If you build modern data pipelines, you know that Apache Iceberg is taking over the data lakehouse ecosystem. But let’s talk about the developer experience: testing Iceberg locally is a nightmare.
Every time you want to verify a schema change, test a new partition spec, or run an integration query, you end up relying on cloud infrastructure. Spinning up AWS EMR, waiting for AWS Glue metadata updates, or querying via Athena is slow, interrupts your flow, and burns cloud budget.
Local dev loops should take seconds, not minutes.
Here is how to set up a full, local, S3-compatible Apache Iceberg + Trino + MinIO stack on your machine in under 60 seconds.
🏗️ The Local Architecture
To replicate a production cloud setup locally, we need three primary layers:
- Storage (S3 replacement): MinIO
- Catalog (Glue replacement): Nessie / Iceberg REST Catalog
- Engine (Athena replacement): Trino
Connecting these three in Docker can be tricky because of authentication and S3 endpoint routing between containers.
I put together a clean, open-source boilerplate so you don't have to debug container networking.
⚡ Quick Start
Prerequisites
- Docker & Docker Compose installed on your machine.
1. Clone the repository
git clone https://github.com/ofirnakdai/local-iceberg-trino-stack.git
cd local-iceberg-trino-stack
2. Spin up the cluster
docker-compose up -d
Wait ~30-60 seconds for the containers to initialize.
3. Access the Services
- MinIO Console: http://localhost:9001 (User: minioadmin | Pass: minioadmin)
- Trino UI: http://localhost:8080 (User: admin)
4. Run your first Iceberg Query
Connect to Trino using the Trino CLI, DBeaver, or any standard SQL client pointing to localhost:8080.
-- Create a new Iceberg schema
CREATE SCHEMA iceberg.default WITH (location = 's3a://lakehouse-data/');
-- Create a basic Iceberg table
CREATE TABLE iceberg.default.events (
event_id VARCHAR,
event_time TIMESTAMP(6),
user_id VARCHAR
) WITH (
format = 'PARQUET',
partitioning = ARRAY['day(event_time)']
);
If you open the MinIO UI now, you’ll see the generated Parquet files and Iceberg metadata JSONs inside your local bucket!
🚀 Taking It Further (CI/CD & Synthetic Data)
A local Docker stack is great for manual testing, but what about automated testing in your CI/CD pipeline or generating realistic synthetic data at scale?
If you want to skip building the infrastructure tools yourself, check out LakehouseKit Pro.
It expands this stack into a complete production-testing ecosystem:
🐍 Python Mock Data SDK: Generate 1M+ synthetic Iceberg rows matching your schema in seconds.
⚡ GitHub Actions CI/CD Starter Kit: Run automated pytest integration tests on PRs (with zero AWS cost).
🏗️ Enterprise IaC Blueprints: Ready-to-use Terraform & Helm templates to jump from local Docker straight to AWS production.
🔗 Useful Links
📦 Free GitHub Stack: https://github.com/ofirnakdai/local-iceberg-trino-stack
⚡LakehouseKit Pro: https://ofirnakdai.gumroad.com/l/lakehousekit
How are you currently handling local testing for Apache Iceberg? Drop your setup or questions in the comments below! 👇
Top comments (0)