<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: אופיר נקדאי</title>
    <description>The latest articles on DEV Community by אופיר נקדאי (@ofirnakdai).</description>
    <link>https://dev.to/ofirnakdai</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4061044%2F573b1283-6152-48e8-bf5b-a330187fb950.png</url>
      <title>DEV Community: אופיר נקדאי</title>
      <link>https://dev.to/ofirnakdai</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/ofirnakdai"/>
    <language>en</language>
    <item>
      <title>How to Run Apache Iceberg &amp; Trino Locally in 60 Seconds (Zero AWS Cost)</title>
      <dc:creator>אופיר נקדאי</dc:creator>
      <pubDate>Mon, 03 Aug 2026 16:52:10 +0000</pubDate>
      <link>https://dev.to/ofirnakdai/how-to-run-apache-iceberg-trino-locally-in-60-seconds-zero-aws-cost-5acc</link>
      <guid>https://dev.to/ofirnakdai/how-to-run-apache-iceberg-trino-locally-in-60-seconds-zero-aws-cost-5acc</guid>
      <description>&lt;p&gt;If you build modern data pipelines, you know that &lt;strong&gt;Apache Iceberg&lt;/strong&gt; is taking over the data lakehouse ecosystem. But let’s talk about the developer experience: &lt;strong&gt;testing Iceberg locally is a nightmare.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Every time you want to verify a schema change, test a new partition spec, or run an integration query, you end up relying on cloud infrastructure. Spinning up AWS EMR, waiting for AWS Glue metadata updates, or querying via Athena is slow, interrupts your flow, and burns cloud budget.&lt;/p&gt;

&lt;p&gt;Local dev loops should take seconds, not minutes.&lt;/p&gt;

&lt;p&gt;Here is how to set up a full, local, S3-compatible &lt;strong&gt;Apache Iceberg + Trino + MinIO&lt;/strong&gt; stack on your machine in under 60 seconds.&lt;/p&gt;




&lt;h2&gt;
  
  
  🏗️ The Local Architecture
&lt;/h2&gt;

&lt;p&gt;To replicate a production cloud setup locally, we need three primary layers:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Storage (S3 replacement):&lt;/strong&gt; MinIO &lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Catalog (Glue replacement):&lt;/strong&gt; Nessie / Iceberg REST Catalog&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Engine (Athena replacement):&lt;/strong&gt; Trino&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Connecting these three in Docker can be tricky because of authentication and S3 endpoint routing between containers. &lt;/p&gt;

&lt;p&gt;I put together a clean, open-source boilerplate so you don't have to debug container networking.&lt;/p&gt;




&lt;h2&gt;
  
  
  ⚡ Quick Start
&lt;/h2&gt;

&lt;h3&gt;
  
  
  Prerequisites
&lt;/h3&gt;

&lt;ul&gt;
&lt;li&gt;Docker &amp;amp; Docker Compose installed on your machine.&lt;/li&gt;
&lt;/ul&gt;

&lt;h3&gt;
  
  
  1. Clone the repository
&lt;/h3&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;git clone https://github.com/ofirnakdai/local-iceberg-trino-stack.git
&lt;span class="nb"&gt;cd &lt;/span&gt;local-iceberg-trino-stack
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h3&gt;
  
  
  2. Spin up the cluster
&lt;/h3&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;docker-compose up &lt;span class="nt"&gt;-d&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;em&gt;Wait ~30-60 seconds for the containers to initialize.&lt;/em&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  3. Access the Services
&lt;/h3&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;MinIO Console:&lt;/strong&gt; &lt;a href="http://localhost:9001" rel="noopener noreferrer"&gt;http://localhost:9001&lt;/a&gt; (User: minioadmin | Pass: minioadmin)&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Trino UI:&lt;/strong&gt; &lt;a href="http://localhost:8080" rel="noopener noreferrer"&gt;http://localhost:8080&lt;/a&gt; (User: admin)&lt;/li&gt;
&lt;/ul&gt;

&lt;h3&gt;
  
  
  4. Run your first Iceberg Query
&lt;/h3&gt;

&lt;p&gt;Connect to Trino using the Trino CLI, DBeaver, or any standard SQL client pointing to localhost:8080.&lt;/p&gt;

&lt;p&gt;-- Create a new Iceberg schema&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight sql"&gt;&lt;code&gt;&lt;span class="k"&gt;CREATE&lt;/span&gt; &lt;span class="k"&gt;SCHEMA&lt;/span&gt; &lt;span class="n"&gt;iceberg&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="k"&gt;default&lt;/span&gt; &lt;span class="k"&gt;WITH&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="k"&gt;location&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="s1"&gt;'s3a://lakehouse-data/'&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;-- Create a basic Iceberg table&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight sql"&gt;&lt;code&gt;&lt;span class="k"&gt;CREATE&lt;/span&gt; &lt;span class="k"&gt;TABLE&lt;/span&gt; &lt;span class="n"&gt;iceberg&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="k"&gt;default&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;events&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;
    &lt;span class="n"&gt;event_id&lt;/span&gt; &lt;span class="nb"&gt;VARCHAR&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;event_time&lt;/span&gt; &lt;span class="nb"&gt;TIMESTAMP&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;6&lt;/span&gt;&lt;span class="p"&gt;),&lt;/span&gt;
    &lt;span class="n"&gt;user_id&lt;/span&gt; &lt;span class="nb"&gt;VARCHAR&lt;/span&gt;
&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;WITH&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;
    &lt;span class="n"&gt;format&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="s1"&gt;'PARQUET'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;partitioning&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;ARRAY&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s1"&gt;'day(event_time)'&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;
&lt;span class="p"&gt;);&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;If you open the MinIO UI now, you’ll see the generated Parquet files and Iceberg metadata JSONs inside your local bucket!&lt;/p&gt;

&lt;h2&gt;
  
  
  🚀 Taking It Further (CI/CD &amp;amp; Synthetic Data)
&lt;/h2&gt;

&lt;p&gt;A local Docker stack is great for manual testing, but what about automated testing in your CI/CD pipeline or generating realistic synthetic data at scale?&lt;/p&gt;

&lt;p&gt;If you want to skip building the infrastructure tools yourself, check out LakehouseKit Pro.&lt;/p&gt;

&lt;p&gt;It expands this stack into a complete production-testing ecosystem:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;&lt;p&gt;🐍 &lt;strong&gt;Python Mock Data SDK&lt;/strong&gt;: Generate 1M+ synthetic Iceberg rows matching your schema in seconds.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;⚡ &lt;strong&gt;GitHub Actions CI/CD Starter Kit&lt;/strong&gt;: Run automated pytest integration tests on PRs (with zero AWS cost).&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;🏗️ &lt;strong&gt;Enterprise IaC Blueprints&lt;/strong&gt;: Ready-to-use Terraform &amp;amp; Helm templates to jump from local Docker straight to AWS production.&lt;/p&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  🔗 Useful Links
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;&lt;p&gt;📦 &lt;strong&gt;Free GitHub Stack&lt;/strong&gt;: &lt;a href="https://github.com/ofirnakdai/local-iceberg-trino-stack" rel="noopener noreferrer"&gt;https://github.com/ofirnakdai/local-iceberg-trino-stack&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;⚡&lt;strong&gt;LakehouseKit Pro&lt;/strong&gt;: &lt;a href="https://ofirnakdai.gumroad.com/l/lakehousekit" rel="noopener noreferrer"&gt;https://ofirnakdai.gumroad.com/l/lakehousekit&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;How are you currently handling local testing for Apache Iceberg? Drop your setup or questions in the comments below! 👇&lt;/p&gt;

</description>
      <category>dataengineering</category>
      <category>python</category>
      <category>docker</category>
      <category>architecture</category>
    </item>
  </channel>
</rss>
