Building Indonesia's Industrial Data Layer: From Power BI to DuckDB Lakehouses and Multi-Agent AI

Eko Suryahadi — Tue, 13 Oct 2026 13:02:24 +0000

Building Indonesia's Industrial Data Layer From Power BI Dashboards to Out‑of‑Core Engines and Multi‑Agent AI

Most industrial data projects fail at the same point: the data sits in the plant, the decisions sit in the boardroom, and nobody bridges the two.
I've spent years working that messy middle — Oil & Gas production analytics, EPC project controls, and AI systems that turn field data into decisions.
This is the architecture, the benchmarks, and the portfolio.

The Problem
Indonesia's heavy industry runs on a fragmented data stack. The production engineer tracks well output in a spreadsheet. The project scheduler manages a Gantt chart in MS Project. The HSE supervisor files safety reports in a web form. The PMO director wants a single view but gets three different PowerPoint decks.

Traditional BI tools (Power BI, Tableau) work for aggregated reporting, but they cannot:

Process millions of rows without hammering the source database
Run predictive models (delay forecast, cost overrun flags) in the same pipeline
Handle multi-user real-time collaboration on project schedules
Execute autonomous agents that monitor, alert, and act

We needed something else.

The Architecture: Three Layers
Layer 1: Out‑of‑Core Lakehouse (GaungNusa)

The core insight: memory is the bottleneck in a $50/month VPS. Traditional data platforms assume infinite RAM. Industrial datasets (production logs, inspection records, supply chain movements) regularly exceed 5 million rows.

Tech stack:

DuckDB 1.1.0 (pure-JS fallback, no native binary conflicts)
Polars/Arrow vectorized memory processing
Snappy Parquet on MinIO S3 (Bronze → Silver → Gold tiering)
Prisma ORM on PostgreSQL for the business layer

Benchmark: Aggregated 5,000,000 synthetic rows via DuckDB with only +0.12 MB Node.js heap delta — the entire operation spilled to SSD, not memory.

flowchart LR
A[Raw Parquet] --> B[Bronze / MinIO]
B --> C[DuckDB Out-of-Core]
C --> D[Silver / MinIO]
D --> E[Polars Arrow Pipeline]
E --> F[Gold / MinIO]
F --> G[Next.js API Route]
G --> H[React Dashboard]

Layer 2: Project Scheduling & Cost Control (Kurvaup)
MS Project is the standard in EPC contracting, but it's single-user, Windows-only, and expensive per seat. We built a web-native replacement.

Core capabilities:

Recursive Critical Path Method (CPM) with custom non-working calendar exceptions
4-way dependency graphs (FS, SS, FF, SF + Lag)
S-Curve baseline variance tracking (EV, PV, AC)
Earned Value Analysis: SPI, CPI, CBS‑WBS reconciliation
Multi-contractor milestone valuation

Why it matters for EPC contractors: A medium refinery project runs 2,500+ activities across 5+ contractors. The scheduler reconciles 6 different MS Project files weekly. Kurvaup collapses this into a real-time multi-user view with automated delay propagation.

Layer 3: AI Agents & MCP Integration

The Model Context Protocol (MCP) connects LLMs directly to enterprise data without brittle RAG pipelines:

Batu ERP Agent: Natural-language queries against inventory, POs, production milestones
Valoris Agentic Suite: Multi-agent autonomous framework for operational analysis and report synthesis
MCP Power BI Model Bridge: Automated DAX measure optimization, schema validation, and documentation via MCP server
QueryGuard: SQL AST injection prevention on autonomous query channels

The Portfolio (50+ Production Repositories)

System	Domain	Engine	Status
Kurvaup	EPC Scheduling	Next.js + DuckDB	Live
GaungNusa Lakehouse	Data Lakehouse	DuckDB/Polars/MinIO	Live
Safety Analytics Pro	O&G HSE	FastAPI + React	Live
EPC Delay Predictor	Project Controls	XGBoost + Next.js	Live
Valoris AI Agents	Operational AI	LangChain/Claude/MCP	Live
Batu ERP Agent	Manufacturing	AI Agent + API	Live
ControlCheck	QA/QC Field	Next.js + PostgreSQL	Pilot
ESG AI Carbon	Sustainability	Python + API	Pilot
DTI Dashboard Builder	Analytics	Low-Code Builder	Live
SC-EPC Supply Chain	Material Tracking	Next.js + Prisma	Live
Cost Control SaaS	EVM/CBS	SaaS Platform	Beta
Band Procurement	Procurement	MVP	Beta
TalentaDesk	Manpower	Next.js + PostgreSQL	Live

The Business Case

For investors: Indonesia's infrastructure spend will exceed IDR 400 trillion in 2026 (PUPR budget + SOE capex). Every rupiah needs tracking, scheduling, and compliance. Current tools are fragmented (MS Project + Excel + Power BI + WhatsApp). A unified decision layer for EPC/O&G operators in Southeast Asia is a greenfield opportunity.

For clients: An EPC contractor managing 10 projects simultaneously can reduce delay-related cost exposure by an estimated $750K-1.2M/year with predictive delay flags appearing 3-4 weeks before conventional reports detect them. Safety compliance audit time drops from 5 days to 4 hours with AI-assisted report synthesis.

The Tech Lead

Eko Suryahadi — Industrial Data & AI Specialist / EPC Tech Lead.

Production out-of-core data systems, mathematical project scheduling engines, and AI agent engineering. Built end-to-end: from Power BI delivery models to DuckDB lakehouses to autonomous multi-agent frameworks, with Oil & Gas and EPC domain depth.

I don't build dashboards. I build the decision layer between field data and management actions, end to end.

Let's Talk

Portfolio: ekosuryahadi.web.id
Email: e.suryahadi@gmail.com
Location: Indonesia (UTC+7) — Remote & On-Site Heavy Industry Advisory

Open to select consulting engagements: data engine architecture, EPC scheduling systems, and custom AI agent deployments.

DEV Community: Eko Suryahadi

Building Indonesia's Industrial Data Layer: From Power BI to DuckDB Lakehouses and Multi-Agent AI

The Portfolio (50+ Production Repositories)

The Business Case

The Tech Lead

Let's Talk