DEV Community

Cover image for The AI-Safe Pipeline From PostgreSQL to Governed Data
Burnside Project
Burnside Project

Posted on

The AI-Safe Pipeline From PostgreSQL to Governed Data

Build a one-way pipeline from PostgreSQL into an air-gapped, governed data zone — where access is controlled at the database, table, and column level.

The New Problem: AI + Production Databases

Modern stacks are quietly introducing a dangerous pattern:
AI Agent → PostgreSQL

Even when wrapped with APIs, proxies, or query layers, the reality is:

  • The agent still reaches production
  • It still executes queries
  • It still operates inside your blast radius

This is fundamentally unsafe.

Not because AI is malicious — but because it is non-deterministic.

The Design Principle

pg-cdc is built around a simple rule:

AI should never have network access to production databases.

Instead of securing access…

We remove it entirely.

What pg-cdc Actually Is

pg-cdc is:

  • A PostgreSQL logical replication (WAL) consumer
  • A Parquet writer with compaction
  • A governance-aware ingestion layer
  • A security boundary between production and consumers

The Air Gap

The most important property:

pg-cdc creates a physical and logical air gap

Properties

  • No return path
  • WAL is unidirectional
  • Storage is append/immutable
  • No database credentials
  • Consumers use IAM
  • No connection strings
  • No shared execution environment
  • No queries run on PostgreSQL
  • Governed surface only
  • Access mediated via catalog + tags
  • Auditable: Every read can be tracked

If You’re Building This Kind of System

Think in layers:

  • Production system (OLTP) → isolated
  • CDC boundary (pg-cdc) → one-way
  • Governed storage → immutable + tagged
  • Query layer → controlled access
  • AI / analytics → consumers

I’d love to hear — these are exactly the problems we’re solving.

Repository: GitHub repo

Connect:LinkedIn

Book a demo:

PostgreSQL #CDC #DataEngineering #AIInfrastructure #Lakehouse #Security #MLOps #DuckDB #Parquet #OpenSource

Top comments (0)