Building a PostgreSQL Change-Event Relay
PostgreSQL logical replication has become a popular foundation for event-driven systems, search synchronization, auditing, and data integration pipelines. Most discussions, however, focus on how to consume logical replication.
Much less attention is given to what happens after a change has been decoded.
That turned out to be the more interesting engineering problem.
The Problem
Many applications need to react whenever data changes.
Typical examples include:
- sending webhooks
- synchronizing search indexes
- updating caches
- triggering automation
- maintaining audit history
- feeding analytics or AI pipelines
Common approaches are:
- application callbacks
- database triggers
LISTEN/NOTIFY
These approaches work well for many use cases, but they also couple integration logic directly to the application or database.
PostgreSQL already records an ordered, durable history of committed transactions through logical replication.
The question becomes:
What should sit between PostgreSQL change events and downstream systems?
The Relay Problem
Once change events leave PostgreSQL, a relay has an entirely different set of responsibilities.
It needs to answer questions like:
- How are failed deliveries retried?
- How are checkpoints persisted?
- How are dead letters handled?
- How are failed events replayed?
- How is routing configured?
- How is operational state inspected?
- What metrics should operators monitor?
None of these concerns belong to PostgreSQL itself.
They're operational concerns.
Ruby Is the Implementation. PostgreSQL Is the Interface.
One question I've already been asked is:
Can this be used if my application isn't written in Ruby?
Yes.
Although Mammoth is implemented in Ruby, the architectural boundary is PostgreSQL logical replication.
The applications producing database changes can be written in any language or framework:
- Ruby on Rails
- Spring Boot
- ASP.NET
- Django
- Laravel
- Phoenix
- Express
- Go
- or anything else that writes to PostgreSQL
Mammoth simply observes committed changes and delivers them downstream.
From the application's perspective, the implementation language of the relay is largely irrelevant.
The database becomes the integration boundary.
A Deliberately Narrow Responsibility
One design goal was to keep Mammoth's responsibility intentionally small.
Upstream components handle:
- logical replication transport
-
pgoutputprotocol parsing - tuple decoding
- normalization into shared change-event objects
Mammoth begins only after normalized change events already exist.
Its responsibility is limited to downstream delivery concerns:
- webhook fanout
- routing by schema, table, and operation
- checkpoint persistence
- retry policies
- dead-letter handling
- replay
- operational state
- observability
Keeping this boundary narrow makes each layer easier to understand, test, and evolve independently.
Why a Relay Instead of a CDC Platform?
I intentionally describe Mammoth as a change-event relay rather than a complete CDC platform.
Its responsibility is deliberately narrow. It doesn't own PostgreSQL protocol parsing, logical replication transport, or event normalization. Those concerns belong upstream.
Mammoth focuses on what happens after normalized change events exist: reliably delivering them to downstream systems while maintaining the operational state required to recover from failures.
Keeping that boundary explicit has made the implementation considerably simpler and easier to reason about.
Operational State
Rather than introducing another distributed dependency, Mammoth currently stores operational state locally using SQLite.
This includes:
- checkpoints
- delivered envelopes
- dead letters
- schema migrations
The goal is to keep operational state easy to inspect, back up, recover, and replay without requiring additional infrastructure.
Deployment
Mammoth is designed to be self-hosted.
Current deployment options include:
- Docker
- Kubernetes (Helm)
- YAML configuration with JSON Schema validation
The current Kubernetes deployment intentionally uses a single active replica because one PostgreSQL logical replication slot maps to one active subscriber.
That constraint comes from PostgreSQL itself rather than the implementation.
Current Status
The current OSS implementation includes:
- webhook fanout
- configurable routing
- retry policies
- checkpoint persistence
- dead-letter persistence
- replay support
- health and metrics endpoints
- extension points for future destination, runtime, and operational-state adapters
The focus so far has been reliability and operational simplicity rather than adding more destinations.
Looking for Feedback
Although Mammoth is implemented in Ruby, I'm interested in feedback from a much broader audience:
- PostgreSQL engineers
- DevOps and platform engineers
- SREs
- distributed systems practitioners
- people operating self-hosted infrastructure
Some questions I'm currently thinking about:
- Is SQLite the right choice for operational state?
- What production metrics would you expect from a relay?
- What failure modes should be considered?
- Is the current deployment model reasonable?
- What capabilities would you expect before trusting a relay like this in production?
Technical feedback, design critiques, and alternative approaches are all welcome.
Repository:
https://github.com/kanutocd/mammoth
Documentation:
Top comments (0)