I Gave Every Pull Request Its Own Database

Git-style branching for Postgres, wired into Jenkins, with a real repo you can run.

Every team has a dev database story. Ours went like this: one shared Postgres that five developers treated like a timeshare. Someone's migration failed halfway. Someone else was "just testing" against it. By Thursday it had the structural integrity of a sandcastle, and by Friday deploy we were all staring at a migration that had passed CI beautifully against a completely empty schema.

We don't really test migrations. We run migration-shaped SQL against tables with zero rows, then wonder why ADD COLUMN ... NOT NULL locks the orders table for nine minutes in production. The migration wasn't the problem. The test never proved anything.

Why not branch the database too?

We branch code for every pull request without thinking. But the database is this precious, monolithic thing you can't copy, so every PR shares one staging instance and developers queue behind each other.

What if a database branch was as cheap as a git branch?

That's what Databricks Lakebase is built for. It's managed Postgres where storage is separated from compute, and a branch is a first-class object: an instant, copy-on-write clone of production. You get a full, data-inclusive copy in seconds, on an endpoint that scales to zero when nobody's using it. Your throwaway test environment costs nothing while it sleeps.

So I built a complete, runnable Jenkins pipeline around it. Opening a pull request spins up a real database branch, migrations run against production-shaped data, tests assert against real rows, and nothing touches production until a DBA approves the exact migration that already passed CI. The DBA approval step is doing more work than it looks, and I'll explain why below.

The repo is here: https://github.com/harishshenoy552/lakebase-cicd

And if you'd rather watch than read, here's the 9-minute walkthrough:

https://www.youtube.com/watch?v=e2g8YtMcqFg

How the pipeline actually works

There are two paths, and the Jenkinsfile keeps them honest.

Pull request opened: Jenkins creates a branch called ci-pr-42 (or whatever your PR number is) straight from production. Copy-on-write, instant, full data. Then it applies the PR's migration to that branch and runs the test suite against it, not against an empty schema but against your actual rows. When the build finishes, pass or fail, the branch is torn down in post { always }, so nothing lingers and no orphaned branches keep eating storage.

Merged to main: the pipeline does something different. It skips the branch/test stages and parks at a DBA approval gate. A human from the dba-team clicks approve, and only then does the promote stage apply the migration to production. The DBA isn't reviewing SQL in the abstract. They're approving the exact migration artifact that already survived CI against production-shaped data.

Everyone works in one pipeline, off the same migration artifact and the same shape of data.

One thing I made sure of: the Jenkinsfile is just a thin wrapper around shell scripts, so the exact same commands run on your laptop and in CI. create_branch.sh, migrate.sh, test.sh, promote.sh, teardown.sh. What you run locally at your desk is what Jenkins runs.

And Jenkins is just the example here. Underneath it's a set of shell scripts, so the same pipeline works with whatever your team already uses: Azure DevOps, GitHub Actions, GitLab CI, CircleCI, or anything else that can run a shell. Swap the orchestrator, keep the scripts.

The repo even includes screenshots from a real Jenkins run of all five stages, so you can see the whole thing working before you wire it up.

The test that matters most

The demo scenario is an orders service. The seed data customers are Ada Lovelace, Grace Hopper, and Alan Turing.

Migration 002 adds a fulfillment_status column (NOT NULL DEFAULT 'pending', plus an index). And the test suite has this assertion:

def test_existing_rows_were_backfilled(conn):
    # The pre-existing production rows must have been backfilled with the default.
    # This is the assertion that fails on a naive ADD COLUMN NOT NULL without a
    # default - and the reason testing against real data (not an empty table) matters.

It checks that the pre-existing orders actually got backfilled with the default. If your migration was the naive ADD COLUMN NOT NULL without a default, this test fails on the branch, safely, before it ever reaches production. On an empty test schema, that same migration passes without complaint and then causes an outage on Friday.

That one test is the core argument for data-inclusive branches.

There's also a pure-SQL fallback (assertions.sql) for locked-down Jenkins agents where you can't install Python dependencies, and a Liquibase path if you want versioned changelogs with rollback instead of plain SQL files. Pick one per project.

What about the DBAs?

The obvious concern: you're letting developers spin up copies of production?

They're isolated copies, not access to production itself. And the promotion path still goes through a human gate restricted to the DBA group. DBAs actually get more signal than before. Instead of reviewing a SQL file in a ticket queue and guessing at the blast radius, they approve a migration that has already been executed and tested against a copy of their data.

Also worth noting: auth is via short-lived OAuth tokens from a service principal. There is no static database password sitting in a Jenkins credential store.

How it compares

Per-PR data-inclusive copy: Oracle does RMAN restore (hours). SQL Server does restore (slow). Aurora has fast clone (closest). Lakebase is instant copy-on-write.

Idle cost of test env: Oracle and SQL Server charge full license plus compute. Aurora is billed while running. Lakebase scales to zero.

Branch as first-class object: Oracle and SQL Server, no. Aurora has clones, not branches. Lakebase, yes.

Licensing cost of throwaway envs: Oracle is per-core, SQL Server per-core/CAL. Aurora has no license but compute adds up. Lakebase is usage-based with no license cost.

Aurora's fast clone is the closest alternative. The rest are slower, more expensive, or both.

Try it in about 3 minutes

git clone https://github.com/harishshenoy552/lakebase-cicd.git
cd lakebase-cicd
cp .env.example .env   # set your Databricks profile
source .env
./scripts/bootstrap_production.sh            # one-time: project + seed data
BRANCH_ID=ci-pr-42 ./scripts/create_branch.sh
BRANCH_ID=ci-pr-42 ./scripts/migrate.sh
BRANCH_ID=ci-pr-42 ./scripts/test.sh         # real rows, real assertions
./scripts/promote.sh                          # the "merge"
BRANCH_ID=ci-pr-42 ./scripts/teardown.sh

You'll need a Databricks workspace with Lakebase (Autoscaling tier), the Databricks CLI, psql, jq, and Python. The README has the full prerequisites and the Jenkins wiring instructions: multibranch pipeline, service principal, restrict the approval gate to your DBA group.

The point

We spent a decade making code branching instant and free. Databases never got the same treatment, so we built rituals around them instead: the staging database nobody trusts, the DBA ticket queue, the careful Friday deploy.

Copy-on-write branching takes away the usual excuse for skipping all that. A database branch can be as disposable as a git branch, migrations can run against real data, and DBAs get their time back for work that actually needs a DBA instead of rubber-stamping SQL files.

The repo is public and the video is linked above if you want to see it running.

---

Originally published on Medium: https://medium.com/@harishshenoy552/i-gave-every-pull-request-its-own-database-ac8212f7c713 · Repo: https://github.com/harishshenoy552/lakebase-cicd · Video: https://www.youtube.com/watch?v=e2g8YtMcqFg