Story 1
A long time ago, when I was working as a developer at an e-commerce company, we received an urgent Slack message: something had gone terribly wrong — customers were being charged extra money.
What happened?
The client had a subscription-based web store. A tech lead updated the staging database but forgot (or didn’t realize) that cron jobs were still enabled. Instead of cleaning the data, he simply created a dump from production and imported it into staging.
As a result, subscription billing ran again.
Around $150,000 was charged from customers’ accounts.
It was chaos. Thousands of apologies. Weeks of refunds. Damage control calls every day.
After that incident, the company introduced strict rules:
- Always clean production dumps before importing.
- Disable all cron jobs.
- Replace external service credentials with sandbox ones.
But it was too late.
The reputation damage had already been done.
Story 2
When it happened, I was just a developer and wasn’t involved in that process — but I remembered the lesson.
When I became a Tech Lead, I made sure developers never received backups with raw production data. Staging and dev servers were also updated with trimmed backups only.
Later, when I moved to a CTO role, I implemented internal tooling to prevent exactly these kinds of incidents.
Did it help? Partially.
A developer from an offshore team wasn’t aware of our processes and updated the staging database with production data. This broke the search entirely, since it was powered by a third-party service, and all search index links were pointing to the staging website.
All customers were redirected to the staging website, which, of course, affected the sales.
We fixed it quickly. And since it wasn’t our fault operationally, our reputation stayed intact.
But the pattern was obvious.
The problem wasn’t people.
The problem was the process.
The Real Problem
I started asking friends in other companies how they handle database sharing.
The answers were surprisingly similar:
- “We have some internal scripts.”
- “We trim data manually.”
- “We have rules — but it’s up to developers to follow them.”
In most cases, production data handling depends on human discipline.
And humans make mistakes.
Looking for a Solution
Back in 2020, I tried to find a proper solution.
There were almost no universal tools available. Existing solutions were either:
- Enterprise-only
- Stack-specific
- Or extremely limited
So I decided to build one.
Solution
VeilDB (https://veildb.com/) is an open-source solution that helps anonymize and mask sensitive data from databases and safely share processed dumps with your team.
You can easily:
- replace user’s name with some value
- email → faker replacement,
- generate random numbers for a credit card.
This perfectly works for outsourcing teams, e-commerce, and agencies managing multiple client environments. It also works for individual projects, ensuring developers don’t get access to the real data.
VeilDB is fully open-source.
👉 GitHub: https://github.com/veildb-tech/service
👉 Documentation: https://veildb.gitbook.io/
👉 Quick Start Guide: https://veildb.gitbook.io/veildb-docs/getting-started/
How does it work?
It consists of three layers:
1. Service Layer (UI)
A web interface where you configure:
- Masking rules
- Permissions
- Schedules
- Webhooks
2. Agent Layer
A CLI-based agent installed on your server that processes databases.
Important design principle:
The Service layer never receives actual data — only database schema.
The Agent:
- Creates a dump
- Restores it inside isolated Docker infrastructure
- Applies masking rules
- Generates a sanitized dump
- Removes temporary data
It never works directly with the original database.
Multiple agents can be connected to a single Service instance.
3. Client Layer
A lightweight CLI tool installed on developers’ machines to securely download processed dumps.
How the Workflow Looks
- Admin configures masking rules and schedules in the UI.
- The Agent checks the queue and retrieves processing rules.
- The Agent creates and processes a dump in isolation.
- Developers download only sanitized backups.
For developers, the flow is simple.
Under the hood, several critical security checks happen:
- The Service verifies whether the developer has access to a specific database.
- The Service issues a temporary token.
- The Client uses this token to download the dump from the Agent.
- The Agent validates the token before allowing the download.
- All communication is encrypted.
Conclusion
Setting up VeilDB may take some initial effort.
But it’s a small price compared to:
- Charging customers by mistake
- Corrupting external services
- Leaking sensitive data
- Damaging your company’s reputation
Production data is powerful.
It should never be shared without protection.
Top comments (0)