Most GDPR work that lands on engineers is not about cookie banners or privacy policies. It comes down to two duties that are easy to state and hard to implement: do not keep personal data longer than you need it, and delete it properly when you are supposed to.
Both are retention and deletion problems, and both are where real systems quietly fall out of compliance. This is a practical guide to getting them right, including the part almost every plan forgets: your backups.
One disclaimer up front. I am an engineer, not a lawyer, and this is implementation guidance, not legal advice. The exact retention periods and legal bases for your product are a question for your DPO or counsel. What follows is how to build the machinery once those decisions are made.
Duty one: storage limitation (stop keeping data forever)
GDPR's storage limitation principle says personal data should be kept only as long as it serves the purpose you collected it for. In engineering terms: every category of personal data needs a defined lifetime and something that enforces it.
The trap is that systems default to keeping everything forever. Disk is cheap, deletes feel risky, and "we might need it later" wins every argument. So the first, unglamorous step is a retention schedule: a table of what personal data you hold, why, and for how long.
| Data | Purpose | Retention |
|---|---|---|
| Account profile | Provide the service | Life of account + 30 days |
| Order history | Legal and tax obligation | 6 to 10 years (jurisdiction dependent) |
| Support messages | Customer service | 2 years |
| Server and access logs | Security and debugging | 30 to 90 days |
| Analytics events | Product analytics | 14 months |
The retention column is a legal decision. The point for engineers is that once it exists, it becomes a spec you can implement and test against.
Enforcing it: automated purging
A retention policy that relies on someone remembering to delete things is not a policy. Automate it:
- A scheduled job (nightly or weekly) that hard-deletes records past their retention window.
- For time-series data (logs, events, analytics), use the storage engine's own TTL or partition dropping rather than row-by-row deletes. Drop the old partition, expire the index, set the bucket lifecycle rule. It is faster and it cannot silently miss rows.
-- example: purge support messages older than 2 years
DELETE FROM support_messages
WHERE created_at < now() - interval '2 years';
Soft delete is not deletion
A common mistake: marking a row deleted_at = now() and calling it done. A soft delete hides data from the application, but the personal data is still there, fully readable in the database. That is fine as an intermediate state (a grace period, an undo window), but it does not satisfy retention or erasure on its own. Something has to hard-delete it eventually.
Duty two: the right to erasure
When a user exercises their right to erasure (the "right to be forgotten"), you generally have 30 days to actually remove their personal data. The difficulty is almost never the main users row. It is that personal data has spread.
By the time a product is real, a single user's data lives in many places:
- the primary database (and its read replicas)
- a search index (Elasticsearch, Algolia, OpenSearch)
- caches (Redis, a CDN)
- object storage (avatars, uploads, exports)
- the analytics pipeline and warehouse
- third-party processors (email, payments, support, error tracking)
- log storage
- backups
Erasure means cascading the delete across all of these, not just the row. Two things make this survivable:
- A data map. You cannot delete what you have not written down. Maintain a list of every system that stores personal data and how to delete from each. This is the same map your retention schedule needs.
- Delegate to your processors. For third parties, you usually do not delete their copy yourself, you call their deletion API or rely on their contractual retention. Stripe, your email provider, your error tracker each have a deletion mechanism. Erasure includes triggering theirs.
A clean implementation is an erasure workflow: one job that fans out deletion to every system on the map, records what it did, and is idempotent so it can be re-run if one downstream call fails.
The part everyone forgets: backups
Here is the question that stops most erasure plans cold. You have deleted the user from production, from the search index, from object storage, from every processor. But last night's database backup still contains them. So does the one from the night before, and every one going back weeks. Have you actually complied?
You cannot realistically open each backup, surgically remove one user, and repackage it. That would defeat the purpose of backups and risk corrupting them. Regulators know this, and the accepted engineering answers are these:
Bounded backup retention. If your backups rotate on a defined schedule (say 30 days) and then are destroyed, the deleted user's data ages out of the backup set within that window. Document this. "Backups are retained for 30 days, after which erased data is permanently gone" is a defensible, common position. The key is that the window is finite and enforced, not "we keep backups forever."
Crypto-shredding. Encrypt each user's personal data with a per-user key. To erase the user, you delete their key. The ciphertext may still sit in old backups, but without the key it is unrecoverable, which for practical and regulatory purposes is deletion. This is the strongest approach and the one to reach for when your backup retention is long or legally required to be.
A restore-and-re-erase procedure. Accept that backups still contain the data, and commit in writing that if you ever restore a backup, you immediately re-run the erasure job for anyone who requested deletion in the meantime. Keep the list of erasure requests so this is possible. Weaker than the first two, but honest and workable.
The wrong answer is to have no answer. "We delete from the database" while unbounded backups quietly retain everyone forever is the gap auditors and breach investigations find first.
Do not forget your logs
Logs are the most overlooked store of personal data. An email address in an error message, an IP in an access log, a full request body captured during debugging: all of it is personal data, subject to the same retention and erasure duties, and scattered across a log system that is designed to be append-only.
You will not run per-user deletes across a log firehose, so handle logs at the source:
- Do not log PII in the first place. Redact emails, tokens, and request bodies before they are written. This is by far the cheapest control.
- Set a short retention. 30 to 90 days covers almost all security and debugging needs. Configure it and let old logs expire automatically.
- Treat "we can search two years of logs" as a liability, not a feature.
A short checklist
- A written retention schedule: every category of personal data, its purpose, its lifetime.
- Automated purging that enforces the schedule (TTLs, partition drops, scheduled jobs).
- A data map: every system that stores personal data and how to delete from each.
- An idempotent erasure workflow that cascades across all of them, including processors.
- A documented, defensible answer for backups: bounded retention, crypto-shredding, or restore-and-re-erase.
- No PII in logs, and short log retention.
None of this is exotic engineering. It is mostly bookkeeping (knowing where the data is) plus a few scheduled jobs. The reason it so often fails is not technical difficulty, it is that nobody owns the map. Write the map down, automate the deletes, and decide your backup story on purpose rather than by accident. That is the difference between a system that is compliant and one that merely looks compliant until someone asks.
Originally published on jguillaumesio.com
Top comments (0)