DEV Community

Serguey Asael Shinder
Serguey Asael Shinder

Posted on

Collect Less Than You Are Allowed To

Storage is cheap,
so the default became keep everything.

Every event.

Every request body.

Every version of every row,
ten years deep,
because deleting takes a decision
and keeping takes none.

That default is going to age badly.

Data you hold is not an asset
until somebody uses it.

Until then it is stock,
and stock carries a cost
that arrives all at once.

A breach does not care
that the records are from 2016.

It exposes them at today's prices.

A subject access request
does not stop at the live table.

It reaches every copy,
including the ones sitting
in an analytics warehouse
that nobody ever mapped.

Due diligence asks what you hold
and on what basis,
and the honest answer
in most companies
is that nobody knows.

Now watch where the rules are going.

Every year the questions
get more specific.

What do you keep.

Why.

For how long.

Where.

Who can read it.

How do you prove you deleted it.

Nothing is moving
in the other direction.

So the future rewards the teams
who can answer quickly,
and answering is far easier
when there is less to describe.

The practical version
is not glamorous.

Give every table a retention period
at the moment you create it,
before it holds anything at all.

Make the deletion a job
that genuinely deletes,
not a flag
that hides a row from one query.

Log the field you need,
not the entire payload
because that was easier to write.

And when somebody wants to collect
a new thing just in case,
ask what decision it will change.

If there is no answer,
you are buying a cost
with no benefit
and a very long tail.

The teams who look prudent
in five years
will not be the ones
who kept the most.

They will be the ones
who can say, on one page,
exactly what they hold
and exactly when it goes.

Collect what you use.

Keep what you must.

Delete the rest on a schedule
nobody had to ask you for.

– Serguey Asael Shinder

Top comments (0)