Every time Article 22 comes up, the same argument starts. Lawyers debate what "meaningful information about the logic involved" is supposed to mean, and engineers start reaching for SHAP plots.
I think both camps are overcomplicating it. I've been on the receiving end of this requirement — fintech, hiring tools, the usual suspects — and what actually gets you in trouble is never "the model is a black box." It's "we can't find the decision you're asking about."
Here's the article in one breath: if you make a decision about someone purely automatically, and it has legal or similarly significant effects on them, they're entitled to understand the logic behind it and to contest it. Okay. Now translate that into engineering tasks:
Can you retrieve the one decision? Not the model, not the endpoint. The decision about this person on this date. I've watched teams spend a full day just figuring out which table it lives in.
Can you reconstruct the inputs? If your features come from a warehouse table that has since been rebuilt, "we don't know anymore" is not an answer a regulator accepts. You need the snapshot of what the system actually saw at the time.
Can you explain it at the right altitude? Nobody wants your weights. The person contesting wants: "You scored 0.31 against a threshold of 0.5. The main reasons you fell short: income ratio and account age." Two sentences. Comprehensible to a human who has never heard of gradient descent.
Can a human actually contest it — and is the contest recorded? The appeal becomes part of that decision's history. If the review happens over email and dies there, you've lost the thread.
Is any of this tamper-proof? The one everybody forgets. The moment a complaint lands, your logs stop being operational data and become contested evidence. If someone with DB access can quietly edit a row from six months ago, items 1–4 are worthless. Append-only, with verifiable integrity. That's the ask, and it's boring on purpose.
Notice what's not on this list: interpretability research. No attention-visualization paper is going to save you under Article 22. What saves you is treating each automated decision as a first-class, immutable event — written once, inputs snapshotted, explanation recorded, and impossible to touch afterward.
The hard part isn't technical. It's that most pipelines were built to serve predictions, not to testify about them. Retrofitting auditability onto a live system is miserable. I've done it. Budget more than you think.
So here's the test: pick one automated decision your system made last week. Could you produce items 1–3 for it today, in a form a regulator would accept?
If your honest answer is "probably, let me check with the data team" — that check is the whole game. Start there.
Top comments (0)