DEV Community

Thang Ngo
Thang Ngo

Posted on

TRUSTED PETS: a local pipeline that protects data while keeping it useful

Any team that shares data hits the same problem. If you remove too little, people can be re-identified. If you remove too much, the data is no longer worth sharing.

At Since AI Hackathon 2026 in Turku we took on the TrustED PETs Transformer challenge, whose brief was protect the data, keep its value. We got nine sensitive synthetic datasets (health, mobility, finance, cybersecurity, customer service and more), 27 sharing scenarios, and an evaluator that runs real re-identification attacks on whatever you release.

Our answer is TRUSTED PETS, a privacy pipeline that runs entirely on your own machine.

👉 Read the full project write-up

The pipeline

Detect → Select → Search → Transform → Evaluate

  • Detect labels every column with a privacy role, a subtype, a GDPR Art. 9 flag and a confidence score. It also finds PII spans in free text and works out how tables link together, including identifiers that encode other attributes. A Finnish HETU, for example, encodes birth date and sex.
  • Select combines those labels with a sharing scenario and makes a rule-based plan with one privacy-enhancing technique per column.
  • Search coarsens that plan step by step until the k-anonymity, l-diversity and t-closeness targets are met and the scenario's utility tasks still work.
  • Transform writes the protected files, loads them back and re-validates them. It publishes only output that passes every check.
  • Evaluate scores the release with the organisers' pets-eval toolkit and compares it with their baselines.

Results that beat the baselines

The detector is tested leave-one-dataset-out, with column and table names hidden. That is the honest test, because names give a lot away.

Column labels, macro-F1 Free-text PII, F2
Organisers' baselines 0.401 0.184
TRUSTED PETS (v3) 0.774 0.998

Recall on names, phone numbers, addresses and national IDs in free text is 0.999. The health scenario asks for 0.95.

On the full customer-service dataset, the pipeline runs end to end to a validated release. Residual national IDs, payment cards, IBANs, phone numbers and emails are all 0%, and the key utility tasks score close to 1.0.

Things we cared about

  • Fully local. No data leaves the machine. There is an optional local 27B decision model for columns the rules can't settle, running on Apple Silicon via MLX. Without it, the rule-based and ML tiers still run.
  • No cheating on detection. A --guard mode refuses any read of gold labels, ground truth or holdout data during protection. Only the final, report-only scorecard may read them.
  • Provenance everywhere. Every plan records digests of the inputs, the detector and its artifacts. Transform refuses to run if anything changed since the plan was made.
  • Honest about failure. If the targets can't be met, the run says so (target_unmet, review_required) and does not quietly lower the bar. On the health data, for example, it stops for a human to review some columns instead of guessing.
  • A UI like a CI run. A React dashboard shows each stage as a step: green for passed, red for failed, yellow when it waits for a reviewer. You can open each step to see labels, the chosen technique per column, risk targets against what was achieved, and the final scorecard.

Tech stack

Python (pandas, numpy, scikit-learn) for the pipeline. React 19 with TypeScript, Vite, Tailwind CSS 4 and Ant Design 6 for the UI. A local MLX model as the optional LLM tier.


The project page has the architecture, screenshots and the full story of how we got there:

👉 TRUSTED PETS: privacy pipeline (project page)

All data in this project is synthetic.

Top comments (0)