<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Thang Ngo</title>
    <description>The latest articles on DEV Community by Thang Ngo (@thang_ngo).</description>
    <link>https://dev.to/thang_ngo</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F1718882%2Fe1afbbf6-72ef-4224-af14-781c5ff8b30c.jpg</url>
      <title>DEV Community: Thang Ngo</title>
      <link>https://dev.to/thang_ngo</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/thang_ngo"/>
    <language>en</language>
    <item>
      <title>TRUSTED PETS: a local pipeline that protects data while keeping it useful</title>
      <dc:creator>Thang Ngo</dc:creator>
      <pubDate>Tue, 17 Nov 2026 21:53:48 +0000</pubDate>
      <link>https://dev.to/thang_ngo/trusted-pets-a-local-pipeline-that-protects-data-while-keeping-it-useful-414c</link>
      <guid>https://dev.to/thang_ngo/trusted-pets-a-local-pipeline-that-protects-data-while-keeping-it-useful-414c</guid>
      <description>&lt;p&gt;Any team that shares data hits the same problem. If you remove too little, people can be re-identified. If you remove too much, the data is no longer worth sharing.&lt;/p&gt;

&lt;p&gt;At &lt;strong&gt;Since AI Hackathon 2026&lt;/strong&gt; in Turku we took on the &lt;strong&gt;TrustED PETs Transformer&lt;/strong&gt; challenge, whose brief was &lt;em&gt;protect the data, keep its value&lt;/em&gt;. We got nine sensitive synthetic datasets (health, mobility, finance, cybersecurity, customer service and more), 27 sharing scenarios, and an evaluator that runs real re-identification attacks on whatever you release.&lt;/p&gt;

&lt;p&gt;Our answer is &lt;strong&gt;TRUSTED PETS&lt;/strong&gt;, a privacy pipeline that runs entirely on your own machine.&lt;/p&gt;

&lt;p&gt;👉 &lt;strong&gt;&lt;a href="https://poserdungeon-portfolio.vercel.app/work/trusted-pets-privacy-pipeline" rel="noopener noreferrer"&gt;Read the full project write-up&lt;/a&gt;&lt;/strong&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  The pipeline
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;Detect → Select → Search → Transform → Evaluate&lt;/strong&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Detect&lt;/strong&gt; labels every column with a privacy role, a subtype, a GDPR Art. 9 flag and a confidence score. It also finds PII spans in free text and works out how tables link together, including identifiers that encode other attributes. A Finnish HETU, for example, encodes birth date and sex.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Select&lt;/strong&gt; combines those labels with a sharing scenario and makes a rule-based plan with one privacy-enhancing technique per column.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Search&lt;/strong&gt; coarsens that plan step by step until the k-anonymity, l-diversity and t-closeness targets are met &lt;em&gt;and&lt;/em&gt; the scenario's utility tasks still work.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Transform&lt;/strong&gt; writes the protected files, loads them back and re-validates them. It publishes only output that passes every check.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Evaluate&lt;/strong&gt; scores the release with the organisers' &lt;code&gt;pets-eval&lt;/code&gt; toolkit and compares it with their baselines.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Results that beat the baselines
&lt;/h2&gt;

&lt;p&gt;The detector is tested leave-one-dataset-out, with column and table names hidden. That is the honest test, because names give a lot away.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;&lt;/th&gt;
&lt;th&gt;Column labels, macro-F1&lt;/th&gt;
&lt;th&gt;Free-text PII, F2&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Organisers' baselines&lt;/td&gt;
&lt;td&gt;0.401&lt;/td&gt;
&lt;td&gt;0.184&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;TRUSTED PETS (v3)&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;0.774&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;0.998&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Recall on names, phone numbers, addresses and national IDs in free text is &lt;strong&gt;0.999&lt;/strong&gt;. The health scenario asks for 0.95.&lt;/p&gt;

&lt;p&gt;On the full customer-service dataset, the pipeline runs end to end to a &lt;strong&gt;validated&lt;/strong&gt; release. Residual national IDs, payment cards, IBANs, phone numbers and emails are all &lt;strong&gt;0%&lt;/strong&gt;, and the key utility tasks score close to 1.0.&lt;/p&gt;

&lt;h2&gt;
  
  
  Things we cared about
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Fully local.&lt;/strong&gt; No data leaves the machine. There is an optional local 27B decision model for columns the rules can't settle, running on Apple Silicon via MLX. Without it, the rule-based and ML tiers still run.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;No cheating on detection.&lt;/strong&gt; A &lt;code&gt;--guard&lt;/code&gt; mode refuses any read of gold labels, ground truth or holdout data during protection. Only the final, report-only scorecard may read them.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Provenance everywhere.&lt;/strong&gt; Every plan records digests of the inputs, the detector and its artifacts. Transform refuses to run if anything changed since the plan was made.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Honest about failure.&lt;/strong&gt; If the targets can't be met, the run says so (&lt;code&gt;target_unmet&lt;/code&gt;, &lt;code&gt;review_required&lt;/code&gt;) and does not quietly lower the bar. On the health data, for example, it stops for a human to review some columns instead of guessing.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;A UI like a CI run.&lt;/strong&gt; A React dashboard shows each stage as a step: green for passed, red for failed, yellow when it waits for a reviewer. You can open each step to see labels, the chosen technique per column, risk targets against what was achieved, and the final scorecard.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Tech stack
&lt;/h2&gt;

&lt;p&gt;Python (pandas, numpy, scikit-learn) for the pipeline. React 19 with TypeScript, Vite, Tailwind CSS 4 and Ant Design 6 for the UI. A local MLX model as the optional LLM tier.&lt;/p&gt;




&lt;p&gt;The project page has the architecture, screenshots and the full story of how we got there:&lt;/p&gt;

&lt;h3&gt;
  
  
  👉 &lt;a href="https://poserdungeon-portfolio.vercel.app/work/trusted-pets-privacy-pipeline" rel="noopener noreferrer"&gt;TRUSTED PETS: privacy pipeline (project page)&lt;/a&gt;
&lt;/h3&gt;

&lt;p&gt;&lt;em&gt;All data in this project is synthetic.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>privacy</category>
      <category>python</category>
      <category>datascience</category>
      <category>hackathon</category>
    </item>
  </channel>
</rss>
