DEV Community

Lv Shao Mu
Lv Shao Mu

Posted on Originally published at github.com

I masked a dataset for an LLM and kept every checksum valid — here's the approach

#python #data #privacy #tutorial

Sharing data with an LLM is a data-leak problem with an extra twist: the model is a reader, not a database. If you scrub a spreadsheet with ***, the model has nothing to reason about. If you replace every cell with random noise, joins break and the analysis collapses.

This post walks through the middle path: generate fake values that keep the original shape, so the file still imports, validates, and joins — without any real value surviving.

What breaks with regex-replace

Take a customer export with a phone, a Chinese resident ID, and a bank card:

name,phone,id_no,bank,email
张三,13812345678,11010519491231002X,6222025644974749,zhangsan@example.com
Enter fullscreen mode Exit fullscreen mode

A naive scrubber produces:

name,phone,id_no,bank,email
***,***********,***,***,***
Enter fullscreen mode Exit fullscreen mode

Four failures at once:

  1. Importers reject the row. Phone columns have length checks.
  2. Checksums die. A Chinese ID's last digit is an ISO 7064 MOD 11-2 check character; a bank card ends with a Luhn digit. Random digits fail both.
  3. Joins break. If two rows referenced the same customer by phone, they now reference two different asterisk blobs.
  4. Reruns disagree. Fuzz each run separately and two environments get two different datasets.

The fix: fake values that pass validation

Mask by generating a deterministic fake per kind:

name,phone,id_no,bank,email
张三,13893525401,494626517976288038,6222025644974749,u72765aad12@example.com
Enter fullscreen mode Exit fullscreen mode

Wait — the bank card looks identical. That's the point. The fake preserves:

  • Phone: 11 digits, 1[3-9] prefix.
  • CN ID: 17 digits plus a valid ISO 7064 check character. 198436564585937260 passes sum(digit[i] * W[i]) % 11 against the same table the importer uses.
  • Bank card: same length, same BIN prefix (622202), and a Luhn-valid check digit — the payment sandbox accepts it.
  • Email: hashed local part, same domain, so you can still count distinct senders per domain.
  • IPv4: mapped into 10.x.x.x private space.

And the whole mapping is keyed:

fake = HMAC-SHA256(secret, "kind:value") → truncated into the value's own alphabet
Enter fullscreen mode Exit fullscreen mode

Same secret + same input → same output. Different secret → different output. Without the secret, the published file alone doesn't let you recover the mapping.

Why checksum validation matters more than it sounds

The most common complaint about PII scrubbers is false positives. \d{18} matches order numbers. \d{16} matches SKUs. The difference between "88 real findings" and "400 red herrings" is whether you validate before flagging:

  • a real CN ID satisfies its check character;
  • a real bank card satisfies Luhn;
  • an 18-digit order number satisfies neither.

Validate first, mask only what passes. Skip validation only with an explicit --loose flag, which exists to catch forged numbers — and says so in its output.

CLI walkthrough

Single file, zero dependencies beyond the Python standard library:

git clone https://github.com/lvshaomu/piimask
cd piimask

# one file, with a report
python piimask.py --in customers.csv --out clean/ --report

# a folder, recursively
python piimask.py --in exports/ --out clean/

# keep the shape instead of generating fakes
python piimask.py --in dump.json --out clean/ --policy mask
Enter fullscreen mode Exit fullscreen mode

Output:

     8  sample.csv -> clean/sample.masked.csv

piimask report
========================================
policy        : fake
detectors     : email, id_card_cn, phone_cn, bank_card, ipv4
replacements  : 8
by kind       : email=2, id_card_cn=2, ipv4=2, phone_cn=2
Enter fullscreen mode Exit fullscreen mode

Exit codes matter for CI: 0 ran fine, 2 means --strict found a file with zero replacements. A silent no-op is a failure, not a success.

Tests

The repo ships 18 assertions, including the ones that matter:

PASS  构造的身份证通过 ISO 7064
PASS  掩码后身份证仍通过校验: 198436564585937260
PASS  掩码后银行卡通过 Luhn: 6222025644974749
PASS  掩码后银行卡保留 BIN 前缀
PASS  同 secret 重跑结果一致
PASS  不同 secret 输出不同
PASS  同值两次掩码一致（可 JOIN）
PASS  CLI 掩码后原身份证不再出现
PASS  JSON 结构保留（键仍在）
Enter fullscreen mode Exit fullscreen mode

The two-sided property — "originals gone, fakes still valid" — is what makes a masked file usable downstream.

Where this fits in an LLM workflow

A practical pattern for sharing data with a model:

  1. Mask first with a deterministic keyed mapping (--secret-file key).
  2. Send the masked file — the model sees realistic structure.
  3. Reason about joins — the same customer maps to the same fake everywhere, so "count orders per customer" still works.
  4. Restore only if you must — keep the secret; without it, the output is not reversible.

The failure mode this avoids: scrubbing to *** and then wondering why the model can't tell which rows belong to the same account.

Limits

  • The detectors are shape + checksum based. Free-text Chinese names are deliberately not guessed, because that's exactly how false positives happen. Column headers drive name detection in the larger variant.
  • .xlsx and .docx need a slightly bigger tool; this repo stays text/CSV/JSON on purpose, standard library only.
  • Deterministic ≠ anonymous against someone who has both the file and the secret. Keep the secret with the restore duty.

The repo: lvshaomu/piimask — MIT, single file, no dependencies. A fuller variant with .xlsx/.docx and four policies lives at csvtriage.com/p/datamask.html.

Top comments (0)