DEV Community

Cover image for pg_anon caught 1 of my 8 PII columns. My schema isn't in English.
Efrain Garay
Efrain Garay

Posted on Originally published at efraingaray.com

pg_anon caught 1 of my 8 PII columns. My schema isn't in English.

pg_anon found 1 of the 8 personal-data columns in my PostgreSQL database. The one it caught was email, and only because "email" is spelled the same in Spanish and English.

The other seven — nombre, apellido, telefono, direccion, fecha_nac, tarjeta_ult4 and rut — walked straight through, unmasked.

pg_anon is the open TantorLabs tool that masks personal data in PostgreSQL: it scans the database, flags the sensitive columns, and dumps a masked copy. Like pg_dump, but covering the sensitive parts on the way out. Exactly what you want before handing a colleague a copy of production. So I fed it a Chilean database and watched it miss almost everything — no error, no warning. It finished successfully and handed me a dump with names and national IDs still in cleartext.

What the scan actually does

Two filters, in order. First it reads each column's name against a set of regexes (^email$, ^phone$, ^ssn$…). To the columns left over, it opens the data and tries patterns on the value (an email's @, a card's 16 digits). Whatever no filter catches passes through.

The rules in the demo meta-dict it ships with are written for English schemas. Mine aren't.

1 of 8

Column Holds Stock rules
email email
nombre first name
apellido surname
telefono phone
direccion address
fecha_nac birth date
tarjeta_ult4 card digits
rut national ID

email got caught by name (it's an English word) and confirmed by its @. Everything else has a Spanish name no stock rule looks for.

The rut is the clearest miss. A RUT looks like 7917183-2: seven or eight digits, a dash, and a mod-11 check digit that can be the letter K. The only national-ID rule pg_anon ships with is ssn. It has no idea what a RUT is — and it won't know a cpf (Brazil), dni (Spain, Argentina), curp (Mexico), nif (Portugal) or aadhaar (India) either. If your schema isn't American, the defaults miss your most sensitive column.

The fix: a few lines of Spanish

You teach it. Column names in your language, plus a content regex for the ID so it's caught even in a column not named rut:

{
  "field": {
    "rules": ["^nombre$", "^apellido$", "^email$", "^telefono$",
              "^rut$", "^direccion$", "^fecha_nac$", "^tarjeta_ult4$"],
  },
  "data_regex": {
    "rules": [
      r"[\w.-]+@[\w-]+\.\w+",   # email
      r"^(\+?56)?9\d{8}$",     # Chilean mobile
      r"^\d{5,8}-[\dkK]$",      # RUT with check digit
    ],
  },
}
Enter fullscreen mode Exit fullscreen mode

With that, recall went from 1 of 8 to 8 of 8, no false positives — it still left the non-personal columns alone. The point isn't "it went up when I wrote the rules" (obviously). It's how cheap the fix was, and that a masking tool ships blind to any schema that isn't English.

I left the full Spanish meta-dict (with the RUT regex) in a public repo so nobody starts from scratch: pg-anon-rules-es.


This is the short version. The full write-up covers what didn't fit: whether foreign keys survive the restore, the trap where a blanket SHA-256 keeps uniqueness but produces "emails" with no @, and how much slower it runs than a plain pg_dump. Numbers, diagrams and a 47-second breakdown: pg_anon found my database's emails, but not the RUT.

What does the national ID look like where you are — and would a stock masking tool catch it? Drop the format (or the regex) in the comments and I'll add it to the repo.

Top comments (0)