If you have ever written a PII detector for European documents, you have probably started with regular expressions. Eleven digits for a Polish PESEL, sixteen alphanumerics for an Italian codice fiscale, a country code and up to 30 characters for an IBAN. The patterns are easy. The problem is that they match far too much: invoice numbers, order IDs, phone numbers and timestamps all fit \d{11}.
Most national identifiers carry a check digit precisely so that typos can be caught. That same property makes them a strong signal for detection: a random 11-digit number passes the PESEL checksum only about one time in ten, and an IBAN-shaped string passes MOD 97 roughly one time in ninety-seven. Combining the pattern with the checksum turns a noisy match into a confident one, even when the number appears with no label next to it.
Below are five algorithms in plain Python, each tested against python-stdnum, which is what you should use in production.
IBAN: ISO 13616, MOD 97-10
Move the first four characters to the end, turn letters into numbers (A=10 to Z=35), and the result modulo 97 must be 1.
def iban_ok(value: str) -> bool:
s = value.replace(" ", "").upper()
if len(s) < 15 or not s[:2].isalpha() or not s[2:4].isdigit():
return False
rearranged = s[4:] + s[:4]
digits = "".join(str(int(c, 36)) for c in rearranged)
return int(digits) % 97 == 1
int(c, 36) is a neat shortcut: it maps 0-9 to themselves and A-Z to 10 to 35. A complete validator also checks the length expected for each country (27 for Italy, 22 for Germany, 19 for Slovenia).
Italian codice fiscale: odd and even positions
The last character is a letter computed from the first fifteen. Characters in odd positions (1st, 3rd, 5th...) are converted with an irregular table; characters in even positions use their plain value (digits as numbers, A=0 to Z=25). The sum modulo 26 gives the check letter.
ODD = {**{str(i): v for i, v in enumerate([1, 0, 5, 7, 9, 13, 15, 17, 19, 21])},
**dict(zip("ABCDEFGHIJKLMNOPQRSTUVWXYZ",
[1, 0, 5, 7, 9, 13, 15, 17, 19, 21, 2, 4, 18, 20,
11, 3, 6, 8, 12, 14, 16, 10, 22, 25, 24, 23]))}
def codice_fiscale_ok(value: str) -> bool:
s = value.upper()
if len(s) != 16 or not s.isalnum():
return False
total = 0
for i, c in enumerate(s[:15]):
if i % 2 == 0: # odd position, 1-based
total += ODD[c]
else:
total += int(c) if c.isdigit() else ord(c) - ord("A")
return chr(ord("A") + total % 26) == s[15]
Watch out for homocodes: when two people would get the same code, some digits are replaced by letters (L for 0, M for 1 and so on). The check above still works because it treats letters and digits uniformly, but a regex that expects digits in fixed positions will miss them.
German Steuer-IdNr: ISO 7064 MOD 11,10
The German tax ID uses a recursive scheme rather than a weighted sum:
def steuer_id_ok(value: str) -> bool:
if len(value) != 11 or not value.isdigit() or value[0] == "0":
return False
product = 10
for d in value[:10]:
total = (int(d) + product) % 10 or 10
product = (total * 2) % 11
return (11 - product) % 10 == int(value[10])
The real IdNr also has rules on repeated digits in the first ten positions, which this short version leaves out. In a test against python-stdnum on 50,000 random numbers, every ID that stdnum accepted also passed this function.
French NIR (numéro de sécurité sociale): 97 minus the remainder
Thirteen digits plus a two-digit key, where the key is 97 - (number mod 97). Corsica is the trap: departments 2A and 2B are replaced by 19 and 18 before computing.
def nir_ok(value: str) -> bool:
s = value.replace(" ", "").upper()
if len(s) != 15:
return False
body, key = s[:13], s[13:]
body = body.replace("2A", "19").replace("2B", "18")
if not (body.isdigit() and key.isdigit()):
return False
return 97 - int(body) % 97 == int(key)
Polish PESEL: weights 1, 3, 7, 9
def pesel_ok(value: str) -> bool:
if len(value) != 11 or not value.isdigit():
return False
weights = [1, 3, 7, 9, 1, 3, 7, 9, 1, 3]
total = sum(int(d) * w for d, w in zip(value[:10], weights))
return (10 - total % 10) % 10 == int(value[10])
PESEL also encodes the birth date, with the century folded into the month (add 20 for the 2000s). python-stdnum checks the date too, and in our test it rejected about four out of five random numbers that passed the checksum alone. For detection that difference matters: the more structure you verify, the fewer false positives.
Putting it into a detector
The pattern finds candidates, the checksum confirms them, and context only adjusts the score:
- Run a permissive pattern per identifier type over the text, tolerating spaces and dashes.
- Normalise each candidate and run the checksum. A pass is a strong match even without a label such as "PESEL:" nearby.
- A fail is not always a discard: OCR errors and typos are common in scanned documents, so a labelled candidate that fails can still be flagged for review.
This is the approach we use in Redactic, a redaction tool for European documents, alongside machine learning models for names and addresses that have no checksum to lean on. Identifiers with check digits are the one part of PII detection where you can be close to certain, so it is worth getting them right.
If you work with scanned PDFs, remember that detection is only half the job: a black box drawn over text often hides nothing, because the characters stay in the file underneath.
Top comments (0)