Three open models, from 17M to 68M parameters, built for a single job: finding personal information in text accurately enough for production and fast enough to run on every request.
Highlights
- 88.3% F1 on Privacy-Bench, a benchmark of human-annotated business documents, with a 68M-parameter model. PII-Tracer (596M) scores 86.5%, GLiNER2-PII (205M) 83.7% and nvidia GLiNER-PII (570M) 66.4%.
- The smallest model, at 17M parameters, is within half a point of PII-Tracer at 1/35 of its size.
- 37% fewer false alarms than PII-Tracer while finding almost as much personal data.
- 12 types of PII, from names and emails to government IDs, secrets and IP addresses.
- Openly available on Hugging Face under Apache-2.0: pii-extractor-68M, pii-extractor-32M and pii-extractor-17M.
Every message sent to an LLM, every document added to a search index and every support ticket written to a log raises the same question: does this contain personal information, and exactly where? The check runs on every request, so its speed and cost set the floor for everything behind it. Yet the strongest PII detectors available today carry hundreds of millions of parameters, and general-purpose LLMs carry far more.
PII Extractor is a family of three compact models purpose-built to extract personal data from the text you send it, and light enough to fine-tune for your own use case. At 68M parameters, it is the most accurate PII detector we tested on real business documents. At 17M, it processes 1,000 tokens in 0.17 seconds on four CPU threads, with no GPU at all.
Why PII detection belongs in a small model
Finding PII is not open-ended generation. A model reads a text once and decides, for every word, whether it belongs to a name, an email address, an account number or nothing at all. The answer is a set of character positions that can be redacted, masked or routed.
That makes it a natural fit for a small encoder. An encoder reads the whole text in both directions at once, so every word is judged in its full context: in "Will will sign the contract on Monday", the first "Will" is a name and the second is not. There is no text to generate and no output to parse, and the cost of a call depends only on the length of the input.
Meet PII Extractor
PII Extractor is fine-tuned from Ettin [6], an open encoder family from Johns Hopkins University with an architecture similar to ModernBERT [7]. The three models are built on Ettin-68M, Ettin-32M and Ettin-17M.
Each model reads up to 8K tokens in a single pass, splits longer texts automatically, and returns every piece of personal data with its exact position and one of 12 types:
| Type | What it covers |
|---|---|
| PERSON | Names, including first names alone, initials and nicknames |
| Email addresses, including written-out forms | |
| PHONE | Phone and fax numbers |
| ADDRESS | Street addresses; a city or country on its own is not labelled |
| DATE | Dates of birth |
| URL | Web addresses, with or without https://
|
| ACCOUNT_NUMBER | Bank, card, IBAN and customer account numbers |
| GOV_ID | Social security, passport, national ID, tax ID and driving licence numbers |
| USERNAME | Usernames and handles |
| SECRET | Passwords, PINs, access codes, API keys and tokens |
| IP_ADDRESS | IPv4 and IPv6 addresses |
| OTHER_ID | Employee, patient, member and device IDs |
Leading accuracy on business documents
On Privacy-Bench, 656 pages of real business documents annotated by people, PII Extractor-68M scores 88.3% F1, ahead of PII-Tracer at 86.5%, GLiNER2-PII at 83.7% and nvidia GLiNER-PII at 66.4%. PII Extractor-32M also scores above all three, at 18× smaller than PII-Tracer.
| Model | Size | Privacy-Bench | PII-TRACE | GPU, 1,000 tokens | CPU, 1,000 tokens |
|---|---|---|---|---|---|
| PII Extractor-68M | 68M | 88.3% | 82.4% | 23 ms | 1.17 s |
| PII Extractor-32M | 32M | 87.5% | 84.6% | 16 ms | 0.40 s |
| PII Extractor-17M | 17M | 86.0% | 81.2% | 15 ms | 0.17 s |
| PII-Tracer [3] | 596M | 86.5% | 92.6% | 271 ms | 9.15 s |
| GLiNER2-PII [4] | 205M | 83.7% | 62.2% | 199 ms | 3.68 s |
| nvidia GLiNER-PII [8] | 570M | 66.4% | 75.1% | 429 ms | 9.45 s |
Emails and phone numbers are close to solved for every model except nvidia GLiNER-PII, which marks the names inside an email address such as megan.donovan@lilly.com as two names and scores 34% on emails. Names sit at 86–88% for both PII Extractor and PII-Tracer. Addresses are where PII Extractor pulls furthest ahead: 63% for the 68M model, against 51% for PII-Tracer and 8% for GLiNER2-PII. Account numbers and links remain hard for every model, and appear only 167 and 21 times in the benchmark.
n/a: PII-Tracer does not support usernames, so it is not scored on them. Dates and secrets are not shown because Privacy-Bench does not annotate them.
On PII-TRACE, a benchmark of multi-turn conversations released alongside PII-Tracer, PII-Tracer leads with 92.6% against 84.6% for PII Extractor-32M, mostly because that benchmark also labels meeting dates and reference codes, which PII Extractor deliberately leaves unmarked.
Fewer false alarms, cleaner redaction
An F1 score combines two different failures. A missed value leaks personal data. A false alarm hides ordinary text and makes a document harder to use, which is why over-eager detectors tend to get switched off.
Out of 8,490 PII values in Privacy-Bench, PII Extractor-68M and PII-Tracer miss a similar number (613 and 567). The difference is in what they flag that isn't PII: 342 false alarms for PII Extractor-68M against 546 for PII-Tracer, 37% fewer. GLiNER2-PII makes the fewest false alarms but misses about one value in five. nvidia GLiNER-PII misses the fewest values (538) but raises 3,194 false alarms, more than nine times as many as PII Extractor-68M.
Fast enough for every request, even without a GPU
We timed every model on the same 50 text samples at each of three lengths, 1,000, 2,000 and 3,000 tokens, and report the median time per sample.
On a T4 GPU, all three PII Extractor models stay under 70 milliseconds per sample, even at 3,000 tokens. PII-Tracer takes 271 milliseconds at 1,000 tokens and almost 2 seconds at 3,000. nvidia GLiNER-PII is the slowest at 1,000 tokens, at 429 milliseconds, and takes 1.6 seconds at 3,000.
Without a GPU, PII Extractor-17M takes 0.17 seconds at 1,000 tokens and 0.48 seconds at 3,000, against 9.15 and 37.37 seconds for PII-Tracer. That makes it practical on ordinary application servers.
Which size to choose
- PII Extractor-68M for the highest accuracy, on a GPU.
- PII Extractor-32M for the best balance of accuracy and speed.
- PII Extractor-17M for CPU-only servers and the lowest cost.
How we evaluated
We ran every model ourselves, on the same texts, with the same scoring. Privacy-Bench [1] contains 656 pages of Word and PDF business documents with 8,490 PII values annotated by people. PII-TRACE [2] contains 500 multi-turn conversations with 2,442 annotated values.
A prediction counts as correct only when its start, end and type all match the annotation. Each benchmark's types were mapped to ours as below, and every model was scored on the types that both the benchmark and the model support.
| PII Extractor | Privacy-Bench | PII-TRACE |
|---|---|---|
| PERSON | NAME_GIVEN, NAME_FAMILY | private_person |
| EMAIL_ADDRESS | private_email | |
| PHONE | PHONE_NUMBER | private_phone |
| ADDRESS | LOCATION_ADDRESS | private_address |
| URL | URL | private_url |
| ACCOUNT_NUMBER | ACCOUNT_NUMBER | account_number |
| USERNAME | USERNAME | Not annotated |
| DATE | Not annotated | private_date |
| SECRET | Not annotated | secret |
GOV_ID, IP_ADDRESS and OTHER_ID are not annotated in either benchmark, so this evaluation does not cover them.
Latency is the median of 50 text samples per length, processed one at a time on a Tesla T4 GPU and on four CPU threads. PII-TRACE scores use its public subset, so they are not comparable with the official test results in the PII-Tracer paper [2].
Limitations
- English only. The models were trained and tested on English text.
- Our definitions of PII. Sensitive attributes such as health, religion or ethnicity are not detected in this version.
- Rare types. Usernames, account numbers and links are found less reliably than names, emails and phones.
What's next
We're bringing dynamic labels at run time to our super-efficient, lightweight models: describe the PII types you need when you call the model, in the style of GLiNER [5], and get them back with no retraining, at a size that still runs on every request.
Try PII Extractor
All three models are on Hugging Face under the Apache-2.0 licence: pii-extractor-68M, pii-extractor-32M and pii-extractor-17M.
Need PII detection tuned to your own definitions, documents or languages? Get in touch with our team at cobuildx.ai/contact.
References
Benchmarks
- TonicAI. Privacy-Bench (human-annotated documents). https://huggingface.co/datasets/TonicAI/Privacy-Bench
- K. Zhang et al. PII-TRACE: A Benchmark for Context-Aware PII Detection in Multi-Turn LLM Conversations. arXiv:2609.22200, 2026. https://huggingface.co/datasets/perplexity-ai/PII-TRACE
Models and methods
- Perplexity AI. PII-Tracer. https://huggingface.co/perplexity-ai/PII-Tracer
- Fastino. GLiNER2-PII. https://huggingface.co/fastino/gliner2-privacy-filter-PII-multi
- U. Zaratiana et al. GLiNER: Generalist Model for Named Entity Recognition using Bidirectional Transformer. NAACL 2024. arXiv:2311.08526.
- O. Weller et al. Seq vs Seq: An Open Suite of Paired Encoders and Decoders (Ettin). arXiv:2507.11412, 2025.
- B. Warner et al. Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inference (ModernBERT). arXiv:2412.13663, 2024.
Other baselines
- NVIDIA. gliner-PII. https://huggingface.co/nvidia/gliner-PII
Originally published at cobuildx.ai.
Top comments (0)