How we built a PII masking layer for LLM APIs — local detection, reversible tokens, one line to integrate

Dhroov Gupta — Mon, 25 May 2026 17:23:59 +0000

If you're building LLM features on top of OpenAI or Anthropic, you're almost certainly sending raw user data to a third-party model provider. Names, emails, phone numbers, tax IDs, health records — whatever your users type, it goes straight to the API.

Here's the uncomfortable part: every attempt to fix this problem seems to make it worse. The most obvious fix — sending your text to a cloud anonymisation service first — means you're solving a data privacy problem by sending your sensitive data to another third party.

I was talking to a healthtech team recently that had been blocked from using GPT-4 for clinical notes for months. Not because the engineers didn't want to — they did. Legal wouldn't sign off because every API call meant patient data leaving their infrastructure. The problem wasn't capability. It was the missing privacy boundary between their data and the LLM.

Armos is that boundary. A local detection and masking layer that sits between your application and the LLM API — PII never leaves your server, and real values are restored in the response automatically.

This is how it works under the hood.

The problem with the obvious approaches

Option 1: Regex scrubbing

Fast to write, breaks constantly. Email regexes miss edge cases. Names are impossible. You end up with a pile of patterns that need constant maintenance and still let things through.

Option 2: Send everything to a cloud anonymisation API

Same problem, different server. You haven't kept the data in-house — you've just added a hop.

Option 3: Build it yourself with Presidio

Microsoft's Presidio is excellent — it's what powers Armos's detection. But it's detection only. You still need to build the masking layer, the vault, the de-masking logic, and wire it into your SDK calls. That's a week of work for a first pass and months of edge cases.

What Armos does instead

Three steps, all local:

1. Detect

Presidio + spaCy runs on the text before it leaves your process. No network call. No data sent anywhere during detection.

2. Mask with reversible tokens

Detected entities are replaced with deterministic tokens:

"Patient John Smith, Aadhaar 2345 6789 0123"
→
"Patient [PII:NAME:c4587843], Aadhaar [PII:AADHAAR:473adcf3]"

The token format encodes the entity type and a hash of the original value. Same value always maps to the same token — so if "John Smith" appears twice, it gets the same token both times, and the LLM can reason about it consistently.

3. Restore

After the LLM responds, the library scans the output for tokens and swaps them back. Your application receives the original text. The model never saw the real values.

The token vault

Tokens need to map back to real values. The library keeps a vault — a simple key-value store — inside the process by default, with an optional Redis backend for cross-process persistence.

# In-memory (default)
client = ArmosOpenAI(OpenAI())

# Redis-backed — tokens survive across requests and processes
client = ArmosOpenAI(OpenAI(), store="redis", redis_url="redis://...")

The vault never leaves your infrastructure. Armos has no server. There's no telemetry, no cloud component.

The integration

This is the entire change to existing code:

# Before
from openai import OpenAI
client = OpenAI()

# After
from openai import OpenAI
from armos import ArmosOpenAI
client = ArmosOpenAI(OpenAI())

Everything downstream works identically — same method signatures, same response objects. The masking and de-masking happen invisibly inside the privacy layer.

What gets detected

10 entity types out of the box:

Names — via spaCy NER (en_core_web_lg)
Email, phone, credit card, IP — Presidio built-ins
Aadhaar, PAN — custom regex recognisers (Indian identifiers that no existing tool handles reliably)
SSN, IBAN — Presidio built-ins with checksum validation
API keys — custom pattern recogniser for OpenAI, AWS, GitHub key formats

Accuracy

I ran a 1,000-sample benchmark across all entity types:

Entity	Accuracy
Email	100%
Aadhaar	100%
PAN	100%
SSN	100%
IBAN	100%
Credit card	100%
Phone	100%
API keys	100%
IP address	99.8%
Person name	96.4%

The 3.6% miss rate on names is entirely Indian names — en_core_web_lg was trained predominantly on Western text. I'm working on a supplemental approach for this.

What's next

Streaming support (stream=True currently passes through unmasked)
Async clients (AsyncOpenAI, AsyncAnthropic)
LangChain and LlamaIndex integrations

The library is early and I'm actively looking for teams using LLMs on sensitive data who want to trial it and shape where it goes.

GitHub: github.com/armos-ai/armos-python
Docs: armos.dev

pip install armos

If you're hitting this problem or have thoughts on the approach, I'd love to hear from you in the comments.

Distributed Rate Limiter

Dhroov Gupta — Fri, 28 Apr 2023 22:03:04 +0000

Introduction

I recently created a rate limiter library for the distributed systems that can be used to control and limit the number of requests made within a specific period of time.

The Idea

The idea behind the library is to create a token bucket that replenishes tokens at a certain rate. Each time a request is made, the library checks if there are enough tokens available in the bucket. If there are, it removes a token and allows the request. If not, it rejects the request.

The Solution

I implemented the token bucket algorithm using Redis as a distributed storage system. The library, called dist-rate, takes an options object that includes the number of tokens in the token bucket, the duration for which the tokens are replenished, and an instance of the Redis client to use for distributed locking and storage.

Usage

To use the dist-rate-limiter library in your project, simply install it via npm:

npm install dist-rate-limiter

To use the package, create an instance of the DistRate class and call the execute() method with a unique ID for each request. The method returns a boolean indicating whether the request should be allowed or not.

import IORedis from "ioredis";
import { DistRate } from "distrate";

const redisClient = new IORedis.Cluster([...]);

const rateLimiter = new DistRate({
  tokens: 10,
  duration: 60,
  redisClient,
});

const allowed = await rateLimiter.execute("user123");

You can then use the rateLimiter instance to control and limit the number of requests made to your API or server.

Links

The dist-rate library is available on both GitHub and npm. You can find the GitHub repository here:

Github - https://github.com/Dhroov7/distRate
NPM - https://www.npmjs.com/package/dist-rate

HacktoberFest 2019

Dhroov Gupta — Wed, 25 Sep 2019 18:52:08 +0000

Hey Everyone,
As we all know HacktoberFest 2019 is starting in 6 days.
So, for new open source developers i've created a repository having easy and beginner friendly issues, so that every new comer can contribute and earn the T-shirt from DigitalOcean...isn't that great!! :)

Link to the repository:
Click here

DEV Community: Dhroov Gupta