DEV Community

Cover image for Building an Air-Gapped, <25ms Local Privacy Gateway for LLMs (HIPAA, GDPR etc)
Vishal Prajapati
Vishal Prajapati

Posted on

Building an Air-Gapped, <25ms Local Privacy Gateway for LLMs (HIPAA, GDPR etc)

#security #ai #python

Building an Air-Gapped, <25ms Local Privacy Gateway for LLMs (HIPAA, GDPR, SOC 2)

Over the past year, nearly every enterprise engineering team has attempted to build with frontier models like OpenAI, Claude, or Azure OpenAI.

Yet, a staggering number of these projects never make it to production. Why?

Compliance.

The moment customer names, Social Security Numbers, credit card numbers, or protected health information (PHI) enter the prompt pipeline, legal and compliance teams hit the emergency brakes:

"Under HIPAA, GDPR, and SOC 2, we are strictly prohibited from transmitting unmasked customer data outside our private network perimeter."

Many commercial "AI privacy solutions" attempt to solve this by asking you to route your raw data through their cloud proxy. But replacing one third-party risk with another isn't real enterprise security.

In this article, I will break down the architecture and implementation of PII Guardrail Studio — a 100% local, air-gapped reverse privacy proxy and encrypted token vault designed to run entirely inside your private VPC.

🛡️ Core Architectural Principles

When designing a privacy gateway for production LLM pipelines, three constraints are non-negotiable:

1. 100% Offline & Air-Gapped (Zero Telemetry)

The gateway must operate without phoning home to any external license server, analytics endpoint, or cloud dependency. If deployed in an isolated Kubernetes pod with network_mode: none, it must function with zero degradation.

2. Sub-25ms In-Memory Detection Latency

Running heavy NLP models locally often introduces hundreds of milliseconds of overhead. By leveraging a C-optimized, regex-compiled boundary engine with 30+ entity recognizers, detection and substitution run in under 25ms on standard commodity hardware.

3. Bi-Directional Reversible Tokenization

Masking PII is only half the battle. If a user asks:

"What medication should be prescribed to Patient Robert Vance?"

The model cannot answer accurately if the patient's identity is completely stripped. Instead, the proxy performs cryptographic token substitution:

[Raw Sensitive Prompt]
       │
       ▼
[Local Privacy Gateway (<25ms)]
       │ (Scrubbed with <PERSON_1>, <SSN_1>)
       ▼
[External LLM API (OpenAI/Claude)]
       │ (Response contains <PERSON_1>)
       ▼
[Local Gateway Token Vault]
       │ (Restores <PERSON_1> ➔ "Robert Vance")
       ▼
[Final User Response]
Enter fullscreen mode Exit fullscreen mode

🔐 The Encrypted Token Vault (SQLCipher AES-256)

When the gateway maps "Robert Vance" to , where is that mapping stored?

In PII Guardrail Studio, mappings are never held in plain text. They are committed to a local SQLCipher database encrypted at rest with AES-256, verified via SHA-256 digests, and bound offline using Ed25519 node-locking.

Even if a malicious actor accesses the physical disk or container volume, the mapping table is unreadable without the node key.

🚀 Quickstart: Running It Locally in 30 Seconds

The gateway can be deployed in two primary ways:

Option A: Via pip (Python 3.10+) 

# Install the official PyPI package
pip install piiguardrails

# Boot the engine and launch the Studio UI
piiguardrails
Enter fullscreen mode Exit fullscreen mode

This immediately initializes the encrypted vault and launches the interactive dashboard at http://localhost:8000.

Option B: Via Docker (Production VPC & Kubernetes) 

docker run -d -p 8000:8000 \
  -v $(pwd)/data:/app/data \
  --name pii-guardrail-studio \
  piiguardrails/enterprise-pii-guardrail:latest
Enter fullscreen mode Exit fullscreen mode

💻 Code Example: Intercepting Prompts in Python

Once the gateway is running at localhost:8000, you can integrate it into any existing Python pipeline using standard httpx or requests:

import httpx

# 1. Raw prompt containing sensitive PII/PHI
raw_prompt = """
Patient Sarah Lin (DOB: 1984-06-12, SSN: 394-20-8192) was admitted to St. Jude Memorial.
Contact her at slin@stjude-health.org regarding medical charts.
"""

# 2. Intercept and mask before calling external APIs
mask_response = httpx.post("http://localhost:8000/mask", json={
    "text": raw_prompt
})

masked_data = mask_response.json()
print("Masked Prompt for OpenAI/Claude:")
print(masked_data["masked_text"])
Enter fullscreen mode Exit fullscreen mode

Output: 

Patient <PERSON_1> (DOB: <DOB_1>, SSN: <SSN_1>) was admitted to <HOSPITAL_1>.
Contact her at <EMAIL_1> regarding medical charts.
Enter fullscreen mode Exit fullscreen mode

Unmasking the Model Response:

When the model returns its completion containing , simply pass it back to the local unmask endpoint:

llm_reply = "Follow up with <PERSON_1> regarding dietary restrictions."

unmask_response = httpx.post("http://localhost:8000/unmask", json={
    "text": llm_reply
})

print("Restored Response:")
print(unmask_response.json()["unmasked_text"])
# Output: "Follow up with Sarah Lin regarding dietary restrictions."
Enter fullscreen mode Exit fullscreen mode

🎁 Community Launch Gift

To celebrate the v2.0 release, you can grab a Free 6-Month Enterprise Evaluation Key (valid through March 31, 2027) directly on the homepage. It unlocks unlimited request throughput, unrestricted payload size, and all 30+ entity recognizers.

🔗 Resources & Getting Involved

If you are building privacy-sensitive LLM applications, try running it locally and let me know your thoughts on the detection engine and roadmap!

Top comments (0)