DEV Community

Cover image for Every LLM Prompt You Send Is Plaintext. Here's How to Fix That Before the EU Makes You.
CloakLLM
CloakLLM

Posted on

Every LLM Prompt You Send Is Plaintext. Here's How to Fix That Before the EU Makes You.

#ai #security #python #javascript

Your LLM calls are unencrypted confessions.

Every time you call litellm.completion() or openai.chat.completions.create(), the provider receives your prompt in full plaintext. Names, emails, SSNs, API keys, medical records - all of it sitting in someone else's logs.

That's been a privacy risk for years. In 5 months, it becomes illegal.

August 2, 2026

The EU AI Act enters enforcement. Article 12 mandates tamper-evident audit logs for AI systems - not console.log(), not a JSON file you append to. Logs that regulators can mathematically verify haven't been altered.

The penalty: up to 7% of global annual revenue.

If you use LLMs and handle EU data, you need:

  1. PII never reaches the provider (or you need explicit consent per entity)
  2. Every AI interaction logged in a verifiable audit trail

Most teams have neither. I built CloakLLM to fix both.

What CloakLLM Does

CloakLLM is open-source middleware that sits between your app and any LLM provider. Python, Node.js, and MCP for Claude Desktop.

It does three things:

Detects PII - three layers deep:

  • spaCy NER for names, orgs, locations (Python)
  • Regex for emails, SSNs, credit cards, API keys, IBANs, phones, IPs, JWTs (both SDKs)
  • Local LLM via Ollama (opt-in) - catches context-dependent PII that regex misses: addresses, medical terms, financial data, national IDs. Your data never leaves your machine.

Cloaks it with context-preserving tokens:

Your app:      "Email sarah.j@techcorp.io about Project Falcon"
Provider sees: "Email [EMAIL_0] about Project Falcon"
You receive:   Original email restored in the response
Enter fullscreen mode Exit fullscreen mode

The LLM still understands the prompt. It just never sees real data.

Logs everything to a hash-chained audit trail:

{
  "seq": 42,
  "event_type": "sanitize",
  "entity_count": 3,
  "prompt_hash": "sha256:9f86d0...",
  "prev_hash": "sha256:7c4d2e...",
  "entry_hash": "sha256:b5e8f3..."
}
Enter fullscreen mode Exit fullscreen mode

Each entry's SHA-256 hash includes the previous entry's hash. Tamper with one log entry, and every subsequent hash breaks. This is what Article 12 actually requires.

Python - One Line with LiteLLM

Works with 100+ providers (Anthropic, OpenAI, Azure, Bedrock, Ollama, etc.) via LiteLLM:

import cloakllm
cloakllm.enable()  # Done.

import litellm
response = litellm.completion(
    model="anthropic/claude-sonnet-4-20250514",
    messages=[{
        "role": "user",
        "content": "Help me email sarah.j@techcorp.io about the Q3 audit"
    }]
)
# Provider never saw the email. Response has it restored.
Enter fullscreen mode Exit fullscreen mode

Node.js - OpenAI SDK 

const cloakllm = require('cloakllm');
const OpenAI = require('openai');

const client = new OpenAI();
cloakllm.enable(client);  // Done.

const response = await client.chat.completions.create({
  model: 'gpt-4o-mini',
  messages: [{
    role: 'user',
    content: 'Write a reminder for sarah.j@techcorp.io about the Q3 audit'
  }]
});
// Provider never saw the email.
Enter fullscreen mode Exit fullscreen mode

Node.js - Vercel AI SDK 

const { createCloakLLMMiddleware } = require('cloakllm');
const { generateText, wrapLanguageModel } = require('ai');
const { openai } = require('@ai-sdk/openai');

const middleware = createCloakLLMMiddleware();
const model = wrapLanguageModel({ model: openai('gpt-4o'), middleware });

const { text } = await generateText({
  model,
  prompt: 'Write a reminder for sarah.j@techcorp.io about the Q3 audit'
});
// Streaming supported - handles tokens split across chunk boundaries
Enter fullscreen mode Exit fullscreen mode

MCP Server - Claude Desktop

CloakLLM also ships as an MCP server with three tools: sanitize, desanitize, and analyze. Add it to your claude_desktop_config.json and every Claude Desktop conversation gets PII cloaking.

Or Use It Standalone 

# Python
from cloakllm import Shield
shield = Shield()
cloaked, token_map = shield.sanitize("Send to john@acme.com, SSN 123-45-6789")
# cloaked: "Send to [EMAIL_0], SSN [SSN_0]"
Enter fullscreen mode Exit fullscreen mode 
// Node.js
const { Shield } = require('cloakllm');
const shield = new Shield();
const [cloaked, tokenMap] = shield.sanitize("Send to john@acme.com, SSN 123-45-6789");
Enter fullscreen mode Exit fullscreen mode

Works with any LLM client, any provider, any framework.

Local LLM Detection

Regex catches structured PII. But "I live at 742 Evergreen Terrace" or "diagnosed with hypertension" - regex can't catch that.

CloakLLM has an opt-in LLM detection layer that runs through local Ollama:

shield = Shield(config=ShieldConfig(
    llm_detection=True,
    llm_model="llama3.2:3b",
))
Enter fullscreen mode Exit fullscreen mode

It detects addresses, medical terms, financial data, national IDs, biometrics, usernames, and passwords - all without your data leaving your machine.

The LLM pass runs after regex, so already-detected entities are skipped. No double counting.

Design Decisions

Regex first, NER second, LLM third. Structured data (emails, SSNs, credit cards) is caught by regex with near-zero latency. spaCy NER runs second for names and orgs. The LLM pass is opt-in and catches everything else. Fast path stays fast.

Overlap detection. If regex catches john@acme.com and NER detects acme.com as ORG, the overlap is caught and the duplicate is skipped.

System prompt injection. Without it, LLMs see [PERSON_0] and ask "what's the real name?" CloakLLM injects a system message telling the model to treat tokens as real values. Only when tokens are present.

Token injection protection. If user input contains [PERSON_0], it gets escaped to fullwidth Unicode brackets before tokenization - preventing attackers from injecting fake tokens to extract other users' PII during desanitization.

try/finally cleanup. Even if the LLM API throws, the token map (which contains PII mappings) is always cleaned up. No PII lingers in process memory.

Vercel streaming. The Vercel AI SDK middleware buffers text-delta chunks and desanitizes on text-end, correctly handling tokens that span chunk boundaries like [EM + AIL_0].

Verify Your Audit Chain 

$ cloakllm verify ./cloakllm_audit/
✅ Audit chain integrity verified - no tampering detected.
Enter fullscreen mode Exit fullscreen mode

If someone edits a log entry:

Entry #40 ✅ → #41 ✅ → #42 ❌ TAMPERED → #43 ❌ BROKEN → ...
Enter fullscreen mode Exit fullscreen mode

Hand this to an auditor.

Install 

# Python
pip install cloakllm                  # standalone
pip install cloakllm[litellm]         # with LiteLLM middleware
python -m spacy download en_core_web_sm

# Node.js
npm install cloakllm
Enter fullscreen mode Exit fullscreen mode

The Numbers

150 tests across Python (62), JS (79), and MCP (9). Security audited with 6 vulnerability classes found and fixed: backreference injection, fake token injection, ReDoS hardening, spaCy model validation, middleware memory cleanup, and custom pattern safety checks. All regression-tested.

Zero runtime dependencies on the JS side. Python depends on spaCy.

What's Next

The roadmap includes LangChain.js integration, OpenTelemetry span emission, RFC 3161 trusted timestamping, sensitivity-based routing (PII → local model, clean → cloud), and an admin dashboard.

The EU AI Act deadline is August 2, 2026. 5 months from today.

→ GitHub: github.com/cloakllm/CloakLLM
→ Python SDK: github.com/cloakllm/CloakLLM-PY | pip install cloakllm
→ Node.js SDK: github.com/cloakllm/CloakLLM-JS | npm install cloakllm
→ MCP Server: github.com/cloakllm/cloakllm-mcp

Top comments (0)