TokenVault – 512KB local-first LLM cost shield & reversible PII vault (Python/Node/Go/Wasm)

Hey HN, I'm Somesh, a 17-year-old builder from India.

While building multi-turn AI apps, I kept running into two problems with existing LLM gateways:

1. Cloud proxy gateways force you to route every prompt through a third-party server (adding 80–200ms of network latency and creating a PII compliance nightmare).
2. Most teams silently lose 30–50% of their OpenAI/Anthropic bill to broken 1,024-token prompt cache prefixes (putting dynamic timestamps/UUIDs at token 0), stuck Period-1/Period-2 agent retry loops, and re-sending massive tool/vision payloads across turns.

So I built TokenVault (https://www.tockenvault.com) using a 5-language specialized stack:

• Rust -> WebAssembly (Wasm) binary kernel in the browser for zero-GC 64-bit FNV-1a 256-token Radix block fingerprinting, Luhn mod-10 card checks, and Shannon entropy secret scanning.
• Native single-file Python SDK (token_vault.py hooking httpx) and Node/Edge SDK (token-vault.js hooking globalThis.fetch) bounded strictly to a 512 KB in-memory ring buffer.
• Go streaming gateway (token_vault_gateway.go) with a 12s TTFT zombie-stream watchdog and bidirectional protocol failover (OpenAI <-> Anthropic <-> Gemini).
• Reversible Ephemeral PII Vault: masks credit cards, emails, DB URIs, and JWTs outbound (<TV_CARD_1>) and re-hydrates them inbound (including across split-token SSE streams).

You can test the client-side forensic log auditor in the browser without signing up (just drop any OpenAI/Anthropic/LangSmith JSONL or CSV trace to get a ghost-leak-optimized.config.json), or run node token-vault.js --self-test locally.

Would love brutal technical feedback from the HN community on the architecture!