Hey Dev community! 👋
I’ve been working on a project to automate routine dispatch chats in the logistics industry using LLMs. The biggest hurdle? Logistics companies are terrified of data leaks (GDPR) and AI hallucinations.
To solve this, I built a "paranoid" Zero Trust architecture around Google Gemini. I’d love your technical feedback on the data flow!
🛠 The Tech Stack
Backend: FastAPI, PostgreSQL, SQLAlchemy.
Frontend: React, Vite, Tailwind.
AI: Google Gemini (strictly constrained to Pydantic Structured Outputs).
🛡️ How the Zero Trust Flow Works
Data Scrubber (Ingestion): Before any prompt hits the LLM, FastAPI intercepts it, strips out all PII (phones, emails, names), and replaces them with tokens (e.g., [PHONE_0]). The real data goes into an encrypted Postgres vault.
LLM Isolation: The LLM receives sanitized text and must reply via strict Pydantic schemas (JSON). No raw text generation is allowed.
Human-in-the-Loop (HITL): If the schema returns requires_human_intervention: true (e.g., hazardous cargo detected), the backend cuts off the AI and pings a human dispatcher.
Deanonymization: The backend swaps the tokens back with real data from the vault before sending the response to the user.
🤔 Where I need your advice:
Are there any glaring logical vulnerabilities in this Anonymizer/Deanonymizer pattern?
For the HITL dashboard, I'm using short-polling (React Query every 5s). At what scale will this database polling become a bottleneck compared to WebSockets?
Any feedback is highly appreciated!
Top comments (0)