Originally published on AI Tech Connect.
The threat model most teams get wrong The mistake is almost never a considered decision. Someone is wiring up a support assistant for a Bengaluru fintech or a Manchester health-tech, they need the model to know that refunds above a certain amount need a manager, and the fastest place to put that rule is the system prompt. It works. It ships. Six months later the prompt is four hundred lines long and contains the pricing ladder, the names of six internal microservices, three real customer transcripts used as few-shot examples, and — in the cases that end badly — a bearer token someone pasted in during a Friday-afternoon debugging session. None of that is visible in the interface, and the invisibility does an enormous amount of psychological work. It feels like configuration. But a system…
Top comments (0)