DEV Community

Vishal Kumar
Vishal Kumar

Posted on

Building a member-support agent that's safe to leave on at 2 AM (the "know when to stop" problem)

A 24/7 support agent for health-plan members is easy to demo and hard to trust. The failure mode isn't "can't answer" — it's a confident wrong answer about someone's coverage. Here's the architecture that makes it deployable.

IntelliBooks Member Support Agent answering a coverage question at 2 AM with human handoff

1. Ground every answer in the member's own documents

RAG over the member's plan (hybrid vector + keyword retrieval, with citations) — the model answers from retrieved coverage text, not its training data. No source, no answer.

2. Redact PHI on the input path

Detection/redaction runs before the model sees anything, so it reasons over the question, not raw identifiers.

3. Make "escalate" a first-class action, not a fallback

A confidence threshold + a real human-in-the-loop handoff: below threshold, the agent routes to a person instead of guessing. This is the single most important design choice — it's what makes "always on" safe.

4. Meet members on their channel

The same agent runs on web, WhatsApp, and SMS via a bot gateway — one agent definition, many surfaces.

5. Audit everything

Every question, retrieval, answer, and escalation is logged immutably — the difference between "trust us" and "here's the transcript" when compliance asks.

The payoff is boring in the best way: members get instant answers to the routine 80%, and staff only see what genuinely needs a human. We ship it as one of several governed healthcare AI agents in IntelliBooks Studio — more at intellibooks.ai/overview.

Top comments (0)