A reader (hi, Jeremy!) recently asked about the prompt-engineering challenges in the AI employee bot I've been writing about — specifically how we balance efficiency with user satisfaction, and whether roles like marketing or HR actually work in practice. Great questions, and they deserve a real answer instead of a vague "it works great!"
Here's what 3 months of running 18 roles in one bot actually taught us.
The core problem: role bleed
When you put 18 roles in one system, the hardest problem isn't the model — it's keeping agents in their lanes. Early on, prompts bled into each other: the copywriter would answer as the accountant, the analyst would start writing sales copy. Every failure felt like the bot "got dumber," but really it was a prompt-collision problem.
What fixed it: a system-prompt registry. Every role gets a frozen, versioned prompt with:
- A strict scope contract (what this role does and, just as important, what it refuses to do)
- Explicit input/output schemas (what data it consumes, what format it returns)
- A shared context layer that the router injects, separate from any role's identity
The router loads only the active role's prompt plus the shared memory summary. Roles stopped colliding almost overnight.
The second problem: prompt rot
System prompts rot faster than anyone warns you. A prompt tuned for one model version silently degrades after an upgrade — the model starts ignoring constraints or formatting output differently. Users experience this as "the bot got worse," with zero error messages.
What fixed it: version-controlled prompts + a small regression suite. Before promoting a new model, we run a fixed set of business scenarios against every role and diff the outputs. This caught several regressions that users would have noticed as quality drift.
Efficiency vs. satisfaction: fallback chains
The efficiency/satisfaction balance is a real trade-off, but it's not binary. The trick that worked best:
- Fast/cheap model first (DeepSeek V4 Flash — roughly 86% cheaper than GPT-4o)
- Escalate to a stronger model only when the task is complex (long context, negotiation, ambiguous requests)
- The user never sees the switch
Result: ~90% of requests served by the cheap model, satisfaction stayed flat, cost dropped a lot. Details on the routing layer are in my previous post.
Do marketing and HR roles actually work?
Short answer: yes, but pick the tasks carefully.
- Marketing roles (drafting, research, campaign briefs, SEO outlines) work surprisingly well — they're mostly structured writing tasks, which LLMs are great at.
- HR roles work best for policy Q&A and onboarding checklists — NOT for actual hiring decisions. We keep a human in the loop for anything consequential.
The roles that get the most daily use, honestly: customer support and data analysis. Those are where the demand is, and where the ROI is highest.
Want the templates?
I'm happy to share the actual prompt template structure (scope contracts, schemas, regression scenarios) in a follow-up. If you're building something similar, drop a comment with what you're working on and I'll prioritize the most useful format.
This is part of an ongoing series on building practical AI employees. Follow me for the architecture deep-dives.
Top comments (0)