In December 2024, Vinnie Giarrusso, principal software engineer at Twilio, described how to add AI to customer communication without making it colder. The worry he named is that assistants take work away and the experience gets less personal. He wants the assistant on the boring technical work and the mundane tasks, so a person can stay with the human part of the conversation.
I think that there's a lot of worry about agents taking work away and resulting in like a less personal experience. But actually, I think it's, I think it's the opposite, I think that there's a huge opportunity here to use these assistants in taking away the boring technical work and the sort of like mundane tasks,that would ordinarily be done by a human.
Vinnie Giarrusso, principal software engineer at Twilio, on Chain of Thought episode 7
Twilio calls them AI assistants on purpose. The industry word is agents. Early on, Vinnie said, the system sits side by side with a person, adding what he called superhuman powers, and depending on the setup a person may accept or reject certain actions. I would keep that accept or reject step on anything the customer cannot undo.
What should the assistant take, and when should a person step in?
I would let the assistant finish checks that are factual once the needed fact is known: order status, the customer's plan, and a single price. Vinnie described the support call where the person is reading a script and still misses the question. If those checks are already done, the person who joins can focus on the human problem, with time to be empathetic. He hopes the customer is not left waiting an hour.
He also drew a line through what an assistant may do alone. He said he had not tested this pattern and was spitballing:
Like there are things that an assistant can be trusted with to do. just kind of fully on their own. And then there are things that at least today, assistants should not be trusted to do fully on their own. especially in the early phase.
Vinnie Giarrusso, principal software engineer at Twilio, on Chain of Thought episode 7
He named the mechanism a stop mode or a request help mode. The assistant works inside a scope, then asks for supervisor approval or checks in on the steps it took. He left the customer-facing wording open. I would send one message that a person has the next step, and I would give that person a report of what was asked, what was answered, and which source was used. Complaints and account changes sit on the human side of the line. The assistant may prepare them. It may not commit them.
Enterprises, he said, will be more measured than the demo cycle. Some teams will treat a new capability as about a year out. If you are building a company on this, read the episodes gathered for AI founders before you promise full autonomy. They focus on defensibility, go to market, and market reality.
How do you stop a confident wrong answer?
Accuracy in this setting is specific. Vinnie's example is the price of a text message in Spain versus another country, when the user never said where they are. The assistant can hold every price row and still not know which one applies. He said accuracy is the assistant asking a follow-up, across one answer or a series of messages, instead of picking a row at random.
The follow-up itself is generative AI: the model produces new text. Sending it on a channel, or changing an account, is a further action. I would not let the function that drafts a reply also commit the account change.
Retrieval is one of the biggest failure modes Vinnie has seen once a system scales. A pipeline from the LangChain docs comes up quickly. After 10, 15, or 20 knowledge sources, the assistant may choose the wrong source, or return top chunks that cannot support the answer. He said context-aware retrieval, informed by Anthropic's contextual research, raised their retrieval accuracy, and he declined to give hard numbers.
On a bad trace, check two things. Which knowledge source did it pick? If that source was right, did the top chunks contain the answer? If you cannot tell, do not add another source.
What should you measure before you widen autonomy?
Vinnie said Twilio put an observability layer in place at day minus one. The team constantly checks context adherence, complexity, accuracy, and tone. Investigating low context or chunk adherence could reveal a mistaken assumption about the case, or an unexpected outcome in a small share of cases. I would narrow what the assistant may do alone until that signal moves.
A platform cannot know every future use. Twilio runs evals on cases that generalize across some customers, but cannot predict every customer use or knowledge source. They open-sourced an evals package in the Twilio Alpha GitHub and NPM namespace. It began as security evals and grew into evals more broadly, on the view that it is more convincing when people try their own questions and find gaps in sources and prompting. I would ship a small shared set plus a way for the domain owner to add cases.
Vinnie said few tools worked equally well for engineers, PMs, and business partners, and that shared view mattered. For a question like which handoffs had weak context adherence, use conversational analytics: someone asks in plain language and gets an answer from the data, without knowing the report or writing a query.
Sketch of the route only. This is not a Twilio system, and it calls no vendor SDK.
type Incoming = {
intent: "status" | "pricing" | "complaint" | "account_change" | "unknown";
missingFacts: string[];
tone: "calm" | "strained" | "angry";
action: "reply" | "lookup" | "change_account";
};
function route(turn: Incoming): "auto" | "clarify" | "human" {
if (turn.action === "change_account" || turn.intent === "complaint") return "human";
if (turn.tone === "angry") return "human";
if (turn.missingFacts.length > 0) return "clarify";
if (turn.intent === "status" || turn.intent === "pricing") return "auto";
return "human";
}
Checklist
- Write two lists before you ship: tasks the assistant may finish alone, and actions that always wait for a person.
- If a required fact is missing, force a clarifying question and block a random choice among price rows or sources.
- On every handoff, pass a short report of what was asked, what was answered, and which knowledge source was used.
- For each failure, check the chosen source and the top chunks, then watch context adherence, complexity, accuracy, and tone.
- Let domain owners add their own eval cases before you expand what the assistant is allowed to do alone.
The full conversation, with the transcript, is on Chain of Thought.
Subscribe to the Chain of Thought newsletter for new episodes and write-ups like this one.
Drafted with AI assistance from the episode transcripts.
Top comments (0)