An agent is a graph

This tutorial was written by Néstor Daza.

---

This is the fourth article in a series about building Claudius, my own Claude-based chatbot (Github). In the previous article, we scaffolded the core components of the app.

We already built the foundation: the data model, an identity the client cannot tamper with, and a health check that proves the application can reach both Atlas and Bedrock. We can now move to the streaming chat backbone, the phase where the conversation finally becomes real and Claudius turns into something you can use every day as your favorite AI tool.

We are going to build the live path, everything that happens between hitting send and tokens arriving on screen. The model runs as a small agent that can pause to search the web, stream its answer back word by word, and let you switch from a cheap model to a strong one in the middle of a thread without losing your place. The next article will be about where the conversation lives once the tokens stop, the persistence, the metering, and the guarded writing operations that keep the whole thing grounded. Here, the question is narrower and more immediate: what actually happens when you hit send?

The agent is not a prompt

The chat engine is not a single call to a model wrapped in a clever prompt; it’s a small state machine built with LangGraph, a design choice that lets the model do more than answer in one shot.

The graph has three nodes. load_context runs but does nothing yet, as a deliberate placeholder where memory retrieval will land in a later phase. agent is the model itself, bound to the tools it is allowed to call. tools is the executor that runs whatever the model asks for. A conditional edge sits after the agent; if the model requests a tool call, control passes to the tools node. If it returns a plain answer, the run ends. The tools node always loops back to the agent.

const builder = new StateGraph(MessagesAnnotation)
  .addNode("load_context", loadContext)
  .addNode("agent", agent)
  .addNode("tools", new ToolNode(tools))
  .addEdge(START, "load_context")
  .addEdge("load_context", "agent")
  .addConditionalEdges("agent", toolsCondition, ["tools", END])
  .addEdge("tools", "agent");

That loop keeps running until the model has what it needs, then returns a natural language answer instead of another tool call. A prompt answers once and stops, a graph keeps working until the task is finished.

Choose the model per run, not per graph

Here is the design decision that took the most thought, and it is invisible in the diagram above. The model is not compiled into the graph.

You might expect the agent node to close over a fixed model instance when the graph is constructed. Instead, the model is chosen per run and passed through LangGraph's configurable channel, the per-invocation configuration that travels alongside the graph state. The agent node reads the inference profile from that channel and builds its model client fresh on every turn.

async function agent(
  state: typeof MessagesAnnotation.State,
  config: RunnableConfig,
): Promise<typeof MessagesAnnotation.Update> {
  const { inferenceProfileId, maxTokens } = readConfigurable(config);
  const model = buildChatModel(
    inferenceProfileId,
    maxTokens !== undefined ? { maxTokens } : {},
  ).bindTools(tools);

  const response = await model.invoke(
    [new SystemMessage(SYSTEM_PROMPT), ...state.messages],
    config,
  );

  return { messages: [response] };
}

readConfigurable pulls the inference profile and token ceiling from the per-run config. buildChatModel constructs the ChatBedrockConverse client for that profile, and bindTools attaches the tool schema so the model knows what it is allowed to call (i.e, web_search). The node then invokes the model with the system prompt followed by the full message history, and returns the model's reply as the one update to the graph state.

Why go to this trouble? Because it is what makes switching models mid-conversation almost free. When you move from Haiku to Sonnet halfway through a thread, nothing about the stored conversation changes. The message history is untouched; on the next turn, we read a different inference profile from the config and build a different client using the same history. There is no graph to rebuild and no checkpoint to migrate. The model is a parameter of the turn, not a property of the conversation.

Model-switching appears in the request the client sends, not in the thread you see on screen. The rendered conversation carries no per-message model label, and the conversation row stores only the current model as a single field, so neither one records which model produced which turn. The request payload does, with each send carrying the conversation identifier, the chosen model, and the new text. Two consecutive sends in one conversation, sharing a conversationId and differing only in modelId, show up like this:

// Turn 3, sent while the selector was on Haiku
{
  "conversationId": "6770f1a2b3c4d5e6f7a8b9c0",
  "modelId": "claude-haiku",
  "text": "Summarize the tradeoffs we just covered."
}
// Turn 4, same conversation, selector now on Sonnet
{
  "conversationId": "6770f1a2b3c4d5e6f7a8b9c0",
  "modelId": "claude-sonnet",
  "text": "Now make the strongest case against that position."
}

The second turn still reads the full prior history that the server holds, so Sonnet answers turn four with turn three already in context, even though turn three ran on Haiku. The modelId is the public catalog identifier that the selector handed the client, paired with the display name on screen; the cross-region inference profile it resolves to remains on the server.

One tool, one predictable shape

The agent has just one tool in this phase, web_search, and it is a thin wrapper over Tavily. It returns exactly three fields per result: title, url, and snippet. Nothing else comes through.

The wrapper is hand-rolled rather than pulled from a library, to have a bit more control over the output shape. The model reads a compact JSON block, and the user interface renders it as a list of sources. Both sides depend on the shape staying fixed. Tavily returns its result text in a field called content, and the wrapper renames it to snippet at that boundary. Replace Tavily with another search service later, and the only code that changes is what's inside that wrapper.

This is a habit worth keeping past this one tool. A predictable shape at a boundary is cheap to build and prevents the rest of the system from developing a quiet dependency on a particular vendor's response format.

Bridging the event gap

The graph produces one kind of stream, the browser expects another; the route between them is a translation layer, the most intricate piece of this phase.

LangGraph emits a flat event log through its streamEvents interface. Model token chunks, tool starts, tool ends, and run boundaries all arrive as a sequence of typed events that you consume in order. The frontend uses the Vercel AI SDK, which expects something different, a typed stream of user-interface message parts. Natural text arrives as a text-start, then a run of text-delta events, then a text-end. Tool activity arrives as tool-input and tool-output parts. The two formats describe the same conversation, but they are not the same format. We need to close the gap between them.

A small state machine takes care of it. It walks the LangGraph event log and emits AI SDK parts as it goes.

for await (const ev of events) {
  switch (ev.event) {
    case "on_chat_model_stream": {
      const chunk = ev.data?.chunk as BaseMessage | undefined;
      const delta = chunk?.text ?? "";
      if (delta.length > 0) {
        if (textId === null) {
          textId = `text-${textSegment++}`;
          writer.write({ type: "text-start", id: textId });
        }
        writer.write({ type: "text-delta", id: textId, delta });
        assistantText += delta;
      }
      break;
    }
    case "on_chat_model_end": {
      endText();
      const output = ev.data?.output as
        | { usage_metadata?: UsageMetadata }
        | undefined;
      addUsage(totals, output?.usage_metadata);
      break;
    }
    case "on_tool_start": {
      writer.write({ type: "tool-input-start", toolCallId: ev.run_id, toolName: ev.name, dynamic: true });
      writer.write({ type: "tool-input-available", toolCallId: ev.run_id, toolName: ev.name, input: ev.data?.input ?? {}, dynamic: true });
      break;
    }
    case "on_tool_end": {
      writer.write({ type: "tool-output-available", toolCallId: ev.run_id, output: parseToolOutput(ev.data?.output), dynamic: true });
      break;
    }
  }
}

Read it event by event. On a model token chunk (on_chat_model_stream), if no text part is open yet, the machine opens one with a fresh identifier and a text-start, then writes the token as a text-delta and appends it to the running answer it keeps in assistantText. On the model turn ending (on_chat_model_end), endText closes the open part, emitting the text-end and resetting the identifier to null so the next token will open a new segment; the same handler pulls the token usage out of the event and adds it to the running totals, which the companion article picks up. On a tool starting (on_tool_start), the machine emits the tool-input parts. On a tool ending (on_tool_end), it parses the tool's JSON output back into a structured part that the renderer can show as sources.

Because the model finishes its turn before the tools it requires run, on_chat_model_end always fires ahead of on_tool_start. So each tool call sits cleanly between closed text segments, never inside an open one. The textSegment counter increments each time, so the answer the model writes after reading its search results opens as a fresh segment rather than reopening the one that came before the search. The renderer sees a clean sequence: natural text, then a tool call with its sources, then more text.

The tool parts are flagged dynamic. The web_search tool is not registered on the client with a schema, so as far as the AI SDK is concerned, its part is a dynamic tool, one whose shape is not known in advance. Marking it dynamic on the server lets the client assemble it into a cleanly typed part that the renderer can switch on, even though the client was never handed the tool's signature.

The conversation identifier rides back on a transient data part. When you send the first message in a brand-new conversation, the server creates the conversation row and needs to tell the client its identifier, without forcing a second request to get it. So it writes a one-off data-conversation part at the very top of the stream.

writer.write({ type: "start" });
writer.write({
  type: "data-conversation",
  data: { id: threadId, title: conversationTitle },
  transient: true,
});
writer.write({ type: "start-step" });

The client reads that part, adopts the identifier, updates the URL, and shows the new conversation in the sidebar, the part itself never becoming a message in the thread. transient marks it as out-of-band, a signal rather than content. No extra round trip, and the conversation is addressable from the first token onward.

The user sees what the agent did

The payoff of all that plumbing is visible on screen. Tokens animate in as they arrive, so the answer builds in front of you instead of appearing in one block after a wait. When the agent decides to search, a live line appears reading "Searching the web," and once results return, it collapses into an expandable list of the sources the tool found, each with its title and link.

That last behavior is a principle that runs through the whole product. Every tool call is visible and inspectable. The user should always be able to see what the agent did on their behalf, not only the polished answer that came out the other end. The event bridge is what makes this possible, because it carries tool activity through as first-class parts rather than burying it inside the model's turn.

There is one more thing to notice about the round trip, and it sets up the second half of the backbone. When you send a message, the client sends only the new text. It does not send back the transcript of everything said before. It cannot, really, because it does not hold the authoritative history. It holds only what it has rendered on screen. The full conversation lives elsewhere, and the model reads it from there on every turn.

Where it lives, and why one small identity decision makes reopening a week-old conversation as cheap as opening a fresh one, is the theme of the next article. The agent is a graph. The conversation, it turns out, is a database.