Command R's Multi-Step Tool Use Loop

#ai #machinelearning #tutorial #llm

Multi-step tool use is not a feature of the API. It is a loop you write around a single-turn endpoint, and the only thing the API gives you is a reliable termination condition. Getting that condition right is the whole of it.

The shape of the loop

One request to /v2/chat produces one assistant turn. That turn either answers, or asks for tools to be run. The loop is:

Send the conversation plus the tool definitions.
If the response has no tool_calls, it is the answer. Stop.
Append the assistant message — including tool_plan and tool_calls — to the conversation.
Execute each call and append one tool-role message per call, each carrying the matching tool_call_id.
Go to 1.

“Multi-step” means the model can use the result of one tool to decide what to call next: look up a customer, then use their region to query the right warehouse. Nothing in the request expresses that plan. The model rediscovers it each turn from the growing message array, which is why every message must go back — dropping the assistant turn that made the calls leaves results in the conversation that answer a question nobody asked.

Step 1: define the tools

v2 tools are JSON Schema under function.parameters. Two real ones, where the second depends on the first:

const tools = [
  {
    type: "function",
    function: {
      name: "get_customer",
      description: "Look up a customer by their order id. Returns region and tier.",
      parameters: {
        type: "object",
        properties: {
          order_id: { type: "string", description: "Order id, e.g. A-1204" },
        },
        required: ["order_id"],
      },
    },
  },
  {
    type: "function",
    function: {
      name: "warehouse_status",
      description: "Current dispatch delay in days for a warehouse region.",
      parameters: {
        type: "object",
        properties: {
          region: { type: "string", enum: ["utrecht", "rotterdam", "amsterdam"] },
        },
        required: ["region"],
      },
    },
  },
];

const impls = {
  get_customer: async ({ order_id }) => ({ order_id, region: "utrecht", tier: "business" }),
  warehouse_status: async ({ region }) => ({ region, delay_days: 2, updated: "2026-08-11" }),
};

The description fields are the only instruction the model gets about when to call what. “Look up a customer” is worse than “Look up a customer by their order id. Returns region and tier”, because the second tells the model what it will get back, which is what it needs to plan a second step. The enum on region does real work too: it stops the model inventing "Utrecht, NL" and failing your lookup.

Step 2: write the loop

import { CohereClientV2 } from "cohere-ai";

const cohere = new CohereClientV2({ token: process.env.CO_API_KEY });

const MAX_STEPS = 8;

async function runAgent(userMessage) {
  const messages = [
    {
      role: "system",
      content:
        "You are a logistics assistant. Use the tools to answer. " +
        "If a tool returns nothing useful, say so rather than guessing.",
    },
    { role: "user", content: userMessage },
  ];

  for (let step = 0; step < MAX_STEPS; step++) {
    const res = await cohere.chat({
      model: "command-a-03-2025",
      messages,
      tools,
    });

    const msg = res.message;

    // Termination: no tool calls means this turn is the answer.
    if (!msg.toolCalls || msg.toolCalls.length === 0) {
      return { text: msg.content?.[0]?.text ?? "", steps: step, messages };
    }

    // The assistant turn goes back verbatim, tool_plan included.
    messages.push({
      role: "assistant",
      toolPlan: msg.toolPlan,
      toolCalls: msg.toolCalls,
    });

    // One tool message per call, keyed by id.
    for (const call of msg.toolCalls) {
      const impl = impls[call.function.name];
      let output;
      try {
        if (!impl) throw new Error("unknown tool: " + call.function.name);
        output = await impl(JSON.parse(call.function.arguments));
      } catch (err) {
        // Errors go back to the model as data, not thrown to the caller.
        output = { error: String(err.message ?? err) };
      }
      messages.push({
        role: "tool",
        toolCallId: call.id,
        content: [{ type: "document", document: { data: JSON.stringify(output) } }],
      });
    }
  }

  throw new Error("agent did not converge in " + MAX_STEPS + " steps");
}

Step 3: run it

const result = await runAgent(
  "Order A-1204 is late. How long is the delay at the warehouse it ships from?"
);

console.log(result.steps, "tool rounds");
console.log(result.text);

A run that needs both tools goes through three requests. The first returns a tool_plan along the lines of “I will look up the customer for order A-1204 to find their region” and one call to get_customer. The second, now able to see region: "utrecht", calls warehouse_status. The third has no tool calls and returns prose. finish_reason is TOOL_CALL on the first two and COMPLETE on the last — useful in logs, but do not use it as the loop condition. The presence of toolCalls is the condition, because a turn can legitimately contain both text and calls.

Because the tool results were returned as document objects, the final answer is citable: the response carries a citations array whose sources have a type of tool, tying the “two days” in the answer to the call that produced it. That is a genuine difference from most tool-use APIs, where a tool result is opaque text and attribution is your problem.

The guards a naive loop omits

A step cap. MAX_STEPS above. Without it, a model that keeps calling a tool that keeps returning nothing useful runs until your bill or your request timeout notices. Eight is a reasonable starting point for a two-tool agent; count the distribution on your own traffic and set it above the p99.
Tool errors returned as data. The catch block feeds the error back to the model. Throwing instead means a transient 404 kills a conversation the model could have recovered from by trying different arguments. The trade-off is real — a model can loop on a persistent error — which is what the step cap is for.
Arguments parsed defensively. function.arguments is a JSON string. It is schema-constrained and will parse, but a required field the model could not determine may arrive filled with a plausible invention. Validate before you execute anything with side effects.
Growth of the message array. Every round adds an assistant turn and one or more tool results, and all of it is resent and rebilled on the next request. An eight-step run with verbose tool output can spend more on re-reading its own history than on the original prompt. Return the minimum useful object from each tool.
Parallel calls executed in parallel. The loop above awaits each call in turn. When the model issues several independent calls in one turn — which it does — running them concurrently with Promise.all and pushing the results in call order is a straightforward latency win, and order matters only for readability since each message carries its own tool_call_id.

One deliberate omission from the loop above: nothing forces a tool call. Cohere exposes tool_choice on v2 for the cases where you need to require one, or to forbid one and demand prose. It is occasionally the right answer — a first turn that must always retrieve before answering — but forcing a call on every iteration of a loop removes the termination condition, since the loop ends precisely when the model chooses not to call anything. Force on the first request if you must, then leave the choice to the model.

The v1 equivalent of this loop is the same control flow with a different message shape: results go in a tool_results array with the call echoed back rather than in a role-tagged message, as described in the tool_results format. Cohere’s multi-step tool use documentation covers both.

Making a run debuggable

An agent loop that goes wrong goes wrong in the middle, and the final answer tells you almost nothing about where. Four things logged per step turn a mystery into a transcript:

logger.info({
  run_id: runId,
  step,
  tool_plan: msg.toolPlan,                          // what it intended
  calls: msg.toolCalls?.map((c) => ({
    name: c.function.name,
    args: c.function.arguments,                     // what it actually asked for
  })),
  finish_reason: res.finishReason,
  input_tokens: res.usage?.tokens?.inputTokens,     // growth per step
  output_tokens: res.usage?.tokens?.outputTokens,
});

tool_plan is the field with no equivalent elsewhere and it is the most valuable line in that record. When a run calls the wrong tool, the plan usually shows that the model had the right intention and a bad tool description, or the wrong intention from the start — and those have completely different fixes. Reading a sequence of plans across a failed run is the closest thing to a stack trace that an agent has.

The token counts per step are the second thing worth keeping, because they make the cost curve visible. Input tokens should grow steadily as history accumulates; a step where they jump is a tool that returned far more than you thought. Summing input tokens across a run and comparing against the first request is the number that tells you whether your agent is affordable at ten times the volume, and it is invariably larger than people expect for the reason set out above — every step re-reads every previous step.

One further habit: keep the whole messages array on a failed run, not just the log lines. It is the exact input that produced the failure, it can be replayed against a different model or a different tool description without reproducing the original conditions, and it is the only artefact that lets you fix an intermittent agent bug rather than watch for it.