Function Calling: Responses API vs Chat Completions

Magnus — Tue, 13 Oct 2026 05:38:42 +0000

A wire-level comparison of function definitions, Call IDs, result messages, streaming arguments, authorization, idempotency, and route compatibility.

Responses and Chat Completions can both ask an application to run a function, but they represent the tool loop differently. Chat Completions puts tool definitions under tools[].function, returns message.tool_calls, and accepts results as role: "tool" messages. Responses uses flat function definitions, returns typed function_call output items, and accepts function_call_output items linked by call_id.

The model does not execute your application function. Your code must validate the proposed arguments, authorize the operation, execute it exactly as intended, return a result to the model, and prevent duplicate side effects when a request is retried.

The tool loop has four steps

The endpoint-specific JSON differs, but the application state machine is the same:

1. Declare an allowed function and its argument schema
2. Receive one or more model-proposed function calls
3. Validate, authorize, and execute each call in application code
4. Return each result using the call's correlation ID

Only after step four can the model produce an answer grounded in the tool result. If the model requests another tool, the loop repeats with a new call ID.

The OpenAI Function Calling guide describes this as a multi-step exchange. Treating the first tool call as a final answer is a common integration error.

Compare the wire contracts

The main differences are structural rather than conceptual.

Concern	Responses API	Chat Completions
Function definition	Flat `tools[]` item with `name`, `description`, `parameters`, `strict`	`tools[]` item with those fields nested under `function`
Model-proposed call	Typed `function_call` output item	Assistant message `tool_calls[]` entry
Correlation identifier	`call_id`	Tool call `id`, returned as `tool_call_id`
Function name	`function_call.name`	`tool_calls[].function.name`
Arguments	JSON string in `function_call.arguments`	JSON string in `tool_calls[].function.arguments`
Tool result	`function_call_output` input item	Message with `role: "tool"`
Common streamed arguments	Typed function-argument delta events	Indexed `delta.tool_calls[]` fragments
Final text	Typed output items and output-text helpers	`choices[0].message.content`

Do not correlate calls by array position. Parallel calls and streamed chunks can arrive in an order that differs from the order in which your application finishes. The explicit call ID is the stable join key.

Define one strict function schema

The following examples expose a read-only get_delivery_status function. The model may request an order status, but it cannot change an order, issue a refund, or access arbitrary storage.

The argument schema is deliberately narrow:

{
  "type": "object",
  "properties": {
    "order_id": {
      "type": "string",
      "pattern": "^ORDER-[0-9]{4}$"
    }
  },
  "required": [
    "order_id"
  ],
  "additionalProperties": false
}

strict: true asks the supported model to adhere to the function argument schema. The application must still parse and validate the JSON before execution. Pattern support and other JSON Schema features can vary by model or provider, so the same compatibility gate used for Structured Outputs also applies to function arguments.

Implement the loop with the Responses API

This JavaScript example keeps the exchange stateless by returning the model’s output items with the function result. It uses a deterministic local function so the control flow is visible without introducing another external API.

import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.MODELFLARE_API_KEY,
  baseURL: "https://modelflare.dev/v1",
});

const model = process.env.MODEL_ID;
if (!model) throw new Error("MODEL_ID is required");

const orders = new Map([
  ["ORDER-1001", { status: "in_transit", eta: "2026-08-06" }],
]);

const tools = [
  {
    type: "function",
    name: "get_delivery_status",
    description: "Return the current delivery status for one order.",
    parameters: {
      type: "object",
      properties: {
        order_id: { type: "string", pattern: "^ORDER-[0-9]{4}$" },
      },
      required: ["order_id"],
      additionalProperties: false,
    },
    strict: true,
  },
];

function executeTool(name, rawArguments) {
  if (name !== "get_delivery_status") {
    throw new Error(`Tool is not allowed: ${name}`);
  }

  const args = JSON.parse(rawArguments);
  if (!/^ORDER-[0-9]{4}$/.test(args.order_id)) {
    throw new Error("Invalid order_id");
  }

  return orders.get(args.order_id) ?? { status: "not_found" };
}

const first = await client.responses.create({
  model,
  input: "Where is ORDER-1001?",
  tools,
  parallel_tool_calls: false,
});

const calls = first.output.filter((item) => item.type === "function_call");
if (calls.length === 0) {
  console.log(first.output_text);
  process.exit(0);
}

const toolOutputs = calls.map((call) => ({
  type: "function_call_output",
  call_id: call.call_id,
  output: JSON.stringify(executeTool(call.name, call.arguments)),
}));

const final = await client.responses.create({
  model,
  input: [...first.output, ...toolOutputs],
  tools,
  parallel_tool_calls: false,
});

console.log(final.output_text);

For a stateful integration, Responses also supports continuing from a prior response where the route and storage policy permit it. A stateless example makes the complete transcript explicit and avoids assuming that every OpenAI-compatible route stores response state.

The critical fields are call.call_id on the model output and the matching call_id on function_call_output. Returning a result with the wrong ID disconnects the data from the model’s request.

Implement the loop with Chat Completions

The same application function can be used with Chat Completions. The tool definition and result message use the Chat wrapper.

import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.MODELFLARE_API_KEY,
  baseURL: "https://modelflare.dev/v1",
});

const model = process.env.MODEL_ID;
if (!model) throw new Error("MODEL_ID is required");

const orders = new Map([
  ["ORDER-1001", { status: "in_transit", eta: "2026-08-06" }],
]);

const tools = [
  {
    type: "function",
    function: {
      name: "get_delivery_status",
      description: "Return the current delivery status for one order.",
      parameters: {
        type: "object",
        properties: {
          order_id: { type: "string", pattern: "^ORDER-[0-9]{4}$" },
        },
        required: ["order_id"],
        additionalProperties: false,
      },
      strict: true,
    },
  },
];

function executeTool(name, rawArguments) {
  if (name !== "get_delivery_status") {
    throw new Error(`Tool is not allowed: ${name}`);
  }

  const args = JSON.parse(rawArguments);
  if (!/^ORDER-[0-9]{4}$/.test(args.order_id)) {
    throw new Error("Invalid order_id");
  }

  return orders.get(args.order_id) ?? { status: "not_found" };
}

const messages = [
  { role: "user", content: "Where is ORDER-1001?" },
];

const first = await client.chat.completions.create({
  model,
  messages,
  tools,
  parallel_tool_calls: false,
});

const assistant = first.choices[0].message;
const calls = assistant.tool_calls ?? [];
if (calls.length === 0) {
  console.log(assistant.content);
  process.exit(0);
}

messages.push(assistant);
for (const call of calls) {
  const result = executeTool(call.function.name, call.function.arguments);
  messages.push({
    role: "tool",
    tool_call_id: call.id,
    content: JSON.stringify(result),
  });
}

const final = await client.chat.completions.create({
  model,
  messages,
  tools,
  parallel_tool_calls: false,
});

console.log(final.choices[0].message.content);

Here, the first assistant message must be appended to messages before its tool results. Each result uses tool_call_id: call.id. Omitting the assistant tool-call message or returning an unmatched ID produces an invalid conversation history.

Stream arguments as fragments, not complete JSON

In streaming mode, function arguments can arrive in fragments. A chunk that contains {"order is not invalid JSON; it is incomplete JSON.

The safe parser keeps one buffer per call identity:

identify the call or indexed call slot from the event;
append each argument delta to that call’s buffer;
wait for the endpoint’s argument-done or completed signal;
parse the complete string once;
validate and execute only after completion.

Responses uses typed events for function-call arguments and output items. Chat Completions emits fragments in choices[].delta.tool_calls[], where the index identifies the in-progress entry and the final tool call carries its stable ID. Do not execute a function when the first fragment arrives.

The AI API Streaming Guide explains why connection, first event, first effective output, idle timeout, and completion should be measured separately.

Make tool execution safe and idempotent

Strict arguments improve structure, not authorization. Before executing any call:

allow only registered function names;
parse arguments with a size limit;
validate the complete schema in application code;
authorize the current user or workload for the requested resource;
separate read-only tools from tools with side effects;
redact secrets and sensitive output before returning it to the model;
use a deadline and bounded downstream retries;
record a safe operation reference, not raw secrets or private content.

For a side-effecting tool, use an idempotency key derived from a stable application request and call identity. Store the completed result before returning it to the model. If a network retry replays the same call, return the stored result rather than issuing a second refund, message, deployment, or database mutation.

The model-generated function name and arguments are untrusted input. A valid schema cannot grant permission, prove ownership, or replace a transaction boundary.

Decide how to handle multiple calls

The examples set parallel_tool_calls: false to keep the state machine easy to inspect. If parallel calls are enabled:

correlate every result by call ID, never by completion order;
enforce a maximum number of calls per response;
apply a concurrency limit in the application;
define whether one failed tool cancels, blocks, or coexists with successful tools;
return one result for every accepted call;
keep side-effecting operations serialized when their order matters.

Parallel execution can reduce latency for independent reads, but it increases authorization, ordering, retry, and partial-failure complexity. Enable it because the workload benefits, not because the parameter exists.

Understand the Modelflare compatibility boundary

Modelflare’s OpenAI compatibility code preserves strict function definitions and maps the supported function-call shapes between Responses and Chat Completions on eligible OpenAI/Codex routes. That mapping is deliberately narrower than the complete Responses tool surface.

Application-defined function tools can be represented in both formats. Responses-only hosted tools—such as provider-run search, code execution, or other built-in tool types—are not equivalent to application function calling and cannot be assumed to convert into Chat Completions.

Other OpenAI-compatible model families are exposed through raw Chat Completions pass-through unless separately verified. For those routes, the upstream provider decides whether tools, strict, parallel calls, streaming argument fragments, and specific tool_choice shapes are supported.

Before enabling a route, run the complete loop rather than checking only whether the first request returns 200:

Test	Required evidence
One read-only call	Function name, parsed arguments, call ID, result linkage, final text
Invalid arguments	Explicit validation failure with no tool execution
Unknown function	Rejected by the application allowlist
No tool required	Normal final text without a fabricated call
Streaming call	Complete reconstructed arguments and correct call ID
Repeated request	One side effect or one cached result, not duplicate execution
Multiple calls	Correct correlation independent of completion order
Route fallback	Same model, protocol, tool schema, and result contract preserved

Use Reliable AI API Routing to keep route changes explicit, and AI API Key Security to separate workloads and permissions.

Choose the format by the application contract

Use Responses when the application benefits from typed output items, the Responses event model, state-continuation options, or other verified Responses capabilities. Use Chat Completions when the application already owns a stable message transcript and the selected provider’s tool contract is verified. The broader endpoint decision is covered in Responses API vs Chat Completions.

The implementation standard should be the same in both cases: explicit tool allowlists, strict argument schemas, application-side validation, authorization before execution, correlation by call ID, idempotency for side effects, and a complete result returned to the model. Those controls—not the endpoint name—make a function-calling workflow reliable.

Originally published at modelflare.dev.

Disclosure: I'm part of the Modelflare team, which builds the AI API gateway mentioned in this article.

DEV Community: Magnus