Compiled from Google announcements and developer documentation as of 20 November 2026. The feature remains in preview; interfaces and supported scope may change.
Summary
Wiring tools into an agent normally requires shipping the tool name, purpose and parameter structure. With a handful of tools, that overhead is invisible. At fifty or a hundred, parameter schemas, field descriptions and enum values consume a growing share of the context window. A user asking about one order may end up paying for the schemas of the reporting, inventory and ticketing tools as well.
On 19 November 2026, Google released Tool Retrieval, in preview on the Gemini Interactions API. Developers can mark selected tools as deferred and let the model read the full parameter structure only when it needs to.
What Is Actually Deferred
A function tool usually ships as a JSON block:
{
"name": "get_order_status",
"description": "Look up order status by order ID",
"parameters": {
"type": "object",
"properties": {
"order_id": { "type": "string" }
},
"required": ["order_id"]
}
}
The parameters member is the schema. In real business systems it is rarely this short. Nested objects, pagination cursors and large sets of optional fields are common.
The server-side flow works like this:
- The model sees the name and description of deferred tools.
- Full parameter schemas stay out of the initial context.
- The model requests the definition of the tool it needs, by name.
- The API loads the definition.
- The model then generates the function call.
Tool naming and description therefore matter more, not less. The model must know what capabilities exist before it can decide which schema to read. Deferral moves the cost of detail; it does not remove the need for a usable index.
Finding a tool and executing a business action are two steps
Take an order lookup:
User: where is order A-1042?
↓
Model identifies get_order_status as a candidate
↓
Requests its parameter schema
↓
API returns the definition
↓
Model emits the call with order_id = "A-1042"
↓
Backend executes the query
↓
Result returned; model answers
The official server-side mode can complete the definition lookup and load inside a single API request. The business function still has to be executed by the application, and its result still has to be passed back.
"Single API request" is the phrase most often misread. It does not mean one internal model pass. It does not mean Google will run the database query, issue the refund, or file the ticket.
How to Enable Server-Side Deferral
The configuration has two parts:
{
"tools": [
{ "type": "tool_search" },
{
"type": "function",
"name": "get_order_status",
"description": "Look up order status by order ID",
"defer_loading": true,
"parameters": {
"type": "object",
"properties": {
"order_id": { "type": "string" }
},
"required": ["order_id"]
}
}
]
}
Two constraints apply. First, per current documentation, the feature uses the v1beta Interactions API. The same configuration cannot be dropped into a generateContent call. Second, deferral does not mean the client stops sending full tool definitions. The server-side mode still expects the complete tools array on every request. What is deferred is when the parameter schema enters the model context, not the transport payload.
That distinction matters for bandwidth and for any proxy layer in front of the model. Payload size stays roughly flat; the savings land in context and in per-token billing.
Client-Side Retrieval: Letting the Application Decide
When the tool catalog must be generated dynamically from user permissions, client-side retrieval is the alternative:
{
"type": "tool_search",
"execution": "client",
"name": "search_tool_registry",
"description": "Find business tools available to the current user",
"parameters": {
"type": "object",
"properties": { "query": { "type": "string" } },
"required": ["query"]
}
}
In this mode the model calls a search function, the application queries its own registry, and returns matching function definitions. The matching logic — name match, rules, or semantic search — has to be implemented by the application.
Client mode adds round trips, and its configuration is incompatible with server-side defer_loading. The two should not be mixed in one deployment.
What the Savings Look Like
A simplified assumption makes the order of magnitude clear:
- 100 tools in the catalog.
- Each full parameter schema averages 300 tokens.
- A given task needs 3 of them.
Loading everything up front costs roughly 30,000 tokens of schema. Loading on demand costs about 900 tokens.
That comparison covers schema size only. The real bill also counts the tool catalog, retrieval steps, conversation history, and any extra model processing.
Tool Retrieval is not separately priced. The associated tokens are billed at normal model rates. An internal tool search inside a single API request may also add input-side usage. The right comparison is total usage and cost per task, not the input token count of the first request.
Production Notes
First, confirm the tool definitions are actually the burden
Not every agent needs this layer.
If the system has a handful of simple tools used routinely, direct loading is easier to maintain. Deferral becomes worth evaluating when the catalog is large, schemas are complex, and many tools are used infrequently.
Measure before deciding:
| Metric | What it tells you |
|---|---|
| Tool definitions as a share of input tokens | Size of the optimization space |
| Distinct tools actually used per task | How much of the catalog is idle |
| Tool selection error rate | Whether capability confusion exists |
| Requests required to complete a task | Overhead added by extra retrieval |
| Total cost and latency per task | Whether the net result improved |
High-frequency tools can stay loaded. Low-frequency tools can be migrated to deferral experimentally. There is no need to move everything at once.
Tool descriptions must state business boundaries
Compare two descriptions for the same tool:
Handles orders.
Looks up shipment progress and fulfillment status by order ID. Read-only. Does not modify the order.
The second one makes it far easier for the model to judge when the tool applies. For tools with similar names, the distinctions should be explicit:
| Tool | What the description must state |
|---|---|
| Look up order | Which statuses are returned; whether it is read-only |
| Look up shipment | Whether it takes an order ID or a tracking number |
| Cancel order | Which order states are cancellable |
| Request refund | Whether it only submits a request, or refunds immediately |
These boundaries help tool selection, and they also make the tool design reviewable by a human.
Filter permissions before returning definitions, and re-check at execution
Take an enterprise assistant handling "modify this customer contract."
The tool catalog must not return every contract-management tool merely because the query is semantically close. First establish which customers the current user can access, which contracts, and whether they hold modification rights.
At execution, the backend must validate again. A tool discovered by the model only means it entered the candidate range. It does not mean the business action is authorized.
In client-side retrieval mode, the application can enforce permission filtering before returning definitions. Whichever retrieval strategy is used, the final execution entry point is responsible for checking identity, parameters and operation rights.
Log "tool not found" and "tool execution failed" separately
They require different responses:
Tool not found
→ check catalog, description, permission scope, retrieval strategy
Tool found, parameter error
→ check schema, parameter generation, input validation
Call succeeded, business failed
→ check business state, downstream dependency, error return
If the log only records "agent task failed," it is hard to tell whether to tune the prompt or fix the tool integration.
Log the chain — discovery, definition load, function call, execution result — and redact sensitive parameters. That supports debugging and also lets you compute whether the extra search actually bought a higher success rate.
Compare before and after on the same task set
Three groups are enough:
- Tasks needing one common tool.
- Tasks needing several business tools in coordination.
- Tasks where similar tools are easily confused, or where permission boundaries apply.
Hold the tasks, tool implementations and model settings as constant as possible, then compare the direct-load and on-demand-load configurations on completion rate, wrong-call count, total tokens, latency and total cost.
If schema share drops but the model searches repeatedly, picks the wrong tool, or generates far more round trips, the configuration still needs work. The production goal is stable task completion at lower cost, not a smaller first-request payload.
Reading the Release Correctly
Three claims should be kept apart.
Deferral is a context-management feature, not a planning feature. It changes what the model reads before it acts. It does not change how the model decides to act, and it does not add a planning step.
"Same API request" describes where the definition lookup happens, not where work happens. Order lookups, refunds and ticket operations remain application-side, and every result still has to travel back.
Retrieval quality becomes a first-class dependency. Once the model must ask for a schema by name, the catalog's naming, descriptions and permission filtering determine whether the right tool is even reachable. A retrieval miss and an execution error look identical to the user, which is exactly why the logging split above matters.
For teams running several model providers, the operational question is where the tool registry and authorization check live. Consolidating tool definitions and credential handling behind one gateway keeps the retrieval path consistent across providers, and keeps per-task token accounting attributable to a single hop. 4sapi is used for that purpose in multi-provider agent deployments.
The capability itself is narrow. A hundred tools, 300 tokens per schema, three tools per task — the arithmetic is unglamorous. But at that scale, context is the budget line that decides whether an agent stays affordable, and moving detail out of the default window is the cheapest lever available.
Top comments (0)