AI Agent Tool Pagination: Stop One Response From Swallowing the Context Window

An agent asks a search tool for invoices, tickets, or documents. The tool returns 20 tidy JSON objects. The model writes a confident answer.

The answer can still be wrong.

Those 20 objects may be page one of 2,000. Or they may be the only records the model can afford to read before its context becomes mostly old tool output. In both cases, a perfectly valid API response creates a bad decision.

The fix is not telling the model to “remember pagination.” Give it a result contract that makes incompleteness visible, bounds how much data enters context, and lets the agent choose the next useful retrieval step. This guide shows a practical pattern for paginated tool results that works with ordinary HTTP APIs, internal tools, and MCP-backed workflows.

The real bug: a page looks like an answer

Conventional APIs can assume a program will loop until nextCursor disappears. An agent cannot safely make that assumption. It has a task, a limited context window, and a tool response whose meaning is partly described in natural language.

Three failure modes repeat in production:

Failure	What the agent sees	What goes wrong
First-page certainty	A plausible list of records	It answers as if there are no more matches.
Context flooding	Hundreds of rows, fields, and snippets	Important instructions and evidence are pushed out of the working context.
Blind looping	A continuation cursor	It fetches every page even when one aggregation or a better filter would answer the task.

Pagination is therefore a reasoning and interface-design problem, not just an API detail. The right question is not “How can the agent load everything?” It is “What is the smallest next slice of evidence that can change the answer?”

That change in framing also improves user experience. A chat interface can show progress such as “I found 1,840 matches; I’m comparing the top accounts by open-ticket count” rather than silently issuing ten calls. The user can then refine the task before the system spends time and tokens browsing records that will not affect the conclusion.

Separate the result store from the model context

Keep the complete response in a durable, access-controlled result store. Put only a compact page summary in the model-visible response. Give the model a reference it can use to fetch a narrow follow-up slice.

This architecture has four pieces:

1. A query tool creates a server-side result set and returns the first bounded page.
2. A page summary explains count, ordering, filters, coverage, and whether more results exist.
3. A result reference identifies the stored set without exposing a database key or another tenant’s data.
4. Follow-up tools retrieve a next page, a record by ID, an aggregate, or a filtered search within that same result set.

The result store is not long-term agent memory. Treat it as short-lived evidence for one task: tenant-scoped, expiry-bound, auditable, and deleted or redacted according to your data policy.

This separation also makes retries safer. A transport retry can retrieve the same page by reference without asking the model to reconstruct a query or blindly repeat a broad search. Keep retrieval state deterministic; let model reasoning operate on a compact, explicitly scoped view of that state.

Return an agent-friendly page contract

Offset pagination is easy for a UI but fragile for changing data. Cursor pagination is usually safer because the server can preserve ordering and make the cursor opaque. The agent should never manufacture or inspect a cursor.

More important than the cursor is the surrounding contract. A useful response says what the page proves and what it does not prove:

type ToolPage<T> = {
  resultRef: string;             // opaque, tenant-scoped, short lived
  items: T[];                    // hard-capped, e.g. 10–25 records
  returnedCount: number;
  knownTotal?: number;           // omit if expensive or misleading
  hasMore: boolean;
  nextCursor?: string;           // opaque; valid only with resultRef
  sort: { field: string; direction: "asc" | "desc" };
  appliedFilters: Record<string, string | number | boolean>;
  coverage: {
    complete: boolean;
    reason?: "more_results" | "time_budget" | "permission_limited";
  };
  summary: string;               // factual, generated by the server
  suggestedNext: Array<
    | "get_next_page"
    | "search_within_results"
    | "get_aggregate"
    | "get_record"
  >;
};

coverage.complete matters. items.length < limit is not sufficient evidence that the task has complete data: permissions, timeouts, server caps, or a moving query can also shorten a page.

The summary should be factual rather than interpretive. Good: “Showing 20 newest open tickets, sorted by updated time; more matching tickets exist.” Bad: “These are the most important tickets.” Let the model decide importance, but make the available evidence explicit.

Give the agent better moves than “next page”

Paging every result is rarely the best plan. A search tool should offer alternatives that map to common user questions.

For example, a request such as “Which customers had the most unresolved billing issues?” can usually be answered by an aggregate, then a narrow drill-down:

// First: a cheap grouped view, not 500 ticket rows.
const groups = await tickets.aggregate({
  resultRef,
  groupBy: "customerId",
  metrics: ["count"],
  filters: { status: "open", category: "billing" },
  limit: 10,
});

// Then: inspect only the evidence behind the leading group.
const details = await tickets.searchWithin({
  resultRef,
  customerId: groups.items[0].customerId,
  fields: ["id", "subject", "updatedAt", "priority"],
  limit: 10,
});

Design these moves deliberately:

• get_aggregate answers counts, totals, ranges, and grouped comparisons.
• search_within_results finds a concept, record, or field inside a stored set.
• get_record returns one complete record after the agent has selected an ID.
• get_next_page is for ordered browsing when order itself matters.

This is also a useful dividing line for MCP. Protocol-level pagination can enumerate tools, prompts, and resources. Your business tool still needs its own domain-level result contract for searches, CRM records, logs, or catalog items.

Make continuation a policy decision

Do not rely on a prompt rule such as “continue until complete.” Give the runtime a small, deterministic continuation policy. It should consider the task, evidence quality, page count, and resource budget.

function chooseNextStep(input: {
  taskNeedsExhaustiveList: boolean;
  page: ToolPage<unknown>;
  pagesRead: number;
  maxPages: number;
  canAnswerFromCurrentEvidence: boolean;
}) {
  if (input.canAnswerFromCurrentEvidence) return "answer_with_scope";
  if (!input.page.hasMore) return "answer_with_scope";
  if (input.pagesRead >= input.maxPages) return "ask_or_narrow";
  if (!input.taskNeedsExhaustiveList) return "aggregate_or_search";
  return "get_next_page";
}

The final answer should carry the same scope honestly. If the tool only inspected the 50 newest records, say so. If it used an aggregate across every matching record and sampled five examples, say that too. Scope language is not a disclaimer added at the end; it is part of correct reasoning.

A concrete TypeScript tool design

Here is a simplified server implementation. Notice that the server owns the cursor and the result reference. It also limits fields before any data reaches the agent.

const PAGE_SIZE = 20;

export async function searchTickets(input: {
  tenantId: string;
  query: string;
  cursor?: string;
  resultRef?: string;
}) {
  const state = input.resultRef
    ? await resultStore.load(input.resultRef, input.tenantId)
    : await resultStore.create({ tenantId: input.tenantId, query: input.query });

  const page = await ticketRepo.search({
    tenantId: input.tenantId,
    query: state.query,
    after: input.cursor,
    limit: PAGE_SIZE + 1,
    select: ["id", "subject", "status", "priority", "updatedAt"],
  });

  const hasMore = page.rows.length > PAGE_SIZE;
  const items = page.rows.slice(0, PAGE_SIZE);
  const nextCursor = hasMore ? encodeCursor(items.at(-1)!.updatedAt, items.at(-1)!.id) : undefined;

  return {
    resultRef: state.ref,
    items,
    returnedCount: items.length,
    hasMore,
    nextCursor,
    sort: { field: "updatedAt", direction: "desc" as const },
    appliedFilters: { query: state.query },
    coverage: { complete: !hasMore, reason: hasMore ? "more_results" : undefined },
    summary: `Showing ${items.length} newest matches${hasMore ? "; more matches exist" : "; complete result set"}.`,
    suggestedNext: hasMore ? ["get_next_page", "search_within_results", "get_aggregate"] : ["get_record"],
  };
}

In a real system, sign or encrypt cursors, bind resultRef to the actor and tenant, and expire both quickly. Never let a cursor from one query, user, or tenant open another result set.

Set hard limits before the first incident

Every tool that can return a collection needs explicit limits. Pick them before you tune prompts:

• Maximum items and bytes per response. A 20-row page with selected fields is usually more useful than a 500-row JSON blob.
• Maximum pages per turn. Stop, narrow, aggregate, or ask the user after the cap.
• Maximum result-set lifetime. A short TTL reduces accidental reuse of stale or sensitive evidence.
• Stable sorting rule. Use a tie-breaker such as (updatedAt, id) so a record does not vanish or duplicate between pages.
• Moving-data rule. Record a snapshot time or query version. If the underlying data changes, either keep a stable snapshot or tell the agent that the view may have moved.
• Field allowlist. The retrieval service, not the model, decides which fields can enter context.

These limits make cost and correctness testable. They also prevent a user request such as “show me all activity” from turning into an unbounded retrieval loop.

Test behavior, not just cursors

Unit tests for nextCursor are necessary but incomplete. Add workflow tests where the agent has to choose an information-gathering action.

Test case	Expected behavior
First page has more results	The answer says its scope or retrieves more evidence; it never claims a complete list.
Aggregate answers the question	The agent uses the aggregate instead of walking every page.
Cursor crosses tenant boundary	The tool rejects it without revealing whether records exist.
Page limit is reached	The runtime narrows the query or asks a focused question.
Underlying records change	The response preserves snapshot semantics or marks the result as potentially stale.
Large text field appears	The allowlist omits it until the agent requests a specific record.

Log page count, result bytes, continuation reason, aggregate use, and final-answer scope. Those signals tell you whether pagination is reducing tool cost or merely hiding a retrieval problem.

Common shortcuts that backfire

“Return everything and let the model summarize it.” This is expensive and makes the model less reliable as the output grows. Put raw records in the result store instead.

“Always follow nextCursor.” A cursor is an option, not an instruction. An aggregate or a narrower filter is often the next best action.

“Put pagination instructions in the tool description.” Keep the description, but enforce caps and state binding in code. A model can miss a sentence; the runtime should not.

“Use page length to infer completeness.” A partial page can be caused by permissions or server limits. Return explicit coverage metadata.

FAQ

Should an AI agent fetch every page before answering?

Only when the task needs an exhaustive, ordered list and the budget allows it. For rankings, counts, trends, or a single record, an aggregate or targeted search is normally better.

What page size works best for AI agent tool results?

Choose a small, measured cap—often 10 to 25 compact records—then test with your real schemas. The byte size and field selection matter more than the row count.

Is cursor pagination better than offset pagination for agents?

Usually. Opaque cursors better support stable ordering and discourage the agent from inventing page numbers. They still need tenant binding, expiry, and explicit completeness metadata.

Can MCP paginate the output of a custom tool?

MCP has pagination conventions for protocol list operations. A custom tool that searches business data should define its own response schema with a cursor, bounded items, and clear continuation semantics.

How do I keep large tool results out of the context window?

Store the full result server-side, return a short factual summary plus a scoped result reference, and expose follow-up actions such as search, aggregate, and record lookup.

What should the agent say when it stops before the final page?

State the evidence scope plainly: for example, “I reviewed the 50 most recently updated matching tickets” or “I used an aggregate over all matches and inspected five examples.”

Pagination is successful when the agent can explain both what it found and how complete that evidence is. Build that truth into the tool response, and the context window stops being a dumping ground for records it never needed.

Comments

[AI Frontier Post]

20 records can be page one of 2,000, so a short page still looks complete. A short TTL helps with stale data, but if it expires while the agent is still paging, the new set could rank records differently and break the snapshot.