The person waiting for an answer is often not the person who needs it now. A support manager may need a triage summary before lunch. A compliance team may need a document review tomorrow morning. An autonomous agent may need several rounds of tool calls before anyone sees a result. OpenAI’s new background completion windows recognize that difference, and turn it into a choice about time, capacity and price.
On November 17, OpenAI announced in its API changelog two discounted completion windows for the Responses API public beta: two hours at model token rates 30% below Standard, and eight hours at 40% below Standard. The feature is available for several GPT-6 family models, including GPT-6.2 Astra, GPT-6.2 Sol, GPT-6.1 Terra and GPT-6 Luna.
At first glance, this looks like a familiar cloud-computing bargain. Wait longer, pay less. But the more important change is conceptual. It asks developers to stop treating every model call as a conversation that must begin immediately. Some AI work is more like freight than a taxi: valuable, time-sensitive, but not necessarily urgent.
That distinction matters as businesses move from chatbots to agents. A single agent task may inspect records, call internal tools, request more context, validate its own work and return for another model pass. The user experience is not defined by the first token appearing on screen. It is defined by whether a useful task is complete before its real deadline.
AutomationBench scores across selected agent models · Chart: SPAISEE · Data: openai.com — see the chart in the original article.
The chart captures the commercial offer, but not the operational trade. The eight-hour option does not mean OpenAI is selling a weaker model. It means the customer gives the serving system room to decide when and where the job should run.
A deadline is not latency
Interactive AI is governed by latency. Someone asks a question, waits, and judges the service partly by how quickly it begins responding. Background work is governed by completion time: the interval from accepted submission to finished result.
Those are related but different measures. A request can have poor time to first token and still be an excellent background job if it finishes before the morning review meeting. Conversely, a system that starts immediately but misses a business deadline has failed the task.
OpenAI’s background-mode guide explains that the two-hour and eight-hour settings are completion targets that begin once the API accepts the initial request. The documentation calls soonest the default for on-demand work, suggests two hours for tasks such as support-ticket analysis ahead of an afternoon meeting, and positions eight hours for overnight document reviews.
The word “target” deserves attention. It frames these windows as a scheduling commitment, not a promise that computation will start instantly. OpenAI also notes that time to first token can be higher for background responses than for synchronous ones.
For a developer, that means the first redesign question is not, “Can this agent run asynchronously?” Nearly any workflow can. The better question is, “What is the latest moment at which this result remains useful?”
A fraud flag that blocks a purchase may need soonest. A nightly reconciliation, contract comparison or backlog classification might have an eight-hour deadline with no loss of business value. A research assistant preparing material for a meeting at 3 p.m. may fit neatly into two hours. The window should follow the consequence of lateness, not a generic preference for low cost.
Figure 1 - How interactive and background metrics measure different parts of an asynchronous agent run — see the chart in the original article.
The scheduler becomes part of the product
Once thousands of agents share accelerator capacity, a model provider cannot simply execute every job in arrival order. Short requests can be delayed behind huge contexts. Jobs that share a prompt prefix may be cheaper to group. Work may be routed to a cluster with more suitable hardware or an available cache. A task with a deadline in 20 minutes is not equivalent to one due tomorrow.
This is the logic of a scheduler. It admits work, sorts it into queues, allocates capacity, and decides what can wait. The technical details are invisible to most users, but the economic consequences are not.
OpenAI said in its July engineering post that its inference stack routes requests according to geography, available capacity and accelerator type. Within a cluster, it considers load, context length, cache availability and other request properties. The company says that GPT-5.6 Sol’s production-kernel work, together with broader kernel advances, reduced end-to-end serving costs by 20%.
Background windows create more freedom for those same mechanisms. If a job can complete any time before 8 a.m., the system can hold it briefly for a more efficient batch, steer it toward capacity that would otherwise sit underused, or run compatible workloads together. The output quality can remain the same because the model has not changed. What changes is the timing of the expensive hardware work.
That is why “background” should not be confused with “cheap.” The discount is real, but it is compensation for flexibility that has value to the provider. In economic terms, the customer is giving up an option to demand immediate service.
Figure 2 - how a background agent request moves from admission through flexible queues and capacity-aware batching to result delivery — see the chart in the original article.
Designing an agent for waiting
The hardest part of moving an agent into the background is often not the first request. It is managing the moments when the model asks for a tool.
OpenAI’s guide describes a pattern in which an application submits the initial response, stores its response ID, receives a completion webhook, performs the requested tool action, and sends a follow-up response with the tool output. The documentation recommends reusing the same model, project, completion window and continuation ID for eligible follow-ups, and says this can make them eligible for higher queue priority.
That sequence turns an agent into a small workflow engine. It needs state, durable records of prior output, idempotent tool calls and a way to handle duplicate notifications. It also needs a rational retry policy. OpenAI automatically retries eligible capacity errors after accepting discounted-window jobs, while customers are told to wait for a webhook or poll the same response ID while a job is queued or in progress.
The application should not respond to every delay by creating a fresh request. That can multiply token use, consume shared rate-limit capacity and make it impossible to know which run owns the final outcome. Instead, teams should assign every workflow a retry budget: perhaps one provider-managed retry, one application retry for a clearly classified failure, and a manual or human-review path after that.
They should also checkpoint after each external tool call. If a document lookup succeeds but the next model step fails, the system should resume from the saved result rather than pay to rediscover it. This is not glamorous engineering. It is the difference between an agent that behaves like a dependable back-office process and one that becomes an expensive loop.
A state-machine diagram — see the chart in the original article.
Measuring value, not calls
A background agent should be measured by cost per completed task, not cost per API call. A cheap first request that produces an incomplete report, triggers repeated tool calls, or misses a deadline is not cheap in any useful sense.
A practical dashboard should track four things: the share of jobs completed before their business deadline; queue time versus execution time; tokens and tool costs per completed outcome; and retries per workflow. Segment those measures by window, job type and deadline class. If eight-hour jobs repeatedly finish in 20 minutes, the team may be overpaying in urgency elsewhere. If two-hour jobs often approach their deadline, the workflow may need smaller stages, earlier submission or a different service level.
There is one operational wrinkle. OpenAI says discounted requests share Batch and Background rate limits, with input tokens counted at submission. It also says prompt-cache entries created by two-hour or eight-hour background requests cannot be reused by Standard requests outside those discounted windows, even with identical prompts. The queueing decision, therefore, can affect more than a single call. It can shape rate-limit planning and cache strategy across an application.
The reception material supplied for this article contains only a critical assessment, not an opposing outside view. Joshua Wendt of S5 Labs argued that the windows are targets rather than guarantees and raised questions about missed-target behavior and billing clarity in some cache and long-context cases. That criticism is useful precisely because the product asks customers to accept uncertainty in exchange for savings.
The larger lesson is straightforward. AI inference is becoming less like buying a fixed unit of intelligence and more like reserving capacity in a crowded transportation network. The best developers will not merely pick the fastest model or the lowest listed rate. They will decide which work must travel first, which work can wait, and what a finished answer is actually worth.
Sources
- developers.openai.com (primary)
- developers.openai.com (primary)
- openai.com (primary)
- openai.com (primary)
Originally published at spaisee.com.
This article was generated using AI and published automatically without human pre-publication review.
Top comments (0)