🚀 New: chi (χ) — an open-source autoresearch harness for fleets of LLM coding agents. Read the announcement.

OpenAI's Discounted Background Windows Restart the Clock on Every Tool Call

GPT-6.2 launches without Flex processing. Its replacement, the Responses API's 2h and 8h background completion windows, sets a completion target per response, not per agent run, so a tool-calling workflow's worst case grows with every turn. It is also async-only, shares the Batch queue limit, and keeps a separate prompt cache.

Contents

If you move a tool-calling workflow onto OpenAI’s new discounted background windows, your latency budget is the window multiplied by the number of model turns, not the window. The background mode guide says the completion target “starts when the API accepts the initial request and applies to that response.” Each follow-up you send after running a tool is a new response with its own 2h or 8h target. An overnight agent that makes six tool calls on the 8h window has six 8-hour targets lined up end to end.

That matters more now than it would have a month ago, because for the newest models this is the only discounted path that works with multi-turn workflows.

What changed on November 17

The API changelog for November 17 added two things together: GPT-6.2 Astra and Sol, and a public beta of background completion windows. The background guide connects them: “Starting with GPT-6.2 Astra and Sol, new models launch without Flex processing (service_tier: flex).” The pricing page agrees. Its Flex table lists GPT-6.1 Terra, GPT-6 Luna and older models, and has no GPT-6.2 row.

The replacement is a field on the Responses API create call. From the API reference:

{
  "model": "gpt-6.2-sol",
  "input": "...",
  "background": true,
  "service_tier": "default",
  "background_options": {
    "completion_window": "8h",
    "continuation_id": "contract-review-run-0412"
  }
}

completion_window takes soonest (the default, no discount), 2h or 8h. The discounted windows require background: true, Standard processing, and stream either false or omitted. The beta covers GPT-6.2 Astra, GPT-6.2 Sol, GPT-6.1 Terra and GPT-6 Luna.

Flex was a slow call. This is a job.

Flex processing was a synchronous request. You set service_tier: "flex", raised your client timeout (the Flex guide notes the SDK default is 10 minutes), and handled 429 Resource Unavailable, which isn’t billed. The call site stayed the same shape: send a request, wait, get an answer.

A 2h or 8h request doesn’t return an answer. It returns a response ID in queued status, and you find out it finished from a response.completed webhook or by polling GET /v1/responses/{id}. Streaming is not allowed on these windows at all. After acceptance, the API “automatically retries eligible capacity errors,” so the 429 handling moves server-side and the waiting moves to you.

In practice, anything that used Flex inside a request handler or a synchronous worker has to become a persisted job: store the response ID, receive a signed webhook, deduplicate by the webhook-id header as the guide instructs, check that event.data.id matches the ID you saved, then fetch the response. If you already run Celery or a similar queue, this is a new task type with a callback, not a parameter change.

Why the deadline compounds

A Batch API job has one clock. The Batch guide gives it a 24-hour window, and a job that misses it moves to expired, with finished requests in the output file and the rest cancelled. That works because every request in the input file is known up front.

An agent run can’t be submitted up front. The model asks for a tool, your code runs it, and only then can you send the next request. The background guide’s own example is an 8h document review where the first response requests a document lookup and a follow-up carries the function_call_output. Each of those is a separate response, so each gets its own target.

continuation_id is how OpenAI tries to soften this. If a follow-up reuses the window, continuation ID, model and project, it qualifies “for higher queue priority.” The reference describes the field as “a customer-provided opaque identifier used to prioritize related background requests,” and adds that it “does not add prior history to the input.” The guide is explicit that this is not a schedule: “Start times depend on scheduling.” Priority shortens the expected wait for turn two. It doesn’t fold turn two into turn one’s target.

The docs also say the API “targets completion” within the window. Unlike Batch, they don’t document a status for a background response that misses its target. Treat the window as a scheduling hint with a discount attached, not an SLA, and put your own wall-clock deadline on the run as a whole.

The response may not outlive the run

A long run creates a second problem. Follow-ups normally chain with previous_response_id, but that only works while the earlier response can still be retrieved. On projects with Zero Data Retention, the data controls page says the API treats store as false for 2h and 8h requests and only stores application state temporarily.

The guide’s instructions for this case are specific. Save the original input and the full response output while you can still fetch it. If the prior response has expired, drop previous_response_id and replay the original input plus every saved output item in order, “including any encrypted reasoning items and the original function call,” then append the new function_call_output. So your job table needs to keep the model’s output items, not just the final text, for as long as the run might last.

Queue limits, cache and service tier

The 2h and 8h windows share the Batch/Background Rate Limit with Batch API jobs. That limit caps total queued input tokens per model, counted at submission. A large overnight batch can use up the headroom your background agents were counting on, and the reverse is also true. Requests on soonest use the normal per-minute limits instead.

The prompt cache is partitioned. The guide says Standard requests outside the discounted windows “can’t reuse prompt cache entries created by 2h or 8h background requests, even when the prompt is identical.” If your interactive traffic and your overnight traffic share a long system prompt, moving the overnight half to 8h doesn’t warm the cache for the daytime half.

service_tier: "auto" is a trap. With auto, the request gets the discount “only if the API selects Standard.” Set default explicitly so a cost model doesn’t depend on a routing decision you can’t see.

The price ladder

The discounts are stated as percentages off Standard. Here they are applied to GPT-6.2 Sol’s short-context Standard rates per million tokens, with the Batch row taken directly from the pricing page:

PathInputOutputTurnaroundMulti-turn tools
Standard$2.00$10.00SynchronousYes
2h window (30% off)$1.40$7.00Target per responseYes, one target per turn
8h window (40% off)$1.20$6.00Target per responseYes, one target per turn
Batch API$1.00$5.0024h per jobNo

Flex used to sit at half of Standard on the models that offer it: GPT-6.1 Terra is $1.00 input on Standard and $0.50 on Flex. Moving a Flex workload to GPT-6.2 on the 8h window means a smaller discount on a newer model. That can still be the right trade, but compare the bill per finished task, not per token.

Which path to use

If every request is known before you start and none depends on another’s output, Batch is still cheaper and has one clock for the whole job. Classification, extraction and eval runs belong there.

If the work is a loop in which the model calls tools and your code answers, the 2h and 8h windows are the discounted option on GPT-6.2. Budget for the worst case of turns times window, put an outer deadline on the run, send a stable continuation_id, and store every output item so you can rebuild the conversation if the chain expires.

If someone is waiting on the result, use soonest or synchronous Standard and pay full price. A 30% discount doesn’t help a user who gave up an hour ago.

Sources