Contents
Every team building an LLM agent ends up writing the same unglamorous code twice: a loop that tracks conversation state, a scheme for summarizing old context before the window fills up, and some way to fan work out to sub-tasks without losing track of what each one returned. OpenAI shipped Codex and ChatGPT for Work by building exactly that infrastructure at scale, and on September 10 it opened public beta of the Agents API, which puts that same harness behind a single API call instead of behind a product. That’s a meaningfully different thing to ship than another model. It’s an admission that the orchestration layer around a model is now valuable enough, and hard enough to get right, to sell on its own.
What the API actually manages versus what you still own
OpenAI runs sessions, context compaction, and recovery; you supply tools and pick where the code executes. A session is a durable agent instance that survives across turns, so you send a task, the harness runs a loop of tool calls and reasoning, and you can steer it mid-turn or resume it hours later without rebuilding the conversation from scratch. The split is deliberate: the harness owns the parts everyone rebuilds badly, and your application owns the parts that make your agent yours, which tools it can call, what data it can see, and where its sandbox actually lives, OpenAI-hosted, your own infrastructure, or a partner like E2B, Modal, or Cloudflare.
Compaction is what lets a session outlive a single context window. Instead of a client-side script that truncates old messages, the harness automatically compacts earlier context as a session approaches its limit, preserving what the agent needs to keep working and discarding the rest. Combined with tool search, which loads tool definitions on demand instead of stuffing every schema into the prompt up front, the practical effect is a session that can run for hours without the developer writing a single line of memory-management code.
Subagents get first-class support with a concurrency cap you set. Setting multi_agent.enabled with a max_concurrent_subagents value lets the main agent break a task into independent pieces, delegate them, and merge the results, each subagent holding its own context so it doesn’t drown in the parent’s history. One customer quoted in the announcement reported an evaluation score climbing from 0.71 to 0.85 and a 4x latency reduction once they stopped hand-rolling that coordination. That is the number worth remembering here, not the throughput of any one model call: multi-step orchestration was the bottleneck, not inference.
Try it: same task, two ways to split the work
Hand-rolled: one loop, one context, sequential
Move the slider to run it.
Agents API: subagents in parallel, capped concurrency
Move the slider to run it.
Push the concurrency slider up and the managed side collapses six independent investigations into fewer batches while each subagent still gets its own clean context, no history bleed between the deployment log and the dependency graph. The hand-rolled side doesn’t have that option, it just keeps stepping through the list one task at a time in a single shared context that grows with every step.
What I’d watch next
The part I’m most interested in isn’t the harness itself, it’s the pricing model underneath it: no markup beyond token and tool usage, and the harness ships as an open-source project on GitHub that anyone can read or fork. That’s a bet that the moat isn’t the orchestration code, it’s the fact that OpenAI operates it at Codex scale and keeps improving it with every model release, so your agent gets better without you touching your integration. If that holds, the next place to look is whether Anthropic or Google answer with something similarly unbundled, a managed harness you can point at your own sandbox, rather than keeping orchestration locked inside a single hosted product. I’d rather build against that primitive than rewrite my agent loop again for the next model.