Contents
Anthropic published a detailed account this week of an agent it has quietly been running in production for months: Claude Tag sitting inside the on-call Slack channel for its own Continuous Integration team, watching for failures, and posting the first diagnosis before a human even opens their laptop. The team also released the setup as an open-source “on-call kit” on GitHub, so any team running Claude Code or Claude Tag can wire up something similar in an afternoon.
This matters because CI incident response is one of the least glamorous, highest-cost parts of running software at scale. A flaky suite or a bad feature flag can burn an engineer’s evening and delay every merge behind it, and the fix is usually mundane once someone actually finds it. What Anthropic describes is not a chatbot bolted onto Slack. It is a full detect, triage, resolve, and verify loop with memory, tool access, and a self-improving knowledge base, running unattended against real production alerts.
Three things worth understanding
The system runs on persistent memory files, not a fixed prompt. Every incident gets written into a lessons.md file with what happened, the root cause, and the gotcha worth remembering, and every new investigation starts by reading it first. When a pattern repeats often enough, it graduates into a dedicated investigation skill, a markdown file with the exact steps a human would take for that bug class. One entry that stuck out in the writeup: after an engineer jumped to a conclusion from a config file before checking metrics, Claude wrote a lesson about that habit, “query the data first, then theorize.”
Detection and escalation split into deterministic and agentic paths. A root oncall.md file encodes hard rules, such as paging only if the error rate crosses a threshold for a sustained window outside a known deploy window, which keeps the page-or-wait decision predictable. Everything past that point, chasing the actual cause across Grafana, logs, PagerDuty, GitHub, and Kubernetes, runs as a dynamic workflow. An orchestrator agent spins up executor subagents per data source, and they report back into one synthesized situation report. Anthropic says the median time to a first evidence-grounded analysis is 14 minutes, with some root causes named in under 4.
Resolution goes as far as opening a pull request. Most of the team’s deployments sit behind feature flags, and a separate agent, running with the on-call engineer’s own permissions, manages canary rollout and can ramp a flag up or down automatically. For everything else, the more common outcome is a PR that a human reviews and merges, which keeps a named owner and an approval gate in the loop instead of letting the agent push straight to production.
Try it: simulate the on-call loop
Below is a scripted replay of the shape of an incident from detection to handoff, based on the timings Anthropic published. Nothing here calls a live model, it is a fixed sequence, but it is worth stepping through slowly to see how little idle time sits between each stage.
Click "Simulate incident" to watch a CI failure move through detection, triage, resolution, and handoff.
Where this leaves teams running their own CI
The part worth stealing is not the AI, it is the shape of the system: a durable memory file, alert rules a human can audit, and an escalation path that starts agentic and only pages a person once there is a grounded hypothesis to hand them. That structure would improve most on-call rotations even with a much less capable model behind it. The open-source kit lowers the cost of finding out, since it ships a runnable test fixture against a fictional team’s incident history that finishes in about ten minutes. If your team already runs Claude Code or has Claude Tag in Slack, the honest next step is not to build a bespoke agent from scratch, it is to point the setup kit at your own postmortems and see how much of that 14 minute median you can actually hit.