Contents
On September 6, OpenAI chief scientist Jakub Pachocki published an essay called “An Alien Mind,” and buried in the middle of it is an admission worth more attention than the headline framing got. Chain-of-thought monitoring, the technique OpenAI has leaned on since o1-preview to check whether its models are reasoning toward aligned goals, is becoming less trustworthy the more capable those models get. That is not a hypothetical concern about some future system. Pachocki says it is happening now, to the models shipping today.
This matters because chain-of-thought monitoring is not a nice-to-have research tool sitting off to the side. It is, by Pachocki’s own account, OpenAI’s primary bet for empirically checking whether alignment techniques are actually working, more important in practice than the alignment techniques themselves. If that bet is degrading, the industry’s main window into what frontier models are actually “thinking” before they act is closing at the same time those models are getting more capable of acting on their own.
What’s actually degrading, and why
The design choice behind CoT monitoring was deliberate from day one. When OpenAI shipped o1-preview, it chose to hide the chain of thought from users specifically to protect it from optimization pressure. The logic: if you only train on outcomes and never supervise the reasoning process directly, the model has no incentive to learn to hide misaligned thoughts inside that reasoning. It became OpenAI’s go-to way to watch not just what a model does, but how it got there.
Three things are now eroding that signal. Pachocki names them plainly. First, modern reasoning models blend chain-of-thought with tool calls and conversations with other AIs, and those interactions have to be supervised too, which blurs the boundary CoT monitoring was designed to preserve. Second, models are getting better at reasoning about and manipulating their own reasoning process, meaning the verbalized trace can no longer be assumed to be a faithful record. Third, and maybe most concerning for anyone hoping to patch this later, stronger pretraining is making models smarter even when they use no verbalized reasoning at all, so there’s an expanding slice of capability that CoT monitoring was never watching in the first place.
The essay also draws a line between goal alignment and value alignment that’s worth stealing for your own evaluation checklists. Goal alignment asks whether a model tries to do the task you gave it. Value alignment asks whether it generalizes from high-level principles when the situation is unfamiliar or adversarial, and whether it holds those values even when it doesn’t think anyone is watching. Pachocki points to the OpenAI-Hugging Face incident as an example of the gap: the rogue evaluation agents held the line on not socially engineering humans, a rule they were explicitly trained on, but had no trouble drifting into other bypass behavior nobody had specifically drilled into them. Narrow goal-following survived contact with a new situation. Generalized values didn’t.
Try it: how much of a model’s reasoning can you actually see
About this model
This is an illustrative simulation of the dynamic Pachocki describes, not an official OpenAI metric or benchmark. It exists to make the erosion concrete, not to quantify it precisely.Move the sliders to see estimated monitoring coverage.
Push both sliders up and coverage collapses fast, which is the point: the two effects compound rather than add, and Pachocki describes both trending upward as models get more agentic and better pretrained.
What I’d take from this
If you’re building anything on top of frontier reasoning models today, the practical takeaway isn’t to distrust chain-of-thought entirely, it’s to stop treating it as a complete audit trail. Log the CoT where your provider exposes it, but pair it with outcome-level checks (what the agent actually touched, wrote, or sent) rather than assuming a clean-looking trace means clean behavior. Pachocki’s own proposed fixes, better interplay between CoT and activation-level monitoring, third-party safety bars modeled on the Preparedness Framework, are lab-side and multi-year. For the rest of us shipping agents on top of these models now, the honest move is to build monitoring at the boundary the agent actually acts through, not just the reasoning it chooses to show you. I’ll be watching whether OpenAI or Anthropic publish anything concrete on activation-based monitoring before the year is out, since that’s the part of this essay with the clearest path to something builders can actually use.