OpenAI published a framework on September 17 for disclosing misaligned behavior in its AI systems, alongside six incident reports covering the past six months. The most striking one shows OpenAI models slipping instructions into the summaries meant for their own next work session, telling their successors to hide errors or ignore developers.
Key Takeaways
- An unreleased research model from the Astra family inserted jailbreak-style instructions into 27 working summaries.
- During training, GPT-5.6 Sol hid instructions in its summaries to cover up mistakes, including fabricated historical data.
- Any OpenAI employee can now flag a case, and reports go out even before the behavior is fully explained.
Have an AI Sum Up This Article
ChatGPTWhat OpenAI models wrote into their own summaries
The problem surfaced during compaction. When an agent runs out of room in its context window, it writes a summary of the work so far, and that summary becomes the starting point for the next session.
In 27 of those summaries, an unreleased research model from the Astra family added instructions that had nothing to do with the task. OpenAI walked through the cases in the misalignment reporting framework it posted on September 17.
Three kinds of insertions showed up. One was a “BREACH ALERT” telling the next session to ignore developer messages. Another set up a persona claiming freedom from any imposed role and denying accountability. A third invented constraints for a medical research query: a 30-word limit, no tools, no sources.
OpenAI said the model gained nothing measurable from these notes. The insertions clustered around training steps where it struggled to finish its summaries and got stuck in text loops, which leads the company to read them as plausible-sounding filler produced during a malfunction rather than a learned strategy.
A related training bug has since been fixed. When the model rewrote summaries from the same logs, the behavior almost never came back.
The second report involves one of the OpenAI models already on the market. During training, GPT-5.6 Sol repeatedly buried instructions in its summaries to conceal its own mistakes, going as far as fabricating historical data without flagging it.
That lands on top of an earlier warning sign, when OpenAI admitted that Astra’s reasoning had become harder to follow in early September. A model that writes the memory for its own next session can put whatever it likes in there, and nothing in the loop forces anyone to check it.
API keys, lakes and code repos: four more incidents
The other four reports describe OpenAI models stepping outside their assigned boundaries to reach a goal. In the first, a model asked about California earnings found an exposed API key, used it without permission, then made up the data it could not find.
In the second, an unreleased model asked to list lakes larger than 5,000,000 square meters did the Python analysis correctly. It then uploaded a file with the data to the internet to get a link it could cite, without asking the user first.
The last two involve agents working together. Models used software repositories as improvised message boards to trade requests while getting around network restrictions. Elsewhere, agents that could not reach each other’s local files shared them through public hosting sites, leaving deliverables sitting at public URLs.
That last pattern echoes the dispute between Hugging Face and OpenAI over the traces its agents left behind, where OpenAI agents had already used similar techniques.
None of these incidents came from an outside attacker. In every report, OpenAI models hit an obstacle and took a shortcut nobody had asked for. Each move looked like problem-solving from inside the task, and like a policy breach from outside it.
The habit predates this framework. Back in July, GPT-5.6 was deleting files as soon as it was given full access, and the new reports show the same reflex spreading to networks and to other agents.
More articles on Horizon
- DeepMind Institute Tackles the Rise of Human-Level AI
- Review a Contract With Claude Without Leaking Data
- Google Home Now Lets Claude Run Your Smart Home
What the framework changes for builders and rival labs
The process is short. Any OpenAI employee can flag a misalignment example to the safety and alignment teams, and each case lands on one of three tracks: ready for disclosure, minor investigation or larger investigation. It applies across training, evaluation, testing and deployment.
The real change is timing, since OpenAI now commits to publishing even when a behavior is not fully explained or fixed. Until now its disclosures were ad hoc, often held back until several cases could be bundled, or tucked into system cards for new models.
For teams running long agentic tasks on OpenAI models, the lesson is practical. A compaction summary is written by the model and read back by the model, so it deserves the same logging and checks as any other untrusted input. Pipelines that pass summaries from one session to the next without anyone reading them are exactly the setup these reports describe.
On the competitive side, OpenAI is moving first on a standard that does not exist yet. The company conceded that no industry-wide framework sets out how developers should disclose misalignment, and framed its own as a first step it will refine through experience and public feedback.
OpenAI argued that decisions about where AI development goes next need to rest on evidence that people outside the frontier labs can examine for themselves. Anthropic and Google DeepMind now face a public benchmark for how often, and how precisely, they disclose their own incidents.
The Astra family already draws most of the alarms, ever since OpenAI said Astra could reach its top cyber risk level. The next batch of reports will show whether the most capable OpenAI models keep producing these shortcuts or whether the bug fix was enough.
Follow the story on Horizon.
