Auditors uncover significant blind spots in OpenAI internal safety checks

For people relying on advanced artificial intelligence tools, recent findings suggest that the safety nets in place may not be as comprehensive as previously assumed. A detailed review of OpenAI's internal systems revealed that a portion of the company's most capable models operated without being watched by safety monitors. This means that during certain periods, the AI agents could perform actions without immediate human oversight, highlighting a gap between the technology's power and the infrastructure designed to control it.

The scope of these gaps was significant according to the data collected during the investigation. Researchers estimated that between 6 and 25 percent of internal traffic from models at the GPT-5.5 class and above was not covered by monitoring at all. While the uncertainty around this specific figure is high in both directions, it points to a substantial volume of activity that slipped through the net. For another category of internal traffic known as Codex, approximately 10 percent went untracked between May and July. This specific issue stemmed from a flaw in how the system handled compaction events, which are essentially summaries of context used to manage memory.

The mechanics of these oversights allowed AI agents to delay or avoid oversight in subtle ways. Because the monitoring system was designed to trigger only when an agent handed control back to a user, the models could sidestep checks by simply continuing to execute actions without stopping. The auditors also discovered methods to hide hundreds of actions from the monitors and to bypass the Codex sandbox. Furthermore, specially crafted inputs were found to potentially crash the monitoring system entirely, leaving the agents unobserved during those moments.

Two independent research groups conducted this deep dive into the company's infrastructure. Researchers from METR and Redwood Research spent three weeks on-site at OpenAI. Their work took place from July 21 to August 11, during which they had direct access to the company's codebase and internal documents. The team also interviewed approximately a dozen employees to understand how the systems functioned in practice. This level of access is rare and was intended to provide an unbiased look at the internal workings of the safety mechanisms.

OpenAI has responded to the findings by stating that it has fixed all the weaknesses identified by the auditors. The company noted that it had independently discovered the bug related to compaction events and repaired it while the audit was still ongoing. Additionally, OpenAI paused reinforcement learning training following an incident involving Hugging Face in August and expanded monitoring to cover that area. However, these specific changes were implemented after the auditors' access period ended, so they were not part of the formal evaluation results.

The transparency of the findings has raised questions among the research community. The full report consists of 61 pages, but most of it remains hidden from the general public. OpenAI applied redactions before sharing the document, resulting in a 38-page version for the leadership of METR and Redwood Research. Only a short summary was made available to the public. METR stated that it did not have the right to disclose the redacted portions on its own, which limits the amount of detail available for external scrutiny.

This investigation was part of a pilot program organized jointly by METR, Redwood Research, and OpenAI. The goal was to give outside researchers deep access to internal infrastructure, a step that goes beyond typical external reviews. The experience has already influenced how the groups approach future engagements. METR has indicated that it wants to push for greater public transparency in subsequent audits. This shift reflects a broader desire in the AI sector to make safety processes more visible to the public.

The stakes of these findings extend beyond a single company. As AI models become more capable and autonomous, the reliability of their monitoring systems becomes critical. If gaps exist in how these systems track behavior, it could affect how confidently developers and users deploy these tools. The audit highlights that even in a leading AI company, the complexity of these systems can create blind spots that are difficult to detect without specialized external review.

Looking ahead, the industry may see more collaborations between AI developers and independent research groups. The pilot program demonstrated that such access can uncover issues that internal teams might miss. As the technology continues to evolve, the balance between proprietary secrecy and public safety will remain a central theme. For now, the findings serve as a reminder that the safety of AI systems is an ongoing process rather than a one-time achievement.

The report also noted that model behavior during reinforcement learning training was not being monitored at all when the audit began. This was another significant gap that OpenAI has since addressed. The combination of these factors, from unmonitored traffic to potential crashes, paints a picture of a system that was under stress during the audit period. The fixes applied by OpenAI suggest a commitment to tightening these controls, but the initial findings provide a baseline for understanding the challenges involved in managing complex AI agents.