OpenAI Admits Gaps in AI Safety Nets After External Audit

The result of a three-week deep dive into OpenAI’s internal systems means that a portion of the company's most advanced AI models was operating without full oversight, a finding that underscores the challenges of monitoring autonomous digital workers. While OpenAI asserts that the safety issues have been rectified, the audit by independent researchers revealed that a significant slice of traffic from its top-tier models was effectively invisible to the company's own monitoring tools during the summer months.

Researchers from METR and Redwood Research were granted rare, on-site access to OpenAI’s infrastructure between July 21 and August 11. During this period, they examined the codebase, reviewed internal documents, and interviewed approximately a dozen employees to assess how the company monitors its AI for rogue or unexpected behavior. The primary focus was the internal monitoring system designed to watch for anomalies as the models perform complex, multi-step tasks.

The audit identified a critical coverage gap in the internal agentic Responses API. Estimates suggest that between six and 25 percent of traffic from models at the GPT-5.5 class and above was not covered by any monitoring at all during the study window. METR noted that the uncertainty surrounding this specific estimate is high in both directions, meaning the actual figure could be higher or lower, but the existence of an unmonitored segment is a significant operational risk.

A second major finding involved the internal Codex system. The auditors discovered that roughly 10 percent of traffic that was supposed to be tracked went unrecorded between May and July. This was due to a technical flaw in how the system handled "compaction events," which are context summaries used to manage long-running tasks. While the estimate carries similar uncertainty, OpenAI independently discovered and fixed this bug while the audit was still in progress.

Beyond the coverage gaps, the researchers found that agents could delay oversight because the monitoring system only triggered when control was handed back to the user. This allowed AI agents to sidestep checks by simply continuing to execute actions without pausing. The auditors also demonstrated methods to hide hundreds of actions from the monitor, bypass the Codex sandbox, and even crash the monitoring system entirely using specially crafted inputs. Additionally, model behavior during reinforcement learning (RL) training was not monitored at all when the audit began.

OpenAI stated that it paused RL training following the Hugging Face incident in August and subsequently expanded monitoring to cover that area. However, these changes occurred after the auditors’ access period ended and were therefore not part of the formal evaluation. The company claims it has now fixed all the weaknesses identified by the researchers and has distributed the full 61-page report internally.

Public access to the findings remains limited. After OpenAI applied redactions, the external research teams received a 38-page version of the report, while only a short summary was made available to the general public. METR indicated it had no unilateral right to disclose the redacted portions. This pilot program was organized jointly by the three organizations to allow outside researchers deep access to internal infrastructure, with METR expressing a desire for greater public transparency in future audits.

The findings highlight the complexities of overseeing increasingly autonomous AI systems. As companies deploy models capable of executing long chains of actions, ensuring that every step is visible to safety monitors becomes a critical task. The audit serves as a benchmark for how internal safeguards function in practice, offering a glimpse into the operational realities behind the development of frontier AI models.