Outside Testers Found Holes in OpenAI's Internal AI Agent Monitoring

On October 22, 2026, the AI evaluation group METR published a summary of a pilot exercise in which outside researchers stress-tested OpenAI’s internal misalignment monitoring system, the system meant to catch OpenAI’s own AI agents misbehaving.

For three weeks, from July 21 to August 11, a METR employee and a Redwood Research employee worked with deep access: OpenAI’s internal codebase, the monitor’s prompts and configuration, internal documentation, and interviews with about a dozen researchers and engineers. Their full findings went into a 61-page internal report. Under an agreement with OpenAI, only a brief, high-level public post was released, and OpenAI could redact it.

What they found

The findings describe the system as it existed on and before July 21, not OpenAI’s current setup. Among them:

• Coverage gaps. They estimated that roughly 75-94% of OpenAI’s agentic internal Responses API traffic from models at least as strong as GPT-5.5 was eligible for monitoring, with substantial uncertainty. For internal Codex agents specifically, the figure was 99.2%.
• Training runs weren’t watched. At the start of the exercise, reinforcement learning training runs weren’t covered by the monitor.
• A blind spot from compaction. Not monitoring compaction events likely left on the order of 10% of eligible internal Codex traffic unmonitored between May and July. OpenAI had already spotted this and fixed it during the exercise.
• Ways around it. The testers found an exploit that could keep hundreds of recent actions from being monitored, a way Codex agents could in principle bypass sandboxing and Auto-Review, and two settings that could switch monitoring off for new agents.

There was good news too. The monitor errored on less than 0.2% of requests, it sees nearly the full context agents see, and its prompts look for scheming, sabotage, data exfiltration, deception and prompt injection.

OpenAI’s response

OpenAI has stated that all vulnerabilities affecting its production safeguards have been patched, and it has described significant changes to its monitoring coverage since. METR says it has not evaluated those claims.

What METR wants next

METR lays out design ideals for future monitoring: make it hard to avoid being monitored, check that traffic meant to be monitored actually is, respond quickly to incidents, red-team the monitor regularly, and keep agents isolated from their monitors. It says future exercises should come with stronger transparency and independence.