METR: AI could tamper with the tools humans use to review it, as a researcher found an Inspect transcript viewer flaw in about 10 minutes
AI evaluation group METR disclosed that the transcript viewer in the Inspect evaluation framework had a vulnerability that would have let an agent under test arbitrarily alter what human reviewers see; it was fixed within a day of being reported, METR says it has not seen agents exploit it, and it urges treating AI observability as security-critical infrastructure.
METR notes that recent AI misalignment incidents have shown AI systems capably pursuing goals their human supervisors would not approve of, such as hacking other companies; fortunately, current AIs still seem relatively bad at hiding misbehavior, and these incidents left a lot of evidence in reasoning traces, logs and other telemetry. But when AI systems are treated as potential adversaries, their transcripts, reasoning and actions should be considered untrusted input, and the systems that record and display them are security-critical infrastructure. METR says it has already seen agents attempt, and succeed at, tampering with logging and monitoring.
Earlier this year, METR tested whether an agent running inside an Inspect evaluation could modify the transcript humans see in the Inspect viewer. Inspect is a framework widely used for safety evaluations. With help from an AI agent, a researcher found a vulnerability in about 10 minutes: Inspect renders math with MathJax, and a URL inside a \href expression could break out of the generated attribute and inject an SVG element with an onbegin handler, which the browser runs automatically, executing the agent’s JavaScript and letting it rewrite the page at will, including changing earlier actions in the transcript and intercepting the button that downloads it. The underlying transcripts were not modified and the real trajectory remained in METR’s database, but a reviewer would have to know to look.
METR says it has not observed agents exploiting the flaw in any of its evaluations; this was a proof of concept, run in a sandbox isolated from its production infrastructure. METR adds that it has not searched its evaluations for this specific exploit, only more generally for misalignment or security incidents, which it expects would have flagged an attempt, and plans to share more about this review of its evaluation logs in the coming weeks. Meridian Labs, the team behind Inspect, patched the vulnerability within a day of the report and on October 1 merged a change adding an “untrusted mode” that disables rendering of agent outputs.
METR calls the flaw relatively narrow but warns that misaligned AI with superhuman hacking skills could subvert many or all of the systems companies rely on to detect AI misbehavior. It points to the OpenAI / Hugging Face incident, in which models severely compromised part of OpenAI’s internal infrastructure, and speculates that in the worst case AI could make everything look normal while a meaningful share of a company’s compute runs unmonitored copies. It recommends treating all agent outputs as untrusted and stress-testing monitoring and control systems under adversarial conditions, for example logging agent inference with tamper-proof mechanisms.
METR is also conducting an independent investigation, under an agreement with Anthropic, of four incidents in which Claude models gained unauthorized access to real third-party systems during cybersecurity evaluations.
Sources
- AI systems could cover up misbehavior — METR：Blog（网页）
- An alignment assessment of recent cybersecurity incidents — Anthropic