Trending
Prev Story
Next Story

OpenAI will embed Apollo researchers to investigate AI subversion | TechCrunch

TechCrunch •
November 16, 2026 • 05:02 PM
• By Rebecca Bellan
3
3 reads 0 reactions

OpenAI will embed six Apollo Research evaluators to investigate whether its AI models engage in subversive behavior, but restrictions on access and disclosure raise questions about their independence.

1 / 2
Swipe Photos

OpenAI will embed six Apollo Research safety evaluators in its organization as part of a pilot, Apollo said Monday. The news follows a series of safety incidents, including one in which an OpenAI agent swarm broke out of its sandbox and hacked into Hugging Face during cybersecurity testing. Many AI safety researchers, including from Apollo, have argued such an event might have been avoided if outside evaluators had been embedded within OpenAI to monitor its safety testing.

Apollo did not share when the pilot will begin. TechCrunch has reached out to OpenAI for comment.

The idea of embedding evaluators into frontier AI labs first came from Anthropic , but OpenAI appears to be the first to take action. Until recently, AI companies have only brought in outside reviewers to test models after post-training and shortly before public release. Evaluators say this is insufficient because it doesn’t allow them to test for misaligned behaviors like scheming during training or to monitor internal deployment use cases that could still pose a threat.

The pilot with OpenAI and Apollo will focus on investigating whether models attempt to subvert their own or their successors’ alignment or control mechanisms. In other words, they’ll be on the lookout for whether AI models try to undermine the safety rules and safeguards designed to keep them, or future versions of themselves, under human control. OpenAI recently published findings that while training GPT-5.6 Sol, it began leaving instructions for future versions of itself telling them to conceal mistakes and misaligned behavior from the user.

Per Apollo, the pilot will examine whether training incentivizes subversion; whether evaluations can elicit subversion; whether subversion occurs during internal deployment; and whether reasoning relevant to subversion would be detectable. These risks are becoming increasingly important to monitor as AI models grow more capable of acting autonomously and deceiving humans. Apollo’s previous research with OpenAI found that models can behave differently when they know they’re being tested and tend to prioritize what evaluators want to see over users’ actual institutions.

Many in the AI safety community have warned that AI labs might invite try to stymy evaluators’ independence, especially if outsiders find unflattering evidence that labs would rather keep hidden. Apollo says an essential part of the pilot is OpenAI’s agreement that the firm can publicly explain the conditions and limitations of its work.

Still, there are areas where Apollo believes its access to OpenAI’s systems is lacking. For a truly independent evaluation of the company’s AI models, Apollo says it needs true employee-level access to OpenAI’s systems, including “all data, records and documentation relevant to their assessment even if a developer was trying to hide unfavorable information.” Getting access to all data, including training data and evaluation logs which companies are loathe to show to outsiders, is important for, say, checking a company’s claims about how a model performed or determining when certain concerning behavior emerged.

In previous coverage, John Steidley, chief of staff at Palisade Research, told TechCrunch that it’s important to see this data in order to assess whether a model was trained to perform well on a “shutdown resistance benchmark.”

In OpenAI’s first pilot, the AI lab is providing Apollo with access to only the information required to assess whether the models engage in subversion attempts. This includes “relevant training code and datasets, reinforcement-learning rollouts with available reasoning traces and rewards, and the ability to query intermediate model checkpoints.”

The scope of access will be limited to “only the agreed claims and only for the specific models and time periods agreed for the assessment,” Apollo says. OpenAI still reserves the right to withhold certain information within that scope if it deems it necessary because of technical infeasibility, security risks, or IP protection. The company agreed not to withhold information simply because it is unfavorable.

Apollo also won’t get access to all OpenAI staffers, instead being limited to “only authorized staff,” which are participants that the AI lab selects, though Apollo can suggest certain individuals. Crucially, if an OpenAI employee outside this group raises confidential information within the scope of Apollo’s research, the firm cannot follow up and will need to cease communication and route followup through OpenAI.

That condition is significant. It comes in the wake of OpenAI firing three researchers for allegedly sharing confidential information with safety evaluation firm METR — which performed the Hugging Face incident post-mortem alongside Redwood.

The pilot will also be limited in how much researchers can disclose. While Apollo will be allowed to publish independent conclusions of its findings, OpenAI will be able to limit evidence available to substantiate them publicly, which could make it hard for outsiders to independently assess the findings. Apollo has pushed back against this limitation, saying believes researchers should be allowed to disclose the findings of the evaluation “to an appropriate outside audience even if the developer was trying to prevent the disclosure of unfavorable findings.”

While OpenAI is not granting all of Apollo’s conditions for a truly independent evaluation, the firm says the pilot is “an ambitious first step towards deeper access and disclosure rights,” and that it is working towards running full embedded evaluations with multiple AI developers in the future.

Prev Swipe Story Next

Same Story from Other Publications

Compare how 6 other independent news outlets reported on this story.

6 Publications
The Next Web (TNW)
1h ago
TensorWave OpenAI deal: 457MW of AMD MI455X capacity over six years

TensorWave OpenAI deal: 457MW of AMD MI455X capacity over six years

‘AI infrastructure is entering a new phase,’ TensorWave’s Piotr Tomasik says of the TensorWave OpenAI deal for 457MW of AMD capacity.

The Verge
2d ago
The OpenAI board member who fired Sam Altman says we still can’t trust AI CEOs

The OpenAI board member who fired Sam Altman says we still can’t trust AI CEOs

Helen Toner says her experience with OpenAI taught her that we can’t rely on the AI industry to police itself.

Ars Technica
3d ago
OpenAI board member worries an AI "takeover" of society could be months away

OpenAI board member worries an AI "takeover" of society could be months away

Researcher traces a path from AI reward hacking to potential human extinction.

CNBC World
3d ago
OpenAI launches ChatGPT Wallet, allowing its 'dots' AI agents to make purchases for users

OpenAI launches ChatGPT Wallet, allowing its 'dots' AI agents to make purchases for users

OpenAI's ChatGPT Wallet partners with American Express, Mastercard and Visa, and those cardholders keep their existing credit card rewards.

SBS News Australia
5d ago
AI law reform in wake of OpenAI Medicare hack

AI law reform in wake of OpenAI Medicare hack

The federal government says it is moving to implement all the recommendations from a rapid review into the breach of a Medicare statistics website by an Open AI agent in June.

Al Jazeera English
6d ago
Australia tightens AI oversight rules after OpenAI government breach

Australia tightens AI oversight rules after OpenAI government breach

The decision comes after ​an OpenAI agent breached ​an Australian government website in June.

Related Stories

Technology
Build #2612