← Back to the dossier

16 November 2026 · OpenAI · Apollo · METR · Transluce · Goodfire

OpenAI lets four outsiders into the training run, on site and on an OpenAI laptop

OpenAI has signed agreements with four independent assessors — Apollo Research, METR, Transluce and Goodfire — for a first phase of "embedded assessments". They get access to some of the same internal material OpenAI's own researchers use: model checkpoints, source code, training and evaluation data, monitoring systems and the people who built them. They work on site, on OpenAI devices. The board's Safety and Security Committee receives the unredacted reports. OpenAI keeps the right to redact and to ask for a delay before publication.

ArgumentConfirmedMore than one independent source, or a primary document.

What happened

The four assessments ask different questions. Apollo looks at whether a model could work against its developer's intentions and whether the safeguards would notice. It lists four claims to test: that training did not incentivise subversion, that strong evaluations and red teaming do not elicit it, that models have not attempted it during internal use, and that the reasoning behind it would be detectable. Apollo gets public and unreleased models, checkpoints, training data, evaluation results, intervention records, monitors and flagged behaviour.

METR takes the largest reinforcement learning run OpenAI has done and looks for misaligned behaviour its existing tests may have missed, with its own priorities and its own search strategy inside an agreed scope. It has also agreed in principle to a broader monitor assessment, after a three-week exercise in which it red-teamed OpenAI's monitoring with access to the codebase. Goodfire embeds interpretability researchers to analyse model internals, mainly for OpenAI's own team. Transluce audits the quality and coverage of post-training alignment evaluations, with a follow-on of about eight weeks in early 2027.

The protocols are the part worth reading twice. Assessors with sensitive access work at OpenAI offices on OpenAI devices. Employee conversations are off the record, but assessors can request a written statement in which OpenAI attests to, corrects or declines to confirm what they were told. Conflicts of interest and cool-off periods are disclosed. The Safety and Security Committee can speak to assessors privately, and a disagreement that survives escalation to the Safety Advisory Group and executives goes to that committee.

OpenAI ties the arrangement to the White House Accord on Super Intelligence, where it agreed to independent assessment of its safety controls. It also discloses who is paid: Apollo and Goodfire accept compensation that is not contingent on results; METR and Transluce take no money but may accept API credits or compute.

How it workedtechnical detail

An ordinary external evaluation gives a lab's model to a tester through an API. The tester sees inputs and outputs and nothing else. An embedded assessment gives the tester the training run: which checkpoints existed, what the reward looked like, what the monitors flagged, and which engineer made which decision. That is the difference between testing the finished product and watching it being made.

The compromise is structural. Access is deep, but it happens on the lab's hardware, under the lab's security rules, with conversations that cannot be quoted. The attestation pathway is the workaround: an assessor can publish "OpenAI confirmed this" or "OpenAI declined to confirm this". That is weaker than a transcript and stronger than nothing. Whether it is enough depends on how often the second sentence appears in a report.

What may follow

This is the most concrete answer any lab has given to the question the rest of this dossier keeps asking: who checks the checker. It is also a bet that transparency can be granted from the inside without becoming publicity. The test arrives with the first report that says something OpenAI would rather not read. On the same day, the market was already selling models with the safety layer removed: that product. Two answers to the same problem, one institutional and one commercial.

What I do not know

None of the assessments has reported yet, so this entry records an arrangement, not a finding. The open questions are the ones that decide whether it means anything: what counts as a failure that must be published, how long a "responsible disclosure delay" can run, whether the Safety and Security Committee is genuinely independent of the executives whose work it oversees, and whether any raw evidence will ever be public. The publication terms say assessors keep their independence but OpenAI may redact confidential information.

Read next

Sources

  1. OpenAI: Our first phase of embedded assessmentsprimary · main source

    16 November 2026. The four assessors and their scopes, the access list, the on-site and device rules, the Safety and Security Committee's role, the off-the-record and attestation pathway, the conflict and cool-off rules, the publication terms and the compensation disclosure.

  2. METR: red-teaming OpenAI agent monitoringresearch · not read end to end yet

    22 October 2026. The three-week monitor exercise OpenAI refers to. Linked from the announcement; I did not read it.

Comments