What will OpenAI’s new outside safety evaluator tell the public?

The four-month pilot lets Apollo Research say what OpenAI withheld and explain whether redactions affected its conclusions. Read the full story on CryptoSlateUnderstand the crypto stories that matter and what to watch next. Get The Catalyst in your inbox

What will OpenAI’s new outside safety evaluator tell the public?

OpenAI is bringing in Apollo Research, an outside AI safety group, for a four-month investigation into whether OpenAI’s models try to bypass safeguards. Apollo can disclose any limits OpenAI places on its access to evidence. OpenAI can redact confidential supporting evidence from the public reports, and Apollo can explain how any redactions affect its conclusions. The agreement protects Apollo’s verdict on each claim from redaction.

Six Apollo evaluators will work inside OpenAI. They will examine whether training encourages models to undermine safeguards, whether safety tests can draw out that behavior, whether it happens during internal use, and whether evaluators can detect the reasoning behind it.

OpenAI confirmed the engagement on November 16 as part of a first phase involving four outside assessors. OpenAI says Apollo is paid for its work and that compensation does not depend on the assessment’s results.

Related Reading

For the training inquiry, Apollo says it will have access to relevant code and datasets and be able to query intermediate training checkpoints. It will also get relevant reinforcement-learning records, including rewards and available reasoning traces. Access details for the other three inquiries were still to be agreed when Apollo announced the pilot on November 16.

The review covers agreed claims, models and time periods. OpenAI can withhold even in-scope material when it considers that reasonably necessary because of technical limits, security risks or intellectual property protection. It must tell Apollo that material was withheld and give written reasons. Where OpenAI considers it feasible and relevant, it will also identify the categories and approximate volume of withheld records. Unfavorable information alone cannot justify withholding access.

Related Reading

Apollo plans two public reports. The first will address whether training encouraged models to undermine safeguards. The second will cover safety testing, internal use and whether the reasoning behind such behavior can be detected. OpenAI can review the reports for confidential material, and its commitments bar it from requesting a redaction solely because a finding is unfavorable. The Safety and Security Committee of the OpenAI Foundation’s board will receive unredacted copies.

Originally published by cryptoslate Aggregated for informational purposes. All rights belong to the original publisher.
← Back to all news