Priorities and principles for effective third party assessments

OpenAI emphasizes the importance of third-party assessments in ensuring AI safety, accountability, and transparency for frontier models. These assessments provide independent scrutiny of safety claims, risk evaluations, and safeguards across training, deployment, and incident response. The company highlights its long-standing collaboration with third-party assessors and integration of such assessments into its Preparedness Framework practices. Deep access, including technical safeguards, chain-of-thought visibility, and confidential data, is provided to enable rigorous evaluations and challenge assumptions.

The proposed priorities focus on independent technical safety assessments by private and non-profit organizations, complementing government-led testing efforts. OpenAI stresses the need for strong independence, scientific rigor, robust security, and clear responsibilities in these assessments. Labs and assessors share responsibility for balancing meaningful scrutiny with the protection of sensitive information. Shared international standards for safety and security practices are advocated to ensure consistency and effectiveness in evaluations.

Four priority areas for deeper assessment are introduced, targeting specific safety questions such as the validity of safety claims, adequacy of evaluations, and robustness of safeguards under realistic conditions. Assessments may vary in duration, from weeks to months, and are generally launch-agnostic, focusing on long-term examination of safety claims. The work aims to inform both pre-deployment and ongoing safety practices, with a focus on structured safety cases and evidence-based arguments.

OpenAI defines safety claims as assertions about model capabilities or safeguards tied to specific risks, while safety cases are structured arguments supported by evidence. The company proposes assessing safety cases across training, evaluation, and deployment, with multiple assessors examining different components. Key questions include whether evidence substantiates safety claims, whether safeguards are robust to adversarial testing, and whether evaluations adequately cover Preparedness risk categories such as chemical, biological, cybersecurity, and AI self-improvement risks.