The Path Ahead

Independent AI evaluation is becoming increasingly central1,2,3,4,5 to managing AI risks, but the field still has significant gaps in standards, funding mechanisms and talent. To fill these gaps, the AI Evaluator Forum is pursuing multiple lines of effort in the coming months.

Standards development

The Forum will build on its existing AEF-1 standard, to develop shared norms for embedded evaluators, including strengthened provisions on conflicts of interest, to help ensure independence, access, and transparency. The effort will draw on input from AEF’s actively expanding membership as well as a broader open consortium of practitioners, academics, and researchers.

Independence-preserving funding

The Forum will advance mechanisms to fund public-interest evaluators, such as pooled funding, that let them conduct highly rigorous evaluations at the necessary scale and speed, without compromising their impartiality. AEF members will collaborate on these approaches to help grow a pluralistic ecosystem of independent evaluators that keeps pace with multi-trillion-dollar companies.

Talent pipeline

The Forum will help develop a pipeline of qualified candidates to be placed in evaluation organizations. The AEF is committed to expanding the field of independent AI evaluation and ensuring its member organizations can compete with frontier companies for top talent.

AEF-1 standard