Anthropic said it has started to give employee-like access to a team from METR, a nonprofit that evaluates frontier AI models.[1] The arrangement gives METR employees company hardware, office space, and access to internal tools roughly on par with what Anthropic’s own risk assessment teams use, according to Anthropic.[1] METR will publish its analysis of Anthropic’s risk levels, incidents, and practices, and Anthropic cannot veto publication but may require redactions for narrowly defined categories like customer data or model architecture information.[1][2] That access aims to close the gap between what the company and outside evaluators understand about its systems, Anthropic said, comparing the model to bank supervision, where regulators embed examiners inside financial institutions.[1]
METR is classified as a third-tier evaluator under Anthropic’s three-tier system, the only one so far with this level of access, according to TechCrunch.[2][3] Anthropic said SecureBio is considered a first-tier evaluator, so its access is more limited, and the company is working with Accenture.[3][1] Anthropic said it is also in early discussions with Hugging Face and other organizations.[1][3]
Anthropic committed to stronger independent, third-party assessment of its models and safety practices last month, including by signing the White House Accord on Super Intelligence, in which companies pledged to partner with independent external auditors or evaluators, the company said.[1] Dario Amodei, Anthropic’s CEO, introduced the idea in a September essay as a step toward slowing model development.[3][2] Anthropic said last month that it would fund Accenture's work directly, and said long-term funding should come from pooled or government sources.[3] Anthropic plans to work with different evaluators under different funding arrangements.[3]