Six METR evaluators now have access comparable to Anthropic’s risk teams and may publish assessments without a company veto, subject to redactions.
Anthropic said on October 15 that six employees of the nonprofit evaluator METR have received company hardware, office space, and access to internal tools and systems broadly comparable to that available to its own risk-assessment teams. METR is expected to publish analyses of Anthropic’s risk levels, incidents, or practices, giving a small external team a closer view of the developer’s operations.
The access is extensive but not unrestricted. Anthropic excludes a small number of predefined categories, including customer data, attorney-client privileged information, and details of model architectures. Outside those areas, employees are encouraged to share information with the METR team as they would with a colleague working on alignment risk. Anthropic says architectural information creates a high risk of inadvertent leaks and is usually of limited relevance, but may be shared with explicit internal approval when highly relevant.
Anthropic says it cannot veto METR’s publication, although it may require material in the protected categories to be removed. METR can publicly describe requested redactions and say whether it considers them justified, provided it does not disclose the protected information. The company says some safety information cannot be released publicly because it could help malicious actors or expose commercially sensitive details such as compute or training information.
The program is limited to a small METR group operating under enforceable agreements and Anthropic’s security standards. Other METR employees are separated from the embedded team’s work, including researchers involved in different projects with Anthropic. According to the company, the arrangement is not a route for other organizations to obtain Anthropic information or data belonging to its customers and partners.
METR’s engagement is the third and highest tier in a framework Anthropic is developing for external evaluators. First-tier investigations are narrowly scoped, with Anthropic deciding what access to provide and approving publication. Second-tier evaluators receive more information and can publish without approval, subject to specified removals. Anthropic also says SecureBio is reviewing the chemical and biological weapons chapter of its August Risk Report, while Accenture and Faculty are working toward a broader partnership spanning model evaluation, red teaming, alignment assessments, and safeguard testing.
The arrangement is intended to reduce the information gap between a frontier-model developer and outside evaluators without publicly exposing sensitive customer, technical, security, or commercial information. Its practical independence will depend on whether METR receives enough evidence to support its conclusions despite the access exclusions and permitted redactions. Anthropic’s announcement describes the framework, not its effectiveness: results from the new embedded evaluation have not yet been published.
| Tier | Access | Publication |
|---|---|---|
| First | Narrowly scoped review; Anthropic determines the access provided | Requires Anthropic’s approval |
| Second | Broader access to information | No approval required; specified information categories may be removed |
| Third | Access comparable to that of internal risk-assessment teams | No veto; predefined exclusions still apply |
Sources
Originally published by AI Feed: Anthropic Gives Six METR Evaluators Employee-Like Access.
Top comments (0)