The fragile science of AI benchmarking

The drive toward embedded evaluators—independent observers placed within AI organizations—gained momentum following a series of safety incidents across the sector. Yet, the efficacy of this approach remains under scrutiny. When Anthropic appointed Accenture to audit its systems, the move triggered widespread skepticism regarding the consulting firm’s technical qualifications for such a complex task. Solaiman argues that the current reliance on these tests is premature, noting that without open access to model weights, independent verification of results is effectively impossible.

Beyond the question of neutrality, the technical scope of evaluation is shifting from the model itself to the surrounding ecosystem. Modern assessments must now account for the harness—the specific interface or tool used to access the AI—which can significantly inflate performance metrics. A less capable model paired with a superior harness may easily outperform a frontier model, complicating comparative analysis. Because agentic evaluations are both costly and technically demanding, the industry currently operates in a fragmented state. Establishing a cohesive, industry-wide standard for these benchmarks is essential if they are to serve as a reliable tool for gauging technological progress rather than a marketing veneer.