Artificial Analysis changed its Intelligence Index on September 4, 2026. Private, held-out evaluations now make up 40% of the overall weighting, up from 20% in v4.1. The company says keeping these tests private makes it harder for model developers to optimize specifically for leaderboard questions.
One new evaluation, AA-Briefcase, gives models multi-week knowledge work projects involving linked tasks and thousands of source files. It grades task success, analysis and presentation quality. The aim is to test an agent working through a substantial project rather than answering isolated questions.
GDP.pdf, developed by Surge AI, tests document reasoning across 100 PDFs totaling 4,592 pages. The tasks span ten professional domains. Grading uses 1,275 expert-written criteria, and the headline all-pass rate counts a task as successful only when every criterion is satisfied.
In the September v4.2 results, Artificial Analysis reports a 33.2% all-pass rate for GPT-6 Astra on GDP.pdf, followed by GPT-5.6 Sol at 28.2% and Claude Fable 5.1 at 26.2%. Those percentages measure complete tasks, not the share of individual criteria answered correctly.
The update also drops GPQA Diamond, which Artificial Analysis considers saturated, and changes parts of its grading infrastructure. A rise or fall in a model's overall Index score across versions can therefore reflect the revised tests as well as a change in the model being measured.