Benchmarks
Coding
Real code: fixing, writing and refactoring.
| Benchmark | Metric | Source | Models | Latest | Status |
|---|---|---|---|---|---|
| Aider Polyglot | Percent correct | Epoch AI | 66 | Scored | |
| Code Migration v1.1 | Accuracy (%) | Vals AI | 29 | 2026-11-09 | Scored |
| CursorBench | Accuracy | Epoch AI | 90 | Scored | |
| DeepSWE | Accuracy | Epoch AI | 69 | Scored | |
| FrontierCode | Accuracy | Epoch AI | 46 | Scored | |
| FrontierSWE | Accuracy | Epoch AI | 23 | Scored | |
| GSO | Accuracy | Epoch AI | 36 | Scored | |
| IOI | Accuracy (%) | Vals AI | 46 | 2026-10-29 | Scored |
| LiveBench 2.0 Coding | Score (%) | LiveBench | 32 | 2026-11-06 | Scored |
| LiveCodeBench (Vals run) | Accuracy (%) | Vals AI | 131 | 2026-09-01 | Scored |
| MirrorCode | Accuracy | Epoch AI | 45 | 2026-10-30 | Scored |
| ProgramBench | Accuracy (%) | Vals AI | 65 | 2026-11-09 | Scored |
| SWE-bench Multimodal | Accuracy (%) | Vals AI | 22 | 2026-11-09 | Scored |
| SWE-bench Verified (Epoch run) | Accuracy | Epoch AI | 33 | 2026-06-25 | Scored |
| SWE-bench Verified (Vals run) | Accuracy (%) | Vals AI | 84 | 2026-09-01 | Scored |
| SWE-rebench | Resolved rate (%) | SWE-rebench | 13 | 2026-07-01 | Scored |
| SciCode | Accuracy | Epoch AI | 238 | Scored | |
| WeirdML v3 | Accuracy | Epoch AI | 26 | Scored |
Agents
Multi-step tool use: terminals, browsers, computers.
| Benchmark | Metric | Source | Models | Latest | Status |
|---|---|---|---|---|---|
| APEX-Agents | Accuracy | Epoch AI | 65 | Scored | |
| BALROG | Accuracy | Epoch AI | 39 | Scored | |
| BFCL v4 | Accuracy (%) | BFCL | 53 | 2026-04-12 | Scored |
| CyberBench v1.1 | Accuracy (%) | Vals AI | 43 | 2026-11-05 | Scored |
| DeepResearch Bench | Accuracy | Epoch AI | 41 | Scored | |
| EMB v1.1 (Excel financial modeling) | Accuracy (%) | Vals AI | 39 | 2026-11-09 | Scored |
| Finance Agent v2.1 | Accuracy (%) | Vals AI | 37 | 2026-11-09 | Scored |
| HLAB-Vals 1.1 (legal agent) | Accuracy (%) | Vals AI | 35 | 2026-11-09 | Scored |
| LiveBench 2.0 Agentic | Score (%) | LiveBench | 32 | 2026-11-06 | Scored |
| LiveBench 2.0 Browser Use | Score (%) | LiveBench | 32 | 2026-11-06 | Scored |
| LiveBench 2.0 Research | Score (%) | LiveBench | 32 | 2026-11-06 | Scored |
| METR 50% Time Horizon | Minutes at 50% success | METR | 24 | Scored | |
| METR task suite | Accuracy | Epoch AI | 44 | Scored | |
| OSWorld (v1) | Success rate | Epoch AI | 9 | Scored | |
| OSWorld 2.0 | Accuracy | Epoch AI | 14 | Scored | |
| OSWorld-Verified | Success rate (%) | OSWorld-Verified | 16 | 2026-08-01 | Scored |
| Remote Labor Index | Accuracy | Epoch AI | 18 | Scored | |
| SRE Bench | Accuracy (%) | Vals AI | 35 | 2026-11-05 | Scored |
| SkillsBench | Accuracy (%) | Vals AI | 33 | 2026-10-29 | Scored |
| Tax Agent Bench | Accuracy (%) | Vals AI | 80 | 2026-11-09 | Scored |
| Terminal-Bench 2.0 | Accuracy | Epoch AI | 51 | 2026-04-23 | Scored |
| Terminal-Bench 2.1 (Vals run) | Accuracy (%) | Vals AI | 71 | 2026-10-01 | Scored |
| Terminal-Bench 4 | Accuracy (%), 5 trials | Terminal-Bench | 35 | 2026-09-29 | Scored |
| Terminal-Bench 4.0 (Vals run) | Accuracy (%) | Vals AI | 61 | 2026-11-09 | Scored |
| Terminal-Bench-Science | Accuracy (%) | Vals AI | 45 | 2026-11-09 | Scored |
| Vending-Bench 2 | Final balance (USD) | Epoch AI | 72 | Scored | |
| tau2-bench Airline | pass^1 (%) | tau2-bench | 8 | 2026-05-26 | Scored |
| tau2-bench Banking Knowledge | pass^1 (%) | tau2-bench | 27 | 2026-08-03 | Scored |
| tau2-bench Retail | pass^1 (%) | tau2-bench | 8 | 2026-05-26 | Scored |
| tau2-bench Telecom | pass^1 (%) | tau2-bench | 8 | 2026-05-26 | Scored |
Reasoning
Hard novel problems and expert science.
| Benchmark | Metric | Source | Models | Latest | Status |
|---|---|---|---|---|---|
| ARC-AGI-1 | Accuracy | Epoch AI | 266 | Scored | |
| ARC-AGI-1 (ARC Prize verified) | Semi-private set score | ARC Prize | 254 | Scored | |
| ARC-AGI-2 | Accuracy | Epoch AI | 256 | Scored | |
| ARC-AGI-2 (ARC Prize verified) | Semi-private set score | ARC Prize | 252 | Scored | |
| ARC-AGI-3 | Semi-private games score | ARC Prize | 77 | Scored | |
| BioMysteryBench | Accuracy (%) | Vals AI | 31 | 2026-11-09 | Scored |
| Chess Puzzles | Accuracy | Epoch AI | 161 | 2026-11-06 | Scored |
| CritPt | Accuracy | Epoch AI | 268 | Scored | |
| EnigmaEval | Accuracy | Epoch AI | 43 | Scored | |
| Extended NYT Connections | Puzzle score (%) | Lech Mazur | 129 | 2026-11-10 | Scored |
| GPQA Diamond | Accuracy | Epoch AI | 317 | 2026-11-06 | Scored |
| GPQA Diamond (Vals run) | Accuracy (%) | Vals AI | 126 | 2026-09-01 | Scored |
| Kagi Reasoning Benchmark | Accuracy (%) | Kagi | 104 | Scored | |
| LiveBench 2.0 Data Analysis | Score (%) | LiveBench | 32 | 2026-11-06 | Scored |
| LiveBench 2.0 Reasoning | Score (%) | LiveBench | 32 | 2026-11-06 | Scored |
| Mystery Game Puzzles | Accuracy | Epoch AI | 146 | 2026-11-06 | Scored |
| MysteryMechanism | Accuracy (%) | Vals AI | 30 | 2026-11-09 | Scored |
| SimpleBench | Accuracy | Epoch AI | 95 | Scored | |
| SimpleBench | Score (%), 5 runs | SimpleBench | 101 | 2026-11-10 | Scored |
Math
Competition and research math.
| Benchmark | Metric | Source | Models | Latest | Status |
|---|---|---|---|---|---|
| FrontierMath Tier 4 | Accuracy | Epoch AI | 81 | 2026-11-06 | Scored |
| FrontierMath Tier 4 (2025 set) | Accuracy | Epoch AI | 70 | 2026-06-08 | Scored |
| FrontierMath Tiers 1-3 | Accuracy | Epoch AI | 125 | 2026-11-06 | Scored |
| FrontierMath Tiers 1-3 (2025 set) | Accuracy | Epoch AI | 96 | 2026-06-08 | Scored |
| MATH Level 5 | Accuracy | Epoch AI | 100 | 2025-10-30 | Scored |
| MathArena ArXivMath (latest month) | Accuracy (%) | MathArena | 5 | Scored | |
| MathArena BrokenArXiv (latest month) | Accuracy (%) | MathArena | 5 | Scored | |
| MathArena Final-Answer Competitions | Mean accuracy (%) | MathArena | 27 | Scored | |
| OTIS Mock AIME 2024-2025 | Accuracy | Epoch AI | 303 | 2026-11-06 | Scored |
| ProofBench | Accuracy | Epoch AI | 85 | Scored | |
| ProofBench v1.1 | Accuracy (%) | Vals AI | 52 | 2026-11-09 | Scored |
Knowledge
Factual accuracy and expert knowledge.
| Benchmark | Metric | Source | Models | Latest | Status |
|---|---|---|---|---|---|
| Humanity's Last Exam | Accuracy | Epoch AI | 49 | Scored | |
| Legal Research Bench | Accuracy (%) | Vals AI | 87 | 2026-11-09 | Scored |
| MMLU-Pro (Vals run) | Accuracy (%) | Vals AI | 126 | 2026-09-01 | Scored |
| SimpleQA Verified | Accuracy | Epoch AI | 103 | 2026-11-06 | Scored |
Frontend
Web interfaces people prefer in blind tests.
| Benchmark | Metric | Source | Models | Latest | Status |
|---|---|---|---|---|---|
| LMArena WebDev | Elo | LMArena | 143 | 2026-11-09 | Scored |
| LiveBench 2.0 Design | Score (%) | LiveBench | 32 | 2026-11-06 | Scored |
| Vibe Code Bench v1.1 | Accuracy (%) | Vals AI | 119 | 2026-11-09 | Scored |
General
Blind human preference on everyday chat.
| Benchmark | Metric | Source | Models | Latest | Status |
|---|---|---|---|---|---|
| Creative Story-Writing (Lech Mazur) | Win chance (%) | Lech Mazur | 58 | 2026-10-23 | Scored |
| EQ-Bench 4 | Judge Elo | EQ-Bench | 32 | 2026-10-24 | Scored |
| EQ-Bench Creative Writing v3 | Judge Elo | EQ-Bench | 115 | Scored | |
| EQ-Bench Longform Creative Writing | Rubric score (0-100) | EQ-Bench | 115 | Scored | |
| LLM Debate Benchmark | Elo | Lech Mazur | 58 | 2026-10-26 | Scored |
| LMArena Text | Elo, style controlled | LMArena | 281 | 2026-11-09 | Scored |
Multimodal
Understanding images, documents and charts.
| Benchmark | Metric | Source | Models | Latest | Status |
|---|---|---|---|---|---|
| GeoBench | Accuracy | Epoch AI | 30 | Scored | |
| LMArena Vision | Elo, style controlled | LMArena | 137 | 2026-11-09 | Scored |
| MMMU-Pro (Vals run) | Accuracy (%) | Vals AI | 89 | 2026-09-01 | Scored |
| VPCT | Accuracy | Epoch AI | 36 | Scored |