| ARC-AGI-1 |
12.3% |
Epoch AI |
A stronger configuration result was used |
| ARC-AGI-1 |
12.3% |
Epoch AI |
A stronger configuration result was used |
| ARC-AGI-1 |
78.7% |
Epoch AI |
A stronger configuration result was used |
| ARC-AGI-1 |
72.7% |
Epoch AI |
A stronger configuration result was used |
| ARC-AGI-1 |
55.7% |
Epoch AI |
A stronger configuration result was used |
| ARC-AGI-1 (ARC Prize verified) |
12.3% |
ARC Prize |
A stronger configuration result was used |
| ARC-AGI-1 (ARC Prize verified) |
78.7% |
ARC Prize |
A stronger configuration result was used |
| ARC-AGI-1 (ARC Prize verified) |
72.7% |
ARC Prize |
A stronger configuration result was used |
| ARC-AGI-1 (ARC Prize verified) |
55.7% |
ARC Prize |
A stronger configuration result was used |
| ARC-AGI-2 |
0.8% |
Epoch AI |
A stronger configuration result was used |
| ARC-AGI-2 |
43.3% |
Epoch AI |
A stronger configuration result was used |
| ARC-AGI-2 |
26.7% |
Epoch AI |
A stronger configuration result was used |
| ARC-AGI-2 |
9.7% |
Epoch AI |
A stronger configuration result was used |
| ARC-AGI-2 (ARC Prize verified) |
0.8% |
ARC Prize |
A stronger configuration result was used |
| ARC-AGI-2 (ARC Prize verified) |
43.3% |
ARC Prize |
A stronger configuration result was used |
| ARC-AGI-2 (ARC Prize verified) |
26.7% |
ARC Prize |
A stronger configuration result was used |
| ARC-AGI-2 (ARC Prize verified) |
9.7% |
ARC Prize |
A stronger configuration result was used |
| BFCL v4 |
45.3% |
BFCL |
A stronger configuration result was used |
| Chess Puzzles |
2.0% |
Epoch AI |
A stronger configuration result was used |
| Chess Puzzles |
45.0% |
Epoch AI |
A stronger configuration result was used |
| Chess Puzzles |
36.0% |
Epoch AI |
A stronger configuration result was used |
| Chess Puzzles |
26.0% |
Epoch AI |
A stronger configuration result was used |
| Extended NYT Connections |
14.5% |
Lech Mazur |
A stronger configuration result was used |
| Extended NYT Connections |
68.1% |
Lech Mazur |
A stronger configuration result was used |
| Extended NYT Connections |
60.6% |
Lech Mazur |
A stronger configuration result was used |
| Extended NYT Connections |
54.2% |
Lech Mazur |
A stronger configuration result was used |
| FrontierMath Tier 4 (2025 set) |
18.8% |
Epoch AI |
A stronger configuration result was used |
| FrontierMath Tier 4 (2025 set) |
16.7% |
Epoch AI |
A stronger configuration result was used |
| FrontierMath Tier 4 (2025 set) |
6.3% |
Epoch AI |
A stronger configuration result was used |
| FrontierMath Tiers 1-3 (2025 set) |
40.3% |
Epoch AI |
A stronger configuration result was used |
| FrontierMath Tiers 1-3 (2025 set) |
36.9% |
Epoch AI |
A stronger configuration result was used |
| FrontierMath Tiers 1-3 (2025 set) |
26.6% |
Epoch AI |
A stronger configuration result was used |
| GPQA Diamond |
73.2% |
Epoch AI |
A stronger configuration result was used |
| GPQA Diamond |
88.2% |
Epoch AI |
A stronger configuration result was used |
| GPQA Diamond |
87.9% |
Epoch AI |
A stronger configuration result was used |
| GPQA Diamond |
82.7% |
Epoch AI |
A stronger configuration result was used |
| LMArena Text |
1,435 |
LMArena |
A stronger configuration result was used |
| LMArena Vision |
1,228 |
LMArena |
A stronger configuration result was used |
| Mystery Game Puzzles |
14.0% |
Epoch AI |
A stronger configuration result was used |
| Mystery Game Puzzles |
22.0% |
Epoch AI |
A stronger configuration result was used |
| Mystery Game Puzzles |
10.0% |
Epoch AI |
A stronger configuration result was used |
| OTIS Mock AIME 2024-2025 |
62.2% |
Epoch AI |
A stronger configuration result was used |
| OTIS Mock AIME 2024-2025 |
96.1% |
Epoch AI |
A stronger configuration result was used |
| OTIS Mock AIME 2024-2025 |
93.9% |
Epoch AI |
A stronger configuration result was used |
| OTIS Mock AIME 2024-2025 |
78.9% |
Epoch AI |
A stronger configuration result was used |
| Remote Labor Index |
2.1% |
Epoch AI |
A stronger configuration result was used |
| SimpleBench |
45.8% |
Epoch AI |
A stronger configuration result was used |
| SimpleQA Verified |
34.3% |
Epoch AI |
A stronger configuration result was used |
| SimpleQA Verified |
32.7% |
Epoch AI |
A stronger configuration result was used |
| SimpleQA Verified |
32.8% |
Epoch AI |
A stronger configuration result was used |
| Terminal-Bench 2.0 |
62.9% |
Epoch AI |
A stronger configuration result was used |
| Terminal-Bench 2.0 |
60.7% |
Epoch AI |
A stronger configuration result was used |
| Terminal-Bench 2.0 |
54.0% |
Epoch AI |
A stronger configuration result was used |
| Terminal-Bench 2.0 |
64.9% |
Epoch AI |
A stronger configuration result was used |
| Terminal-Bench 2.0 |
62.9% |
Epoch AI |
A stronger configuration result was used |
| VPCT |
67.0% |
Epoch AI |
A stronger configuration result was used |
| tau2-bench Airline |
52.5% |
tau2-bench |
A stronger configuration result was used |
| tau2-bench Banking Knowledge |
12.6% |
tau2-bench |
A stronger configuration result was used |
| tau2-bench Retail |
75.0% |
tau2-bench |
A stronger configuration result was used |
| tau2-bench Telecom |
57.2% |
tau2-bench |
A stronger configuration result was used |