o3
| Released | 2025-04-16 |
|---|---|
| Price per 1M | $2.00 in · $8.00 out |
| Overall score | 51.3 · rank 102 |
| Context window | 200,000 tokens |
| Provider | OpenAI |
| Weights | Proprietary |
Scores
| Dimension | Score | Rank | Benchmarks |
|---|---|---|---|
| Overall: Balanced | 51.3 | 102 | 31 |
| Overall: Coding | 50.5 | 106 | 17 |
| Overall: Reasoning | 52.2 | 96 | 16 |
| Overall: Agentic work | 49.4 | 118 | 17 |
| Overall: Frontend | 51.2 | 79 | 8 |
| Coding | 51.1 | 110 | 4 |
| Agents | 48.0 | 118 | 4 |
| Reasoning | 52.0 | 98 | 9 |
| Math | 51.9 | 95 | 4 |
| Knowledge | 53.4 | 75 | 3 |
| General | 54.4 | 71 | 3 |
| Multimodal | 51.7 | 67 | 4 |
| Value vs price | −0.2 | 121 |
Configurations
| Configuration | Balanced | Benchmarks |
|---|---|---|
| Unspecified setting | 50.4 ±2.4 | 9 |
| High | 52.4 ±2.6 | 20 |
| Low | 49.3 ±5.0 | 6 |
| Medium | 50.1 ±1.4 | 16 |
Evidence
Coding 51.1 ±2.2
| Benchmark | Result | Source | Date |
|---|---|---|---|
| Aider Polyglot | 81.3% | Epoch AI | 2026-11-11 |
| GSO | 8.8% | Epoch AI | 2026-11-11 |
| SWE-bench Verified (Epoch run) | 62.3% | Epoch AI | 2026-02-12 |
| LiveCodeBench (Vals run) | 83.9% | Vals AI | 2026-09-01 |
Agents 48.0 ±2.1
| Benchmark | Result | Source | Date |
|---|---|---|---|
| BFCL v4 | 63.1% | BFCL | 2026-04-12 |
| METR 50% Time Horizon | $120 | METR | 2026-11-11 |
| OSWorld-Verified | 23.0% | OSWorld-Verified | 2025-07-28 |
| METR task suite | 65.4% | Epoch AI | 2026-11-11 |
| DeepResearch Bench | 45.2% | Epoch AI | 2026-11-11 |
| OSWorld (v1) | 23.0% | Epoch AI | 2026-11-11 |
Reasoning 52.0 ±1.8
| Benchmark | Result | Source | Date |
|---|---|---|---|
| ARC-AGI-2 | 6.5% | Epoch AI | 2026-11-11 |
| ARC-AGI-2 (ARC Prize verified) | 6.5% | ARC Prize | 2026-11-11 |
| CritPt | 1.1% | Epoch AI | 2026-11-11 |
| ARC-AGI-1 (ARC Prize verified) | 60.8% | ARC Prize | 2026-11-11 |
| ARC-AGI-1 | 60.8% | Epoch AI | 2026-11-11 |
| EnigmaEval | 13.1% | Epoch AI | 2026-11-11 |
| GPQA Diamond | 81.8% | Epoch AI | 2025-04-16 |
| GPQA Diamond (Vals run) | 84.1% | Vals AI | 2026-09-01 |
| Kagi Reasoning Benchmark | 67.6% | Kagi | 2026-11-11 |
| Chess Puzzles | 37.0% | Epoch AI | 2026-10-13 |
| SimpleBench | 53.1% | SimpleBench | 2026-11-11 |
| SimpleBench | 53.1% | Epoch AI | 2026-11-11 |
| Mystery Game Puzzles | 29.0% | Epoch AI | 2026-08-27 |
Math 51.9 ±1.4
| Benchmark | Result | Source | Date |
|---|---|---|---|
| MATH Level 5 | 97.8% | Epoch AI | 2025-04-16 |
| FrontierMath Tiers 1-3 | 33.3% | Epoch AI | 2026-08-27 |
| FrontierMath Tier 4 (2025 set) | 2.1% | Epoch AI | 2025-07-01 |
| OTIS Mock AIME 2024-2025 | 84.4% | Epoch AI | 2026-08-07 |
| FrontierMath Tiers 1-3 (2025 set) | 18.7% | Epoch AI | 2025-11-16 |
Knowledge 53.4 ±2.7
| Benchmark | Result | Source | Date |
|---|---|---|---|
| SimpleQA Verified | 49.4% | Epoch AI | 2026-08-27 |
| Humanity's Last Exam | 20.3% | Epoch AI | 2026-11-11 |
| MMLU-Pro (Vals run) | 85.6% | Vals AI | 2026-09-01 |
General 54.4 ±3.2
| Benchmark | Result | Source | Date |
|---|---|---|---|
| LMArena Text | 1,432 | LMArena | 2026-11-09 |
| EQ-Bench Creative Writing v3 | 1,675 | EQ-Bench | 2026-11-11 |
| EQ-Bench Longform Creative Writing | 62.7% | EQ-Bench | 2026-11-11 |
Multimodal 51.7 ±2.8
| Benchmark | Result | Source | Date |
|---|---|---|---|
| LMArena Vision | 1,216 | LMArena | 2026-11-09 |
| MMMU-Pro (Vals run) | 80.4% | Vals AI | 2026-09-01 |
| VPCT | 52.0% | Epoch AI | 2026-11-11 |
| GeoBench | 74.0% | Epoch AI | 2026-11-11 |
Results not used in the score
| Benchmark | Result | Source | Why not used |
|---|---|---|---|
| ARC-AGI-1 | 53.8% | Epoch AI | A stronger configuration result was used |
| ARC-AGI-1 | 41.5% | Epoch AI | A stronger configuration result was used |
| ARC-AGI-1 (ARC Prize verified) | 53.8% | ARC Prize | A stronger configuration result was used |
| ARC-AGI-1 (ARC Prize verified) | 41.5% | ARC Prize | A stronger configuration result was used |
| ARC-AGI-2 | 3.0% | Epoch AI | A stronger configuration result was used |
| ARC-AGI-2 | 2.0% | Epoch AI | A stronger configuration result was used |
| ARC-AGI-2 (ARC Prize verified) | 3.0% | ARC Prize | A stronger configuration result was used |
| ARC-AGI-2 (ARC Prize verified) | 2.0% | ARC Prize | A stronger configuration result was used |
| Aider Polyglot | 76.9% | Epoch AI | A stronger configuration result was used |
| Aider Polyglot | 76.9% | Epoch AI | A stronger configuration result was used |
| BFCL v4 | 48.6% | BFCL | A stronger configuration result was used |
| Chess Puzzles | 36.0% | Epoch AI | A stronger configuration result was used |
| Chess Puzzles | 36.0% | Epoch AI | A stronger configuration result was used |
| EnigmaEval | 11.9% | Epoch AI | A stronger configuration result was used |
| FrontierMath Tiers 1-3 | 29.8% | Epoch AI | A stronger configuration result was used |
| FrontierMath Tiers 1-3 | 19.3% | Epoch AI | A stronger configuration result was used |
| FrontierMath Tiers 1-3 (2025 set) | 16.9% | Epoch AI | A stronger configuration result was used |
| FrontierMath Tiers 1-3 (2025 set) | 9.7% | Epoch AI | A stronger configuration result was used |
| GPQA Diamond | 80.8% | Epoch AI | A stronger configuration result was used |
| GPQA Diamond | 79.8% | Epoch AI | A stronger configuration result was used |
| GeoBench | 60.0% | Epoch AI | A stronger configuration result was used |
| Humanity's Last Exam | 19.2% | Epoch AI | A stronger configuration result was used |
| METR task suite | 63.6% | Epoch AI | A stronger configuration result was used |
| Mystery Game Puzzles | 23.0% | Epoch AI | A stronger configuration result was used |
| OSWorld (v1) | 17.2% | Epoch AI | A stronger configuration result was used |
| OSWorld (v1) | 9.1% | Epoch AI | A stronger configuration result was used |
| OSWorld-Verified | 9.1% | OSWorld-Verified | A stronger configuration result was used |
| OSWorld-Verified | 17.2% | OSWorld-Verified | A stronger configuration result was used |
| OTIS Mock AIME 2024-2025 | 60.0% | Epoch AI | A stronger configuration result was used |
| OTIS Mock AIME 2024-2025 | 83.9% | Epoch AI | A stronger configuration result was used |
Names matched to this model
o3, o3-2025-04-16