MMLU-Pro (Vals run)
Ten-option multiple-choice questions across 14 academic subjects. Near saturation. Run by Vals AI.
| Category | Knowledge |
|---|---|
| Metric | Accuracy (%) |
| Source | Vals AI (evaluator) |
| License | None stated |
| Benchmark page | www.vals.ai |
| Status | Scored |
| Weight | 0.6 within Knowledge |
| Fit | Discrimination 0.25, noise 0.10 |
Results
| # | Model | Provider | Result | Date |
|---|---|---|---|---|
| 1 | Claude Fable 5.1 Max | Anthropic | 92.4% | 2026-09-01 |
| 2 | Claude Opus 5 Max | Anthropic | 91.6% | 2026-09-01 |
| 3 | Claude Fable 5 Max | Anthropic | 91.5% | 2026-09-01 |
| 4 | Gemini 3.1 Pro Preview High | 91.0% | 2026-09-01 | |
| 5 | Gemini 3.8 Flash High | 90.2% | 2026-09-01 | |
| 6 | Gemini 3.7 Flash High | 90.1% | 2026-09-01 | |
| 7 | Gemini 3 Pro Preview | 90.1% | 2026-09-01 | |
| 8 | Claude Opus 4.7 Max | Anthropic | 89.9% | 2026-09-01 |
| 9 | Claude Opus 4.8 Max | Anthropic | 89.6% | 2026-09-01 |
| 10 | Gemini 3.5 Flash High | 89.5% | 2026-09-01 | |
| 11 | Grok 4.6 High | xAI | 89.4% | 2026-09-01 |
| 12 | Qwen3.7 Max | Alibaba | 89.3% | 2026-09-01 |
| 13 | Gemini 3.6 Flash High | 89.3% | 2026-09-01 | |
| 14 | Grok 4.5 High | xAI | 89.2% | 2026-09-01 |
| 15 | Claude Opus 4.6 Thinking | Anthropic | 89.1% | 2026-09-01 |
| 16 | GPT-5.6 Sol Max | OpenAI | 89.1% | 2026-09-01 |
| 17 | Muse Spark 1.1 Extra High | Meta | 88.7% | 2026-09-01 |
| 18 | Qwen3.8 Max | Alibaba | 88.6% | 2026-09-01 |
| 19 | Gemini 3 Flash Preview High | 88.6% | 2026-09-01 | |
| 20 | Muse Spark 1.2 Extra High | Meta | 88.3% | 2026-09-01 |
| 21 | GPT-5.5 Extra High | OpenAI | 88.1% | 2026-09-01 |
| 22 | Kimi K3 Max | Moonshot AI | 88.0% | 2026-09-01 |
| 23 | Claude Opus 4.1 Thinking | Anthropic | 87.9% | 2026-09-01 |
| 24 | Qwen3.6 Plus | Alibaba | 87.7% | 2026-09-01 |
| 25 | Kimi K2.6 | Moonshot AI | 87.6% | 2026-09-01 |
| 26 | Claude Sonnet 5 Max | Anthropic | 87.5% | 2026-09-01 |
| 27 | GPT-5.4 Extra High | OpenAI | 87.5% | 2026-09-01 |
| 28 | Claude Sonnet 4.5 Thinking | Anthropic | 87.4% | 2026-09-01 |
| 29 | Claude Sonnet 4.6 Max | Anthropic | 87.3% | 2026-09-01 |
| 30 | Muse Spark | Meta | 87.3% | 2026-09-01 |
| 31 | Claude Opus 4.5 Thinking | Anthropic | 87.3% | 2026-09-01 |
| 32 | DeepSeek V4 Pro Max | DeepSeek | 87.2% | 2026-09-01 |
| 33 | Claude Opus 4.1 | Anthropic | 87.2% | 2026-09-01 |
| 34 | Qwen3.5 Plus Thinking | Alibaba | 87.2% | 2026-09-01 |
| 35 | MiniMax-M2.1 | MiniMax | 87.0% | 2026-09-01 |
| 36 | DeepSeek V4 Pro 0813 Max | DeepSeek | 87.0% | 2026-09-01 |
| 37 | GLM-5.1 | Zhipu AI | 86.9% | 2026-09-01 |
| 38 | GLM-5.3 Max | Zhipu AI | 86.8% | 2026-09-01 |
| 39 | GLM-5.2 | Zhipu AI | 86.7% | 2026-09-01 |
| 40 | GPT-5.6 Terra Extra High | OpenAI | 86.7% | 2026-09-01 |
| 41 | GPT-5 High | OpenAI | 86.5% | 2026-09-01 |
| 42 | GPT-5.1 High | OpenAI | 86.4% | 2026-09-01 |
| 43 | Inkling | Thinking Machines | 86.3% | 2026-09-01 |
| 44 | Grok 4.20 (Reasoning) | xAI | 86.3% | 2026-09-01 |
| 45 | Gemini 3.1 Flash Lite Preview | 86.2% | 2026-09-01 | |
| 46 | GPT-5.2 Extra High | OpenAI | 86.2% | 2026-09-01 |
| 47 | DeepSeek V4 Flash 0731 High | DeepSeek | 86.2% | 2026-09-01 |
| 48 | Claude Opus 4 | Anthropic | 86.2% | 2026-09-01 |
| 49 | GLM-5.3-Flash Max | Zhipu AI | 86.1% | 2026-09-01 |
| 50 | GPT-5.6 Luna Max | OpenAI | 86.0% | 2026-09-01 |
| 51 | GLM-5 Thinking | Zhipu AI | 86.0% | 2026-09-01 |
| 52 | Kimi K2.5 Thinking | Moonshot AI | 85.9% | 2026-09-01 |
| 53 | Grok 4.3 High | xAI | 85.8% | 2026-09-01 |
| 54 | Gemini 3.5 Flash Lite High | 85.8% | 2026-09-01 | |
| 55 | Nemotron 3 Ultra 550B A55B | NVIDIA | 85.8% | 2026-09-01 |
| 56 | o3 High | OpenAI | 85.6% | 2026-09-01 |
| 57 | Claude Opus 4.5 High | Anthropic | 85.6% | 2026-09-01 |
| 58 | Inkling Small | Thinking Machines | 85.6% | 2026-09-01 |
| 59 | Grok 4 | xAI | 85.3% | 2026-09-01 |
| 60 | Qwen3 Max | Alibaba | 85.0% | 2026-09-01 |
| 61 | DeepSeek V3.2 Thinking | DeepSeek | 84.9% | 2026-09-01 |
| 62 | MiMo-V2.5-Pro | Xiaomi | 84.6% | 2026-09-01 |
| 63 | GPT-5.4 mini Extra High | OpenAI | 84.6% | 2026-09-01 |
| 64 | Qwen3.8 27B Extra High | Alibaba | 84.3% | 2026-09-01 |
| 65 | MiniMax-M3 | MiniMax | 84.2% | 2026-09-01 |
| 66 | Grok 4.1 Fast (Reasoning) | xAI | 84.2% | 2026-09-01 |
| 67 | Qwen3.5 Flash | Alibaba | 84.1% | 2026-09-01 |
| 68 | Gemini 2.5 Pro Preview 06-05 | 84.1% | 2026-09-01 | |
| 69 | Claude Sonnet 4 Thinking | Anthropic | 83.9% | 2026-09-01 |
| 70 | Gemini 2.5 Flash (Sep 2025) | 83.7% | 2026-09-01 | |
| 71 | Gemini 2.5 Flash (Sep 2025) Thinking | 83.7% | 2026-09-01 | |
| 72 | Qwen3 Max Preview | Alibaba | 83.5% | 2026-09-01 |
| 73 | o1 High | OpenAI | 83.5% | 2026-09-01 |
| 74 | DeepSeek-R1 | DeepSeek | 83.2% | 2026-09-01 |
| 75 | DeepSeek V3.2 | DeepSeek | 83.1% | 2026-09-01 |
| 76 | MiMo-V2.5 | Xiaomi | 82.9% | 2026-09-01 |
| 77 | GLM-4.7 | Zhipu AI | 82.7% | 2026-09-01 |
| 78 | Claude Sonnet 3.7 Thinking | Anthropic | 82.7% | 2026-09-01 |
| 79 | GPT-5 Mini High | OpenAI | 82.2% | 2026-09-01 |
| 80 | GLM-4.6 | Zhipu AI | 82.2% | 2026-09-01 |
| 81 | Qwen3 235B-A22B | Alibaba | 81.2% | 2026-09-01 |
| 82 | GLM-4.5 | Zhipu AI | 81.2% | 2026-09-01 |
| 83 | Kimi K2 Thinking | Moonshot AI | 81.1% | 2026-09-01 |
| 84 | Claude Sonnet 3.7 | Anthropic | 80.7% | 2026-09-01 |
| 85 | o4-mini High | OpenAI | 80.6% | 2026-09-01 |
| 86 | GPT-4.1 High | OpenAI | 80.5% | 2026-09-01 |
| 87 | MiniMax-M2.7 | MiniMax | 80.4% | 2026-09-01 |
| 88 | MiniMax-M2.5 | MiniMax | 80.1% | 2026-09-01 |
| 89 | Grok 3 | xAI | 79.9% | 2026-09-01 |
| 90 | Mistral Large 3 | Mistral | 79.8% | 2026-09-01 |
| 91 | X-Ai/Grok-4-Fast-Reasoning | xAI | 79.7% | 2026-09-01 |
| 92 | DeepSeek V3 0324 | DeepSeek | 79.5% | 2026-09-01 |
| 93 | Claude Sonnet 4 | Anthropic | 79.4% | 2026-09-01 |
| 94 | Kimi K2 Instruct | Moonshot AI | 79.4% | 2026-09-01 |
| 95 | GPT OSS 120B | OpenAI | 79.2% | 2026-09-01 |
| 96 | Gemini 2.5 Flash-Lite Preview 09-2025 Thinking | 79.1% | 2026-09-01 | |
| 97 | Claude Haiku 4.5 Thinking | Anthropic | 78.7% | 2026-09-01 |
| 98 | o3-mini High | OpenAI | 78.7% | 2026-09-01 |
| 99 | Gemini 2.5 Flash-Lite Preview 09-2025 | 78.6% | 2026-09-01 | |
| 100 | Claude Sonnet 3.5 v2 | Anthropic | 78.4% | 2026-09-01 |
| 101 | Gemini 2.0 Flash | 77.4% | 2026-09-01 | |
| 102 | GPT-4.1 mini High | OpenAI | 77.2% | 2026-09-01 |
| 103 | GPT-5.4 nano High | OpenAI | 77.2% | 2026-09-01 |
| 104 | GPT-5 Nano High | OpenAI | 76.1% | 2026-09-01 |
| 105 | Grok-2 (Dec 2024) | xAI | 75.5% | 2026-09-01 |
| 106 | Mistral Medium 3.5 High | Mistral | 75.3% | 2026-09-01 |
| 107 | Gemini 1.5 Pro (Sept 2024) | 75.3% | 2026-09-01 | |
| 108 | Mistral Medium 3 | Mistral | 75.3% | 2026-09-01 |
| 109 | Grok 4.1 Fast | xAI | 75.2% | 2026-09-01 |
| 110 | GPT-4o (2024-08-06) High | OpenAI | 74.1% | 2026-09-01 |
| 111 | DeepSeek-V3 | DeepSeek | 73.8% | 2026-09-01 |
| 112 | GPT-4o (2024-11-20) High | OpenAI | 72.6% | 2026-09-01 |
| 113 | GPT OSS 20B | OpenAI | 71.6% | 2026-09-01 |
| 114 | X-Ai/Grok-4-Fast-Non-Reasoning | xAI | 70.3% | 2026-09-01 |
| 115 | Mistral Large 2.1 | Mistral | 69.7% | 2026-09-01 |
| 116 | Command A | Cohere | 69.2% | 2026-09-01 |
| 117 | Laguna XS.2 | Poolside | 69.1% | 2026-09-01 |
| 118 | Laguna M.1 | Poolside | 68.8% | 2026-09-01 |
| 119 | Magistral Medium 1.2 | Mistral | 68.7% | 2026-09-01 |
| 120 | Mistral Small 3.1 | Mistral | 66.0% | 2026-09-01 |
| 121 | Gemini 1.5 Flash (Sep 2024) | 65.6% | 2026-09-01 | |
| 122 | mistral-small-2402 | Unknown | 64.4% | 2026-09-01 |
| 123 | Claude Haiku 3.5 | Anthropic | 64.1% | 2026-09-01 |
| 124 | GPT-4.1 nano High | OpenAI | 63.5% | 2026-09-01 |
| 125 | GPT-4o mini High | OpenAI | 62.7% | 2026-09-01 |
| 126 | Magistral Small 1.2 | Mistral | 62.1% | 2026-09-01 |