allbenched allbenched Allbenched allbenched allbenched AllBenched allbenched allbenched allbenched

Benchmarks / Multimodal

MMMU-Pro (Vals run)

College-level questions that need both an image and text to answer. Run by Vals AI.

CategoryMultimodal
MetricAccuracy (%)
SourceVals AI (evaluator)
LicenseNone stated
Benchmark pagewww.vals.ai
StatusScored
Weight0.9 within Multimodal
FitDiscrimination 0.40, noise 0.11

Results

#ModelProviderResultDate
1 Claude Fable 5.1 Max Anthropic 90.6% 2026-09-01
2 Claude Opus 5 Max Anthropic 89.9% 2026-09-01
3 Claude Fable 5 Max Anthropic 89.3% 2026-09-01
4 Gemini 3.8 Flash High Google 89.1% 2026-09-01
5 Gemini 3.7 Flash High Google 89.0% 2026-09-01
6 GPT-5.6 Sol Max OpenAI 88.8% 2026-09-01
7 Gemini 3.6 Flash High Google 88.4% 2026-09-01
8 Gemini 3.5 Flash High Google 88.3% 2026-09-01
9 GPT-5.5 Extra High OpenAI 88.3% 2026-09-01
10 Gemini 3.1 Pro Preview High Google 88.2% 2026-09-01
11 Kimi K3 Moonshot AI 88.2% 2026-09-01
12 Qwen3.8 Max Alibaba 88.0% 2026-09-01
13 Gemini 3 Flash Preview High Google 87.6% 2026-09-01
14 GPT-5.4 Extra High OpenAI 87.5% 2026-09-01
15 Gemini 3 Pro Preview High Google 87.5% 2026-09-01
16 Muse Spark Meta 87.4% 2026-09-01
17 GPT-5.2 Extra High OpenAI 86.7% 2026-09-01
18 Claude Opus 4.8 Max Anthropic 86.6% 2026-09-01
19 Muse Spark 1.1 Extra High Meta 86.6% 2026-09-01
20 GPT-5.6 Terra Max OpenAI 86.5% 2026-09-01
21 Kimi K2.6 Moonshot AI 86.3% 2026-09-01
22 Muse Spark 1.2 Extra High Meta 86.1% 2026-09-01
23 GLM-5.3-Flash Max Zhipu AI 86.0% 2026-09-01
24 Claude Opus 4.7 Max Anthropic 85.5% 2026-09-01
25 GPT-5.6 Luna Max OpenAI 85.0% 2026-09-01
26 Kimi K2.5 Thinking Moonshot AI 84.3% 2026-09-01
27 Qwen3.6 Plus Alibaba 84.2% 2026-09-01
28 Qwen3.8 27B Extra High Alibaba 83.9% 2026-09-01
29 Claude Opus 4.6 Thinking Anthropic 83.9% 2026-09-01
30 Gemini 3.5 Flash Lite High Google 83.6% 2026-09-01
31 Claude Sonnet 4.6 Max Anthropic 83.6% 2026-09-01
32 Grok 4.20 (Reasoning) xAI 83.5% 2026-09-01
33 GPT-5.1 High OpenAI 83.2% 2026-09-01
34 Grok 4.3 xAI 83.1% 2026-09-01
35 Claude Sonnet 5 Max Anthropic 83.0% 2026-09-01
36 Claude Opus 4.5 Thinking Anthropic 82.9% 2026-09-01
37 Gemini 3.1 Flash Lite Preview High Google 82.5% 2026-09-01
38 Qwen3.5 Flash Alibaba 81.9% 2026-09-01
39 GPT-5 High OpenAI 81.5% 2026-09-01
40 Gemini 2.5 Pro Preview 06-05 Google 81.3% 2026-09-01
41 MiniMax-M3 MiniMax 81.2% 2026-09-01
42 Claude Opus 4.5 High Anthropic 81.1% 2026-09-01
43 Gemini 2.5 Flash (Sep 2025) Thinking Google 80.8% 2026-09-01
44 o3 High OpenAI 80.4% 2026-09-01
45 MiMo-V2.5 Xiaomi 80.0% 2026-09-01
46 o4-mini High OpenAI 79.7% 2026-09-01
47 Gemini 2.5 Flash (Sep 2025) Google 79.5% 2026-09-01
48 Claude Sonnet 4.5 Thinking Anthropic 79.3% 2026-09-01
49 GPT-5.4 mini Extra High OpenAI 79.2% 2026-09-01
50 GPT-5 Mini High OpenAI 78.9% 2026-09-01
51 Inkling Thinking Machines 78.5% 2026-09-01
52 Claude Opus 4.1 Thinking Anthropic 77.5% 2026-09-01
53 o1 High OpenAI 77.4% 2026-09-01
54 Grok 4 xAI 76.3% 2026-09-01
55 Gemini 2.5 Flash-Lite Preview 09-2025 Thinking Google 75.4% 2026-09-01
56 Claude Sonnet 3.7 Thinking Anthropic 75.1% 2026-09-01
57 Claude Sonnet 4 Thinking Anthropic 74.9% 2026-09-01
58 Claude Opus 4.1 Anthropic 73.7% 2026-09-01
59 GPT-5.4 nano High OpenAI 73.6% 2026-09-01
60 Claude Opus 4 Anthropic 73.3% 2026-09-01
61 X-Ai/Grok-4-Fast-Reasoning xAI 72.8% 2026-09-01
62 Grok 4.1 Fast (Reasoning) xAI 72.7% 2026-09-01
63 Gemini 2.5 Flash-Lite Preview 09-2025 Google 72.5% 2026-09-01
64 Claude Sonnet 4 Anthropic 72.4% 2026-09-01
65 GPT-4.1 High OpenAI 72.4% 2026-09-01
66 Gemini 2.5 Flash (Apr 2025) Thinking Google 71.9% 2026-09-01
67 Claude Sonnet 3.7 Anthropic 71.5% 2026-09-01
68 GPT-5 Nano High OpenAI 70.9% 2026-09-01
69 GPT-4.1 mini High OpenAI 70.5% 2026-09-01
70 Gemini 2.0 Flash Google 69.8% 2026-09-01
71 Claude Sonnet 3.5 v2 Anthropic 68.8% 2026-09-01
72 Gemini 2.5 Flash (Apr 2025) Google 68.0% 2026-09-01
73 Mistral Large 3 Mistral 66.2% 2026-09-01
74 Gemini 1.5 Pro (Sept 2024) Google 65.5% 2026-09-01
75 Magistral Small 1.2 Mistral 65.2% 2026-09-01
76 Magistral Medium 1.2 Mistral 64.6% 2026-09-01
77 GPT-4o (2024-08-06) High OpenAI 64.0% 2026-09-01
78 Grok 4.1 Fast xAI 63.7% 2026-09-01
79 X-Ai/Grok-4-Fast-Non-Reasoning xAI 63.4% 2026-09-01
80 Mistral Medium 3 Mistral 63.0% 2026-09-01
81 GPT-4o (2024-11-20) High OpenAI 62.2% 2026-09-01
82 Grok 4.5 High xAI 61.8% 2026-09-01
83 Mistral Small 3.1 Mistral 60.1% 2026-09-01
84 Grok-2 (Dec 2024) xAI 57.3% 2026-09-01
85 Gemini 1.5 Flash (Sep 2024) Google 57.2% 2026-09-01
86 GPT-4o mini High OpenAI 56.6% 2026-09-01
87 GPT-4.1 nano High OpenAI 55.1% 2026-09-01
88 Claude Haiku 4.5 Thinking Anthropic 46.1% 2026-09-01
89 Qwen3.5 Plus Thinking Alibaba 22.8% 2026-09-01