OTIS Mock AIME 2024-2025

Competition math problems in the style of the AIME, written after most training cutoffs. Run by Epoch AI.

CategoryMath
MetricAccuracy
SourceEpoch AI (evaluator)
LicenseCC BY 4.0
Benchmark pageepoch.ai
StatusScored
Weight1.0 within Math
FitDiscrimination 2.21, noise 0.55

Results

#ModelProviderResultDate
1 Kimi K3.1 Pro Max Moonshot AI 100.0% 2026-11-06
2 Gemini 4 Carbon High Google 100.0% 2026-11-02
3 GPT-6.1 Terra Max OpenAI 100.0% 2026-10-26
4 Gemini 4 Argon High Google 100.0% 2026-10-13
5 GPT-6.1 Sol Max OpenAI 100.0% 2026-09-29
6 Claude Sonnet 5.5 Max Anthropic 100.0% 2026-09-29
7 GPT-6 Sol Max OpenAI 100.0% 2026-09-22
8 Claude Opus 5.5 Max Anthropic 100.0% 2026-09-22
9 Qwen3.8 Max 0902 Extra High Alibaba 100.0% 2026-09-02
10 GPT-6 Astra Max OpenAI 100.0% 2026-08-30
11 Claude Fable 5 High Anthropic 100.0% 2026-08-06
12 GPT-5.6 Sol Max OpenAI 100.0% 2026-07-09
13 GPT-5.5 Pro Extra High OpenAI 100.0% 2026-04-24
14 GPT-5.5 Extra High OpenAI 100.0% 2026-04-24
15 Claude Fable 5.1 Max Anthropic 100.0% 2026-09-01
16 Claude Fable 5.2 Max Anthropic 99.7% 2026-11-06
17 Claude Fable 5 Max Anthropic 99.7% 2026-06-10
18 GPT-5.6 Terra Max OpenAI 99.7% 2026-07-09
19 Mistral Large 4 High Mistral 99.4% 2026-11-02
20 DeepSeek V4.1 Flash 1017 Max DeepSeek 99.4% 2026-10-19
21 Qwen3.8 Max Extra High Alibaba 99.4% 2026-08-04
22 Muse Wonder Max Meta 99.4% 2026-10-12
23 Doubao Seed Evolving (Nov 2026) High ByteDance 99.2% 2026-11-02
24 Grok 4.6 Extra High xAI 99.2% 2026-08-14
25 Muse Spark 1.3 Extra High Meta 99.2% 2026-09-17
26 GPT-6 Luna Max OpenAI 98.9% 2026-09-22
27 Gemini 3.8 Flash High Google 98.9% 2026-09-02
28 Claude Opus 5 Max Anthropic 98.9% 2026-07-24
29 Claude Haiku 5.5 Extra High Anthropic 98.9% 2026-10-07
30 DeepSeek V4 Pro 0813 Max DeepSeek 98.6% 2026-08-18
31 Mistral Large 4 Preview High Mistral 98.3% 2026-10-12
32 DeepSeek V4.1 Flash Max DeepSeek 98.3% 2026-10-08
33 GPT-5.6 Luna Max OpenAI 98.3% 2026-07-09
34 Claude Opus 4.8 Max Anthropic 98.3% 2026-06-07
35 Muse Spark Open Extra High Meta 98.1% 2026-10-15
36 Grok 4.7 Extra High xAI 98.1% 2026-09-22
37 Claude Opus 4.7 Extra High Anthropic 97.8% 2026-04-17
38 Claude Opus 5 Anthropic 97.8% 2026-08-06
39 Grok 4.6 High xAI 97.8% 2026-08-12
40 Claude Opus 4.8 Low Anthropic 97.8% 2026-08-06
41 Claude Fable 5 Low Anthropic 97.8% 2026-08-06
42 GPT-5.4 High OpenAI 97.8% 2026-07-15
43 Grok 4.5 High xAI 97.8% 2026-07-08
44 Gemini 3.7 Flash High Google 97.2% 2026-08-14
45 Kimi K3 Max Moonshot AI 97.2% 2026-07-16
46 Claude Haiku 5.5 High Anthropic 97.2% 2026-10-07
47 DeepSeek V4 Pro Max DeepSeek 96.7% 2026-06-17
48 Kimi K2.6 Moonshot AI 96.1% 2026-05-02
49 Doubao Seed 2.1 Turbo High ByteDance 96.1% 2026-10-30
50 GPT-5.2 High OpenAI 96.1% 2025-12-11
51 GPT-5.2 Extra High OpenAI 96.1% 2025-12-13
52 Gemini 3.1 Pro Preview Google 95.6% 2026-02-20
53 Kimi K2.7 Code Moonshot AI 95.6% 2026-08-07
54 Qwen3.7 Max Alibaba 95.6% 2026-08-07
55 Doubao Seed 2.1 Pro High ByteDance 95.6% 2026-10-30
56 GPT-5.6 Sol Low OpenAI 95.6% 2026-08-07
57 Gemini 3.1 Pro Preview High Google 95.6% 2026-08-06
58 Gemini 3 Flash Preview High Google 95.6% 2026-08-06
59 GPT-5.4 Medium OpenAI 95.6% 2026-07-15
60 Gemini 3.5 Flash High Google 95.6% 2026-05-25
61 DeepSeek V4 Pro High DeepSeek 95.6% 2026-08-06
62 GPT-5.4 Extra High OpenAI 95.3% 2026-03-06
63 Claude Sonnet 5 Extra High Anthropic 94.7% 2026-07-01
64 DeepSeek V4 Flash 0731 Max DeepSeek 94.4% 2026-08-02
65 Claude Opus 4.6 Thinking Anthropic 94.4% 2026-02-06
66 Gemini 3.6 Flash High Google 94.2% 2026-08-02
67 Doubao-Seed-2.0 Pro High ByteDance 93.9% 2026-10-30
68 GLM-5.3-Flash Max Zhipu AI 93.9% 2026-08-26
69 GPT-5.2 Medium OpenAI 93.9% 2025-12-11
70 GLM-5.1 Zhipu AI 93.3% 2026-08-10
71 Qwen3.7 Plus Alibaba 93.3% 2026-08-07
72 Qwen3.6 Plus Alibaba 93.3% 2026-08-07
73 Kimi K3 High Moonshot AI 93.3% 2026-08-07
74 Claude Opus 5 Low Anthropic 93.3% 2026-08-06
75 Grok 4.3 High xAI 93.3% 2026-06-17
76 Gemini 3 Flash Preview Google 92.8% 2025-12-17
77 Grok 4.20 (Reasoning) xAI 92.2% 2026-07-13
78 Kimi K2.5 Moonshot AI 92.2% 2026-02-02
79 Gemini 3 Pro Preview Google 91.4% 2025-11-19
80 GPT-5 High OpenAI 91.4% 2025-10-29
81 Qwen3.6 27B Alibaba 91.1% 2026-08-07
82 Qwen3.6 Max Preview Alibaba 91.1% 2026-08-07
83 Doubao Seed 2.0 Lite (Apr 2026) High ByteDance 91.1% 2026-10-30
84 GLM-5.3 Max Zhipu AI 91.1% 2026-08-24
85 Claude Opus 4.6 Max Anthropic 91.1% 2026-08-06
86 Inkling Small Extra High Thinking Machines 90.0% 2026-08-14
87 Muse Spark Meta 88.9% 2026-04-08
88 Qwen3.5 397B-A17B Alibaba 88.9% 2026-08-07
89 GPT-5.4 mini Extra High OpenAI 88.9% 2026-08-07
90 GPT-5.6 Terra Low OpenAI 88.9% 2026-08-07
91 Gemini 3.5 Flash Low Google 88.9% 2026-08-06
92 Inkling Extra High Thinking Machines 88.9% 2026-08-05
93 GPT OSS 120B High OpenAI 88.9% 2025-12-11
94 GPT-5.1 High OpenAI 88.6% 2025-11-13
95 DeepSeek Reasoner DeepSeek 87.8% 2025-12-16
96 GPT-5.4 nano High OpenAI 87.8% 2026-04-14
97 GPT-5.4 mini High OpenAI 87.2% 2026-04-15
98 GPT-5 Medium OpenAI 87.2% 2025-08-07
99 Qwen3 235B-A22B Thinking 2507 Alibaba 86.7% 2025-12-10
100 Qwen3.7 Flash Alibaba 86.7% 2026-08-07
101 Qwen3.5 Plus Alibaba 86.7% 2026-08-07
102 Qwen3.6 35B-A3B Alibaba 86.7% 2026-08-07
103 Nemotron 3 Ultra NVIDIA 86.7% 2026-08-10
104 Claude Opus 4.7 Max Anthropic 86.7% 2026-08-06
105 GPT-5 Mini High OpenAI 86.7% 2025-10-30
106 GLM-5.2 Max Zhipu AI 86.4% 2026-06-25
107 Claude Opus 4.5 Thinking Anthropic 86.1% 2025-11-24
108 Claude Sonnet 4.6 Thinking Anthropic 85.8% 2026-02-20
109 GPT-5.1 Medium OpenAI 85.6% 2025-11-17
110 Gemini 2.5 Pro Google 84.7% 2025-11-16
111 Qwen3.6 Flash Alibaba 84.4% 2026-08-07
112 Qwen3.5 Flash Alibaba 84.4% 2026-08-07
113 o3 Medium OpenAI 84.4% 2026-08-07
114 Claude Opus 4.8 Non-reasoning Anthropic 84.4% 2026-08-06
115 GPT-5.4 Low OpenAI 84.4% 2026-07-15
116 GPT-5.5 Low OpenAI 84.4% 2026-08-07
117 Grok 4 xAI 84.0% 2026-11-11
118 o3 High OpenAI 83.9% 2025-04-16
119 GLM-4.7 Zhipu AI 83.3% 2026-01-29
120 Kimi K2 Thinking Turbo Moonshot AI 83.1% 2025-11-13
121 Qwen3.5 397B-A17B Non-reasoning Alibaba 82.2% 2026-08-07
122 Gemma 4 26B A4B IT Minimal Google 82.2% 2026-08-28
123 Gemini 3.6 Flash Low Google 82.2% 2026-08-07
124 Claude Sonnet 4.6 Medium Anthropic 82.2% 2026-07-13
125 o4-mini High OpenAI 81.7% 2025-04-16
126 GPT-5 Nano High OpenAI 81.1% 2025-10-31
127 GLM-5 Zhipu AI 80.0% 2026-02-12
128 Gemini 3.6 Flash Minimal Google 80.0% 2026-08-07
129 Qwen3.7 Plus Non-reasoning Alibaba 80.0% 2026-08-07
130 Gemini 3.1 Flash Lite High Google 80.0% 2026-08-06
131 Claude Sonnet 5 Max Anthropic 80.0% 2026-08-06
132 Gemini 3.5 Flash Minimal Google 80.0% 2026-07-15
133 GPT-5.2 Low OpenAI 78.9% 2025-12-11
134 GPT-5 Mini Medium OpenAI 78.3% 2025-08-07
135 Qwen3.7 Flash Non-reasoning Alibaba 77.8% 2026-08-07
136 Claude Sonnet 4.5 Thinking Anthropic 77.8% 2025-10-28
137 Grok-3 mini High xAI 77.8% 2025-04-10
138 o3-mini High OpenAI 76.9% 2025-02-27
139 GLM-5.2 Low Zhipu AI 75.6% 2026-08-10
140 Claude Sonnet 4.6 High Anthropic 75.6% 2026-07-13
141 GPT-5 Nano Medium OpenAI 74.2% 2025-08-07
142 Qwen3 Max Alibaba 73.3% 2025-10-06
143 o4-mini Medium OpenAI 73.3% 2026-08-07
144 Gemma 4 31B IT Minimal Google 73.3% 2026-08-06
145 o1 High OpenAI 73.3% 2026-07-15
146 o1 Medium OpenAI 73.3% 2025-02-27
147 Gemini 2.5 Flash (Apr 2025) Google 73.1% 2025-04-22
148 MiniMax-M3 MiniMax 71.1% 2026-08-10
149 Gemini 3.5 Flash Lite High Google 71.1% 2026-08-06
150 Claude Sonnet 4.6 Max Anthropic 71.1% 2026-08-06
151 Claude Sonnet 4 Thinking Anthropic 71.1% 2025-05-22
152 Gemini 2.5 Flash Preview 05-20 Google 70.8% 2025-05-20
153 Qwen3-30B-A3B-Thinking (Jul 2025) Alibaba 70.3% 2026-08-30
154 Qwen3.5 35B-A3B Non-reasoning Alibaba 70.0% 2026-08-30
155 Seed 1.6 ByteDance 69.4% 2026-10-30
156 GPT-5.6 Sol Non-reasoning OpenAI 68.9% 2026-08-07
157 Kimi K3 Low Moonshot AI 68.9% 2026-08-07
158 Qwen3.6 35B-A3B Non-reasoning Alibaba 68.9% 2026-08-07
159 GPT-5.4 nano Low OpenAI 68.9% 2026-08-07
160 Claude Opus 4.1 Thinking Anthropic 68.9% 2025-08-05
161 GPT-5.5 Instant OpenAI 68.1% 2026-08-02
162 Seed-OSS-36B-Instruct ByteDance 67.5% 2026-08-28
163 Qwen3 32B Alibaba 66.9% 2026-08-30
164 Qwen3.6 27B Non-reasoning Alibaba 66.7% 2026-08-07
165 GPT-5.6 Luna Low OpenAI 66.7% 2026-08-07
166 Claude Haiku 4.5 Thinking Anthropic 66.7% 2025-10-22
167 DeepSeek R1 0528 DeepSeek 66.4% 2025-05-29
168 Qwen3-14B Alibaba 66.4% 2026-08-28
169 GPT OSS 20B Medium OpenAI 65.3% 2026-08-27
170 Claude Opus 4 Thinking Anthropic 64.4% 2025-05-28
171 o3-mini Medium OpenAI 63.9% 2025-02-25
172 GPT-5.1 Low OpenAI 63.9% 2025-11-25
173 Qwen3 30B A3B Alibaba 62.8% 2026-08-28
174 Qwen3 30B-A3B Instruct 2507 Alibaba 62.2% 2026-08-30
175 GPT-5.2 Non-reasoning OpenAI 62.2% 2026-07-13
176 Grok-3 mini Low xAI 62.2% 2025-04-10
177 Qwen3.5 9B Non-reasoning Alibaba 61.7% 2026-08-27
178 Gemini 3.5 Flash Lite Low Google 60.0% 2026-08-06
179 o3 Low OpenAI 60.0% 2026-07-15
180 QwQ 32B Alibaba 59.2% 2026-08-28
181 GLM-4.7-Flash Zhipu AI 58.3% 2026-08-30
182 Gemini 2.0 Flash Thinking (Jan 2025) Google 57.8% 2025-03-07
183 GPT-5.5 Non-reasoning OpenAI 57.8% 2026-08-07
184 GPT-5.4 Non-reasoning OpenAI 57.8% 2026-07-15
185 o4-mini Low OpenAI 57.8% 2026-07-13
186 Claude Sonnet 3.7 Thinking Anthropic 57.8% 2025-03-13
187 Qwen3 8B Alibaba 56.1% 2026-08-27
188 Qwen3.5 4B Non-reasoning Alibaba 55.8% 2026-08-28
189 DeepSeek-R1-Distill-Qwen-32B DeepSeek 55.6% 2026-08-28
190 Grok 3 xAI 55.6% 2025-04-10
191 GPT-5 Mini Minimal OpenAI 55.6% 2026-08-07
192 Qwen3.5 35B-A3B Alibaba 54.4% 2026-08-30
193 DeepSeek-R1 DeepSeek 53.3% 2025-02-26
194 GPT-5.6 Terra Non-reasoning OpenAI 53.3% 2026-08-07
195 o1 Low OpenAI 53.3% 2026-07-15
196 qwen3-4b-instruct-2507 Unknown 52.2% 2026-08-28
197 DeepSeek-R1-Distill-Llama-70B DeepSeek 51.4% 2025-03-07
198 Gemini 3.5 Flash Lite Minimal Google 51.1% 2026-08-06
199 GPT OSS 20B High OpenAI 50.8% 2026-08-27
200 DeepSeek-R1-Distill-Qwen-14B DeepSeek 50.6% 2026-08-28
201 DeepSeek Chat DeepSeek 48.9% 2026-07-16
202 Claude Opus 4.5 Anthropic 48.1% 2025-11-24
203 o1-mini High OpenAI 46.9% 2025-03-06
204 GPT-5.4 nano Non-reasoning OpenAI 46.7% 2026-08-07
205 GPT-5 Minimal OpenAI 46.7% 2026-07-20
206 GPT-5 Nano Low OpenAI 46.7% 2026-07-13
207 DeepSeek V4 Pro Non-reasoning DeepSeek 46.7% 2026-08-06
208 GPT-4.1 mini OpenAI 44.7% 2025-04-14
209 o1-mini Medium OpenAI 44.7% 2025-03-06
210 Gemini 3.1 Flash Lite Low Google 44.4% 2026-08-06
211 o3-mini Low OpenAI 44.4% 2026-07-20
212 DeepSeek R1 0528 Qwen3 8B DeepSeek 43.9% 2026-08-28
213 Qwen3.5 9B Alibaba 42.8% 2026-08-27
214 Claude Opus 4 Anthropic 42.2% 2025-05-22
215 GPT OSS 20B Low OpenAI 40.3% 2026-08-27
216 Claude Opus 4.1 Anthropic 40.0% 2025-08-05
217 GPT-5.6 Luna Non-reasoning OpenAI 40.0% 2026-08-07
218 GPT-4.1 OpenAI 38.3% 2025-04-14
219 DeepSeek V3 0324 DeepSeek 37.8% 2025-04-01
220 GPT-4.5 OpenAI 37.8% 2025-02-28
221 GPT-5.1 Non-reasoning OpenAI 37.8% 2026-08-07
222 Gemini 3.1 Flash Lite Minimal Google 37.8% 2026-08-06
223 Claude Haiku 4.5 Anthropic 35.8% 2025-10-16
224 Claude Sonnet 4.5 Anthropic 35.6% 2025-09-29
225 GPT-5 Nano Minimal OpenAI 35.6% 2026-07-13
226 Mistral Medium 3 Mistral 32.2% 2025-05-07
227 Gemini 2.0 Flash Google 31.1% 2025-02-25
228 o1-preview OpenAI 31.1% 2025-03-07
229 Mistral Small 3.2 Mistral 30.3% 2026-08-30
230 Magistral Small Mistral 30.0% 2026-08-06
231 Claude Sonnet 4 Anthropic 28.9% 2025-05-22
232 GPT-4.1 nano OpenAI 28.9% 2025-04-14
233 GLM-5.2 Non-reasoning Zhipu AI 28.9% 2026-08-10
234 Magistral Small 1.2 Mistral 28.1% 2026-08-30
235 MiniMax-M3 Non-reasoning MiniMax 26.7% 2026-08-10
236 GPT-5.4 mini Non-reasoning OpenAI 26.7% 2026-08-07
237 Qwen3-14B Non-reasoning Alibaba 25.8% 2026-08-30
238 Qwen3 30B A3B Non-reasoning Alibaba 25.6% 2026-08-30
239 GLM-4.7-Flash Non-reasoning Zhipu AI 25.0% 2026-08-28
240 Gemini 1.5 Pro (Sept 2024) Google 23.1% 2025-02-25
241 Qwen3 32B Non-reasoning Alibaba 23.1% 2026-08-30
242 Gemma 3 27B IT Google 22.5% 2026-08-27
243 Qwen3 8B Non-reasoning Alibaba 22.2% 2026-08-27
244 Claude Sonnet 3.7 Anthropic 21.9% 2025-02-25
245 DeepSeek-R1-Distill-Qwen-1.5B DeepSeek 21.4% 2026-08-28
246 Llama 4 Maverick Meta 20.6% 2025-04-08
247 Qwen Plus Alibaba 17.8% 2025-04-07
248 Gemma 3 12B IT Google 16.7% 2026-08-28
249 Gemini 1.5 Flash (Sep 2024) Google 16.3% 2025-02-25
250 Qwen Max Alibaba 16.1% 2025-04-01
251 DeepSeek-V3 DeepSeek 15.8% 2025-02-25
252 Phi-4 Microsoft 13.8% 2025-02-25
253 Grok-2 (Dec 2024) xAI 11.5% 2025-02-25
254 Llama 3.1-405B Meta 9.7% 2025-02-25
255 Mistral Large 2407 Mistral 8.5% 2025-02-25
256 Claude Sonnet 3.5 v2 Anthropic 8.5% 2025-02-25
257 Qwen2.5-VL 72B Instruct Alibaba 8.1% 2025-02-25
258 Qwen3-1.7B Non-reasoning Alibaba 8.1% 2026-08-28
259 Mistral Large 2.1 Mistral 7.8% 2025-02-25
260 Llama 4 Scout Meta 7.8% 2025-04-08
261 Gemma 3 4B IT Google 7.5% 2026-08-28
262 Qwen2.5-32B Alibaba 7.4% 2025-02-25
263 GPT-4o mini OpenAI 6.9% 2025-07-30
264 Gemini 1.5 Pro (May 2024) Google 6.8% 2025-02-25
265 Mistral Small 3 Mistral 6.7% 2026-08-30
266 GPT-4 Turbo OpenAI 6.7% 2025-02-27
267 Claude 3.5 Sonnet Anthropic 6.5% 2025-02-25
268 GPT-4o (2024-08-06) OpenAI 6.4% 2025-02-25
269 GPT-4o (2024-05-13) OpenAI 6.3% 2025-02-25
270 GPT-4o (2024-11-20) OpenAI 6.3% 2025-02-25
271 Qwen Turbo Alibaba 6.1% 2025-04-07
272 Mistral Small 3.1 Mistral 5.8% 2025-03-18
273 Llama-3.3-70B-Instruct Meta 5.1% 2025-02-25
274 Claude 3 Opus Anthropic 4.7% 2025-02-25
275 Gemini 1.5 Flash 8B Google 4.6% 2025-02-25
276 Tulu 3 (Tülu 3) 70B Ai2 4.4% 2025-03-07
277 Claude Haiku 3.5 Anthropic 4.3% 2025-02-25
278 Llama 3-70B Meta 4.3% 2025-02-25
279 Gemini 1.5 Flash (May 2024) Google 3.9% 2025-02-25
280 Llama-3.1-70B-Instruct Meta 3.6% 2025-02-25
281 granite-4.0-micro Unknown 2.8% 2026-08-30
282 Llama-3.2-90B-Vision-Instruct Meta 2.6% 2025-02-25
283 Qwen2.5 7B Instruct Alibaba 2.5% 2026-08-30
284 Claude 2 Anthropic 2.5% 2025-03-07
285 Claude 3 Sonnet Anthropic 2.5% 2025-02-25
286 Hermes 2 Theta Llama-3 70B Nous Research 2.5% 2025-04-01
287 GPT-3.5-turbo OpenAI 2.2% 2026-08-07
288 Mistral Large Mistral 1.9% 2025-02-25
289 Claude 2.1 Anthropic 1.9% 2025-03-07
290 Llama 3-8B Meta 1.9% 2026-08-30
291 Claude Haiku 3 Anthropic 1.8% 2025-02-25
292 Llama-3.1-8B-Instruct Meta 1.7% 2026-08-27
293 Gemma 2 27B Google 1.4% 2025-03-07
294 GPT-4 OpenAI 1.1% 2025-10-23
295 Gemini 1.0 Pro Google 1.1% 2025-02-25
296 Gemma 3 1B Google 1.1% 2026-08-28
297 DeepSeek LLM 67B DeepSeek 0.8% 2026-08-28
298 granite-4.0-1b Unknown 0.8% 2026-08-30
299 Llama 3.2 1B Instruct Meta 0.6% 2026-08-30
300 Gemma 2 9B Google 0.6% 2025-03-07
301 Mistral 7B Instruct v0.3 Mistral 0.3% 2026-08-30
302 granite-4.0-350m Unknown 0.3% 2026-08-28
303 Llama 2-70B Meta 0.0% 2025-02-25