SciCode

Scientific research coding problems from physics, chemistry, biology and math.

CategoryCoding
MetricAccuracy
SourceEpoch AI (evaluator)
LicenseCC BY 4.0
Benchmark pagescicode-bench.github.io
StatusScored
Weight0.8 within Coding
FitDiscrimination 0.26, noise 0.11

Results

#ModelProviderResultDate
1 Claude Fable 5.2 Max Anthropic 67.7% 2026-11-11
2 Claude Opus 5.5 Max Anthropic 66.9% 2026-11-11
3 Claude Fable 5.2 Extra High Anthropic 65.6% 2026-11-11
4 Claude Opus 5.5 Extra High Anthropic 65.1% 2026-11-11
5 Gemini 4 Carbon High Google 64.2% 2026-11-11
6 Claude Fable 5.1 Max Anthropic 63.1% 2026-11-11
7 Gemini 4 Argon High Google 61.8% 2026-11-11
8 Claude Sonnet 5.5 Max Anthropic 61.0% 2026-11-11
9 Claude Fable 5 Max Anthropic 61.0% 2026-11-11
10 MiMo-V2.6-Pro Xiaomi 60.9% 2026-11-11
11 Claude Fable 5.1 Extra High Anthropic 60.9% 2026-11-11
12 Kimi K3.1 Pro Moonshot AI 60.8% 2026-11-11
13 Claude Opus 5.5 High Anthropic 60.4% 2026-11-11
14 Muse Wonder Max Meta 60.3% 2026-11-11
15 Gemini 3.7 Flash Medium Google 59.8% 2026-11-11
16 Muse Spark 1.3 Extra High Meta 59.7% 2026-11-11
17 Claude Fable 5.2 Medium Anthropic 59.7% 2026-11-11
18 Kimi K3 Max Moonshot AI 59.5% 2026-11-11
19 Claude Fable 5.2 High Anthropic 59.5% 2026-11-11
20 Claude Opus 5.5 Medium Anthropic 59.3% 2026-11-11
21 GLM-5.3 Max Zhipu AI 59.0% 2026-11-11
22 Step 5 Preview StepFun 58.9% 2026-11-11
23 Muse Spark 1.1 Meta 58.8% 2026-11-11
24 Muse Spark 1.3 Max Meta 58.8% 2026-11-11
25 Muse Spark 1.1 Extra High Meta 58.8% 2026-11-11
26 Kimi K3 Moonshot AI 58.7% 2026-11-11
27 Gemini 3.1 Pro Preview Google 58.7% 2026-11-11
28 Claude Fable 5.1 High Anthropic 58.7% 2026-11-11
29 Claude Opus 5.5 Low Anthropic 58.6% 2026-11-11
30 Muse Spark Open Extra High Meta 58.5% 2026-11-11
31 GPT-5.6 Sol High OpenAI 57.8% 2026-11-11
32 Grok 4.7 High xAI 57.8% 2026-11-11
33 Claude Fable 5.2 Low Anthropic 57.8% 2026-11-11
34 GPT-6 Sol Max OpenAI 57.6% 2026-11-11
35 Grok 4.7 Extra High xAI 57.4% 2026-11-11
36 Muse Spark 1.2 Extra High Meta 57.4% 2026-11-11
37 GPT-5.6 Sol Medium OpenAI 57.4% 2026-11-11
38 Claude Sonnet 5.5 Extra High Anthropic 57.3% 2026-11-11
39 Gemini 3.7 Flash High Google 57.2% 2026-11-11
40 GPT-5.6 Sol Max OpenAI 57.1% 2026-11-11
41 GPT-5.6 Sol Extra High OpenAI 57.1% 2026-11-11
42 Claude Fable 5.1 Low Anthropic 56.7% 2026-11-11
43 Gemini 3.8 Flash High Google 56.6% 2026-11-11
44 GPT-5.4 Extra High OpenAI 56.6% 2026-11-11
45 GPT-6 Astra Max OpenAI 56.5% 2026-11-11
46 Grok 4.6 High xAI 56.5% 2026-11-11
47 GPT-5.6 Sol Low OpenAI 56.4% 2026-11-11
48 Claude Opus 5 Max Anthropic 56.4% 2026-11-11
49 Claude Fable 5.1 Medium Anthropic 56.4% 2026-11-11
50 GPT-5.5 High OpenAI 56.1% 2026-11-11
51 DeepSeek V4.1 Flash 1017 Max DeepSeek 55.9% 2026-11-11
52 Grok 4.6 Medium xAI 55.9% 2026-11-11
53 GPT-5.5 Extra High OpenAI 55.8% 2026-11-11
54 GPT-6.1 Sol High OpenAI 55.8% 2026-11-11
55 GPT-6 Astra Extra High OpenAI 55.7% 2026-11-11
56 GPT-6.1 Sol Extra High OpenAI 55.7% 2026-11-11
57 GPT-6 Astra High OpenAI 55.4% 2026-11-11
58 GPT-6.1 Terra Max OpenAI 55.1% 2026-11-11
59 Gemini 3.8 Flash Medium Google 55.1% 2026-11-11
60 GPT-6 Sol Extra High OpenAI 55.1% 2026-11-11
61 Claude Haiku 5.5 Max Anthropic 55.0% 2026-11-11
62 GPT-5.6 Terra Max OpenAI 55.0% 2026-11-11
63 Grok 4.5 High xAI 55.0% 2026-11-11
64 Claude Opus 5 Extra High Anthropic 55.0% 2026-11-11
65 Gemini 3.8 Flash Low Google 55.0% 2026-11-11
66 Grok 4.7 Low xAI 54.9% 2026-11-11
67 GPT-6 Sol High OpenAI 54.9% 2026-11-11
68 GPT-6 Luna Max OpenAI 54.6% 2026-11-11
69 Claude Opus 4.7 Max Anthropic 54.5% 2026-11-11
70 GPT-5.5 Medium OpenAI 54.5% 2026-11-11
71 Claude Opus 5 High Anthropic 54.3% 2026-11-11
72 Claude Sonnet 5 High Anthropic 54.3% 2026-11-11
73 Mistral Large 4 Preview Mistral 54.2% 2026-11-11
74 GPT-6.1 Sol Max OpenAI 54.2% 2026-11-11
75 GPT-6 Astra Medium OpenAI 54.2% 2026-11-11
76 Claude Sonnet 5 Extra High Anthropic 54.1% 2026-11-11
77 GPT-6 Astra Low OpenAI 54.1% 2026-11-11
78 Gemini 3.5 Flash High Google 53.9% 2026-11-11
79 GPT-6 Sol Medium OpenAI 53.8% 2026-11-11
80 Claude Sonnet 5.5 High Anthropic 53.7% 2026-11-11
81 Claude Sonnet 5 Max Anthropic 53.6% 2026-11-11
82 GPT-5.6 Luna Max OpenAI 53.6% 2026-11-11
83 Gemini 3.7 Flash Low Google 53.6% 2026-11-11
84 Kimi K2.6 Moonshot AI 53.5% 2026-11-11
85 Claude Opus 4.8 Max Anthropic 53.5% 2026-11-11
86 GPT-6 Astra Non-reasoning OpenAI 53.5% 2026-11-11
87 Gemini 3.6 Flash High Google 53.4% 2026-11-11
88 GPT-6.1 Terra Extra High OpenAI 53.4% 2026-11-11
89 Qwen3.8 Max Alibaba 53.2% 2026-11-11
90 GPT-6.1 Sol Medium OpenAI 53.2% 2026-11-11
91 GPT-6.1 Sol Low OpenAI 53.2% 2026-11-11
92 Grok 4.6 Extra High xAI 53.0% 2026-11-11
93 Claude Sonnet 5.5 Medium Anthropic 52.9% 2026-11-11
94 Kimi K3 Low Moonshot AI 52.7% 2026-11-11
95 GPT-5.5 Instant OpenAI 52.6% 2026-11-11
96 GPT-5.6 Terra High OpenAI 52.4% 2026-11-11
97 GPT-6.1 Terra Low OpenAI 52.4% 2026-11-11
98 Inkling 2 Small Thinking Machines 52.3% 2026-11-11
99 GPT-5.6 Terra Extra High OpenAI 52.3% 2026-11-11
100 Qwen3.8 Max 0902 Alibaba 52.1% 2026-11-11
101 GPT-5.4 mini Extra High OpenAI 52.1% 2026-11-11
102 DeepSeek V4.1 Flash Max DeepSeek 51.9% 2026-11-11
103 Claude Haiku 5.5 Extra High Anthropic 51.7% 2026-11-11
104 GPT-6 Luna Extra High OpenAI 51.7% 2026-11-11
105 GLM-5.3-Flash Zhipu AI 51.6% 2026-11-11
106 GPT-5.5 Low OpenAI 51.6% 2026-11-11
107 GPT-5.6 Luna High OpenAI 51.6% 2026-11-11
108 Claude Sonnet 5 Medium Anthropic 51.6% 2026-11-11
109 GPT-6.1 Terra Medium OpenAI 51.6% 2026-11-11
110 GPT-6.1 Terra High OpenAI 51.6% 2026-11-11
111 Muse Spark Meta 51.5% 2026-11-11
112 MiMo-V2.6-Flash Xiaomi 51.3% 2026-11-11
113 GLM-5.2 Max Zhipu AI 51.2% 2026-11-11
114 DeepSeek V4 Pro 0813 Max DeepSeek 51.0% 2026-11-11
115 GPT-6 Luna Medium OpenAI 50.9% 2026-11-11
116 Claude Opus 5 Medium Anthropic 50.7% 2026-11-11
117 GPT-5.6 Terra Medium OpenAI 50.5% 2026-11-11
118 GPT-5.6 Luna Extra High OpenAI 50.5% 2026-11-11
119 GPT-6 Luna High OpenAI 50.4% 2026-11-11
120 Grok Build 0.1 xAI 50.2% 2026-11-11
121 MiMo-V2.5-Pro Xiaomi 50.2% 2026-11-11
122 GPT-6 Sol Low OpenAI 50.2% 2026-11-11
123 MiniMax-M2.7 MiniMax 50.1% 2026-11-11
124 Claude Sonnet 5 Low Anthropic 50.1% 2026-11-11
125 DeepSeek V4 Pro Max DeepSeek 50.0% 2026-11-11
126 GPT-5.6 Terra Low OpenAI 49.9% 2026-11-11
127 DeepSeek V4 Flash 0731 Max DeepSeek 49.9% 2026-11-11
128 Inkling Small Thinking Machines 49.7% 2026-11-11
129 Qwen3.7 Max Alibaba 49.5% 2026-11-11
130 Grok 4.6 Low xAI 49.4% 2026-11-11
131 Claude Haiku 5.5 Low Anthropic 49.2% 2026-11-11
132 Claude Sonnet 5.5 Low Anthropic 49.1% 2026-11-11
133 Kimi K2.5 Moonshot AI 49.0% 2026-11-11
134 Claude Haiku 5.5 Medium Anthropic 49.0% 2026-11-11
135 Claude Haiku 5.5 High Anthropic 48.7% 2026-11-11
136 Beam 1.0 Reflection 48.5% 2026-11-11
137 Grok 4.3 High xAI 48.3% 2026-11-11
138 Claude Opus 5 Low Anthropic 48.0% 2026-11-11
139 Kimi K2.7 Code Moonshot AI 47.8% 2026-11-11
140 GPT-6.1 Terra Non-reasoning OpenAI 47.8% 2026-11-11
141 GPT-5.5 Non-reasoning OpenAI 47.3% 2026-11-11
142 GPT-6 Sol Non-reasoning OpenAI 47.3% 2026-11-11
143 GPT-5.4 nano Extra High OpenAI 47.2% 2026-11-11
144 MiniMax-M3 MiniMax 47.1% 2026-11-11
145 GPT-5.6 Sol Non-reasoning OpenAI 47.1% 2026-11-11
146 Inkling Extra High Thinking Machines 47.0% 2026-11-11
147 GPT-6 Luna Low OpenAI 46.9% 2026-11-11
148 Claude Sonnet 4.6 Max Anthropic 46.8% 2026-11-11
149 GPT-5.6 Luna Medium OpenAI 46.8% 2026-11-11
150 Qwen3.8 27B Extra High Alibaba 46.6% 2026-11-11
151 DeepSeek V4 Pro High DeepSeek 46.4% 2026-11-11
152 Gemini 2.5 Pro Google 46.3% 2026-11-11
153 Qwen3.7 Plus Alibaba 46.1% 2026-11-11
154 GPT-5.6 Luna Low OpenAI 46.1% 2026-11-11
155 Claude Sonnet 4.5 Anthropic 45.7% 2026-11-11
156 Gemma 4 31B IT Google 45.5% 2026-11-11
157 GLM-4.7 Zhipu AI 45.1% 2026-11-11
158 GPT-5.6 Terra Non-reasoning OpenAI 45.1% 2026-11-11
159 Ring-2.6-1T Ant Group 45.0% 2026-11-11
160 Muse Glimmer High Meta 44.9% 2026-11-11
161 DeepSeek V4 Flash Max DeepSeek 44.9% 2026-11-11
162 GLM-5.1 Zhipu AI 44.8% 2026-11-11
163 Gemini 3.1 Flash Lite Google 43.4% 2026-11-11
164 GPT-5.1 OpenAI 43.3% 2026-11-11
165 MiMo-V2.5 Xiaomi 43.1% 2026-11-11
166 GPT-6 Luna Non-reasoning OpenAI 43.1% 2026-11-11
167 GPT-5 OpenAI 42.9% 2026-11-11
168 o3-mini High OpenAI 42.8% 2026-11-11
169 Nova 2.0 Pro Preview Medium Amazon 42.7% 2026-11-11
170 Claude Haiku 4.5 Anthropic 42.3% 2026-11-11
171 GLM-5.3 Low Zhipu AI 42.0% 2026-11-11
172 DeepSeek V4 Flash High DeepSeek 42.0% 2026-11-11
173 Qwen3 235B-A22B Thinking 2507 Alibaba 41.4% 2026-11-11
174 Gemini 3.5 Flash Lite Google 41.3% 2026-11-11
175 Cogito v2.1 Deep Cogito 41.0% 2026-11-11
176 Qwen3.6 Plus Alibaba 40.7% 2026-11-11
177 Trinity Large Thinking Arcee AI 40.6% 2026-11-11
178 GPT-4.1 mini OpenAI 40.4% 2026-11-11
179 GPT-5.6 Luna Non-reasoning OpenAI 40.4% 2026-11-11
180 Nemotron 3 Ultra NVIDIA 40.3% 2026-11-11
181 Mistral Medium 3.5 Mistral 40.2% 2026-11-11
182 Step 3.7 Flash StepFun 40.1% 2026-11-11
183 Claude Sonnet 4 Anthropic 40.1% 2026-11-11
184 Gemma 4 26B A4B IT Google 40.1% 2026-11-11
185 Qwen3.8 27B Low Alibaba 40.1% 2026-11-11
186 DeepSeek V4 Pro 0813 Non-reasoning DeepSeek 39.9% 2026-11-11
187 GPT-5 Mini OpenAI 39.2% 2026-11-11
188 Magistral Medium 1.2 Mistral 39.2% 2026-11-11
189 DeepSeek V3 0324 DeepSeek 39.0% 2026-11-11
190 GPT-5 Mini High OpenAI 39.0% 2026-11-11
191 Qwen3.8 27B Medium Alibaba 39.0% 2026-11-11
192 GPT OSS 20B High OpenAI 38.9% 2026-11-11
193 DeepSeek-V3.2-Exp Thinking DeepSeek 38.9% 2026-11-11
194 Mercury 2 Inception 38.7% 2026-11-11
195 Nova 2.0 Pro Preview Low Amazon 38.7% 2026-11-11
196 Command A Plus Cohere 38.5% 2026-11-11
197 Mercury 2.5 Inception 38.5% 2026-11-11
198 DeepSeek V4.1 Flash 1017 Non-reasoning DeepSeek 38.5% 2026-11-11
199 GLM-4.6 Zhipu AI 38.4% 2026-11-11
200 DeepSeek-R1 DeepSeek 38.3% 2026-11-11
201 DeepSeek-V3.1-Terminus DeepSeek 38.0% 2026-11-11
202 Qwen3.6 27B Non-reasoning Alibaba 37.3% 2026-11-11
203 Mistral Large 3 Mistral 36.6% 2026-11-11
204 Qwen3.6 35B-A3B Alibaba 36.6% 2026-11-11
205 Qwen3 Coder Next Alibaba 36.2% 2026-11-11
206 Qwen3.8 27B Non-reasoning Alibaba 36.2% 2026-11-11
207 GLM-5.2 Non-reasoning Zhipu AI 36.1% 2026-11-11
208 Nemotron 3 Super NVIDIA 36.0% 2026-11-11
209 GPT OSS 120B Low OpenAI 36.0% 2026-11-11
210 DeepSeek-V3 DeepSeek 35.8% 2026-11-11
211 Qwen3.5 122B-A10B Non-reasoning Alibaba 35.7% 2026-11-11
212 DeepSeek V4.1 Flash Non-reasoning DeepSeek 35.5% 2026-11-11
213 Qwen3 32B Alibaba 35.4% 2026-11-11
214 Magistral Small 1.2 Mistral 35.2% 2026-11-11
215 GPT OSS 120B High OpenAI 34.0% 2026-11-11
216 Qwen3-30B-A3B-Thinking (Jul 2025) Alibaba 33.0% 2026-11-11
217 Mistral Medium 3.1 Mistral 32.4% 2026-11-11
218 Llama 4 Maverick Meta 31.7% 2026-11-11
219 Qwen3-14B Alibaba 31.6% 2026-11-11
220 Qwen3.5 9B Alibaba 29.5% 2026-11-11
221 Qwen3.5 35B-A3B Non-reasoning Alibaba 29.3% 2026-11-11
222 Devstral Small 2 Mistral 28.8% 2026-11-11
223 Nova 2.0 Pro Preview Non-reasoning Amazon 28.1% 2026-11-11
224 Mistral Small 3.1 Mistral 27.8% 2026-11-11
225 Claude Haiku 3.5 Anthropic 27.4% 2026-11-11
226 Mistral Small 3.2 Mistral 26.4% 2026-11-11
227 Llama-3.3-70B-Instruct Meta 26.0% 2026-11-11
228 GPT-4.1 nano OpenAI 25.9% 2026-11-11
229 MiMo-V2-Flash Xiaomi 25.9% 2026-11-11
230 Granite 4.1 30B IBM 25.8% 2026-11-11
231 Solar Pro 3 Upstage 25.5% 2026-11-11
232 Gemma 3 27B IT Google 23.3% 2026-11-11
233 Qwen3 8B Alibaba 22.6% 2026-11-11
234 Llama 4 Scout Meta 21.3% 2026-11-11
235 Gemma 3 12B IT Google 16.4% 2026-11-11
236 Llama-3.1-8B-Instruct Meta 13.2% 2026-11-11
237 Phi-4-mini Microsoft 10.8% 2026-11-11
238 Qwen3.6 35B-A3B Non-reasoning Alibaba 1.3% 2026-11-11