WeirdML v3

Unusual machine-learning tasks the model solves by writing and running code.

CategoryCoding
MetricAccuracy
SourceEpoch AI (evaluator)
LicenseCC BY 4.0
Benchmark pagehtihle.github.io
StatusScored
Weight0.8 within Coding
FitDiscrimination 1.36, noise 0.38

Results

#ModelProviderResultDate
1 GPT-6 Astra Extra High OpenAI 42.2% 2026-11-11
2 GPT-6.1 Sol Extra High OpenAI 38.3% 2026-11-11
3 Claude Opus 5.5 Extra High Anthropic 31.2% 2026-11-11
4 GPT-6.1 Terra Extra High OpenAI 28.1% 2026-11-11
5 Claude Fable 5.1 Extra High Anthropic 26.0% 2026-11-11
6 Gemini 4 Argon High Google 20.3% 2026-11-11
7 Claude Sonnet 5.5 Extra High Anthropic 19.9% 2026-11-11
8 GPT-6 Sol Extra High OpenAI 19.7% 2026-11-11
9 Claude Opus 5 Extra High Anthropic 19.0% 2026-11-11
10 Claude Fable 5 Extra High Anthropic 15.6% 2026-11-11
11 Muse Wonder Extra High Meta 15.2% 2026-11-11
12 GPT-5.6 Sol Extra High OpenAI 15.0% 2026-11-11
13 Claude Haiku 5.5 Extra High Anthropic 11.7% 2026-11-11
14 Kimi K3.1 Pro Extra High Moonshot AI 9.9% 2026-11-11
15 DeepSeek V4.1 Flash 1017 High DeepSeek 9.2% 2026-11-11
16 Grok 4.7 Extra High xAI 8.5% 2026-11-11
17 GPT-6 Luna Extra High OpenAI 7.9% 2026-11-11
18 Kimi K3 High Moonshot AI 7.5% 2026-11-11
19 Gemini 3.8 Flash High Google 7.4% 2026-11-11
20 Muse Spark 1.3 Extra High Meta 6.7% 2026-11-11
21 Claude Opus 4.5 High Anthropic 6.5% 2026-11-11
22 DeepSeek V4.1 Flash High DeepSeek 6.0% 2026-11-11
23 GPT-5.6 Luna Extra High OpenAI 5.2% 2026-11-11
24 GPT-5.2 Extra High OpenAI 5.0% 2026-11-11
25 Mistral Large 4 Preview High Mistral 4.5% 2026-11-11
26 Inkling Extra High Thinking Machines 3.9% 2026-11-11