MathArena BrokenArXiv (latest month)
Altered statements of recent arXiv results that are subtly false; the model must notice the flaw instead of producing a fake proof. Newest month with at least five models.
| Category | Math |
|---|---|
| Metric | Accuracy (%) |
| Source | MathArena (evaluator) |
| License | None stated |
| Benchmark page | matharena.ai |
| Status | Scored |
| Weight | 0.9 within Math |
| Fit | Discrimination 1.46, noise 0.59 |
Results
| # | Model | Provider | Result | Date |
|---|---|---|---|---|
| 1 | GPT-6.1 Sol Max | OpenAI | 80.3% | 2026-11-11 |
| 2 | Claude Opus 5.5 High | Anthropic | 77.8% | 2026-11-11 |
| 3 | Kimi K3.1 Pro | Moonshot AI | 58.0% | 2026-11-11 |
| 4 | DeepSeek V4.1 Flash 1017 Max | DeepSeek | 49.7% | 2026-11-11 |
| 5 | Gemini 4 Argon | 39.8% | 2026-11-11 |