BALROG
Progress in long text and vision games such as NetHack and Crafter.
| Category | Agents |
|---|---|
| Metric | Accuracy |
| Source | Epoch AI (evaluator) |
| License | CC BY 4.0 |
| Benchmark page | balrogai.com |
| Status | Scored |
| Weight | 0.6 within Agents |
| Fit | Discrimination 0.41, noise 0.26 |
Results
| # | Model | Provider | Result | Date |
|---|---|---|---|---|
| 1 | GPT-6 Astra Max | OpenAI | 68.3% | 2026-11-11 |
| 2 | Claude Opus 5 Max | Anthropic | 63.4% | 2026-11-11 |
| 3 | GPT-5.6 Sol Max | OpenAI | 60.0% | 2026-11-11 |
| 4 | Gemini 3 Pro Preview | 58.1% | 2026-11-11 | |
| 5 | Gemini 3.1 Pro Preview | 57.0% | 2026-11-11 | |
| 6 | GPT-5.6 Terra Max | OpenAI | 53.2% | 2026-11-11 |
| 7 | Gemini 3 Flash Preview | 48.1% | 2026-11-11 | |
| 8 | GPT-5.6 Luna Max | OpenAI | 45.6% | 2026-11-11 |
| 9 | Grok 4 | xAI | 43.6% | 2026-11-11 |
| 10 | Claude Opus 4.5 | Anthropic | 43.5% | 2026-11-11 |
| 11 | Gemini 2.5 Pro Preview 06-05 | 43.3% | 2026-11-11 | |
| 12 | Claude Opus 4.5 Thinking | Anthropic | 43.0% | 2026-11-11 |
| 13 | DeepSeek-R1 | DeepSeek | 34.9% | 2026-11-11 |
| 14 | Gemini 2.5 Flash | 33.5% | 2026-11-11 | |
| 15 | GPT-5 Minimal | OpenAI | 32.8% | 2026-11-11 |
| 16 | Claude Sonnet 3.5 v2 | Anthropic | 32.6% | 2026-11-11 |
| 17 | GPT-4o (2024-05-13) | OpenAI | 32.3% | 2026-11-11 |
| 18 | Claude Haiku 4.5 | Anthropic | 31.2% | 2026-11-11 |
| 19 | Claude Haiku 4.5 Thinking | Anthropic | 31.2% | 2026-11-11 |
| 20 | Grok 3 | xAI | 29.5% | 2026-11-11 |
| 21 | Reka Flash 3 | Reka | 29.2% | 2026-11-11 |
| 22 | Llama-3.1-70B-Instruct | Meta | 27.9% | 2026-11-11 |
| 23 | Llama-3.2-90B-Vision-Instruct | Meta | 27.3% | 2026-11-11 |
| 24 | Llama-3.3-70B-Instruct | Meta | 23.0% | 2026-11-11 |
| 25 | Gemini 1.5 Pro (Sept 2024) | 21.0% | 2026-11-11 | |
| 26 | DeepSeek-R1-Distill-Qwen-32B | DeepSeek | 19.5% | 2026-11-11 |
| 27 | Claude Haiku 3.5 | Anthropic | 19.3% | 2026-11-11 |
| 28 | Mistral Nemo | Mistral | 17.6% | 2026-11-11 |
| 29 | GPT-4o mini | OpenAI | 17.4% | 2026-11-11 |
| 30 | Llama-3.2-11B-Vision-Instruct | Meta | 16.8% | 2026-11-11 |
| 31 | Qwen2.5-VL 72B Instruct | Alibaba | 16.2% | 2026-11-11 |
| 32 | Llama-3.1-8B-Instruct | Meta | 15.1% | 2026-11-11 |
| 33 | Gemini 1.5 Flash (Sep 2024) | 14.6% | 2026-11-11 | |
| 34 | Qwen2-VL-72B-Instruct | Unknown | 12.8% | 2026-11-11 |
| 35 | Phi-4 | Microsoft | 11.6% | 2026-11-11 |
| 36 | Llama 3.2 3B Instruct | Meta | 10.1% | 2026-11-11 |
| 37 | Qwen2.5 7B Instruct | Alibaba | 7.8% | 2026-11-11 |
| 38 | Llama 3.2 1B Instruct | Meta | 6.6% | 2026-11-11 |
| 39 | Qwen2-VL-7B-Instruct | Unknown | 3.7% | 2026-11-11 |