METR task suite
Software and research tasks measured in how long they take skilled humans. Average success across METR tasks.
| Category | Agents |
|---|---|
| Metric | Accuracy |
| Source | Epoch AI (evaluator) |
| License | CC BY 4.0 |
| Benchmark page | metr.org |
| Status | Scored |
| Weight | 1.2 within Agents |
| Fit | Discrimination 0.38, noise 0.16 |
Results
| # | Model | Provider | Result | Date |
|---|---|---|---|---|
| 1 | Claude Mythos Preview | Anthropic | 85.2% | 2026-11-11 |
| 2 | Claude Opus 4.6 | Anthropic | 78.9% | 2026-11-11 |
| 3 | Gemini 3.1 Pro Preview | 77.0% | 2026-11-11 | |
| 4 | GPT-5.2 High | OpenAI | 75.3% | 2026-11-11 |
| 5 | Claude Opus 4.5 Thinking | Anthropic | 75.0% | 2026-11-11 |
| 6 | GPT-5.3 Codex | OpenAI | 74.5% | 2026-11-11 |
| 7 | GPT-5.4 Extra High | OpenAI | 74.3% | 2026-11-11 |
| 8 | Claude Opus 4.5 | Anthropic | 73.0% | 2026-11-11 |
| 9 | Gemini 3 Pro Preview | 71.0% | 2026-11-11 | |
| 10 | GPT-5.1 Codex Max | OpenAI | 70.8% | 2026-11-11 |
| 11 | GPT-5 Medium | OpenAI | 69.6% | 2026-11-11 |
| 12 | GPT-5 High | OpenAI | 69.4% | 2026-11-11 |
| 13 | Claude Sonnet 4.5 Thinking | Anthropic | 67.4% | 2026-11-11 |
| 14 | Claude Opus 4.1 Thinking | Anthropic | 66.8% | 2026-11-11 |
| 15 | Grok 4 | xAI | 66.6% | 2026-11-11 |
| 16 | o3 Medium | OpenAI | 65.4% | 2026-11-11 |
| 17 | o4-mini Medium | OpenAI | 63.9% | 2026-11-11 |
| 18 | Claude Opus 4 Thinking | Anthropic | 63.9% | 2026-11-11 |
| 19 | o3 | OpenAI | 63.6% | 2026-11-11 |
| 20 | Claude Sonnet 4 Thinking | Anthropic | 62.0% | 2026-11-11 |
| 21 | Claude Opus 4.1 | Anthropic | 61.6% | 2026-11-11 |
| 22 | Claude Opus 4 | Anthropic | 61.6% | 2026-11-11 |
| 23 | Claude Sonnet 3.7 Thinking | Anthropic | 60.0% | 2026-11-11 |
| 24 | Kimi K2 Thinking | Moonshot AI | 59.2% | 2026-11-11 |
| 25 | GPT OSS 120B | OpenAI | 56.6% | 2026-11-11 |
| 26 | Claude Sonnet 3.7 | Anthropic | 55.8% | 2026-11-11 |
| 27 | Gemini 2.5 Pro Preview 06-05 | 55.4% | 2026-11-11 | |
| 28 | DeepSeek R1 0528 | DeepSeek | 53.8% | 2026-11-11 |
| 29 | DeepSeek-R1 | DeepSeek | 51.9% | 2026-11-11 |
| 30 | o1 Medium | OpenAI | 51.1% | 2026-11-11 |
| 31 | DeepSeek V3 0324 | DeepSeek | 49.6% | 2026-11-11 |
| 32 | DeepSeek-V3 | DeepSeek | 47.4% | 2026-11-11 |
| 33 | Claude Sonnet 3.5 v2 | Anthropic | 45.2% | 2026-11-11 |
| 34 | o1-preview | OpenAI | 45.1% | 2026-11-11 |
| 35 | GPT-4o (2024-05-13) | OpenAI | 40.8% | 2026-11-11 |
| 36 | Claude 3.5 Sonnet | Anthropic | 40.2% | 2026-11-11 |
| 37 | GPT-4 Turbo | OpenAI | 36.7% | 2026-11-11 |
| 38 | GPT-4 | OpenAI | 36.1% | 2026-11-11 |
| 39 | Qwen2.5-VL 72B Instruct | Alibaba | 35.8% | 2026-11-11 |
| 40 | Qwen2-72B | Alibaba | 29.9% | 2026-11-11 |
| 41 | Claude 3 Opus | Anthropic | 29.5% | 2026-11-11 |
| 42 | GPT-3.5-turbo | OpenAI | 21.5% | 2026-11-11 |
| 43 | davinci-002 | OpenAI | 16.2% | 2026-11-11 |
| 44 | GPT-2 (1.5B) | OpenAI | 10.1% | 2026-11-11 |