tau2-bench Banking Knowledge
Customer-service agent for a bank that must search a knowledge base and use tools while talking to a simulated user. Standard Sierra scaffold, GPT-5.2 user simulator.
| Category | Agents |
|---|---|
| Metric | pass^1 (%) |
| Source | tau2-bench (original) |
| License | MIT |
| Benchmark page | taubench.com |
| Status | Scored |
| Weight | 1.0 within Agents |
| Fit | Discrimination 0.75, noise 0.29 |
Results
| # | Model | Provider | Result | Date |
|---|---|---|---|---|
| 1 | Qwen3.8 Max Extra High | Alibaba | 55.2% | 2026-08-03 |
| 2 | Claude Opus 5 Max | Anthropic | 48.7% | 2026-08-03 |
| 3 | Grok 4.5 High | xAI | 47.9% | 2026-07-24 |
| 4 | GPT-5.6 Sol Extra High | OpenAI | 46.9% | 2026-07-22 |
| 5 | GPT-5.5 Extra High | OpenAI | 44.6% | 2026-07-22 |
| 6 | Muse Spark 1.1 Extra High | Meta | 40.5% | 2026-07-23 |
| 7 | Claude Opus 4.7 Max | Anthropic | 40.2% | 2026-07-23 |
| 8 | Claude Fable 5 Max | Anthropic | 39.7% | 2026-07-23 |
| 9 | Claude Opus 4.8 Max | Anthropic | 39.7% | 2026-07-23 |
| 10 | GPT-5.4 Extra High | OpenAI | 39.4% | 2026-05-06 |
| 11 | Kimi K3 Max | Moonshot AI | 37.1% | 2026-07-24 |
| 12 | GLM-5.2 Extra High | Zhipu AI | 37.1% | 2026-07-24 |
| 13 | GPT-5.2 High | OpenAI | 32.2% | 2026-05-05 |
| 14 | Claude Opus 4.6 Max | Anthropic | 27.3% | 2026-05-06 |
| 15 | Gemini 3 Flash Preview High | 27.3% | 2026-03-02 | |
| 16 | Gemini 3.1 Pro Preview High | 26.0% | 2026-05-05 | |
| 17 | Claude Sonnet 4.5 | Anthropic | 25.3% | 2026-02-24 |
| 18 | Inkling Max | Thinking Machines | 25.0% | 2026-07-24 |
| 19 | Claude Opus 4.5 High | Anthropic | 24.7% | 2026-05-05 |
| 20 | Gemini 3 Pro Preview High | 18.0% | 2026-03-02 | |
| 21 | Grok 4.20 (Reasoning) High | xAI | 18.0% | 2026-05-05 |
| 22 | Grok 4 Fast High | xAI | 15.7% | 2026-05-05 |
| 23 | Gemini 2.5 Pro High | 13.7% | 2026-05-05 | |
| 24 | Grok 4.1 Fast High | xAI | 13.1% | 2026-05-05 |
| 25 | GPT-5.2 Non-reasoning | OpenAI | 12.6% | 2026-05-26 |
| 26 | GLM-5 | Zhipu AI | 9.8% | 2026-02-27 |
| 27 | Qwen3.5 397B-A17B | Alibaba | 9.8% | 2026-02-27 |