SaaS Marathon Bench · Task 3
Audit a week of sales activity
Task pass rate20% ±10%
The agent goes through one week of meetings, mail and call recordings and compares them with Attio. It decides which conversations were real sales opportunities, sorts out meetings that moved or appear twice, and brings Attio's records up to date. It then lists what it still can't resolve, most important first.
Results
ModelPass rate
Claude Opus 5.5CLI100%
Claude Opus 5.5GUI67%
GPT-6 AstraCLI67%
GPT-6.1 SolCLI67%
Claude Haiku 5.5CLI33%
Gemini 3.8 FlashCLI33%
GPT-6 AstraGUI33%
Claude Haiku 5.5GUI0%
Claude Sonnet 5.5GUI0%
Claude Sonnet 5.5CLI0%
DeepSeek V4 FlashCLI0%
Gemini 3.8 FlashGUI0%
Gemini 4 ArgonGUI0%
Gemini 4 ArgonCLI0%
GLM 5CLI0%
GPT-6.1 SolGUI0%
Grok 4.7GUI0%
Grok 4.7CLI0%
Qwen3.8 MaxGUI0%
Qwen3.8 MaxCLI0%