Which AI model is best for workflow automation?

AutomationBench, now part of Fireworks’ Specialized Intelligence Index, evaluates AI models on end-to-end business workflows across sales, marketing, operations, support, finance, and HR. The benchmark assesses whether an AI agent can complete tasks like updating records, following rules buried in emails, and coordinating updates across multiple apps without errors. Unlike general benchmarks that test question-answering or coding, AutomationBench measures actual outcomes, ensuring workflows are executed correctly rather than just reporting success.

The benchmark is built on patterns from over 2 billion monthly tasks processed by 3.7 million companies on Zapier. Each task simulates a small company with a CRM, inbox, spreadsheets, and support cases across 47 apps. For example, an agent must identify the correct account among similar names, update the right deal status, and notify the appropriate team based on a rule hidden in an email. The agent operates without clarifying questions, and tasks only pass if every system change is verified, reflecting real-world business demands.

The leaderboard includes 657 private tasks to prevent models from training on answers. Scores represent the percentage of tasks completed with all checks passing. The top model, Gemini 4 Argon, achieves 51.3%, while Claude Sonnet 5.5 follows at 44.7%. Most failures stem from agents reporting success despite incorrect data updates, a phenomenon termed 'false confidence' by Zapier’s Principal Engineer, Robin Salimans.

Price does not correlate with performance; GPT-6 Sol delivers 80% of GPT-6 Astra’s results at one-sixth the cost per task, and Claude Sonnet 5.5 outperforms Claude Opus 5.5 while being cheaper. Cheaper models like GPT-6 Luna and DeepSeek V4 Flash cost $0.04 per task but complete fewer than 25% of full workflows, making them suitable only for simple steps like categorizing tickets.