Thinking of ACE? We Can Do It with Fewer Tokens
Hugging Face introduces ALTK-Evolve, a system that reduces token usage for LLM agents by delivering only task-relevant guidelines instead of a full playbook, achieving similar or better accuracy at lower cost compared to ACE.
ACE and ALTK-Evolve both enable LLM agents to learn from past failures by storing lessons as reusable guidelines, addressing issues like mis-paginated API calls or incorrect resolutions. While both systems agree on the value of preserving granular experience, they differ in how these lessons are delivered during inference. ACE maintains a single, comprehensive playbook updated incrementally, whereas ALTK-Evolve consolidates lessons into individually retrievable guidelines with support counts to track reliability. The core principle in both systems is to avoid compressing lessons into summaries, instead counting and preserving their distinct contributions to agent performance.
The delivery method drives the token efficiency difference. ACE injects its full playbook at every step, regardless of task or model, while ALTK-Evolve tailors retrieval to the model's capacity—sending only high-support guidelines or a curated subset for weaker models. On a strong model, ALTK-Evolve matches or exceeds ACE's performance at roughly 40% of the inference cost, and on a weaker model, it achieves comparable accuracy at about one-seventh the cost. The approach prioritizes task-specific relevance over exhaustive context, reducing token waste without sacrificing reliability.
Benchmark results on AppWorld tasks show ALTK-Evolve outperforming ACE on Easy, Hard, and Overall metrics for DeepSeek-V3.2, while ACE leads on Medium. For gpt-oss-120b, ACE performs better on Easy and Medium, but ALTK-Evolve wins on Hard tasks and the aggregate. The performance gap highlights how weaker models benefit from selective retrieval, whereas stronger models can utilize more comprehensive guidance. Both systems improve over their no-memory baselines, with ALTK-Evolve demonstrating that calibrated delivery—not just memory volume—drives efficiency gains.
Hugging Face's ALTK-Evolve library, which includes extraction, consolidation, and retrieval pipelines, is available for implementation. The system's advantage lies in its ability to deliver the right amount of guidance at the right time, avoiding the pitfalls of either overwhelming weaker models or under-serving stronger ones. The findings underscore the importance of adaptive memory delivery in optimizing LLM agent performance and cost.