How Much Memory Does Your Agent Actually Need?

Hugging Face’s ALTK-Evolve framework evaluates how much agentic memory—distilled guidelines from past tasks—improves performance across eight models, from a 30B dense model to proprietary systems. The research reveals that memory is not a binary feature but a calibrated dose: strong models with headroom benefit from full guideline sets, while weaker models perform better with compact cores plus task-specific retrieval. Already-saturated models show no measurable gains, regardless of memory size.

The study tested models on AppWorld, a benchmark of 585 multi-step tasks across nine simulated applications, using two scoring metrics: Task Goal Completion (TGC) and stricter Scenario Goal Completion (SGC). For example, DeepSeek-V3.2 (671B MoE) improved TGC by 9.5 percentage points with a full guideline set, while gpt-oss-120b (117B MoE) gained 16.1pp with curated retrieval at 5% lower token cost compared to the full set.

ALTK-Evolve distills guidelines from an agent’s past trajectories—successes and failures—without updating model weights or requiring human annotation. The guidelines are then injected at inference time as either a full set or a curated subset relevant to the task. This approach is portable across models and cost-effective, as prompt caching keeps production costs low even with large guideline sets.

The findings underscore that memory effectiveness varies by model tier, with factors like benchmark headroom, context-window size, architecture, and task distribution influencing outcomes. The practical takeaway is clear: calibrating memory dosage to the model’s capability—rather than maximizing memory—yields the best balance of performance and cost.