Introducing Agent Memory on Databricks

Databricks announced Agent Memory on Agent Bricks, its agent developer platform, designed to store critical facts, preferences, and workflows that agents should retain. The feature is available in Beta and runs on Lakebase, Databricks' serverless Postgres, to ensure high reliability and low latency. Agent Memory is built on the MemCurrent architecture, developed by Databricks Research, which combines three access patterns to manage memory efficiently. Without memory, agents risk repeating mistakes or forgetting user preferences, leading developers to create custom solutions with unresolved design challenges.

Agent Memory addresses these issues by using MemCurrent’s three access patterns: plugins for automatic context logging, on-demand tools for agent calls, and an offline dreaming pipeline to consolidate memory in the background. All operations are served with low latency via Lakebase, Databricks’ serverless Postgres. Internal benchmarks showed that implementing this memory architecture tripled agent pass rates on enterprise tasks while reducing token usage by 31%. The feature integrates with existing Agent Bricks components like agent hosting, Databricks Sandbox, and Unity Gateway.

Agent Memory solves two common problems when agents move from demo to daily use: agents either become expensive due to excessive token usage or fail to personalize without retaining context. The system stores raw session records and distills them into long-term memory in the background, ensuring agents remain efficient and personalized. Everything runs on Lakebase, which scales with agent traffic and operates within the user’s workspace, eliminating the need for additional infrastructure.

Agent Memory is a production implementation of MemCurrent, the memory system detailed in the research paper 'MemCurrent: Designing Memory for Enterprise Agents.' The system uses a two-layer memory approach: raw records of events and distilled, curated memories updated by a background process. This design improved agent performance, reducing errors and token usage while increasing task completion rates. The same infrastructure powers memory in Genie, Databricks’ natural-language business experience. Deployment is streamlined via the Agent Bricks CLI, enabling agents with memory to be set up in minutes.