What Is Agent Memory and How Does It Work?

It is the means by which an AI agent remembers things from one step to another, and from session to session, rather than starting from scratch each time. It stores facts, preferences, and interactions, and recalls the ones it needs.

This is because a model starts afresh with each call by default. Memory enables an agent to maintain continuity, since it can recall your preferences, base actions on previous ones, and avoid repeating itself, which is much of what makes a multi-step agent coherent.

Short-term memory contains the context of the current task and is forgotten once the session ends. Long-term memory holds durable information outside the window and recalls it whenever the agent needs it, so the agent remembers across sessions.

This is usually achieved by storing memories as embeddings and doing a similarity search to find those relevant to the current context, then adding them to the context. It is exactly the retrieval machinery used in RAG.

Storing the wrong things creates noise, memories may become obsolete or even conflicting, and retaining user data raises privacy obligations. Good memory systems are selective, keep data up to date, and handle users' private data responsibly.