What is Agent Memory?

Agent Memory refers to the ability and mechanism of AI agents to store and manage information during the execution of tasks. It is similar to the human memory system, enabling agents to remember past interactions, experiences, and knowledge, and use this information to make better decisions in subsequent tasks. This memory mechanism is essential for achieving continuous learning and handling long-term tasks.

What Agents Need to Memory?

From a technical perspective, the agent's memory is essentially an extension of the limited context of the large model. During the life cycle of the agent, the user or agent will generate a large amount of data, and the context that the AI ​​large model can process is limited, usually 16K to 2M tokens. This means that AI's own context processing capabilities alone cannot directly process such a large amount of data.

From a product perspective, Agent Memory can achieve personalized interactions, maintain contextual consistency, and most importantly, effectively reduce operating costs.

• Personalized interaction: For example, a user asks AI to recommend a movie. If the agent has memory, AI can recommend the user's favorite movie genre based on the user's historical interests, avoid repeating the recommendation of movies that have already been watched, and recommend movies that are more in line with the user's taste based on the user's preferences. This personalized experience can enhance user stickiness and satisfaction and increase usage frequency.
• Maintain contextual coherence: The particularity of natural language interaction requires AI to understand the context, otherwise ambiguous or incoherent responses may be generated even in the same conversation. For example, a user asks "How was the movie last night?" Without memory, AI may not understand which movie the user is referring to. But if AI has memory, it can recall the movie the user watched recently and respond accurately: "You watched "Avengers" last night, and the overall rating was high. What did you think?" This can keep the conversation smooth and relevant, avoiding repeated inquiries and misunderstandings.
• Reduced operating costs: Without memory, AI needs to re-read historical records and perform contextual reasoning for each conversation, which increases computing resource consumption and prolongs response time, affecting user experience. With memory, AI can directly use users' historical information and preferences to provide services, avoiding processing all conversation content from scratch each time. This approach greatly reduces the demand for back-end computing, improves efficiency, and reduces server and storage costs, thereby effectively reducing operating costs.

Difference between RAG and memory

Strictly speaking, memorization is a subset of RAG (Retrieval-Augmented Generation). Both extract information from the outside and incorporate it into the prompts generated by the LLM (Large Language Model), but their application scenarios and goals are different. The core difference is that RAG focuses on knowledge, while memorization focuses on user information.

• Usage scenarios
• RAG: Used to retrieve information in large document collections (such as corporate wikis, technical documents, etc.).
• Memory: Focuses on managing personalized information in user interactions, especially in multi-user environments.
• Information Density
• RAG: processes dense unstructured data (such as text, tables), mainly used for fact retrieval.
• Memory: Processes multiple rounds of conversation data between users and agents, focusing on optimizing the interactive experience.
• Search method
• RAG: Exact document matching via semantic search and embedded retrieval.
• Memory: Focuses on summarizing and compressing key information in interactions to optimize contextual experiences.

Comparison of Common Agent Memory Mechanisms

The following is a comparison of the most popular memory design mechanisms (pictures from Gustavo):

Here is a specific example to help you understand the difference between these memory mechanisms: