AI Agent Memory: How It Works, and When Your Agent Needs It

You correct your AI agent on Monday. On Thursday it makes the same mistake, because it has no idea Monday happened. Every session starts from zero.

AI agent memory fixes that. It’s an external store that sits outside the model, plus rules for three things: what gets written down, how it is stored, and what gets read back into the next session. The model itself doesn’t learn anything. The system around it does.

This guide explains how agent memory works. It walks through one of the most-starred open-source memory projects of 2026, Tencent’s TencentDB Agent Memory. It compares the main options. And it shows the small memory system we run on our own AI agent, with five weeks of real numbers on what it caught. If you’re still deciding what AI agents are good for, start there and come back.

What “memory” means for an AI agent

A language model has a context window: the text it can see right now. When the session ends, that text is gone. Making the window bigger doesn’t fix this. It only makes each call more expensive.

Agent memory is anything that survives the session and can come back later. Most designs split it into three types:

• Episodic memory. What happened, and when. “On 6 October the user said the two New York testimonials are not real clients.”
• Semantic memory. Facts and preferences, with no timestamp attached. “The Search Console property is a URL-prefix property, not a domain property.”
• Procedural memory. How to do things. “Before publishing a post, set tags, set the featured image and check the sitemap.”

A support bot mostly needs semantic memory: who the customer is and what they bought. A coding agent mostly needs procedural memory: how this team ships. An agent that runs a long project needs all three.

How agent memory works: write, store, retrieve

Every memory system runs the same loop, whatever the vendor calls it.

1. Write. Something decides what’s worth keeping. The simplest version saves the whole transcript. Better versions run an extraction step, usually another LLM call, that pulls out facts, decisions and corrections and throws the chatter away. Extraction is where most of a memory system’s running cost goes, because it is one more model call per conversation.

2. Store. The extracted items go somewhere that lasts. Three storage types are common:

3. Retrieve. Before the agent reasons, the system searches memory with the current task and injects the best matches into the prompt. Strong systems combine keyword search (BM25) with vector search, merge the two ranked lists with Reciprocal Rank Fusion (RRF), and then cap the result. The cap matters. Without one, memory grows until it fills the context window it was meant to protect. This is the same retrieval problem behind any RAG and LLM development project, applied to the agent’s own history.

A fourth step, forgetting, is often skipped. Facts go stale: a price changes, a phone number is retired. If memory never decays and nothing marks a fact as replaced, the agent keeps acting on old information with confidence.

A worked example: Tencent’s open-source memory hub

TencentDB Agent Memory is an MIT-licensed project from Tencent Cloud’s database team. On 14 October 2026 it had about 29,000 GitHub stars. The latest stable release was v2.0.2, published 9 October 2026. It’s written in TypeScript and runs self-hosted.

It’s worth studying because it makes four design choices that most teams only discover after their first memory system goes wrong.

Memory grows in layers. Raw conversations are saved as L0. A background process distils them into L1 atoms (single facts, preferences, constraints), then L2 scenarios (knowledge grouped by project), then L3, a long-term profile of the user or team. When retrieving, it starts with the compact L2/L3 layers for a quick picture of the context, and falls back to L1/L0 with BM25 + vector + RRF search only when it needs an exact fact. Results are capped by item count, character budget and timeout.

Memory is more than chat. It manages four asset types: Chat Memory, Skills (reusable procedures with versions and validation rules), a Wiki built from your documents, and a CodeGraph built from your codebase. That covers all three memory types above. CodeGraph maps symbols and call relationships, so an agent can check what a change will affect before it edits anything.

Access is controlled. Each asset is private, team, restricted (per-user, role or agent permissions) or agent. New memories are private by default, and sharing them is a deliberate action. If several people and agents share one memory, this is the part you can’t skip.

It plugs in through a proxy. You point an agent’s base URL at the proxy, with no plugin and no code change. The README lists Claude Code, Codex, OpenClaw, Hermes and Tencent’s own CodeBuddy and WorkBuddy. Documents and code reach the agent as tools it calls when needed (/v3/tools/list, /v3/tools/call), not text pasted into every prompt.

What to keep in mind. The project’s one benchmark, PersonaMem accuracy rising from 48% to 76% with memory switched on, is self-reported in the README and has no independent reproduction yet. The README also says CodeGraph works best with public HTTPS repositories, and that routing memory to the right agent is still mostly manual.

What we run ourselves: a small memory for our SEO agent

We use an AI agent to run SEO and marketing operations for this website. In its first weeks it made mistakes that each cost a session to untangle. It trusted a status code it shouldn’t have. It read the wrong key in a tool’s output and reported a clean result. It mistook a cached test run for a fresh one.

So on 9 September 2026 we gave it a memory. It’s deliberately small: one SQLite file with full-text search (FTS5), and a 519-line Python script using only the standard library. There’s no vector database. A table is reserved for embeddings in case we need them later.

It stores five kinds of record:

• Lessons. A mistake already made once, written as *when* it applies, the *rule*, *why* (what went wrong), and the *source*.
• Decisions the human owner has settled, so the agent never asks again.
• Facts: IDs, access status, measured numbers.
• Open threads with due dates.
• Every past session transcript, saved and searchable.

The design choice that does the most work is where retrieval happens. Hooks fire at four points: session start (load active lessons, recent decisions and open threads), every prompt (search memory with the prompt and inject the top matches), before every shell command, and end of every turn (save the transcript). Sixteen of the lessons carry a pattern that matches the risky command itself. The lesson appears right before the agent repeats the mistake, not somewhere in a long prompt it may skim.

Five weeks of numbers, measured 14 October 2026:

A concrete example from our weekly site check on 14 October: before the agent ran the page-comparison tool, the memory fired a lesson saying a “content changed” flag on every page is a known false alarm. The tool then flagged all 14 pages, and the agent moved on instead of chasing a fake regression. Later in the same check, a speed test failed with a misleading rate-limit error because an API key wasn’t loaded. That became lesson 45 within minutes. The next session will already know.

What keyword search was enough for. Our memory is about one website, with a fixed vocabulary: page names, tool names, error messages. Exact-term search finds those better than vector search would. Once a domain’s wording drifts, such as customer conversations or several products, this stops being true.

Mem0 vs Zep vs Letta vs Tencent: which fits

These four open-source projects come up most when teams look for agent memory. Star counts are from GitHub on 14 October 2026. We left out benchmark scores because the published comparisons disagree with each other.

Two notes. Zep’s own zep repository is mainly examples and integrations; the open-source engine is Graphiti, so that’s the star count shown. And all four are young projects that release often, so check the changelog before you pick one.

Build or adopt? A decision rule

Four questions settle most cases:

1. One agent or many? One agent with one user can get by with a small store, even a single SQLite file. Several agents, or several people sharing an agent, need visibility rules from day one. That is Tencent’s hub, or a careful build of your own.
2. Do facts change over time? If the answer to “what is the client’s current plan?” changes, you need memory that tracks when a fact was true. That points to a graph with time tracking, such as Graphiti.
3. Does anything need access control? Customer data, HR data or anything regulated rules out a single shared prompt. Memory has to be filtered by who is asking before retrieval, not after.
4. Is the vocabulary narrow? Narrow domains (one product, one codebase, one site) work well with keyword search. Broad ones need vector search or a hybrid.

Our rule: adopt an open-source layer when your case matches what it was built for. Build when the value lies in *when* memory is injected, like our command-matching lessons. No framework offers that out of the box, and it took us about 500 lines.

If you’re planning an agent and don’t know which side you’re on, our guide to building a custom AI agent covers the rest of the architecture. Our AI agent development team designs memory as part of the build, not as an add-on later.

Planning an agent that needs to remember? Book a 30-minute memory design review. We’ll map what your agent should keep, where to store it, and what it will cost to run. Book the review

What memory adds to an agent’s cost

Memory brings three cost lines. Only one of them is large.

• Storage. Small. Our five weeks of lessons, decisions and 24 sessions of transcripts sit in one local SQLite file. Hosted vector databases cost more, but storage is rarely the line that hurts.
• Extraction and retrieval calls. This is the recurring cost. Every distillation pass and every embedding is a model call. Layered designs like Tencent’s add a background pipeline that runs after conversations end. Model choice matters a lot here. Our LLM API pricing comparison shows how far token prices vary between models.
• Human review time. This is the cost most teams forget. A memory that writes without review collects wrong facts and repeats them with confidence. Our lessons are written when the owner corrects the agent or the agent catches its own error. Each one records its source, and lessons that stop being true are retired (one so far). Tencent’s hub uses review before sharing for the same reason.

Frequently asked questions

What is AI agent memory?

AI agent memory is a store outside the language model that keeps information between sessions: past events, facts and preferences, and learned procedures. Before each task, the system retrieves the relevant items and adds them to the prompt. After the task, it writes new items back. The model itself is unchanged.

Is a bigger context window the same as memory?

No. A context window holds text for one session and is cleared when that session ends. A larger window also raises the cost of every call. Memory persists across sessions and puts only the relevant items into the window, so the agent knows more and each call costs less.

What is the difference between RAG and agent memory?

RAG retrieves from a fixed body of documents. Agent memory also writes: it adds what the agent learned from conversations, corrections and completed tasks, and should update or retire facts when they change. Most memory systems use RAG-style retrieval, with the read-and-write loop on top.

Do I need a vector database for agent memory?

Not always. When the agent works in a narrow domain with consistent terms, such as one product or one website, keyword full-text search finds exact names and error messages well. Our own agent memory runs on SQLite with no vector database. Add vector search when users describe the same thing in many different ways.

Which open-source agent memory should I choose?

Choose by workload. Mem0 suits adding memory to one existing agent. Zep’s Graphiti suits facts that change over time. Letta suits long-running agents built on its runtime. TencentDB Agent Memory suits teams where several people and agents share memory under access control.

Facts about third-party projects come from their GitHub repositories and READMEs, checked 14 October 2026. Ethersofts has no commercial relationship with any project named here.