What is agent memory?
Agent memory is the information an AI agent keeps outside the language model so it can use it again, later in the same task or in a future session. It includes the working state of the current job and longer-lived stores of facts, past experiences and rules. The agent’s code decides what goes into those stores and which pieces go back into the prompt on the next call.
Memory has to live outside the model because the model keeps nothing. The CoALA paper, which proposed a common framework for language agents, puts it in one line: “Language models are stateless: they do not persist information across calls” arxiv.org. Every time a chatbot seems to recall your name or a past order, some code stored that detail and pasted it back into the prompt.
A 2024 survey of the field calls memory “the key component to support agent-environment interactions” arxiv.org. In practice the word covers several different stores, and each is built differently and fails in its own way.
What are the types of memory in AI agents?
One breakdown that framework docs borrow has four types: working memory for the task in hand, and three long-term stores called episodic, semantic and procedural. It comes from CoALA (Cognitive Architectures for Language Agents), a 2023 paper by Theodore Sumers, Shunyu Yao, Karthik Narasimhan and Thomas Griffiths that borrows its structure from cognitive science and older symbolic AI systems arxiv.org. The paper lists “short-term working memory and several long-term memories: episodic, semantic, and procedural” arxiv.org. LangChain’s memory docs point to the same paper when they map human memory types onto agents docs.langchain.com.
Working memory “maintains active and readily available information as symbolic variables for the current decision cycle”. CoALA says that includes perceptual inputs, knowledge the agent has just reasoned out or retrieved, and its active goals. CoALA treats it as a data structure that persists across model calls within a task, and the prompt is assembled from part of it each time.
Episodic memory “stores experience from earlier decision cycles”: what the agent did, what happened, what feedback it got. The term comes from psychology. In a 2002 review, Endel Tulving wrote that the notion was “first proposed some 30 years ago” and described episodic memory in humans as a system “that enables human beings to remember past experiences” pubmed.ncbi.nlm.nih.gov. In agents, an episode is a stored record of a past run, and LangChain’s docs note that episodic memories “are often implemented through few-shot example prompting” docs.langchain.com.
Semantic memory “stores an agent’s knowledge about the world and itself”. A customer’s preferred language, a supplier’s remittance address and a product’s warranty period are all semantic memories.
Procedural memory is how the agent acts. CoALA says language agents hold it in two forms: “implicit knowledge stored in the LLM weights, and explicit knowledge written in the agent’s code” arxiv.org. The LangChain docs add the prompt to that list, describing procedural memory as “a combination of model weights, agent code, and agent’s prompt” docs.langchain.com. Business rules an agent must follow, such as an approval threshold, belong here too.
Frameworks also sort memory by scope. LangGraph calls thread-scoped memory short-term (“tracks the ongoing conversation by maintaining message history within a session”) and calls memory that is “shared across conversational threads” long-term docs.langchain.com. The two schemes fit together: working memory is short-term, and the other three are long-term.
How do agents write and recall memories?
An agent writes a memory either while it works or in a background job afterwards, and it recalls one by scoring stored items against the current situation and putting the top few into the prompt. The LangChain docs name the two write paths “in the hot path” and “in the background” docs.langchain.com.
One early recall design comes from the 2023 Generative Agents paper, whose simulated townspeople keep a “memory stream, a database that maintains a comprehensive record of an agent’s experience” arxiv.org. Each memory is scored three ways. “Recency assigns a higher score to memory objects that were recently accessed”, importance favours what the agent judged significant, and relevance favours what relates to the current situation. The agents also write reflections, “higher-level, more abstract thoughts” built from many observations, back into the same stream.
MemGPT, also from 2023, treats the context window like RAM and outside storage like disk. In the paper, “External context refers to any information that is held outside of the LLMs fixed context window”. That context “must always be explicitly moved into main context” before the model can use it arxiv.org. Its authors went on to found Letta letta.com. Reflexion showed that an agent can improve across attempts without retraining: its agents “reflect on task feedback signals, then maintain their own reflective text in an episodic memory buffer” arxiv.org.
Where the memory is stored depends on the framework or product: a LangGraph checkpointer for thread state, a database of extracted facts, or a graph. mem0 “stores extracted memories, not a verbatim transcript” by default docs.mem0.ai. Graphiti keeps facts in a graph where “old facts are invalidated” rather than deleted when information changes github.com/getzep. Our comparison of nine memory layers covers how the products differ.
Who should be allowed to write each kind of memory?
The safer the memory type, the more freedom the agent can have to write it. Working and episodic memory can be written by the agent itself. Semantic memory needs checks before other agents rely on it. Procedural memory, which changes how agents act, should change only with a person’s approval.
CoALA backs the top of that ranking. It says learning by writing to procedural memory is possible, but “it is significantly riskier than writing to episodic or semantic memory, as it can easily introduce bugs or allow an agent to subvert its designers’ intentions” arxiv.org. Applied to an accounts payable agent, the four types look like this.
| Memory type | What it holds for an AP agent | How long it should last | Who should write it | What goes wrong if writes are unchecked |
|---|---|---|---|---|
| Working | The invoice being processed, the PO and goods receipt it pulled, its draft match result | One task; cleared when the run ends | The agent, freely | Little beyond the current run, as long as it is not saved as a long-term memory |
| Episodic | Past runs: that last month a reviewer changed the GL code on a similar invoice, and why | Months; prune old or superseded episodes | The agent or the platform, logged automatically | An odd one-off case gets recalled as if it were the normal pattern |
| Semantic | Facts: a supplier's bank details, a vendor's payment terms, which cost centre a recurring bill goes to | Until the fact changes; keep the date it was true from | The agent can propose; a source system or a person confirms | A wrong bank detail or an outdated term gets paid against |
| Procedural | How to act: the match tolerance, which invoices need director approval, a vendor exception with an end date | Until a person changes or retires it; keep prior versions | A named owner approves every change; the agent may only suggest | The agent rewrites its own rules and every later invoice follows the wrong one |
| Trail | Shared company context for business agents: policies, ERP and CRM records, email threads and SOPs parsed into a graph, plus business rules, each cited to document, page and line | Until superseded; prior versions retained so past decisions replay | A named approver signs off; agents can report problems but cannot approve | Built for company context and business rules; we found no mention on its pages of storing an agent's working state, its chat history with users or end-user preferences, so pair it with a framework or memory service for thosetrailhq.com |
Facts in this table were checked against the sources listed at the end of this page, between and .
LangChain’s docs note that in practice “it is fairly uncommon for agents to modify their model weights or rewrite their code. However, it is more common for agents to modify their own prompts” docs.langchain.com. A self-edited prompt is a procedural memory write. For a support bot that tweaks its tone, that is fine. For an agent that pays invoices, an approval step belongs between the suggestion and the change. Our guide on governing a memory layer for LLM agents covers that approval, versioning and audit work in detail.
How is agent memory different from RAG and a long context window?
RAG is one way to read semantic memory, and a context window is where working memory sits during a call. Agent memory is the wider system that decides what to keep and what to load next time.
CoALA says retrieval-augmented methods “can be viewed as retrieving from a semantic memory of unstructured text”, and that such examples use “a fixed, read-only semantic memory” arxiv.org. A memory system adds the write side: facts learned during a run, records of what happened, corrections from reviewers. Trail’s post on what a company brain is makes a related point about storage: “A vector database is one possible storage component of a company brain, not the brain itself.”
A longer context window helps up to a point. CoALA grants that “a longer context window could reduce the importance of long-term memory”. The LangChain docs describe the limits: “A full history may not fit inside an LLM’s context window, resulting in an irrecoverable error”, and “most LLMs still perform poorly over long contexts” even when it does fit docs.langchain.com. Whatever is in the window is also gone when the session ends, and a second agent never sees it.
Which memory types does your agent need?
Start with the smallest set that covers the failures you expect, and add a type only when you can point to a mistake it would prevent. Each added type brings its own store and write path, and more for someone to review.
Choosing agent memory types: a decision rule (ours)
Checklist- Working memory onlyThe task finishes in one session and nothing needs to carry over, such as summarising a document or answering a one-off question. Every agent has this already in its prompt and state.
- Add semantic memoryThe same user, customer or supplier comes back and the agent should not ask for the same facts twice. Store each fact with its source and the date it became true, so it can be checked and expired.
- Add episodic memoryThe agent repeats a task and reviewers keep correcting the same kind of mistake. Store past runs with their outcomes and reuse the good ones as examples.
- Add governed procedural memoryThe agent acts on company policy: payments, approvals, refunds, compliance checks. Keep the rules outside the prompt, give each one an owner and an approval step, and keep old versions so a past decision can be explained.
- Share long-term memory across agentsSeveral agents handle the same process. They should read the same approved facts and rules instead of each keeping its own copy, or they will drift apart.
By that rule, a support assistant that answers questions may stop at semantic memory. A finance or operations agent that acts on policy reaches governed procedural memory, which carries the most risk because one bad rule affects every case it matches.
How do you correct or forget an agent memory?
Mark the old memory as superseded, keep a record of it, and make sure the agent stops reading it. Research has paid it less attention than writing: CoALA notes that “modifying and deleting (a case of “unlearning”) are understudied in recent language agents” arxiv.org.
Graphiti, for one, invalidates old facts instead of deleting them, so you can still ask what was true at an earlier date. In finance, that answers an auditor who asks why an invoice was paid under last quarter’s rule. Personal data may still need real deletion, for example after an erasure request, so check both behaviours before choosing a store.
How Trail holds shared company memory for agents
Trail, the memory and context layer for enterprise AI agents, covers the shared semantic and procedural rows of the table for business agents. Its Context Graph holds “Five kinds of context, connected and cited”: policy documents, ERP and CRM records, computed figures, email and call logs, and SOPs. Approved business rules are stored on that graph, and the page says sources are “Parsed into structure, not flattened into vectors” trailhq.com. Retrieval still uses similarity alongside the graph: the walkthrough says a query “is matched two ways: by similarity and by entity” trailhq.com.
Writes go through a person. Its rule engine page says “Nothing becomes available to an agent because a model inferred it”, and each rule carries “A named approver, with a timestamp” with “Prior versions retained, so past decisions replay” trailhq.com. When a reviewer corrects an agent, Trail “turns the correction into a proposed rule, routes it to the owner, and applies it once approved” trailhq.com. An agent can flag a problem but, in the how-it-works page’s words, “It can report. It cannot approve.” Once approved, a rule reaches “every connected agent” from the next question on trailhq.com. Agents connect through an MCP server, LangChain and LlamaIndex retrievers, or REST and GraphQL trailhq.com.
Where it is not the fit: Trail is built for company context and business rules, and we found no mention on its pages of storing an agent’s working state, its chat history with users or end-user preferences. A LangGraph checkpointer handles thread state, and mem0 or a similar service handles user facts. A support bot that only needs to remember customers has no use for an approval queue, and every rule Trail proposes takes an owner’s time to review.
How we sourced this explainer on agent memory
We read the primary papers for the definitions: CoALA (arXiv 2309.02427, full text), Generative Agents, MemGPT, Reflexion, the 2024 memory survey by Zhang and colleagues, and Tulving’s 2002 review via PubMed. We used the LangChain memory docs for framework terms, mem0, Graphiti and Letta pages for product examples, and Trail’s homepage, rule engine, context graph, how-it-works, walkthrough and comparison pages for every Trail claim. We also read the top-ranking explainers from mem0 and Atlan to see what they cover. All sources were checked on 20 October 2026. The memory-type table and the decision rule are our own synthesis; we did not build or test an agent for this page. Trail publishes this page.
Frequently asked questions
What is agent memory?
Agent memory is the information an AI agent keeps outside the language model so it can use it on a later step or in a later session. It covers the working state of the current task and long-term stores of facts, past experiences and rules. The model itself keeps nothing between calls, so the agent's code decides what to store and what to put back into the prompt.
What are the types of memory in AI agents?
The CoALA paper splits it into four types: working memory for the current task, episodic memory for past experiences, semantic memory for facts, and procedural memory for how the agent should act. Frameworks such as LangGraph also group them by scope, calling thread-scoped memory short-term and anything shared across sessions long-term.
Does a bigger context window replace agent memory?
No. A larger window lets more history fit into one call, but the history still disappears when the session ends, and LangChain's docs note that most models perform poorly over long contexts anyway. Long-term memory is still needed for anything that must carry over to another session, user or agent.
Should an AI agent be allowed to write its own memory?
For notes about the current task and records of what happened, usually yes. For facts other agents will rely on, add checks such as source links and expiry dates. For rules that change how agents act, a person should approve the change first, which is how Trail, the memory and context layer for enterprise AI agents, handles business rules.
Sources checked
Show 19 sources
- Sumers, Yao, Narasimhan, Griffiths: Cognitive Architectures for Language Agents (CoALA), arXiv
- CoALA full text (arXiv HTML)
- Tulving, Episodic memory: from mind to brain, Annual Review of Psychology (2002), PubMed
- Park et al.: Generative Agents, Interactive Simulacra of Human Behavior (arXiv HTML)
- Packer et al.: MemGPT, Towards LLMs as Operating Systems (arXiv)
- MemGPT full text (arXiv HTML)
- Shinn et al.: Reflexion, Language Agents with Verbal Reinforcement Learning (arXiv)
- Zhang et al.: A Survey on the Memory Mechanism of Large Language Model based Agents (arXiv)
- LangChain docs: memory overview
- mem0 docs: how Mem0 works
- Graphiti README
- Letta homepage
- Trail homepage
- Trail Rule Engine
- Trail Context Graph
- Trail Context Graph: how it works
- Trail walkthrough
- Trail comparison
- Trail blog: what is a company brain