What an Agent Remembers: LLM Memory between Context Window and Knowledge Graph

Advanced Topics 1 · Series: Getting Started with n8n

In Article 6 I deliberately did not use the AI Agent node with attached memory. For pure ticket classification it is the wrong tool: a single model call that follows a fixed schema needs no memory of past tickets. That decision was correct, but it left a question open, one that arises the moment an agent has to stay coherent across more than a single call: how does a language model remember anything at all, and which kind of memory does which task require.

This article is the first in a series of Advanced Topics that go deeper into individual subjects from the n8n series. Here the focus is the concept: how a language model remembers, and which kind of memory fits which task. What that looks like in practice is the subject of the next article, where I build into a ticket triage workflow what n8n does not bring out of the box.

Memory is often treated as a function you switch on. In practice it is an architecture decision with consequences for scalability, latency, and reliability. So I draw the lines from the ground up, with the sources behind each claim.

Memory is architecture, not a feature

A language model has two fundamentally different kinds of knowledge. One sits in the weights, frozen at training time. That is the parametric side: it costs no extra latency, it carries general world knowledge, and it cannot be changed without expensive fine-tuning. From the moment training ends, this knowledge begins to age. Whoever relies on the weights for live data gets confidently delivered, but possibly outdated answers.

The other knowledge is injected into the context at runtime. That is the non-parametric side, and it is the actual lever. Everything an agent knows about the current case, the user, or the domain without it being in the weights is handed to it actively. This shifts the developer’s role: instead of merely addressing a stateless model, they manage the state of the application across several steps. That is precisely the point where memory stops being a toggle and becomes an architecture decision.

This first axis, parametric versus non-parametric, describes where knowledge lives. It does not yet answer which function a given memory item serves. That second question is easily conflated with the first, and the conflation produces a fuzzy classification.

The clean taxonomy knows four functions

A common way to sort memory lines up four terms: in-context, external, parametric, and episodic. As an entry point that is catchy, but it mixes two different questions. Three of these terms describe a storage location. In-context is what currently sits in the prompt, external is what comes from a store via retrieval, parametric is what sits in the weights. The fourth, episodic, describes no location but a cognitive function. Episodic memory is therefore not a fourth, parallel type alongside the three. In practice it is implemented as external memory: summarized logs and user profiles, reloaded by a session ID. Location and function belong on two axes, not in one list.

Separate the function out cleanly and an established framework is at hand. CoALA, short for Cognitive Architectures for Language Agents (Sumers et al., 2023, published in TMLR 2024), adopts the cognitive-science breakdown and distinguishes four functions:

Two of these functions fall out of the catchy four-term row: semantic and procedural. The semantic layer in particular is the dominant type for personal assistants, the procedural one for coding agents. More important still is the step that connects episodic and semantic: the consolidation of concrete experiences into generalized facts. An agent that only collects episodes without ever condensing them piles up a chronicle, but it learns nothing. This condensation step, known in the literature as reflection, is the real transition from “experienced it” to “knows it”.

This is no academic detail. Memory as a research field is young and contested. Systems like MemGPT/Letta, Mem0, or Zep/Graphiti openly compete for the best approach, and even the de-facto benchmark LoCoMo is the subject of public methodological disputes. MemPalace, which I wrote about in April, addresses exactly the episodic layer: it stores the conversation transcripts as raw text instead of having an AI summarize them, and on that basis claims an edge over Mem0 and Zep on LongMemEval. Sorting memory into four clean drawers suggests a closure that the subject does not have.

The context window is not memory

A common and tempting assumption goes: if the context window is large enough, I need no memory, I just put everything in. That assumption is not empirically supported. On the contrary, three studies contradict it, and they do so in different ways, which is why it is worth keeping them apart.

The first work is Lost in the Middle (Liu et al., 2023). Models use information at the beginning and the end of a long prompt reliably, in the middle less so. The accuracy curve over position forms a U. When the relevant passage sits in the middle, accuracy drops markedly. Hence the common shorthand that transformers ignore the middle of a dense prompt.

The second work is Chroma’s Context Rot (2025), and it says something more general. Across 18 models, the study shows that performance becomes less reliable as input length grows, independent of position alone and even on trivial tasks. Pure length increase cost about 8 percent accuracy on average (Source: Chroma 2025). This qualifies the clean U: the degradation is primarily length-driven, not chiefly a middle effect. The two studies are often seen in the same context, as if the neglected middle were the cause of the length degradation. But they are two different observations, and only the second supports the conclusion that more tokens harm in themselves.

A third work sharpens the picture. NoLiMa (Adobe Research, ICML 2025) deliberately removes the literal overlap between question and target passage, so the model has to genuinely understand rather than just match keywords. Under that condition the effective usable context length falls far below the advertised windows, in many cases to a few thousand tokens (Source: NoLiMa 2025). The conclusion across all three studies is robust: more tokens are not automatically better, and a large context window is no substitute for a deliberate memory. So if storage in the prompt does not grow without limit, the knowledge has to move outward.

RAG as the external memory

As soon as the data grows too large for the prompt, it is moved into a retrieval layer. The documents sit as embeddings in a vector store, and at runtime a similarity search pulls the matching pieces and places them in the context. This is retrieval-augmented generation, the standard way to ground answers in verifiable facts without fine-tuning. Storage is practically unlimited, and token costs stay predictable because only the relevant pieces go into the model.

Retrieval is often portrayed as the simple, static predecessor of the clever agentic approaches. That underestimates how far the state of the art has come. The open question in RAG is chunking: cut too small and context is lost, cut too large and you blow the token budget and pull in noise. Anthropic’s Contextual Retrieval targets exactly this and prepends to every chunk a short, model-generated context sentence before embedding. The effect is measurable: the failure rate at retrieval drops by 35 percent through contextualized embeddings alone, by 49 percent together with BM25, and by 67 percent when a re-ranker is added on top (Source: Anthropic 2024). Hybrid search from dense vectors and classic keyword matching, followed by a re-ranker that reorders hits by actual relevance, is textbook today. Well-built RAG thus solves many of the problems one would otherwise assign only to the next level of complexity.

Agentic RAG moves control into the model

In agentic RAG it is no longer a hard-wired pipeline that decides what gets retrieved, but the model itself. The agent is given retrieval tools and chooses whether to search, where to look, and whether to start a refined second search after an insufficient hit. For multi-step research, where a single query is not enough and several sources must be compared, that is the right flexibility.

This flexibility is not free. The path through retrieval is no longer deterministic, the same question can take two different routes. Every additional reasoning step costs another model call, which drives up latency and token consumption. And debugging gets harder, because a wrongly chosen search path is not in the code but in a model decision. Anthropic’s line for such systems is therefore to start with the simplest solution and add complexity only when it demonstrably pays off (Source: Anthropic, Building Effective Agents). Agentic RAG is not the default, but an answer to tasks the linear path does not solve. In n8n this approach has a practical advantage: every retrieval step the agent takes is visible in the executions view, that is, which tool was called with which query and what came back. That turns an otherwise opaque multi-step run into something traceable.

GraphRAG is a special tool, not an upgrade

GraphRAG replaces the flat vector store with a knowledge graph. Instead of finding similar-sounding text pieces, the approach maps entities and their relationships and lets you traverse the web of data. Microsoft’s implementation has a model extract entities and relations from the raw text, groups them via community detection, and pre-generates summaries per cluster. The strength lies in global questions, such as overarching themes across hundreds of documents that a pure similarity search cannot bring together.

This strength comes at a price that is rarely named. The extraction is model-driven and expensive, both in the initial indexing and on every update, because communities are recomputed. By comparison, a query over the graph costs a multiple of the tokens of plain RAG, in one benchmark roughly 377 times (Source: GraphRAG-Bench 2025). And the benefit is not universal: on simple fact retrieval GraphRAG performs measurably worse than a plain vector store, because the detour through the graph contributes nothing there. Lighter variants like LightRAG or temporal approaches like Graphiti lower the cost but change nothing about the principle. How high the price turns out, however, depends on the data. Graphify, which translates a code repository into a local knowledge graph, derives the structure deterministically via a tree-sitter parser and only needs the expensive model for unstructured material such as PDFs. With clearly structured data a graph can be built much more cheaply than model-driven extraction suggests. The trade-off stays the same, a build step against better structural questions, only the bill comes out differently. The consensus for 2026 is therefore not “graph beats vector” but adaptive routing: simple queries to the vector store, complex and global ones to the graph. GraphRAG is a tool for a particular class of questions, not the next level up for everything.

Where memory fails under load

A cleanly chosen memory concept is not enough if operation hollows it out under load. Four weak points deserve attention, three of them concern reading, one writing.

The first is context rot over long-running tasks. The longer a conversation runs, the more the model loses sight of the central requirements while still keeping the greeting from the start at hand. The countermeasure is compression: older sections are regularly condensed into a brief state summary, so the important information stays in the reliable zones of the context.

The second is retrieval failures at scale. Semantic similarity is not the same as relevance, and as the corpus grows a search for the k most similar hits (top-k) increasingly pulls in pieces that share keywords but do not carry the needed context. They distract the model and inflate the budget. The countermeasure is the combination already named of hybrid search and a re-ranker that scores relevance before anything reaches the context window.

The third is relevance drift in agent loops. When an agent formulates its own search queries, a single near miss can trigger a chain: the agent uses the noise to shape the next search and drifts further from the original intent with each step. The countermeasure is a guard that checks the confidence of the retrieved data and does not let the agent press on with hits that are too weak, but instead resets the search or asks back. This guard sits on the read path, it filters what comes back from storage.

The fourth weak point is rarely named and is the most dangerous one the moment an agent itself writes into storage. If a run takes a wrong turn and deposits a fabricated fact in the store, later queries pull it back as truth. The error does not vanish on its own, it amplifies, because every query confirms it and the agent builds on it. This is write contamination, memory poisoning, and it is the write-path variant of the compounding errors the agent literature warns about. The read guard from the previous paragraph does not help here, because the nonsense is long since persistent by that point. What is missing is a second guard on the write path. Before committing, an independent instance checks whether the entry matches the expected schema, whether it is backed by a source, and whether confidence is above the threshold. If it fails, it goes into quarantine or a dead-letter branch, not into the store. The established patterns for this are called Self-RAG, where the model critiques its own output before committing, and Corrective RAG, where a dedicated evaluator catches weak results. The basic rule is simple: an AI never writes into long-term memory unvalidated, and every entry carries its provenance.

Behind these four points stands a larger picture that pure retrieval hygiene often misses. Memory is not only additive. What is never forgotten clogs storage and distracts, so forgetting and decay strategies belong to the design as much as storing itself. And the consolidation from episodic to semantic, the reflection step from the taxonomy section, is what turns collected cases into usable knowledge. Under the term context engineering four tasks have emerged that summarize this: write, select, compress, and isolate. Retrieval covers only the select part.

n8n’s native baseline and its limits

Against this background it can be said precisely what n8n brings out of the box. The AI Agent node accepts a memory sub-node. Simple Memory holds the last N interactions as a sliding window in workflow storage, while the variants Redis Chat Memory, Postgres Chat Memory, and MongoDB Chat Memory persist the same history. The Chat Memory Manager additionally allows inserting, reading, or deleting messages deliberately. On the retrieval side, vector store nodes are available for Pinecone, Qdrant, Supabase, PGVector, Milvus, Weaviate, Chroma, and more, plus embedding nodes including local models via Ollama. A vector store can be attached directly as a tool to the agent. For the practice of this series that is a solid baseline.

Measured against this overview, though, the gaps become just as clear. “Memory” in n8n means conversation history, not long-term memory with consolidation. There is no native node that summarizes older transcripts, the window cuts off hard rather than condensing. There is no logic for forgetting or decay, only the hard cutoff or full persistence. Re-ranking is available natively but limited to Cohere. GraphRAG is missing entirely, n8n does classic vector retrieval. n8n’s in-memory vector store (the Simple Vector Store) holds the embeddings only in the process’s working memory, without writing them out. That is enough to try out a RAG flow without external infrastructure, or to query freshly fetched documents once and then discard them. As a lasting knowledge base it does not serve, because the data is lost on a restart and can be evicted under memory pressure. Simple Memory does not work in queue mode, because there is no guarantee that every call hits the same worker. And a guard on the write path, checking what an AI deposits in the store, does not exist.

None of these gaps is a knockout criterion. Each can be closed, with built-in means or an external service. That is exactly the subject of the next article.

Next step

With that the overview is complete: memory is an architecture decision over four cognitive functions, the context window is not storage, retrieval ranges from plain RAG to the expensive knowledge graph, and under load four weak points show up, of which write contamination is the least heeded. n8n covers the basics of this and leaves the demanding parts open. The next Advanced Topic takes the ticket triage workflow from Article 6 and closes these gaps one by one: a summarization memory, a forgetting rule, a custom re-ranker, and a write guard in front of the store.

← Article 6: AI Classification Without the Cloud