Your Agent's Context Window Isn't Memory. That Confusion Is Why It Keeps Forgetting Things.

Two different problems wearing the same word

"Memory" gets used for two unrelated things in agent systems, and the conflation causes real production bugs. The first is the context window: the tokens the model can see in a single call, which resets at the end of a session or gets truncated when the conversation gets long. The second is persistent memory: facts, preferences, and state that need to survive across sessions, get updated when they change, and get retrieved selectively rather than dumped in wholesale. A model with a million-token context window still has zero persistent memory unless something outside the model is deciding what to write down, when to update it, and when to throw it away.

This matters for a buyer evaluating an agent vendor because "our agent has memory" is doing a lot of unearned work in a sales pitch. The real question is narrower: what's the write path, what's the read path, and what happens when two stored facts disagree.

Where this actually breaks

The common failure isn't the model hallucinating from nothing — it's the model reasoning correctly from state that's wrong. A customer tells a support agent their shipping address changed. If the system appends that as a new fact instead of superseding the old one, the agent now has two addresses on file and no principled way to pick between them. Ask it again later and it may pick the stale one, because nothing told it the old fact was dead.

The pattern repeats everywhere agents carry state across turns: a scheduling agent holding two conflicting availability windows, an internal ops agent working from a budget figure that was revised last week but never overwritten, a sales agent citing a price that changed. None of this shows up as an obvious crash. It shows up as the agent confidently giving an answer that was true at some point, just not now — which is a harder bug to catch than an error message, because nothing fails loudly.

The second common failure is retrieval relevance, which is the same problem RAG systems have, applied to an agent's own history: as stored facts accumulate, pulling back the right ones at the right moment gets harder, and an agent with ten thousand stale, half-contradictory memories can perform worse than one with none at all. More storage without a decay or conflict-resolution strategy makes the system slower and less reliable, not smarter.

The architectures in actual use right now

There's no single standard yet, but the production approaches split along a few lines worth knowing before you evaluate a vendor's claims:

• Vector-plus-structured stores (the pattern behind tools like Mem0) extract atomic facts from conversation, embed them, and retrieve by similarity, often layering a lightweight graph on top for relationships between facts. This is the simplest to stand up and the easiest to reason about for straightforward preference and fact storage.
• Temporal knowledge graphs (the approach behind Zep, built on its open-source Graphiti engine) attach a valid-time and a record-time to every fact, so a superseded fact gets marked rather than deleted and the system can answer "what did we believe at that point," not just "what do we believe now." This buys you auditability and correct handling of changing facts, at the cost of more moving parts.
• Self-managed memory blocks (the Letta/MemGPT lineage) give the agent a small number of labeled, size-limited text fields it edits itself as part of its own reasoning, plus a filesystem-style archive it can page through. This keeps the "what does the agent believe right now" surface small and inspectable, but puts more of the correctness burden on the model's own editing judgment.

None of these is strictly better. A support bot that mostly needs "what did this customer already tell us" is well served by the first pattern. A system that has to reason about what changed and when — compliance, billing history, anything audit-sensitive — needs the second. An agent doing long, autonomous multi-step work benefits from the third because it can manage its own working set instead of a backend deciding for it.

What to actually check before you buy or build this

If a vendor or an internal team tells you an agent "remembers" things, ask three concrete questions instead of taking the word at face value:

1. What happens when a fact changes? Does the old value get overwritten, marked superseded, or just left to sit alongside the new one as a second, contradicting entry?
2. What's the retention and decay policy? Facts that are never revisited should lose retrieval priority or get archived — a system with no forgetting mechanism degrades as it accumulates history, it doesn't improve.
3. Can you see what the agent currently "believes" about a given user or case? If there's no way to inspect the current memory state outside of the agent's own answers, you have no way to catch a stale or contradictory fact before a customer does.

The honest framing for a business evaluating this: persistent memory is a data design problem that happens to sit next to a language model, not a model capability. A bigger context window doesn't change any of the three questions above — it just means the model can see more tokens per call, which is a different axis entirely from whether the system remembers the right thing correctly over time.

Takeaway: before trusting an agent with state that persists across sessions, get a straight answer on how it resolves conflicting facts and how old facts get retired — not just whether it "remembers."