How We Tested Memory In AI Agents And What We Took Away
AIThis post was created with the assistance of artificial intelligence (AI).

🔍 Read the full analysis: How We Tested Memory In AI Agents And What We Took Away on ThorstenMeyerAI.com

Buying for a business?Offer from Amazon

Get business pricing on tech for your team

  • Business-only prices and quantity discounts
  • Tax-exempt purchasing
  • Multiple users, one account, clear invoices
As an affiliate, we earn on qualifying purchases.

TL;DR

Anthropic published a research post on November 6, 2026, describing how Claude agents can save, organize and correct information between sessions. Its reported tests suggest that compact, structured memory can outperform sprawling notes, though several results are single runs and have not been peer reviewed. The publisher also describes five problems it encountered while running its own Claude-based agents.

Anthropic published a research post on November 6 describing how its Claude models can write, organize and correct notes saved between conversations. The post reports tests in which a newer model used a smaller, more structured set of game notes than an older model, while earning more badges; the findings offer a look at how persistent memory may affect AI agents, but Anthropic says they come from its own evaluations rather than a peer-reviewed paper.

Anthropic defines agent memory as information a model gains through interactions or assigned tasks and retains between sessions in a form it can read later. That differs from a context window, which holds information available during one session, as well as from changes to model weights and summaries used to continue work when a context window fills up.

The post describes several components: agents write memory notes based on what seems likely to matter again; a later process, which Anthropic calls “dreaming,” checks notes against logs or transcripts and can correct or reorganize them; and structures such as linked wiki pages can make saved information easier to find. Anthropic also describes a memory hierarchy, with a small set of notes loaded at the start of a session and other material retrieved only when needed.

In a Pokémon Red test, Anthropic compared one run each of Claude Sonnet 3.7 and Claude Fable 5 at about 10,000 game steps. Sonnet 3.7 had created 413 files across 225 folders and earned one badge. Fable 5 had nine files in two folders, plus a top-level dashboard file, and earned four badges. Anthropic argues that repeatedly visible game details need not be saved, while plans and useful strategies may need to be recorded. These figures describe single runs, not a broad performance comparison.

At a glance
reportWhen: Research post published November 6, 202…
The developmentAnthropic published a research post on agent memory, reporting evaluations of how Claude models store and retrieve information across sessions.
Thorsten Meyer AI
Agent memory · game test

Infographic 01 · Remember less, remember the right things

What the screen shows does not belong in memory

Money and position can be read off the screen every turn. Plans, strategy and tested rules exist only if the agent writes them down.

Newer model, smaller memory4 badges

Claude Fable 5 wrote nine files in two folders, plus one top-level dashboard file (global.md) with its goal, plan and rules for using memory.

Memory files at about 10,000 steps

One run each, Pokémon Red, a memory folder the model could read and write

  • Claude Sonnet 3.7225 folders · 1 badge413 files
  • Claude Fable 52 folders + global.md · 4 badges9 files

Over 17 steps, Sonnet 3.7 created six files, deleted nine others and took no actions in the game.

413memory files written by the older model
225folders it spread them across
9files written by the newer model
4badges for the newer model, against 1

Source: Anthropic, The components of agent memory (research post, 6 Nov 2026). One run per model, Anthropic’s own evaluation.
Powered by Thorsten Meyer AI

Why Agent Memory Design Matters

When an agent carries information from one session to another, its saved notes can shape later decisions. Memory quality therefore affects continuity: a useful note may prevent the system from repeating work, while a mistaken or outdated one may lead it to repeat an error. Persistent memory is not simply a larger context window; it adds a process for choosing what to retain, how to retrieve it and when to revise it.

Anthropic’s examples point to a practical design trade-off. Saving everything can create clutter, but saving too little can leave an agent without the reasoning or tested rules needed for a later task. The source article says its own Claude-based agents exposed problems including crowded indexes, stale notes, messages crossing between records and detail lost during handoffs. It also reports that dated rules with explanations were more useful in its experience. Those observations are the publisher’s account, not independently verified results.

The results may interest teams building agents that work across sessions, especially where tasks depend on accumulated instructions or project knowledge. But the post does not establish that one memory approach will work best across models, tasks or users. Its findings are evidence about particular tests, not a general measure of agent reliability.

Products Worth Considering

Logitech MX Master 4 Ergonomic Wireless Mouse with Haptic Feedback, Graphite

Logitech MX Master 4 Ergonomic Wireless Mouse with Haptic Feedback, Graphite

LG UltraGear 34G600A-B 34-Inch Curved WQHD Gaming Monitor, 160Hz

LG UltraGear 34G600A-B 34-Inch Curved WQHD Gaming Monitor, 160Hz

As an affiliate, we earn on qualifying purchases.

From Session Context to Saved Notes

The source article distinguishes working memory from persistent memory using the analogy of a desk and a filing cabinet. A model’s context window contains the conversation and other information it can access now; persistent memory is material saved for later sessions. The distinction matters because an assistant without a saving mechanism generally does not carry details from one separate conversation into the next.

Anthropic’s post draws an analogy to human memory, where some sorting of experiences appears to happen during rest, and connects its “dreaming” concept to the idea of sleep-time compute, previously proposed by researchers at Letta and UC Berkeley. In Anthropic’s account, an agent can revisit past logs between sessions, check whether remembered information was supported and prepare notes for future work. The post also describes linked, subject-specific pages as one way to organize those notes.

The source article says Anthropic’s evaluations include a database task from Continual Learning Bench 1.0, where questions began in fresh sessions and a memory folder was carried between them. It reports that Sonnet 4.6 produced untested guesses, while Opus 5 was the first model in the comparison to pair most entries with a rule, evidence for the rule and warnings about traps. The supplied material also says Opus 5.5 and Sonnet 5.5 closed the loop, but does not provide scores for this comparison.

““Dreaming””

— Anthropic, describing its memory process

Limits of Anthropic’s Test Results

The findings are reported in Anthropic’s research post, not a peer-reviewed paper. The source article explicitly notes that several evaluations are single runs. In the Pokémon comparison, each model was represented by one run, so the badge and file counts do not establish a repeatable advantage or isolate memory design as the cause of the difference.

The supplied material does not give full methods, uncertainty ranges or comparative scores for every task. It also does not establish how the approaches perform across different model providers, real-world deployments or longer periods of use. The publisher’s five lessons from its own agents are presented as operational experience; the source says Anthropic did not review or sponsor that account, and the agents in it ran on Claude.

It remains unclear how reliably an agent can identify stale or incorrect notes, how much reviewing memory costs in time or computing resources, and what safeguards are needed when records contain sensitive information. The post’s examples describe approaches and particular results, not answers to all of those deployment questions.

Further Tests Across Tasks

The source material does not identify a scheduled follow-up study or publication date. Anthropic’s post lays out memory components and examples, while the publisher says it operates a small fleet of agents and has drawn practical lessons from their behavior. No next milestone is confirmed in the material provided.

For readers evaluating the claims, the next useful evidence would include repeated trials, fuller reporting of methods and scores, and comparisons across task types and models. Those results could show whether compact notes and transcript checks hold up beyond the examples described here. Until then, the post is best read as a report on Anthropic’s approach and evaluations, with the scope of its conclusions limited by the evidence made public.

Key Questions

What does Anthropic mean by agent memory?

It means information a model gains through interactions or tasks, saves, and can read in a later session. Anthropic distinguishes this from information held only in the current context window.

What happened in the Pokémon Red comparison?

In one run each at about 10,000 game steps, Sonnet 3.7 had created 413 memory files across 225 folders and earned one badge. Fable 5 had nine files in two folders plus a dashboard file and earned four badges. The result is a single-run comparison, not proof of a general performance advantage.

What is “dreaming” in Anthropic’s account?

It is Anthropic’s term for reviewing saved notes against logs or transcripts between sessions, with the aim of correcting mistakes, adding useful information or changing how memory is organized.

Are the reported results independently verified?

The source describes them as Anthropic’s own evaluations, published in a research post rather than a peer-reviewed paper. Several results are single runs, and the source material does not provide independent replication.

Source: ThorstenMeyerAI.com

You May Also Like

How Artificial Intelligence Creates Immersive History Of La Citadelle, 1693

Artificial intelligence has generated an interactive, cinematic reconstruction of the 1693 siege of La Citadelle, blending historical accuracy with artistic storytelling.

How I Use LLMs To Learn Complex Topics

A detailed look at how individuals leverage large language models for learning challenging subjects, highlighting methods, benefits, and challenges.

You Could Have Invented Transformers From Memory

Alex Wang’s essay frames neural-network layers as associative memory stores and estimates how their recall capacity changes from linear maps to attention.

The Trust Shock: What Suspending Fable 5 Means for US AI, Its Rivals, and the World

US government suspends Anthropic’s Fable 5 and Mythos 5, raising questions about trust, regulation, and future AI launches in the US and globally.