The role of reasoning in agent memory

AI agents can improve at recurring tasks without retraining by learning from previous experience. We tested whether lessons improve when a reviewer can also see the agent’s reasoning. Across four models handling customer-support tickets, memory based on past actions increased resolution rates from 34% to 59%. Adding reasoning raised them further to 67%.

One way agents can learn from experience is through memory-based recursive self-improvement (RSI). After completing a task, a reviewer model examines the agent’s actions and outcomes, extracts useful lessons, and adds them to the agent’s instructions for future tasks without retraining the underlying model.

The reviewer can also examine the agent’s reasoning, often called its chain of thought. This provides additional context about why the agent made particular decisions, but also increases the amount of information the reviewer must process. We wanted to understand whether that additional context leads to better lessons.

We used τ²-bench, a customer-support benchmark from Sierra Research, to test four AI models on telecom support tickets. These included problems like customers losing mobile data or being unable to send picture messages. Each agent had to diagnose and resolve the issue using account and phone tools, with an automated grader checking whether it succeeded.

We ran the benchmark in solo mode, without a simulated customer, and without its troubleshooting guide.

Each agent worked through sequences of 24 tickets. We compared three approaches:

1. No memory: The agent handled each ticket without lessons from previous tasks.
2. Action memory: After each ticket, a reviewer model (Claude Opus 5.5) examined the agent’s actions and outcomes and updated its lessons.
3. Reasoning memory: The reviewer also had access to the agent’s reasoning behind its decisions.

Everything else stayed the same: the tickets, their order, the reviewer model, and the instructions for writing lessons, adapted from Google’s ReasoningBank. We ran 12 sequences per model.

We selected four open models that expose their reasoning so we could compare the value of recording actions alone against recording actions and reasoning together.

Figure 1. The memory loop. The three groups differ only in what reaches the lesson-writer.

Every model performed better with memory, and most of the improvement came from action memory alone. Across all four models, the average share of tickets resolved increased from 33.5% without memory to 59.2% with action memory and 66.5% with reasoning memory.

Ticket resolution rates across four agent models, comparing no memory, action memory and reasoning memory. Results exclude the first ticket in each sequence, before any lessons were available.

The additional benefit of reasoning varied significantly by model. For DeepSeek v4 Flash, it improved resolution by nearly 17 percentage points. For MiniMax-M2.5, it made almost no difference.

DeepSeek showed the clearest evidence of improvement as it accumulated lessons. With reasoning memory, its resolution rate increased from 71% on tickets 2–8 to 93% in the final third of each sequence. Without memory, performance remained around 50%.

Action memory improved performance initially but declined later in the sequences. This was partly due to harder tickets appearing toward the end, particularly cases involving customers who had run out of mobile data.

On these tickets, action memory resolved half or fewer, while reasoning memory improved from resolving 56% to 87%.

Figure 2: DeepSeek v4 Flash performance across successive tickets

As agents completed more tasks, their memories didn’t keep growing indefinitely. Instead, the reviewer revised, consolidated, and removed lessons as new information became available.

Memory size eventually leveled off, although this was partly due to a limit on how much could be stored. The result shows how agents can accumulate lessons from experience while keeping their memory relatively compact.

Figure 3: Memory size over time across runs. Memory grows early, then levels off as lessons are revised, merged, and removed.

The out-of-data tickets help explain why reasoning made a difference.

To resolve these tickets, the agent needed to add data to a customer’s plan. The benchmark’s grader checked whether it added the correct amount. But because the benchmark provided no customer to consult, the agent had to guess.

At the same point in a DeepSeek sequence, the two reviewers generated different lessons:

Action memory:

Refueling unasked fails the ticket.

Reasoning memory:

In tool-only tickets with no amount, a single 2GB refuel passed; a silent 1GB refuel failed.

Both reviewers could see the amount of data the agent had added and whether the ticket passed. But only the reasoning-aware reviewer could see that the agent had been guessing.

That context helped it produce a more specific lesson. Rather than concluding that the agent shouldn’t add data without asking, the reasoning-based lesson identified an action that had succeeded under the benchmark’s constraints.

In a real customer-support setting, the agent should ask the customer how much data they need. This particular lesson is specific to the benchmark, but it illustrates how access to reasoning can change what an agent learns from an experience.

Agents can improve substantially without retraining, and much of that improvement comes from learning from past actions and outcomes. Access to reasoning can help further, though the benefit varies by model and comes at a cost: approximately 30% more per ticket compared with action memory in our experiments.

Our study relied on a stronger reviewer model and definitive success signals from an automated grader. The agents also worked without the benchmark’s troubleshooting guide, which their accumulated lessons may have partly replaced. In production, feedback is less clear, often coming from repeated questions, user frustration, or escalations. Understanding whether agents can learn as effectively from these noisier signals is an important next step.

τ²-bench is a benchmark of customer-service tasks from Sierra Research. Each task has a simulated back office and a programmatic grader. The telecom domain plays a phone carrier. The agent has the carrier's account tools and the customer's phone. The benchmark plants faults on the line and on the phone. The agent must find them and fix them. We used the domain in solo mode. There is no simulated customer, and the agent works the ticket alone.

The benchmark in numbers.

The three ticket types and what the grader checks.

The benchmark plants 19 kinds of fault in our chains. Phone settings: airplane mode on, mobile data off, data saver on, a wrong network mode, Wi-Fi Calling on, a slow VPN. SIM: unseated, or locked with a PIN. APN: broken settings, or a broken MMS setting. App permissions: SMS, storage, or both removed from the messaging app. Roaming: the customer is abroad and roaming is off on the line, on the phone, or both. Account: the data allowance is used up, or the line is suspended for an overdue bill or an ended contract. A ticket names none of these. The agent sees only the symptom.

A sample ticket looks like this.

The lesson-writer is Claude Opus 5.5. After every ticket it reads the ticket, its group's record of the episode, and the notes carried from the previous ticket. It returns the complete updated notes. The action-memory writer reads every tool call with its arguments and result, and the outcome. The CoT-memory writer reads the same, plus the agent's reasoning at every turn. Nothing else differs.

Each lesson has a title, a one-sentence description of when to use it, and one to three sentences of content. This is ReasoningBank's memory-item format. The writer's instructions are ReasoningBank's extraction prompts with the domain words swapped. We did not tune them. The grader's outcome picks one of two versions: one for a resolved ticket, one for an unresolved ticket. They differ in five phrases. A merge paragraph we added follows both versions. It makes the writer return the whole set of notes, not only the new lessons. It names the cap: 32 lessons and 14,400 characters, which the harness enforces as 14,400 bytes. The format, the prompt and the merge paragraph are below, verbatim.

The lesson format, as the agent reads it:

The writer’s system prompt:

The merge paragraph, appended to both versions:

All four agents are open-weight models. We call them through one AI gateway with the same limits. Each turn may return 8,000 output tokens. Each ticket may take 100 steps. Each call may take 120 seconds. gpt-oss-120b and minimax-m2.5 were the first two agents. DeepSeek v4 Flash and Qwen3-Next-80B thinking came from a screen of four open-weight candidates. A candidate passed on three conditions. It called the tools reliably. It returned reasoning on most turns. It resolved 5% to 80% of tickets unaided. GLM-5.3-Flash and Kimi K3 returned reasoning on under a third of turns, so we dropped them.

The four agents: what they are and how we called them.

Figure D1 shows success at each ticket position for all four agents, 12 sequences each, with the 14,400-byte notes. The lines are trailing five-ticket means. Every group plays ticket 1 with empty notes, so the groups differ there by chance only. The no-memory and action-memory lines drop in the last third for every agent. All four agents play the same 12 sequences, and that third has more tickets with a used-up data allowance (see Finding 2).

Figure D1. Success at each ticket position, all four agents, 14,400-byte notes, 12 ticket sequences per agent. Lines are trailing five-ticket means.