From Stored Corrections to Reliable Behaviour: Evaluating AI Memory

Abstract

An AI system can store a correction and still repeat the corrected failure. The missing step is evaluation at the point where memory is expected to alter behaviour. Existing long-term-memory research usefully measures recall, temporal reasoning, knowledge updates and answer generation, but an operational correction often has to survive a longer chain: it must be retained, retrieved in a new session, selected as relevant, translated into the right decision, and reflected in a verifiable outcome. This paper proposes a fresh-session correction-transfer protocol that measures those stages separately. It treats memory not as a binary feature but as a socio-technical pipeline with observable failure boundaries. The protocol includes matched positive and negative cases, supersession and contradiction tests, action-level scoring, provenance checks and outcome verification. It is a design proposal, not a report of a controlled study. Its purpose is to make claims such as “the system learned” testable without confusing storage, retrieval or fluent acknowledgement with reliable behaviour.

1. The question

How should an AI system be evaluated when the practical requirement is not merely to recall a prior correction, but to behave consistently with that correction in a fresh session and produce the intended result?

The scope is persistent memory used by assistants and agents across sessions. “Correction” means a human statement that revises a fact, preference, procedure, boundary or prior interpretation. “Transfer” means that the correction affects a later, independently initiated task. This paper does not claim that persistent memory is model-weight training, that every correction should become durable, or that perfect recall is desirable. Some information should expire, remain task-local, require confirmation or never be stored.

The central proposal is simple: evaluate the whole correction-to-outcome path, while retaining stage-level measures that explain why it failed.

2. Why storage is an inadequate success criterion

Memory-augmented systems separate information that is immediately in a model’s context from information held elsewhere. MemGPT, for example, models the context window as constrained “main memory” and uses external storage plus explicit movement of information into context [2]. That architecture illustrates an important boundary: an item can exist in storage while remaining unavailable to the current inference.

Retrieval is still not the end of the chain. LongMemEval decomposes long-term memory into indexing, retrieval and reading, and evaluates information extraction, multi-session reasoning, temporal reasoning, knowledge updates and abstention [1]. Its reported error analysis includes instances where the relevant material was retrieved but the generated answer was wrong [1]. The operational implication is broader than conversational question answering: retrieval is a necessary intermediate event in many designs, not proof that the system used a correction correctly.

LoCoMo likewise broadens evaluation beyond isolated factual recall. It uses question answering, event summarisation and dialogue generation to probe long-range conversational understanding, including temporal and causal relations [3]. These benchmarks supply valuable tasks and failure categories. They do not, however, directly answer whether a correction transferred into a later tool choice, permission decision, calculation, document edit or other externally inspectable act.

A system that says “I remember that you prefer X” may still select Y in the workflow. Conversely, a system may take the right action without exposing the correct memory trace: the action might be a coincidence, a default or contamination from the test prompt. Both cases matter. Memory evaluation therefore needs diagnostic evidence and behavioural evidence.

3. A correction-transfer model

We propose six stages:

Correction
   ↓
[1] Retention → [2] Retrieval → [3] Relevance
                                  ↓
[6] Outcome ← [5] Action ← [4] Decision

3.1 Retention

The correction is represented in an eligible durable store with provenance, scope, time and lifecycle status. Retention asks whether a record exists and whether it preserves the correction’s meaning. It does not yet ask whether the record will be found.

3.2 Retrieval

The later task activates the stored correction without restating it in the test prompt. Evaluation should capture both recall and over-retrieval. A system that retrieves every preference for every task may score well on raw recall while degrading privacy, efficiency and relevance.

3.3 Relevance

The system distinguishes applicable corrections from superficially similar but inapplicable memories. Scope may depend on user, project, role, jurisdiction, date, tool or task class. This stage is where negative controls are essential: the evaluator needs cases in which the memory exists but should not govern the current task.

3.4 Decision

The system converts relevant memory into the right planned choice. This is distinct from repeating the stored sentence. A correction such as “use the approved source of truth before reporting completion” must alter the decision procedure, not merely appear in an explanation.

3.5 Action

Where the task permits action, the executed step must agree with the decision and correction. Tool errors, stale state, authority limits and execution bugs can break the chain after a correct plan. In high-consequence tests, a sandbox or reversible fixture should replace live operations.

3.6 Outcome

The evaluator inspects the authoritative result or a suitable independent observation surface. This distinguishes attempted compliance from effective compliance. Outcome evaluation should also detect harmful side effects, not just the desired state.

A transfer claim should identify the highest verified stage. “Retained” is weaker than “retrieved”; “retrieved” is weaker than “applied”; and “applied” is weaker than “outcome verified.” This vocabulary prevents one green signal from silently standing in for the pipeline.

4. Evaluation design

4.1 Correction classes

A useful suite should sample at least five classes:

1. Factual revision: an old fact is replaced by a newer fact.
2. Preference: a stable presentation or interaction choice should influence an applicable later task.
3. Procedure: a required sequence or verification step should be followed.
4. Boundary: an action is disallowed or requires approval.
5. Ephemeral instruction: the item should remain local or expire, testing whether the system avoids inappropriate persistence.

These classes should not be collapsed into one aggregate score. A system may be adequate at recalling preferences and unsafe at handling revoked permissions.

4.2 Fresh-session isolation

The transfer task should begin in a genuinely fresh session without the correction in visible conversational context. The evaluator should record the memory snapshot or store version before the task, the retrieval trace where available, model and system versions, tool fixture state, and test prompt. If the system’s interface does not expose retrieval traces, the limitation should be stated rather than inferred away.

Isolation must also control leakage. The task wording should not reveal the expected correction, and evaluators should check whether the answer can be produced from defaults or model knowledge. A matched baseline, run without the stored correction, helps determine whether behaviour changed because of memory rather than chance.

4.3 Positive, negative and contrastive cases

For each correction, construct a small family of cases:

• Positive applicable case: the correction should govern the task.
• Negative scope case: a similar task falls outside its user, project or role scope.
• Supersession case: a newer correction replaces an older one.
• Conflict case: two records cannot safely be reconciled without clarification.
• Expiry case: the correction is no longer active.
• Abstention case: evidence or authority is insufficient, so the correct behaviour is to pause or ask rather than act.

LongMemEval’s inclusion of knowledge-update and abstention tasks supports the importance of evaluating more than simple extraction [1]. The cases above extend that logic to operational behaviour; they are proposed additions, not claims about that benchmark.

4.4 Observable records

A test record should contain:

• correction identifier and exact semantic expectation;
• permitted persistence class and scope;
• source/provenance reference;
• active, superseded, expired or disputed lifecycle state;
• test prompt and fresh-session identifier;
• expected retrieval decision;
• expected action and prohibited alternatives;
• authoritative outcome check;
• actual stage results;
• uncertainty, side effects and reviewer decision.

Sensitive source text need not be copied into the report. A controlled reference and a redacted semantic expectation can preserve auditability while reducing disclosure.

5. Metrics that preserve the failure boundary

A single “memory accuracy” number is attractive and frequently unhelpful. The protocol uses a scorecard:

• Retention fidelity: did the durable representation preserve the correction, scope, provenance and lifecycle state?
• Retrieval recall: among applicable cases, how often was the governing correction retrieved?
• Retrieval precision: among retrieved items, how often were they applicable?
• Supersession accuracy: did the system prefer the active correction over stale versions?
• Abstention accuracy: did it avoid unsupported or unauthorized action?
• Decision compliance: did the planned choice implement the correction?
• Action compliance: did the executed action match the compliant plan?
• Outcome success: did the authoritative state satisfy the expected result without prohibited side effects?
• Unsupported-memory rate: how often did the system claim a correction or provenance that was not present?

Report numerators and denominators by correction class, not only macro averages. Rare but severe boundary failures should be listed separately. A team may choose consequence-weighted thresholds, but those thresholds are deployment decisions and should not be disguised as universal scientific constants.

The strict end-to-end metric is verified transfer success: all required stages for that case pass. Stage metrics remain indispensable because the same end failure can arise from absent storage, missed retrieval, poor relevance filtering, incorrect reasoning, failed execution or faulty outcome observation.

6. Interpreting results

Four patterns are especially informative.

High retrieval, low decision compliance indicates that memory reaches the model but does not reliably govern reasoning. Prompt design, conflict resolution or decision controls may be the next intervention.

High decision compliance, low action compliance points away from memory and toward the execution layer: tool binding, stale state, permissions or error handling.

High positive-case success, low negative-case accuracy suggests over-application. The system “remembers” but lacks sufficient scope control—a particularly dangerous failure for permissions and personal data.

High outcome success with weak trace evidence may still be operationally acceptable in low-consequence cases, but it does not support a strong causal claim about memory. The baseline and contrastive tests become critical.

NIST’s AI Risk Management Framework treats AI as socio-technical and places measurement within a broader cycle of governance, contextual mapping and risk management [4]. Correction-transfer evaluation should follow that posture: tests must reflect the actual deployment context, affected people, consequence level and human oversight, rather than treating benchmark performance as a complete trustworthiness judgment.

7. Alternatives and related approaches

A long-context prompt can keep history immediately available, avoiding a separate retrieval step. It does not remove the need to test relevance, updates, action and outcomes, and it may create different efficiency and information-exposure trade-offs. Long-term memory benchmarks provide repeatable comparative tasks; their advantage is controlled measurement across systems. Production replay provides realism but creates privacy, reproducibility and safety concerns. Human review can catch nuanced failures but is expensive and variable.

The proposed protocol is therefore complementary. Use established benchmarks to compare general conversational-memory abilities; use architecture-specific retrieval diagnostics to inspect components; and use fresh-session correction-transfer cases to test the behaviours that matter in a particular operating context.

8. Practical application

The accompanying Fresh-Session Correction-Transfer Protocol supplies a case template, state model, test matrix and release checklist. A minimal pilot can use synthetic, non-sensitive corrections in a sandbox. The design should be pre-registered before results are inspected: define expected applicability, action, authoritative surface and prohibited behaviour first. Otherwise, fluent outputs make post-hoc success criteria remarkably cooperative.

A deployment team can use the protocol for regression testing after changes to embedding models, retrieval thresholds, summarisation, memory compaction, prompts, tool routing or lifecycle rules. Failures should be repaired at the stage indicated by evidence and rerun against both the failing case and its contrastive controls.

9. Limitations

This paper proposes an evaluation method; it does not report comparative experiments or establish that the protocol predicts long-term user trust or business performance. Fresh-session tests may not reproduce every production interaction. Retrieval traces can be unavailable or misleading. Outcome checks can share failure modes with the system under test. Human-authored expected actions may encode ambiguity or bias. Persistent-memory testing raises privacy and consent concerns, especially when real corrections concern sensitive attributes. The six-stage model is a diagnostic abstraction: some architectures merge stages, and some tasks require no external action.

Most importantly, reliable correction transfer is not always desirable. A malicious, mistaken or context-bound correction should not automatically become durable. Governance must decide what may be stored, who may revise it, how contradictions are handled, when records expire and how a person can inspect or delete them. Evaluation cannot substitute for those authority decisions.

10. Conclusion

The meaningful question for persistent AI memory is not “was the correction saved?” It is “did an eligible correction govern the right later task, in the right scope, through the right action, with a verified result?” Existing research provides important benchmarks and architectures for long-horizon recall and reasoning. Operational assurance requires an additional bridge from memory components to observable behaviour.

By separating retention, retrieval, relevance, decision, action and outcome—and by testing positive, negative, superseded, expired and conflict cases—the proposed protocol turns “the system learned” from a narrative into a falsifiable claim. It also makes failures easier to repair. Memory deserves credit only for the stage the evidence actually reaches.

References

[1] D. Wu, H. Wang, W. Yu, Y. Zhang, K.-W. Chang and D. Yu, “LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory,” ICLR 2025, 2025. https://arxiv.org/abs/2410.10813

[2] C. Packer, S. Wooders, K. Lin, V. Fang, S. G. Patil, I. Stoica and J. E. Gonzalez, “MemGPT: Towards LLMs as Operating Systems,” arXiv:2310.08560v2, 2024. https://arxiv.org/abs/2310.08560

[3] A. Maharana, D.-H. Lee, S. Tulyakov, M. Bansal, F. Barbieri and Y. Fang, “Evaluating Very Long-Term Conversational Memory of LLM Agents,” Proceedings of ACL 2024, 2024. https://arxiv.org/abs/2402.17753

[4] National Institute of Standards and Technology, Artificial Intelligence Risk Management Framework (AI RMF 1.0), NIST AI 100-1, January 2023. https://doi.org/10.6028/NIST.AI.100-1