Your Agent Remembers the Old Fact. Can It Use the Update?

A project changes its deployment target. A customer changes their delivery address. A teammate stops owning a service.

The agent has seen the update. Weeks later, it confidently acts on the previous information.

That failure is especially difficult to catch because the answer can look perfectly reasonable. It may contain a real address, a valid command, or the name of someone who genuinely owned the service. The mistake becomes visible only when you check whether that information still applies.

With AML Cycle 2 results approaching, this is one of the questions we want developers to bring to the upcoming leaderboards. How well do memory systems handle a history that keeps changing?

A November update puts changing facts in focus

On November 3, Salesforce published an update on Agentforce Memory. The company described an evaluation using long conversations derived from ConvoMem and run through its live Agentforce system.

Salesforce reports that an updated memory policy recalled 1.8 times as many explicitly stated facts and 2.2 times as many facts that changed over time as the previous policy. According to the company, the underlying architecture stayed the same. One policy change was to treat updated information as a correction to an existing fact, rather than storing it alongside the earlier statement as another independent fact.

These are Salesforce’s results under its own evaluation setup. They make a practical point for developers, though. The rules governing what a system writes and updates can materially affect what the agent later retrieves and uses. Salesforce’s November 3 update

That raises a useful engineering question. When an agent uses an old fact, which part of the system allowed it to remain authoritative?

“Newest wins” breaks surprisingly quickly

Consider a small sequence of project updates.

Ask the agent which database production uses on November 10.

Selecting the most recently mentioned database would produce the wrong answer. Selecting the most recent statement alone also leaves work to do, because the postponement refers to an earlier plan.

The system needs to connect the statements, distinguish a plan from an active configuration, and recover the state that applies to the question.

Now ask a different question.

“Which database were we planning to use after the migration?”

The earlier MySQL statement becomes useful evidence again.

This example suggests that a memory record needs more than a timestamp. Depending on the application, it may need an entity, a scope, an effective date, a source, and a relationship to the record it updates.

Those details help answer different questions about the same history without flattening everything into a single latest value.

Build an evaluation that can locate the failure

A wrong final answer gives you a place to start investigating. To decide what to change, inspect the path from the update to the action.

Suppose the agent recommends a PostgreSQL-specific command after a completed migration to MySQL.

First, check whether the migration update reached persistent memory. A summarizer may have dropped it, or a write policy may have treated it as temporary context.

Next, inspect the retrieved evidence. Did the system return the migration record? Did it return both versions without showing their relationship? Did an older, frequently repeated configuration outrank a short correction?

Then inspect the answer step. The correct update may have been present, while the model still followed the older command because it was more detailed or appeared earlier in the context.

Each case calls for a different intervention. More retrieval results might help when evidence is missing. They can make the problem worse when conflicting records are already overwhelming the answer context.

For a development evaluation, preserve the input history, stored records, retrieved evidence, and final output for each case. A useful record is one that lets another engineer trace the wrong recommendation back to the first step where the necessary information was lost or misinterpreted.

Evaluate the neighboring questions too

A system can learn to answer “What is the current database?” while still mishandling the surrounding history.

After a changed fact, add several related questions to the evaluation.

Ask for the current state. Ask what was true before the change. Ask when the change took effect. Ask why the earlier configuration was replaced. Finally, give the agent a task that requires using the current state.

These questions expose different failure modes. A system that overwrites aggressively may answer the current-state question correctly while losing the history needed to explain an earlier incident. A system that keeps everything may preserve the evidence while leaving the answer model to resolve every conflict.

For coding agents, this distinction matters whenever an old solution remains searchable. A previously successful patch can be valuable historical evidence while being unsuitable for the current version of the project.

The evaluation should make those two conditions distinguishable.

Include temporary and scoped updates

Explicit corrections are a manageable starting point. Real conversations contain less tidy changes.

“I’m working from the London office this week.”

“Use the staging account for this investigation.”

“For this customer, send the report to the regional lead.”

Each statement has a boundary. Applying it outside that boundary can create a persistent error.

An evaluation can vary one condition at a time. Keep the person and task fixed, but move the date beyond the stated period. Keep the date fixed, but switch customers. Keep the customer fixed, but move from staging to production.

This helps reveal whether the system retained the conditions attached to a fact.

Some preferences also expire without an explicit correction. In those cases, a product needs a policy for uncertainty. The agent might ask for confirmation before taking a consequential action. Whether that behavior is appropriate depends on the task, so the evaluation should specify when confirmation is expected and when the available evidence is sufficient.

What we’re watching as AML Cycle 2 approaches

Agent Memory Leaderboard evaluates memory systems through a shared protocol. Participants provide Add and Search interfaces, while the platform handles downstream Answer and Eval. Cycle 2 covers Textual, Coding, and Multimodal Memory, with separate rankings by track and participant division.

The published scope includes knowledge updates and conflict handling in textual memory, historical engineering experience under noise in coding memory, and text–image evidence in multimodal memory. Official Cycle 2 results are expected in mid-November. AML Cycle 2 evaluation scope and schedule

As those results arrive, we’ll be looking at how performance differs across the published tasks and capability dimensions. A strong aggregate score is a starting point for examining where a system handles changing history well and where further investigation is needed.

The November Agentforce update also gives developers a reason to examine their own write policies before making larger architectural changes. A memory system can have the necessary storage and search components while still treating corrections, plans, and temporary instructions too similarly.

Before your next evaluation run, take one ordinary fact from your application and change it halfway through the history. Add a future plan, postpone it, and ask questions about both the current state and the past.

Then give the agent a task that depends on getting the distinction right.

Which record does it use, and can you explain why?