A support agent collects an order number, checks the shipment, answers a question, and continues the conversation. Several messages later, it asks for the order number again.
That repetition often looks like forgetfulness. A common explanation is that the agent “knew” the number and then lost it. The more accurate explanation is simpler: many software agents do not maintain a continuously updated internal memory. They choose each next action from the state supplied for the current turn.
A typical sequence looks like this:
User message
↓
Current conversation state
↓
Agent chooses next step
↓
Tool call: look up order 8472
↓
Tool result: shipped, arriving Friday
↓
Response to user
On a later turn, the agent can reuse 8472 only if that value remains available. It might appear in the recent conversation, in a compact summary, or in an external record maintained by the application. If none of those contains it, the next decision begins without the number.
The crucial misconception is that an agent automatically remembers everything it has previously processed. Processing a value once does not make that value durable. A tool result, for example, is usually an observation delivered during one step. Unless the surrounding system preserves the observation, the agent cannot reliably consult it later.
This distinction becomes visible when conversations grow. Applications often limit how much earlier text is supplied on every turn. Older messages may be omitted or compressed into a summary such as:
The customer asked about a delayed shipment. Delivery status was checked.
That summary preserves the topic but drops the identifier required for another lookup. Asking for the order number again is then a reasonable next step, even though the full transcript shows that the user already provided it.
Reliable agents therefore separate transient context from durable state. Transient context helps interpret the immediate exchange: what the user just said, what a tool just returned, and which action is underway. Durable state stores facts that later steps must recover, such as an order ID, an approved refund amount, or the identifier of an unfinished task.
Durable state also needs rules. Saving every sentence indefinitely creates privacy, accuracy, and cleanup problems. A useful state record is usually explicit and narrow:
case_id: C-193
order_id: 8472
delivery_status: shipped
status_checked_at: 2026-11-18T10:42:00-08:00
The timestamp matters because remembered information can become stale. Reusing an order number may be safe; reusing yesterday’s delivery status without another tool call may not be. Good next-step selection depends on both availability and freshness.
When an agent repeats a question, the right diagnostic is not merely “Why did it forget?” Better questions are: Was the value included in the current turn? Was it saved as structured state? Did summarization remove it? Was the stored value considered too old or too uncertain to trust?
An agent’s continuity comes from the state surrounding each decision. If an order number must survive across turns, hoping that it remains somewhere in the conversation is fragile. Store it deliberately, attach its scope and freshness, and make it available when the next action is chosen.
Top comments (0)