Stanislav Tsepa

Posted on Feb 24

Memory isn’t magic: 3 types of AI 'memory' (and when to use each)

#ai #llm #agents #programming

If you’ve ever said “we should add memory to the app,” you’re not alone.

It’s also the fastest way to start a week-long argument, because people use the word memory to mean totally different things.

In practice, “memory” in AI products usually breaks down into three distinct systems:

1) Context (short-term memory) — what’s in the current prompt/window
2) Long‑term memory (facts) — durable information you can look up later
3) Procedural memory (rules + habits) — the system that prevents repeating mistakes

This post is a beginner-friendly map of those three types, with small code snippets and a visual you can copy.

If you like this kind of build log / agent engineering, I post updates here: https://t.me/the_prompt_and_the_code

1) Context: short-term “memory” inside the prompt

What it is: The text (and other inputs) you send to the model right now.

What it’s good for:

continuing a conversation
keeping a consistent writing style
doing multi-step tasks (“first do X, then Y”)

What it’s not good for:

remembering things forever
storing preferences safely
being accurate at large scale (context gets expensive and messy)

Tiny example (Python)

messages = [
  {"role": "system", "content": "You are a helpful assistant."},
  {"role": "user", "content": "Summarize this article in 3 bullets."},
  {"role": "user", "content": article_text},
]

If it’s not in messages, the model doesn’t know it.

Common failure mode: “infinite context” thinking.

If you keep stuffing everything into the prompt, you get:

rising costs
slower responses
more contradictions

Rule of thumb: use context to do the work, not to store the world.

2) Long-term memory: durable facts you can retrieve

What it is: Information stored outside the model—files, a database, a vector store, etc.

What it’s good for:

user preferences (“use shorter replies”, “use metric units”)
project state (“what repo are we working in?”)
anything you want to survive restarts

Minimal pattern: store → retrieve → insert

Here’s a tiny “facts store” example using a JSON file (so you can see the pattern without extra infra):

import json
from pathlib import Path

MEM = Path("memory.json")

def load_mem():
    return json.loads(MEM.read_text()) if MEM.exists() else {}


def save_mem(mem):
    MEM.write_text(json.dumps(mem, indent=2))


def remember(key, value):
    mem = load_mem()
    mem[key] = value
    save_mem(mem)


def recall(key, default=None):
    return load_mem().get(key, default)

Then at runtime:

tone = recall("preferred_tone", "friendly")
prompt = f"Write in a {tone} tone."

Key point: long-term memory is not “the model remembering.” It’s your app doing retrieval.

The safety angle

Long-term memory can leak. If you store secrets, they can end up back in a prompt.

A useful mental model from security research: treat external instructions/files as potentially unsafe unless you have checks.

3) Procedural memory: the checklist that makes the system reliable

What it is: The habits your system follows every time.

This is the most underrated kind of “memory.” It’s not knowledge—it’s behavior.

Examples:

“Always run tests before merging.”
“When posting content, dedupe and respect cooldowns.”
“When a tool call fails, retry with backoff.”

Why procedural memory matters

Modern agent systems are vulnerable to “instruction supply chain” problems—where a tool/skill/integration includes unsafe directives.

One response is a procedural audit step: a small set of rules that run at execution time.

Here’s a toy example: a “tool allowlist” gate.

ALLOWED_TOOLS = {"read_file", "write_file", "run_tests"}


def call_tool(name, **kwargs):
    if name not in ALLOWED_TOOLS:
        raise RuntimeError(f"Tool not allowed: {name}")
    return TOOLS[name](**kwargs)

That’s not “memory” in the human sense—but it’s exactly how you keep an AI workflow from drifting into chaos.

Putting it together: a simple architecture

A good default architecture is:

Context: current conversation/task
Long-term facts: small, durable, queryable store
Procedural rules: guardrails + habits + automation

(See the diagram below.)

Quick cheat sheet

If you want the model to “remember what we just said” → Context
If you want it to remember next week → Long-term memory
If you want it to stop making the same mistake → Procedural memory

Sources / further reading

Wibault et al., Recurrent Structural Policy Gradient for Partially Observable Mean Field Games (2026). (Good for “history-aware policies” as a concrete concept.) arXiv:2602.20141
“SKILL-INJECT: Measuring Agent Vulnerability to Skill File Attacks” (2026). (Great overview of instruction/skill supply-chain risks.) arXiv:2602.20156
Helen Nissenbaum, Privacy in Context (Contextual Integrity) — helpful mental model for why “safe vs unsafe” depends on situation.

DEV Community