DEV Community: Wojtek Pluta

The Agent Loop Decoded: Three Levels Every Agent Engineer Must Know

Wojtek Pluta — Fri, 19 Jun 2026 09:13:00 +0000

This article is syndicated from the original post on blogs.oracle.com. Read the canonical version there for the latest updates.

Chances are you have already run an agent loop today without naming it.

Every session with a coding companion such as Claude Code, Codex, or Cursor is one: the model reads a request, inspects the repository, edits a file, runs the tests, observes the failures, and edits again until the build passes.

That cycle of reasoning, acting, and observing the result is the agent loop at work, and it now sits at the centre of nearly every production agent system. The agent loop is the repeating cycle a harness runs within a single agent turn: assemble context, invoke the model to reason, act on its decision, and go again until a stop condition ends the run.

This piece unpacks that loop across three levels of understanding.

Level 1 is the minimal loop most developers meet first: an LLM, a handful of tools, and a response.
Level 2 introduces a lifecycle inside the loop, where memory operations turn a stateless process into a reasoning engine with state.
Level 3 pushes operations both inside and outside the loop, where the agent harness becomes a system in its own right.

By the end, you will know which level your system sits at, what breaks when the level and the task are mismatched, and what engineering work moves you up. Every pattern discussed is implemented in the companion notebook, built on Oracle AI Database, so you can run the loop rather than just read about it.

What is an Agent

Figure 1: An agent perceives its environment, reasons with an LLM, acts, and remembers

An agent is a computational system that perceives its environment, reasons about what it perceives, takes actions to achieve a goal, and has some form of memory. That description applies to many things: a thermostat, a chess engine, a human professional. What makes an AI agent distinct is that the reasoning step is handled by a large language model, and the range of possible actions extends well beyond a binary output.

An agent’s architecture consists of two separable layers. The first is the model: the inference engine that does the reasoning. The second is the harness: the code that prepares context, executes tool calls, enforces operational constraints, and persists state. Most agent engineering work happens in the harness, not the model. Understanding that boundary clarifies where failures originate and where interventions are effective.

Figure 2: The two layers of an agent’s architecture: the model and the harness

An agent needs at minimum four things to be useful:

Instructions: a system prompt or goal that tells it what it is trying to accomplish.
Memory: access to information beyond the current message, including prior context, retrieved knowledge, and learned patterns.
The ability to take actions: tool calls, API requests, database writes, or any operation with an external effect.
A reasoning engine: an LLM that looks at context and decides what to do next.

What Is a Loop?

A loop is a control structure that repeats a block of execution until a condition is met. In programming you encounter this everywhere: iterating over a collection, running until a flag is set, calling recursively until a base case is reached.

The agent loop applies that same structure to an LLM-powered system. Rather than processing a user message once and returning a static response, the agent feeds its output back into itself, reasoning, acting, observing the result, and reasoning again, until it determines the task is complete.

Figure 3: The agent loop: assemble context, reason, act, and repeat until a stop condition ends the run

The necessity for loops in agent execution can be derived from the nature of the use cases and tasks agents are applied to. These common use cases can be referred to as application modes: the expected interaction patterns between a user and an agent. There are three:

Assistant
Deep Research
Coding

Take the deep research mode. An agent tasked with finding relevant sources, identifying contradictions across them, and producing a structured summary is not running a single-shot task. It requires the agent to:

Search for relevant sources.
Read and evaluate what it finds.
Identify gaps and contradictions.
Search again to fill in those gaps.
Synthesise everything into a coherent output.

Figure 4: The deep research cycle: search, evaluate, identify gaps, and search again until coverage is sufficient

No single LLM call can do all of that. What is required is the mechanism and scaffolding that allows the model to reason, act, observe the result, reason again, and continue until the task is complete. That mechanism is the agent loop.

Notably, implementations of agent frameworks and harnesses, however opinionated, have shared one thing in common: convergence on a minimal agent loop design. That convergence is arguably not much of a design choice, so much as a logical consequence of the task itself.

The agent loop exists because long-horizon tasks cannot be completed in a single forward pass.

The loop emerging as a design pattern draws a parallel to how humans operate in most organisations: structured cycles of work, review, and feedback that repeat until the objective is met.

Stop Conditions

Loops have to be exited eventually. The programmatic loops taught in computer science classes usually exit in one of two ways: the iteration count for the loop is reached, or a break statement inside the loop triggers an exit.

A well-designed agent loop defines explicit exit criteria. Common examples:

The model produces a final response with no pending tool calls.
A goal-completion check returns true: an objective-specific predicate, not merely the absence of tool calls.
A maximum number of iterations is reached.
A wall-clock timeout expires.
An error occurs that the agent cannot recover from.
The harness identifies a failure mode, such as the agent repeating the same action without progress.
The agent explicitly invokes an exit action or sets a completion flag.

In the notebook accompanying this article, the stop conditions are implemented directly inside the harness:

def call_agent(query, thread_id='1', max_iterations=10, max_execution_time_s=60.0):
    start_time = time.time()
    iteration = 0

    while iteration < max_iterations:
        if time.time() - start_time > max_execution_time_s:
            break  # Wall-clock timeout

        response = call_openai_chat(messages, tools)
        if not response.tool_calls:
            break  # Model produced a terminal message; exit the loop

        # Execute tools, append outputs, continue
        iteration += 1

    # Fallback if max iterations reached
    return 'Max iterations reached; please refine the request.'

The max iterations of the loop is set to 10 by default. This is a guard against the loop running indefinitely, which can incur high operational cost through the increase in token consumption across inference calls. There is also a max_execution_time_s parameter, which adds a temporal guard to the agent loop’s execution.

It is worth noting that a terminal message from the model, one with no further tool calls, ends the agent’s turn. It does not mean the user’s goal has been satisfied. The model may return a clarifying question, a partial result, or a response that requires follow-up. The agent harness is responsible for checking whether the goal is actually complete, not simply whether the model has stopped emitting tool calls. This distinction becomes more consequential as tasks grow in length and complexity, and it is where domain expertise becomes paramount in agent harness engineering.

Failure mode identification deserves its own mention as an exit path. A loop should break not only when work completes but when work stops progressing.

The clearest example is tool call repetition: the agent invokes the same tool with identical arguments for a third consecutive iteration, a strong signal that it is stuck rather than working. A well-instrumented harness keeps a window of recent tool calls, detects the repetition, and exits with a diagnostic instead of spending the remaining iterations on a stalled run. Oscillation between two states belongs to the same family of detectable failures.

Defining the Agent Loop

With the components and the exit criteria established, the definition can now be stated with precision:

The Agent Loop

A cyclical, iterative execution pattern inside a single agent run where the harness repeatedly:

Assembles execution context: system instructions, conversation state, retrieved memory, tool outputs, and any relevant external data.
Invokes a reasoning model to decide what to do next.
Acts: responds to the user, calls tools, writes memory or state, or updates its plan.

Each cycle appends its trace (assistant messages, tool outputs, state updates) to the context and repeats until a termination check ends the run. Context-window pressure and operational safety (timeouts, iteration caps, budget guards) are first-class concerns, not afterthoughts.

Three Levels of the Agent Loop

The agent loop is not a fixed pattern. The simple design presented above evolves as memory, tooling, and opinionated scaffolding are added. The three levels below provide a framework for where a system currently sits and what engineering work lies ahead. Most production failures (agents that repeat themselves, lose context, or produce inconsistent results across sessions) trace back to a mismatch between task complexity and agent level.

Figure 5: The three levels of the agent loop

Level 1: LLM + Tools + Response

At its simplest, the agent loop is an LLM that can call tools and return a response. There is no persistent memory, no external state, and no scaffolding beyond the loop itself. The loop iterates because tool results must be fed back to the model before it can produce a final answer.

The code below demonstrates the pattern most developers encounter when building simple tool-calling agents:

messages = [system_prompt, user_message]
while True:
    response = llm.chat(messages, tools=available_tools)
    if response.tool_calls:
        for call in response.tool_calls:
            result = execute_tool(call.name, call.args)
            messages.append(tool_result(result))
    else:
        return response.content  # Terminal message; exit

Figure 6: Level 1: the minimal tool-calling loop

LangChain’s ReAct agent provides this pattern out of the box. The agent receives an input query, selects a tool, calls it, observes the output, and reasons again, all within a single run:

from langchain.agents import AgentExecutor, create_react_agent
from langchain_openai import ChatOpenAI

llm = ChatOpenAI(model='gpt-4o')
agent = create_react_agent(llm, tools=[search_tool], prompt=prompt)
executor = AgentExecutor(agent=agent, tools=[search_tool], max_iterations=10)
executor.invoke({'input': 'What are the latest AI papers on agent memory?'})

Level 1 is where most developers start, and it is genuinely useful for self-contained tasks. Its limitation is structural: the agent has no recollection of previous conversations. Every run starts cold, the context window is the only memory it has, and it resets completely when the run ends. On any multi-turn or long-horizon task, it will repeat work it already did, lose track of decisions made earlier in the session, and produce output that contradicts its own prior responses.

Level 2: Lifecycle Inside the Loop

At Level 2, operations begin to appear inside the agent loop. Memory is read before the LLM is called, and memory is written after the agent acts. The loop now has a lifecycle. At Level 1, the loop can be seen as a transport mechanism for tool calls. At Level 2, the loop becomes a reasoning engine with state. This is also where the distinction between a memory-augmented agent and a memory-aware agent becomes consequential.

Memory-augmented agents retrieve and inject information into context. They read from memory, but they do not actively manage it. Memory is something that happens to them.
Memory-aware agents treat memory as a first-class engineering concern. They encode, store, retrieve, inject, and forget, actively managing their cognitive state within each run and across sessions. Level 2 is where you begin building memory-aware agents.

This distinction, and the engineering it implies, is the subject of the DeepLearning.AI short course Agent Memory: Building Memory-Aware Agents, built with Oracle, if you want the full overview.

Figure 7: Memory-augmented agents read from memory; memory-aware agents manage it

Level 2 makes context assembly trade-offs immediately visible. Adding more memory types (conversation history, retrieved documents, entity records, workflow patterns) improves grounding and action selection. On the other hand, it also introduces cost: more tokens, higher latency, and a greater risk of injecting irrelevant or stale content that misleads the model rather than informing it.

There are a few failure modes worth mentioning:

Noisy retrieval: semantically similar documents that are not actually relevant to the current query. Mitigation approaches are implemented via relevance thresholds and precision-oriented retrieval strategies such as hybrid search and pre-, post-, and in-filtering methods in retrieval pipelines.
Stale memory: data can quickly become irrelevant in a fast-paced problem domain: cached facts, entity records, or summaries that are no longer accurate. Mitigate with TTL policies and update-on-write patterns.
Tool schema overload: context bloat is a common problem, and it is most prevalent in tool-calling agents with too many tool definitions passed to the model at once, degrading tool selection accuracy. Mitigate with semantic tool retrieval rather than exhaustive enumeration; this is shown in the companion notebook for this piece.

There are more failure modes, and in production these are not edge cases. They are predictable failures that any Level 2 agent will encounter as memory stores grow. Designing mitigation strategies at the start is cheaper than retrofitting fixes later.

Memory operations are common in Level 2 agent loops, mainly because agents at this level are designed for continuity and adaptation. Memory operations are programmatic methods designed to modify data and information within the agent’s system boundary and across other system components such as databases and external stores.

Operation	When It Runs	Purpose
Read conversational memory	Before LLM call	Load prior chat history into context
Read knowledge base	Before LLM call	Inject relevant documents and facts
Read workflow memory	Before LLM call	Surface known action patterns
Read entity memory	Before LLM call	Resolve named references in the query
Write conversational memory	After user message received	Persist the user turn
Write knowledge base	After tool search	Store retrieved results for future runs
Write entity memory	After LLM response	Extract and persist people, places, systems
Write conversational memory	After final response	Persist the assistant turn

In the accompanying notebook, these operations are centralised in a MemoryManager class backed by Oracle AI Database. Before each run, the harness calls all read operations to assemble context. After each run, write operations persist the new information:

# Reads: all run BEFORE the tool-call loop
conv_mem = memory_manager.read_conversational_memory(thread_id)
knowledge = memory_manager.read_knowledge_base(query)
workflows = memory_manager.read_workflow(query)
entities = memory_manager.read_entity(query)
summaries = memory_manager.read_summary_context(thread_id)
context = build_context(
    conv_mem,
    knowledge,
    workflows,
    entities,
    summaries,
)

# Inner tool-call loop
response = run_tool_call_loop(context, tools)

# Writes: all run AFTER the loop exits
memory_manager.write_conversational_memory(thread_id, 'assistant', response)
memory_manager.write_entity(extract_entities(query, response))

The notebook uses six distinct memory types, each stored in Oracle AI Database and each serving a specific cognitive function:

Conversational memory: episodic chat history retrieved by thread ID via a standard SQL table. Exact lookup, no similarity search required.
Knowledge base memory: semantic memory backed by a vector-enabled SQL table with HNSW indexing for similarity search.
Workflow memory: procedural memory storing learned action patterns and tool sequences.
Toolbox memory: a vector-indexed registry of tool definitions enabling semantic discovery rather than exhaustive schema enumeration.
Entity memory: LLM-extracted people, places, and systems, persisted across sessions.
Summary memory: compressed context for long conversations, with just-in-time expansion when the agent needs the full content.

At Level 2, the loop is no longer just executing tools. It is actively managing its own cognitive state.

Level 3: Operations Inside and Outside the Loop

At this point, developers understand not only which operations they require inside the loop; more opinionated scaffolding and harness begin to form around the agent loop itself.

Operations now exist both within the loop and outside it, and there are deliberate architectural choices about which side of the boundary each operation belongs on. This is where agent engineering becomes opinionated, and where context engineering and memory engineering become distinct disciplines with separate concerns.

In a Level 3 agent loop, some operations should be automatic. The agent should never have to decide whether to load its own conversation history. Others should be agent-triggered: the agent decides when to search the web, not the harness.

Getting this boundary wrong produces either context bloat, when too much is loaded automatically, or missed context, when content that should always be present is left to the model’s discretion.

Operation	Programmatic	Agent Triggered	Why
Read conversational memory	Yes	No	The agent always needs its history
Read knowledge base	Yes	No	Relevant documents always loaded at run start
Read workflow base	Yes	No	Known patterns always surfaced before reasoning
Read entity memory	Yes	No	Named references always resolved upfront
Read summary context	Yes	No	Summary IDs always loaded; full content expanded on demand
Expand a summary	No	Yes	Agent decides when it needs the full content
Search the web (Tavily)	No	Yes	Agent decides when stored knowledge is insufficient
Summarise conversation	No	Yes	Agent decides when context needs compaction
Write tool log (offload)	Yes	No	Automatic after every tool call; keeps context lean

Context engineering at Level 3

Three techniques only become necessary at Level 3. Below Level 3, your context is manageable by construction. At Level 3, with memory reads, multiple tool calls, and iterated reasoning, it is not.

Context window monitoring: tracking token usage across iterations to detect when compaction is needed before the window fills and performance degrades.
Conversation compaction: replacing verbose chat history with compressed summaries while preserving originals in the database. The notebook marks messages with a summary_id rather than deleting them, keeping the full record available for audit and on-demand expansion.
Tool output offloading: persisting full tool outputs to a tool log table and replacing them in context with a compact one-line reference.

The tool log pattern is worth examining in detail. A single web search can return three to four thousand tokens of raw results. Without offloading, every subsequent iteration in the same run carries those tokens. With offloading, the context receives only a reference:

def execute_tool(tool_name, tool_args, thread_id):
    raw_output = run_tool(tool_name, tool_args)

    # Full output persisted to the database
    log_id = memory_manager.write_tool_log(
        thread_id=thread_id,
        tool_name=tool_name,
        tool_output=raw_output,
    )

    # Context receives only the compact reference
    return f'[Tool Log ID: {log_id}] Results stored. Call read_tool_log to retrieve.'

Semantic tool discovery

At Level 3, the number of available tools is unlikely to stay small. Passing every tool schema to the model on every iteration is a known failure mode: tool selection accuracy drops as the schema list grows, and token costs climb regardless of how many tools are actually relevant.

The notebook addresses this with a Toolbox: a vector-indexed registry of tool definitions where only semantically relevant tools are retrieved and passed to the model for each query. Tools are registered with LLM-augmented metadata so that embeddings capture intent and use case, not just function signatures:

@toolbox.register_tool(augment=True)  # LLM enriches description for retrieval
def search_tavily(query: str, max_results: int = 5):
    """Search the web and persist results in the knowledge base."""
    ...

# At runtime: only semantically relevant tools passed to the model
relevant_tools = memory_manager.read_toolbox(current_query)

Idempotency and tool reliability

Tool call failures are a production reality. Network errors, rate limits, and transient service issues occur regularly. If the harness retries a failed tool call naively, it risks executing a side-effecting operation twice: writing a record, sending a message, or triggering a payment more than once.

The mitigation is idempotency: assigning each tool call a stable key before execution so that retries can be safely distinguished from duplicate calls. This is harness-level engineering, not model-level reasoning, and it belongs in the Level 3 design.

Prompt caching and message ordering

At Level 3, the harness also starts to affect inference economics through prompt caching. Most LLM providers implement prefix-based caching: if the beginning of a prompt is identical to a recent request, the cached computation can be reused, reducing latency and cost.

The implication for agent design is concrete. Rewriting earlier messages mid-conversation, to clean up history, reorder context, or inject new system instructions inline, breaks prefix stability and degrades cache hit rates. The correct pattern is to append new instructions rather than modifying existing message history. The Codex implementation established this explicitly: old prompts are preserved as exact prefixes of new prompts specifically to maintain caching benefits across long multi-step runs.

Level 3 is where the agent harness becomes a system in its own right. The inner loop, assembling context, invoking the model, and acting, has not changed. What has changed is everything around it: the scaffolding that feeds it, the operational constraints that govern it, and the persistence layer that gives it continuity across time and sessions.

Other Loops the Agent Engineer Should Know

The agent loop does not run in isolation. It sits inside a wider system of loops, and the engineering decisions made inside the agent loop are shaped by what happens in the loops around it.

Three matter most to agent engineers and memory engineers: the training loop that produced the model, the feedback loop that signals whether the system is working, and the human loop that bounds its authority.

Figure 8: The loops interconnected: the training loop produces the model, the agent loop generates experience, and the memory layer routes that experience back as training signal

The training loop

The training loop is the cycle that produced the model in the first place: data collection, gradient updates, evaluation, and release. It operates offline, at a timescale of days or weeks, on curated datasets. The agent loop operates online, in real time, on live interactions.

Today these two loops are largely decoupled. Training happens, weights are frozen, and the agent loop runs on top of those fixed weights. The apparent learning you observe within a session, an agent recalling prior context or adapting to corrections, is not weight updating. It is retrieval. The agent is not learning; it is reading from memory.

This separation defines the boundary of what the agent loop can and cannot accomplish on its own. It can accumulate experience through memory operations. It cannot change the underlying model without a training cycle. Understanding this boundary tells you which problems belong to memory engineering and which require retraining.

The feedback loop

Every action the agent takes produces feedback. Tool results are feedback. User corrections are feedback. Evaluation metrics (hallucination rate, task completion, citation accuracy) are feedback at a system level.

At Level 3, the agent harness begins to make the feedback loop explicit and instrumentable. The notebook’s context window growth chart is a primitive example: watching whether token counts stabilize across runs tells you whether your context engineering is actually working. More sophisticated systems route evaluation signals back into memory stores, marking retrieved content as reliable or unreliable based on downstream outcomes, and gradually improving retrieval quality without retraining.

The feedback loop is what turns an agent into a system that improves over time. Without it, every invocation starts from the same baseline regardless of what the agent has done before.

Human in the loop

Long-horizon tasks regularly reach decision points where the agent lacks the information, authority, or confidence to proceed without human input. The human-in-the-loop pattern introduces a pause condition: the agent surfaces a question or proposed action, waits for review or correction, and then continues.

This is a stop condition of a different kind. Rather than halting because the task is finished, the loop pauses because it has reached the boundary of its autonomous authority. Designing this well involves two things: knowing in advance where those boundaries should sit for a given workflow, and ensuring the agent communicates specifically when it reaches one. A generic request for help is insufficient. The agent must surface a precise description of what information or decision is blocking progress.

Human-in-the-loop is not a safety net for when the agent fails. It is a deliberate architectural decision about where human judgment adds the most value in a system. The agent loop handles what can be reasoned about autonomously. The human loop handles what requires authority, context, or accountability that the agent does not have.

Where This Is Going

The agent loop, the training loop, and the feedback loop are currently operated as separate engineering concerns. That separation is practical, not fundamental. As agents accumulate experience across millions of runs, the information they generate (episodic memories, entity graphs, workflow patterns, evaluation signals, context growth traces) becomes a training signal. The training loop will eventually consume the output of the agent loop, closing the circle.

When that happens, the quality of the memory layer becomes the quality of the training data. Agents with well-engineered memory (clean episodic records, accurately extracted entities, reliable retrieval signals) produce better training signals than agents that let context accumulate without structure.

This convergence has a name. Continual learning is the ability of a model to acquire new knowledge and capabilities from a stream of incoming data over time, without retraining from scratch and without catastrophically forgetting what it has already learned. It is a formal machine learning discipline, not a metaphor, and it is the bridge between the two loops: the agent loop generates the experience, and continual learning is the process by which the training loop absorbs that experience into model weights.

Continual learning in agentic systems is the capacity of an agent to improve over time through the accumulation of high-signal memory units, with the extracted signal applied across three optimization surfaces: token space, weight space, and latent space.

The Union of the Agent Loop and the Training Loop

What connects them is the memory layer.

Oracle AI Database serves as the agent memory core, providing vector search, relational storage, and graph capabilities in a single engine. Memory operations that run inside the agent loop (encoding, storing, retrieving, injecting, and forgetting) produce a durable record of agent experience.

Oracle OCI provides the platform for continuous learning: the infrastructure to retrain models on that accumulated experience at scale, closing the loop from runtime behaviour back into model weights.

The agent loop and the training loop are converging. The memory layer is where they meet.

For engineers building agents today, this means the decisions made about memory architecture are not just operational decisions. They are decisions about what the system will be able to learn from tomorrow. A database that can serve low-latency semantic search at runtime can also serve as the data source for a continuous training pipeline.

Design your memory layer accordingly.

FAQ

What is the agent loop?

The agent loop is the repeating cycle a harness runs within a single agent turn: assemble context, invoke the model to reason, act on its decision, and repeat until a stop condition ends the run. It exists because long-horizon tasks cannot be completed in a single LLM call.

How do you stop an agent loop from running forever?

Define explicit stop conditions in the harness: a terminal message with no pending tool calls, a goal-completion check, an iteration cap, a wall-clock timeout, unrecoverable errors, and failure mode detection such as the agent repeating the same tool call with identical arguments.

What is the difference between a memory-augmented agent and a memory-aware agent?

A memory-augmented agent retrieves and injects information into context but does not manage it; memory is something that happens to the agent. A memory-aware agent encodes, stores, retrieves, injects, and forgets, actively managing its cognitive state within each run and across sessions.

How do I know which level my agent system sits at?

If there is no persistence beyond the context window, it is Level 1. If memory is read before the model call and written after the agent acts, it is Level 2. If there is a deliberate boundary between programmatic and agent-triggered operations, with techniques such as compaction, tool output offloading, and semantic tool discovery, it is Level 3.

What connects the agent loop to the training loop?

The memory layer. Agent runs generate experience: episodic records, entities, workflows, and evaluation signals. With continual learning, that experience becomes training signal. Oracle AI Database stores and serves it inside the agent loop; Oracle OCI provides the platform to retrain models on it. The patterns are implemented in the companion notebook.

From RAG to Memory Systems: Building Stateful AI Architecture

Wojtek Pluta — Fri, 19 Jun 2026 09:12:15 +0000

This article is syndicated from the original post on blogs.oracle.com. Read the canonical version there for the latest updates.

Why retrieval is not memory, and what changes when you build for continuity instead of lookup.

Key Takeaways

RAG is retrieval, not memory. It helps AI find information, but it cannot reliably remember user preferences, past work, or ongoing conversations.
Memory systems add continuity. They store and reuse facts, preferences, policies, and summaries so AI can continue across sessions and interactions.
Different memories need different treatment. Policies, preferences, facts, episodes, and traces should each have their own storage rules, retrieval methods, and lifecycles.
The memory manager is the core component. It controls what gets stored, what gets retrieved, and how prompts are rebuilt each turn while maintaining governance, privacy, and efficiency.

The pattern most teams ship as RAG is roughly four lines of code: embed your documents, embed each user query, pull the top-k nearest neighbors, stuff them into the prompt. It works. It works well enough that it's the default for almost every internal knowledge assistant shipped in the last two years. It also explains why those assistants all feel the same after the first interesting conversation. They forget and they repeat themselves. They cannot learn from what they were just told, and they cannot carry on a meaningful series of conversations.

RAG helps a model look things up. A memory system helps an application remember and continue across turns. The goal is not to keep packing more tokens into prompts. The goal is to build a reusable memory layer that ingests once, distills what's useful, and retrieves the right slice when it's needed. Most teams don't have agent memory, they have retrieval plus prompt inflation.

Try the companion demo: explore the working RAG-to-memory implementation, schemas, and runnable code in the Oracle AI Developer Hub repo.

Check out the walkthrough where we show what has to sit around retrieval to make AI agents actually stateful: typed memory, scoped records, promotion gates, fallback retrieval, prompt assembly, and traces.

How to evolve RAG into a memory system

The short version: keep your retrieval pipeline as one input to a larger loop, and add these five things around it:

Type your memory. Separate policy, preference, fact, episodic, and trace memory into distinct schemas with distinct lifecycles.
Scope every record. Add a tenant_id, user_id, and agent_id, then enforce scope as a hard predicate before ranking, never after.
Put a promotion gate between observations and durable writes. This prevents the store from poisoning itself with everything the model said.
Reassemble the prompt on EVERY turn. Use typed memory rather than accumulating context across turns.
Instrument the loop. Add a trace envelope so you can replay, audit, and improve it.

RAG vs Memory comparison

The rest of this guide walks through each step, the schemas behind them, the queries, the order you should build things in, and the tradeoffs that decide whether it survives production.

Why basic RAG stops short of stateful AI

Pure retrieval architectures fail in five places, usually in the same order.

Multi-turn continuity. The system has no memory of what was just said unless you re-stuff the transcript every turn. Stuff it and you pay tokens and lose model quality to context rot. Skip it and the agent forgets what the user asked three messages ago. Retrieval alone won't fix this.

Resumability. A user closes the tab on Friday and reopens it on Monday. The agent should pick up where they left off. With RAG alone, the only thing that survives is the document corpus. Every preference the user expressed and every partial workflow they were in the middle of is gone.

Long conversations. After a few hundred turns you can't pack the transcript anymore. You need summarization and structured extraction. None of that is retrieval.

User preferences and policy recall. "This user wants JSON." "This tenant formats dates DD/MM/YYYY." "Refunds over $500 require approval." You don't get those by semantic similarity to a user's last message. They're durable rules and parameters that need exact lookup with stable visibility on every turn, scoped to the right user or tenant.

Prompt growth. The default response to all of the above is to throw more into the prompt. More retrieval, more history, more context. The token bill compounds, the model gets lost-in-the-middle, and the system feels slower at exactly the moment it should feel more competent.

If all you need is semantic lookup over documents, basic RAG is enough. If you need continuity and governed recall, you now have a memory system design problem.

What changes when retrieval becomes memory

The shift here isn't bolting a database next to the vector store. What changes is what the storage layer is for and how your agents interact with it.

Retrieval is a query against a corpus you ingested once. The corpus is upstream and the query is downstream. Nothing the model says or does flows back into the corpus.

Memory is a write path. Anything the system observes during a run, or that a user explicitly confirms, can be promoted into a durable store. Each promoted record carries its own scope, its own provenance back to the run that created it, and a retention rule. The same record can be read later from a different turn, a different session, or by a different agent operating under the same access boundary.

A useful frame for the rest of this guide:

Retrieve when you need knowledge
Store when you need continuity
Structure when you need precision
Summarize when you need compression
Govern when you need production trust

There's a popular metaphor for this kind of system: the second brain. The framing matters because most second-brain implementations stop one step short. They give you searchable notes, which is closer to a better filing cabinet than to actual memory. A real second brain is accumulated, reusable memory. Notes get distilled into facts that attach to the entities they describe, and completed work gets summarized into reusable episodes. The same store then serves a chat session for one user, an autonomous agent running on their behalf, and a human searching their own work later, without any of them needing its own copy. The architectural leap is from file retrieval to memory formation. The optimization is from context stuffing to context reuse.

A vector store is not a memory system. The rest of this guide is what's underneath that distinction.

What kinds of memory do AI systems need

Memory types for AI systems

"Add memory" may sound like a single feature, but in practice it refers to several distinct systems. If you don't separate them, you end up with one catch-all store that handles every question poorly.

Anti-pattern: one memory store for every kind of memory. A single vector index where everything goes in. Raw conversation logs sit next to extracted facts. A user preference looks like a passing remark from three weeks ago. A tenant policy is just another row to be ranked. Retrieval can't tell these apart at query time, and neither can the model when the results land in the prompt.

There are five types worth distinguishing. Each one has its own schema and lifecycle, and each calls for a different retrieval strategy. Conflating any two of them produces specific, predictable failure modes.

Policy memory

Procedural rules and constraints. Tenant-scoped or global, versioned, and tightly controlled. Retrieved by exact match, never similarity.

Why it exists: the agent has to follow rules. Compliance constraints, brand guardrails, approval thresholds, security boundaries, etc. These change rarely and deliberately. They aren't suggestions and they aren't searchable.

{
 "$schema": "https://json-schema.org/draft/2020-12/schema",
 "title": "policy_memory",
 "properties": {
 "policy_id": { "type": "string", "maxLength": 64 },
 "tenant_id": { "type": "string", "maxLength": 64 },
 "policy_type": {
 "type": "string", "enum": ["compliance", "guardrail", "approval"]
 },
 "policy_key": { "type": "string", "maxLength": 128 },
 "policy_value": { "type": "object" },
 "version": { "type": "integer", "minimum": 1 },
 "effective_from": { "type": "string", "format": "date-time" },
 "effective_until": { "type": ["string", "null"], "format": "date-time" },
 "created_by": { "type": "string", "maxLength": 64 }
 }
}

Example records:

{ "policy_key": "refund_threshold", "policy_value": { "max_auto_approve_usd": 500 } }
{ "policy_key": "data_residency", "policy_value": { "allowed_regions": ["us-east-1"] } }
{ "policy_key": "tone_guardrail", "policy_value": { "forbidden_phrases": ["risk-free"] } }

Retrieval is WHERE tenant_id = ? AND policy_key = ?. No vectors involved. Policy retrieval that uses similarity is a bug, because you'll silently drift away from the rule that's actually in force.

Preference memory

Stable personalization parameters. User-scoped, sometimes tenant-scoped. Predictable lookup, no ranking.

Why it exists: this is what makes the system feel tailored without the user having to retell it every session. "I want JSON." "Use DD/MM/YYYY." "Be terse." Skip preference retrieval once and the system feels generic.

{
 "$schema": "https://json-schema.org/draft/2020-12/schema",
 "title": "preference_memory",
 "properties": {
 "user_id": { "type": "string", "maxLength": 64 },
 "tenant_id": { "type": "string", "maxLength": 64 },
 "pref_key": { "type": "string", "maxLength": 64 },
 "pref_value": {},
 "source": { "type": "string", "enum": ["user_stated", "inferred", "admin_set"] },
 "confidence": { "type": ["number", "null"], "minimum": 0, "maximum": 1 },
 "updated_at": { "type": "string", "format": "date-time" }
 }
}

Example records:

{ "pref_key": "response_format", "pref_value": "json", "source": "user_stated" }
{ "pref_key": "date_format", "pref_value": "DD/MM/YYYY", "source": "inferred", "confidence": 0.85 }
{ "pref_key": "verbosity", "pref_value": "terse", "source": "user_stated" }

Retrieval is a single keyed lookup that runs every turn. Preferences feed the static prefix of the prompt: the part that should hit the cache and stay stable.

Fact memory

Durable assertions the agent may reuse, with provenance. Where compounding advantage lives, and where design problems get hardest, because every fact you write is a bet about the future.

Why it exists: this is the agent's working knowledge of the world. "The customer's production database is in us-east-1." "Their fiscal year starts April 1." Facts are what let the agent stop asking the same clarifying questions across sessions. They're also what poisons the system when promoted carelessly.

{
 "$schema": "https://json-schema.org/draft/2020-12/schema",
 "title": "fact_memory",
 "properties": {
 "fact_id": { "type": "string", "maxLength": 64 },
 "tenant_id": { "type": "string", "maxLength": 64 },
 "user_id": { "type": ["string", "null"], "maxLength": 64, "description": "NULL for tenant-scoped facts" },
 "agent_id": { "type": ["string", "null"], "maxLength": 64, "description": "NULL for shared facts" },
 "subject": { "type": "string", "maxLength": 256, "description": "entity the fact is about" },
 "predicate": { "type": "string", "maxLength": 64, "description": "kind of fact" },
 "content": { "type": "string", "description": "the fact text" },
 "content_hash": { "type": "string", "maxLength": 64, "description": "SHA-256 for dedup" },
 "embedding": { "type": ["array", "null"], "items": { "type": "number" }, "minItems": 1024, "maxItems": 1024, "description": "nullable; rebuildable" },
 "metadata": { "type": ["object", "null"], "description": "partial-match filters" },
 "status": { "type": "string", "enum": ["provisional", "active", "revoked"], "default": "active" },
 "source_run_id": { "type": "string", "maxLength": 64 },
 "source_turn_id": { "type": ["string", "null"], "maxLength": 64 },
 "confidence": { "type": "number", "minimum": 0, "maximum": 1 },
 "superseded_by": { "type": ["string", "null"], "maxLength": 64, "description": "fact_id of replacement" },
 "expires_at": { "type": ["string", "null"], "format": "date-time" },
 "created_at": { "type": "string", "format": "date-time" }
 },
 "x-indexes": [
 { "name": "idx_fact_scope", "columns": ["tenant_id", "user_id", "agent_id"] },
 { "name": "idx_fact_status", "columns": ["tenant_id", "status"] },
 { "name": "idx_fact_dedup", "columns": ["content_hash", "tenant_id"] }
 ]
}

A few schema details deserve attention before moving on. The embedding column lives in the same row as content and the scope columns, which lets one query plan filter by tenant and search semantically. The content_hash is the dedup primitive: a SHA-256 of normalized content, scoped by (tenant_id, user_id, agent_id), so the same assertion written twice resolves to the same row instead of two rows competing for retrieval. And subject and predicate work as metadata tags for retrieval and grouping, not as the structured pieces of an RDF-style triple; the assertion itself lives in content as prose, which is what the agent reads. (Embedding dimension is model-dependent. The 1024 shown here matches several modern embedding models, but set minItems/maxItems to match the model you actually use.)

Example records:

{
 "subject": "customer:acme-corp",
 "predicate": "infrastructure",
 "content": "Production database is in us-east-1, replicas in us-west-2.",
 "source_run_id": "run_a1b2c3",
 "confidence": 0.95
}

Fact memory needs hybrid retrieval (lexical for "give me facts about subject X," semantic for "give me facts conceptually adjacent"). The retrieval section walks through the scoring.

Episodic memory

Structured summaries of completed work. The summary is a reusable artifact distilled from the trace; the underlying trace is kept separately for replay.

Why it exists: most production tasks are variations on tasks the agent has already completed. Episodic memory is what lets the system recognize "we've done this before" and retrieve the shape of the prior solution rather than re-deriving it from scratch.

{
 "$schema": "https://json-schema.org/draft/2020-12/schema",
 "title": "episodic_memory",
 "properties": {
 "episode_id": { "type": "string", "maxLength": 64 },
 "tenant_id": { "type": "string", "maxLength": 64 },
 "user_id": { "type": ["string", "null"], "maxLength": 64 },
 "task_type": { "type": "string", "maxLength": 64, "examples": ["support_case", "migration"] },
 "title": { "type": "string", "maxLength": 256 },
 "summary": { "type": "string" },
 "outcome": { "type": "string", "examples": ["resolved", "escalated"] },
 "key_steps": { "type": "array", "items": { "type": "string" } },
 "artifacts": { "type": ["object", "null"], "description": "file refs, ticket IDs" },
 "embedding": { "type": ["array", "null"], "items": { "type": "number" }, "minItems": 1024, "maxItems": 1024 },
 "status": { "type": "string", "enum": ["provisional", "active", "revoked"], "default": "active" },
 "source_run_id": { "type": "string", "maxLength": 64 },
 "completed_at": { "type": "string", "format": "date-time" }
 },
 "x-indexes": [
 { "name": "idx_ep_scope", "columns": ["tenant_id", "task_type", "completed_at"] }
 ]
}

Example record:

{
 "task_type": "support_case",
 "title": "Stripe webhook signature mismatch after secret rotation",
 "summary": "User rotated webhook secret in dashboard but kept old secret in env var. Resolved by updating env var and redeploying.",
 "outcome": "resolved",
 "key_steps": [
 "Confirmed signature verification was failing for all events",
 "Compared dashboard secret to env var",
 "Updated env var, redeployed, verified delivery"
 ]
}

Retrieval is hybrid (lexical + vector, fused) over the summary, optionally filtered by task_type.

Trace memory

Raw, append-only execution events. The flight recorder. High-volume, mostly useful for replay and forensics.

Why it exists: you need to be able to reconstruct what happened in a run. What did the agent retrieve, what did it decide, what tools did it call, what did the user say, what did the model say back. Trace memory is the source from which episodic and fact memory get distilled.

{
 "$schema": "https://json-schema.org/draft/2020-12/schema",
 "title": "trace_memory",
 "properties": {
 "trace_id": { "type": "string", "maxLength": 64 },
 "run_id": { "type": "string", "maxLength": 64 },
 "tenant_id": { "type": "string", "maxLength": 64 },
 "user_id": { "type": ["string", "null"], "maxLength": 64 },
 "turn_index": { "type": "integer", "minimum": 0 },
 "event_type": { "type": "string", "enum": ["user_msg", "tool_call", "tool_result", "model_msg"] },
 "payload": { "type": "object" },
 "token_cost": { "type": ["integer", "null"], "minimum": 0 },
 "latency_ms": { "type": ["integer", "null"], "minimum": 0 },
 "created_at": { "type": "string", "format": "date-time" }
 },
 "x-indexes": [
 { "name": "idx_trace_run", "columns": ["run_id", "turn_index"] },
 { "name": "idx_trace_tenant", "columns": ["tenant_id", "created_at"] }
 ]
}

Trace memory is the wrong place to look when you want to know what a user is like. It tells you what a user did. Most teams conflate trace with fact and preference memory and end up doing semantic search across execution logs to infer how to behave. That's how systems start hallucinating their own personality.

How this maps to other taxonomies

A lot of public writing slices memory differently. Cognitive-science framings use semantic, episodic, and procedural. Some product framings add conversational, summary, and entity. Those terms aren't wrong, they just mix structural categories (conversational, summary) with content categories (semantic, entity). For implementation I prefer this five-type split because each type behaves differently in storage and retrieval, and treating them as one thing produces predictable failures.

If you're using the Oracle Agent Memory SDK, it records policy as a guideline, preferences as preference, facts as fact, episodic as memory, and traces as message. Pick a vocabulary and stick with it across your codebase. The categories matter; the names don't.

How should you store agent memory

The shape of the memory drives the store. Four tradeoffs come up over and over.

Short-term (STM) vs Long-term (LTM) memory

Short-term memory means the agent process keeps state in RAM (a Python dict, a Redis cache, a session object) and loses it when the process restarts. Fast, simple, zero friction. The right call for one specific case: ephemeral session state during a single run. Tool outputs, scratch reasoning, intermediate retrieval results, anything that shouldn't outlive the turn.

For everything else it's the wrong call. State that lives only in RAM gives you no replay path when something goes wrong, no audit trail when a regulator asks how a decision got made, no way to enforce retention, and no scoping model to keep tenants apart. Yesterday's session is gone the moment the process restarts.

Long-term memory means the durable layer is a real datastore. The structured schema, the transactional guarantees, and the query language are why it costs more to write to and far more to operate around. It's the only viable choice for policy, preference, fact, and episodic memory.

A real system uses both. Working set in short-term memory during a run, long-term for anything that should survive the run:

class AgentSession:
    def __init__(self, run_id, user_id, tenant_id, agent_id, memory):
        # Ephemeral, discarded at end of run
        self.scratch = {}  # tool outputs, intermediate state
        self.turn_buffer = []  # current turn's events before flush

        # Durable, read from / written to via the memory manager
        self.run_id = run_id
        self.user_id = user_id
        self.agent_id = agent_id
        self.tenant_id = tenant_id
        self.memory = memory

What changes when you move durable state into a database:

Replay becomes possible. You can reproduce yesterday's session by replaying trace memory for that run_id. Without this, debugging is guesswork.
Multi-process and multi-region work correctly. A user starts a session on one server, comes back tomorrow on a different server, and the agent picks up where it left off.
Promotion becomes a real operation. You can't promote a candidate fact to durable memory if there is no durable memory. Everything is equally ephemeral, which means nothing accumulates.
Audit and deletion become tractable. GDPR right-to-forget is a DELETE cascading across stores. With short-term memory it's fiction.
Cost goes up at write time and down at read time. Database writes are slower than dict writes, but database reads of structured memory are far cheaper than re-running extraction over a transcript every turn.

Anti-pattern: no replay. You can't reproduce yesterday's session because the memory store has moved on. This is the bug class that ruins post-mortems. Trace memory written on every turn is the cheapest insurance against it.

Anti-pattern: treating scratchpad as memory. Tool call results, intermediate reasoning, and generated assets all leaking into long-term storage because nothing ever explicitly discards them. Your memory ends up mostly exhaust. The split above (ephemeral RAM for the run, durable database for what should survive) is the discipline that prevents this.

Filesystem vs database memory

A class of memory storage lives on the filesystem: markdown files, AGENTS.md, project notes, structured directories of knowledge. The pattern is everywhere in coding agents (Claude Code, Cursor, Codex), and it works there for a specific reason: the filesystem is already the unit of work. The repo is the scope and the file path is the boundary. Models trained on internet-era developer workflows are unusually competent with developer-native interfaces, which is why filesystems keep showing up in modern agent stacks.

There are two distinctions. The first is between the filesystem as an interface (the small Unix-shaped tool surface of ls, cat, grep, tail, read_range) and as a substrate (the place durable state actually lives). The interface argument is strong: agents are good at composing those primitives, and you don't need bespoke memory APIs when a folder of markdown will do. The substrate argument is much weaker once you leave a single-tenant prototype, because filesystems give you almost none of what shared, reliable memory needs.

What breaks when filesystem becomes the substrate:

No tenant isolation. A misconfigured path traversal or a shared root directory exposes another tenant's data. There is no row-level access control; permissions are filesystem-level, which is a coarse instrument for a fine-grained problem.
No transactional guarantees. Updating a fact and superseding the old one takes two file writes that have to succeed together. They can't. Crash in between and you have inconsistent memory.
No hybrid retrieval in one query. You can grep, or you can embed and search vectors, but not both filtered by the same predicates in the same operation. Once you start maintaining your own indexes and metadata files, you are building a database with fewer guarantees.
No deletion cascade. Removing a fact from a markdown file doesn't invalidate the embedding index that referenced it. GDPR right-to-forget is a much harder operation when truth is scattered across files and projections that point at them.
Concurrency is your problem. Concurrent writes can interleave or overwrite without locking. Locking semantics vary across platforms and network filesystems, and naive concurrent writes silently lose entries.

When to choose which:

Situation	Filesystem	Database
Single-tenant local agent (coding, personal)	Yes	Overkill
Multi-tenant SaaS agent	No	Required
Need transactional updates	No	Required
Need exact + semantic in one query	No	Required
Want users to edit memory by hand	Yes	Possible but harder
Need replay and audit trails	Possible but painful	Native

Local files are not the end state. They're a useful interface for single-developer agents and a poor substrate for everything else.

Single table vs multiple typed tables

Once you've chosen a database, the next decision is how to lay it out. Three patterns come up in practice, with real shipped examples for each.

The simplest is a single store with a type discriminator (a memory_type column or a namespace path). LangGraph's BaseStore takes this approach: a key-value store with hierarchical namespaces, where different memory categories live under different prefixes rather than in different tables. Mem0 sits nearby in spirit, though its implementation spreads memory across a vector store, a graph store, and a key-value store rather than collapsing everything into one row shape. The wins of this pattern are real: schema simplicity, one storage interface for everything, no joins, easy to add a new category. For a single-tenant prototype or a low-volume internal tool, this is the right call.

The middle pattern splits by access pattern. The most common shape is two stores: one for durable memory and a separate store for conversation history or traces. LangGraph does this at the framework level (BaseStore for cross-thread memory, checkpointer for thread-local conversation state). Letta splits further into core memory (in-context blocks), recall memory (conversation history), and archival memory (durable facts in a vector store). Oracle's AI Agent Memory SDK lands here too: its public API exposes three primitives (users and agents, memories, threads), with the underlying storage split across distinct tables for threads, messages, memory records, and actor profiles. The reasoning is consistent across these systems. Traces are append-only, high-volume, and queried by run_id for replay. They share almost nothing at the storage layer with durable memory. Letting traces grow inside the same table as facts means heavy trace writes bloat the working set for every other query.

The third pattern is one table per memory type, the model this article describes. The cost is real: more DDL, more places to touch when something cross-cutting changes, more conceptual surface area for someone new to the codebase. The payoff is that the rules each type follows live in the schema itself. Per-type retention, per-type indexes (no wasted vector index on policy or preference), and per-type access patterns become enforceable without application code having to remember them. The shift from the middle pattern to this one is incremental: you're already splitting traces out, now you split policy and preference out from fact and episodic for the same reason.

A reasonable working rule: start with a single store when the system is a prototype. Split traces or conversation history out as soon as their volume changes the characteristics of the operational store. Move to the typed model when you have more than one tenant, or when policy and preference start needing different retention rules than fact memory. Each split costs something at the time you do it and pays back across every retrieval after.

Exact retrieval vs semantic retrieval

Pure semantic retrieval is a first draft. It works for "find me documents about this concept." It fails for "what is this user's timezone preference," because that's a key lookup and similarity ranking is the wrong tool for a key lookup.

Memory needs both:

Exact retrieval handles preferences, policies, IDs, and scoped lookups
Semantic retrieval handles facts, episodes, and recall over unstructured content
Hybrid retrieval (lexical plus semantic, with reranking) is the production floor for fact and episodic memory

The architectural question is whether your store can do both in one query, or whether you have to compose them across systems. Composing across systems is workable until you need them filtered by the same access-control predicates, at which point a single query plan starts to look attractive.

Transcripts vs summaries

Storing raw transcripts as memory is the most common anti-pattern in this space. Fast to ship, expensive to live with. Transcripts are noisy: false starts, retracted statements, recovered tool errors, intermediate reasoning the user never confirmed. Index them directly and you train your retrieval layer to surface noise.

Summaries are durable memory. Transcripts are source material. Keep transcripts in trace memory for replay, promote structured summaries into episodic memory for reuse. The transcript records what happened. The summary records what should be remembered.

Storage tradeoff at a glance

Memory type	Store shape	Retrieval pattern	Lifecycle	Risk if wrong
Policy	Relational, versioned	Exact match by ID/version	Immutable, deployment-controlled	Silent guardrail drift
Preference	Structured rows or JSON keyed to user	Exact match by user ID	TTL, user-controlled	System feels generic
Fact	Hybrid: structured + vector	Lexical + semantic with rerank	Provenance, decay, revocation	Memory poisoning, drift
Episodic	Structured summaries with vector index	Lexical + Semantic over summaries	Long-lived, policy-gated	Precedent becomes policy
Trace	Append-only event log	Replay by run ID; vector for forensics	Retention-policy bound	No replay, no debugging

How memory systems handle long conversations without exploding context

Bounded-prompt agent memory loop

This is the question that drives most teams toward a real memory system, and the question they usually try to solve in the wrong place. The instinct is to keep the entire conversation in the prompt and trust the model's growing context window to absorb it. That's the dump-the-transcript anti-pattern, and it fails for three compounding reasons.

Context windows aren't free. Cache hits are cheap, cache misses on a million-token prompt aren't, and most production agent workloads have enough volatile content per turn that cache hit rates end up mediocre.

More tokens often make the model worse. Lost-in-the-middle, attention dilution, and context rot all show up in production at long context lengths. The default instinct to give the model everything is the default wrong answer.

Even if cost and quality were free, transcripts are the wrong shape for retrieval. You can't ask a transcript "what does this user prefer?" and get a clean answer. You can only ask the summary you extracted from it.

The pattern that works keeps the raw record in trace memory, but doesn't retrieve from it directly. Periodically distill it into structured artifacts in episodic and fact memory and retrieve from those. The conversation gets shorter over time in the prompt, even as the underlying record gets longer.

Each turn runs a fixed loop:

Append the user message to trace memory
Reassemble the prompt from durable memory (policy + preferences + retrieved facts + retrieved episodes + a short summary of recent turns)
Call the model
Append the response back to trace memory
Extract structured artifacts for the promotion gate

The key move is step 2. The prompt is rebuilt from scratch on every turn. Active policy goes in. Applicable preferences go in. The top-ranked facts and episodes for this query go in. A short summary of recent turns goes in at the tail. The transcript itself stays in trace memory. That is the difference between accumulating context and reassembling it: accumulating grows forever, reassembling stays bounded.

For the extraction step, two patterns work in production. The first is a combined structured-output call: the model returns its reply AND any extraction candidates in one API request, so extraction adds no extra latency or cost to the user-visible turn. The second is a separate extractor LLM that runs as a follow-up call after the reply. That path should move off the request thread (a background task, a queue worker, a separate process) so the user doesn't wait on a second round-trip.

Two retrieval paths, not one

This is the most important structural decision in the system, and it's the one most often missed. Memory retrieval has two modes that look similar from the outside but have completely different requirements underneath.

Known-scope lookup. "Give me all active policies and preferences that apply to this agent for this turn." The query knows exactly what it wants. This is enumeration. Every policy that applies must be returned, no top-k cutoff, no ranking. Cheap, deterministic, runs on every single turn. This is what feeds the model's static prefix.

Semantic discovery. "Find facts and episodes conceptually relevant to this user message." The query doesn't know what it wants until it sees what comes back. Ranking matters. Top-k matters. Score thresholds matter. This is what feeds the volatile tail of the prompt, on demand.

Build both into the manager from day one, because conflating them is the most common mistake in this space.

Path A: Known-scope lookup (startup injection)

When a session starts or an agent spawns, you need every policy and preference that applies, in full. You're not searching by relevance; you're loading the rule book. The right tool is exact-match SQL with a UNION across scope buckets:

-- All policy and preference memory that applies to this turn.
-- Exhaustive, no ranking, no top-k cutoff.
SELECT 'policy' AS kind, policy_key AS key, policy_value AS value
 FROM policy_memory
 WHERE tenant_id = :tenant_id
 AND (effective_until IS NULL OR effective_until > CURRENT_TIMESTAMP)
UNION ALL
SELECT 'preference' AS kind, pref_key AS key, pref_value AS value
 FROM preference_memory
 WHERE tenant_id = :tenant_id
 AND user_id = :user_id
ORDER BY kind, key;

This runs on every turn. It's fast (indexed exact match), exhaustive (no top-k), and deterministic (same inputs, same output). It feeds the static prefix of the prompt: the part that ideally hits the prompt cache and doesn't change between turns.

A couple of things to call out. The query filters by scope before doing anything else, so cross-tenant data is structurally invisible. And there's no embedding involved anywhere in the path. Policy retrieval that uses similarity drifts silently away from the rule that's actually in force, which is one of the worst failure modes in this space.

Path B: Semantic discovery (runtime recall)

When the user sends a message, you need facts and episodes that might be relevant, ranked by how relevant they actually are. This is where hybrid retrieval lives: lexical search for term-exact recall, vector search for conceptual recall, fused into one ranked list:

-- Hybrid: vector similarity + lexical match, scope-filtered before ranking.
WITH vector_hits AS (
 SELECT fact_id, content, source_run_id,
 vector_distance(embedding, VECTOR_EMBEDDING(ALL_MINILM_L12_V2 USING :query AS DATA), 'COSINE') AS vector_distance
 FROM fact_memory
 WHERE tenant_id = :tenant_id
 AND status = 'active'
 AND superseded_by IS NULL
 AND (user_id IS NULL OR user_id = :user_id)
 ORDER BY vector_distance
 FETCH FIRST 20 ROWS ONLY
),
lexical_hits AS (
 SELECT fact_id, content, source_run_id, SCORE(1) AS lexical_score
 FROM fact_memory -- Oracle Text CONTEXT index on content
 WHERE tenant_id = :tenant_id
 AND status = 'active'
 AND CONTAINS(content, :query, 1) > 0
 ORDER BY lexical_score DESC
 FETCH FIRST 20 ROWS ONLY
)
SELECT fact_id, content, source_run_id,
 vector_distance, lexical_score
 FROM vector_hits v
 FULL OUTER JOIN lexical_hits l ON v.fact_id = l.fact_id -- content is CLOB; can't appear in USING/equality joins;

The lexical CTE above uses Oracle Text syntax (CONTAINS, SCORE, the trailing index-id argument). On other engines, you’d swap in the equivalent such as tsvector @@ to_tsquery on Postgres, MATCH...AGAINST on MySQL, match_phrase on Elasticsearch. The hybrid pattern is the same with different operator names. The application layer fuses scores. A weighting that holds up well in production:

def fuse_score(vector_distance, lexical_score, max_lexical):
    # Vector: convert distance to similarity in [0, 1]
    v_sim = 1.0 / (1.0 + vector_distance) if vector_distance is not None else 0.0

    # Lexical: normalize against the max in this batch
    l_sim = (lexical_score / max_lexical) if (lexical_score and max_lexical) else 0.0

    # Weighted: lexical gets more weight when both signals exist
    if vector_distance is not None and lexical_score:
        return 0.4 * v_sim + 0.6 * l_sim

    # Vector-only fallback (no lexical match)
    return v_sim

From running this in production, two settings matter most. Lexical hits get more weight (0.6) than vector similarity (0.4). When both signals fire on the same row, the row is genuinely relevant; when only the vector fires, it's plausibly relevant but should rank lower. And drop everything below a fused score of 0.4, because vector search has a long tail of weakly-related results that hurt more than they help.

Calibrate the result set into tiers the agent can actually use:

def relevance_tier(score):
    if score is None:
        return "standard"  # lexical fallback, no vector signal
    if score >= 0.7:
        return "high"
    if score >= 0.5:
        return "standard"
    return "low"

Returning bare scores asks the agent to do calibration work it’s not good at. Returning relevance: "high" lets the model decide how much weight to give each result, which is something it does well.

Filter before ranking, always

Both paths share one rule: scope filters run before ranking, never after. A query that ranks across all tenants and then filters is a leak waiting to happen. The filter itself works fine; the problem is that the embedding neighborhood was already shaped by data the user shouldn't have seen.

-- WRONG: ranks across all tenants, then filters
WITH ranked AS (
 SELECT * FROM fact_memory
 ORDER BY vector_distance(embedding, :q, 'COSINE')
 FETCH FIRST 100 ROWS ONLY
)
SELECT * FROM ranked WHERE tenant_id = :current_tenant;

-- RIGHT: filters by scope first, ranks within scope
SELECT * FROM fact_memory
 WHERE tenant_id = :current_tenant
 ORDER BY vector_distance(embedding, :q, 'COSINE')
 FETCH FIRST 5 ROWS ONLY;

This is the single most important pattern in this section. It's also the one most teams get wrong, because the ranking-first pattern is what most vector store quickstarts demonstrate.

Cascade fallback

In production, every layer of this can fail. The embedder is offline. The vector index is rebuilding. The lexical index is missing. A real system degrades through tiers rather than throwing:

hybrid (vector + lexical) -> vector-only -> lexical-only -> exact-match LIKE

Each step is still useful, just at lower fidelity. The agent still gets a result. The system stays available. The failure shows up in the trace envelope as a degraded retrieval mode rather than user-visible breakage.

Why a memory manager matters more than another vector store

If you take memory typing seriously, you need something that decides what gets written and where, what gets retrieved when the model asks, and what gets assembled into the prompt on this turn. A vector store doesn't do any of that. The component that does is the memory manager.

Five responsibilities, each of which gets glossed over until production exposes it.

Write: the promotion gate

Decide what enters durable memory and what stays ephemeral. This is the highest-risk operation in the system. Promote everything and the memory store poisons itself; promote nothing and the agent feels amnesiac.
The gate runs three operations in a single transaction:

Classify and scope. Determine the candidate's type and resolve its scope tuple (tenant, user, agent, session).
Dedup by content hash plus scope tuple. If an existing record matches both, return a dedup result instead of writing. The same fact arriving from two runs in the same scope resolves to one row; the same fact in two different scopes ends up as two rows, because they live in different access boundaries.
Type-specific verification. Facts need a confidence above threshold and a source run ID. Contradicting facts route to the supersession path rather than a new write.

Then compute status from scope and type — never from the caller — and write the record with provenance attached. The companion runtime package has the full implementation.

The flow above hides two rules that do real work:

Status assignment: Agent-scoped memory, preferences, and episodes enter as active. Tenant-scoped facts and policies enter as provisional: durable enough to confirm later, but not retrievable by semantic discovery until status flips to active. That gap is what keeps a noisy promotion pipeline out of the recall layer.
Use both halves of the dedup key: Without the scope half, you collapse rows across access boundaries that should stay separate. Without the hash half, the same fact creates competing duplicates in retrieval.

Scope assignment at promotion time is the underrated half of this. If user Jane discovers that Acme's production database is in us-east-1, the fact is logically about Acme, not about Jane. Writing it user-scoped means every other user at Acme has to rediscover it independently and the compounding-advantage story falls apart. Writing it tenant-scoped without a rule lets user-scoped facts leak across users the first time someone gets sloppy. The working heuristic: default to the narrowest scope that matches the candidate (usually user), and only promote to tenant scope when the subject of the fact is the tenant entity itself and the fact has been observed from two independent sources. The promotion gate is where that rule belongs because callers should never be trusted to pick scope on their own.

Before you ship, write down six things for each promotion path: what gets promoted, the granularity, which memory type it lands in, the provenance attached, the decay rule that retires it, and the authority that signed off.

Anti-pattern: user-supplied status. Letting the save call accept a status field guarantees that somebody will eventually pass status="active" from a place you didn't expect. Compute status from scope and type inside the manager, and keep it out of the public API entirely.

Update: invalidation across projections

Memory drifts. Tenant policy changes, user preferences update, a fact gets contradicted by a system of record. Updates need to be coordinated: write the new state, mark the old state superseded, and invalidate any derived projections that cached the old answer. This is one of the places where transactional guarantees stop being a nice-to-have.

Inside a single transaction: write the new fact and get its new ID, mark the old fact superseded with a pointer to the replacement, invalidate any cached retrieval results that referenced the old fact, and emit a fact.superseded event so other downstream projections (search indexes, etc.) update.

Without atomicity, that sequence can interleave with retrieval and produce a window where both the old and new fact are visible. With it, retrieval sees either the old fact or the new one, never both.

Summarize: compression after stabilization

Compress trace memory into episodic and fact memory. This is where transcripts become reusable. It's also where most teams get hurt: summarizing before you stabilize meaning (resolve references, attach provenance, normalize entities) compresses noise faster than signal. Stabilization comes first; the summary comes after.

Pull the full trace for the run, stabilize it (resolve pronouns, normalize entity references, drop retracted statements), then run two extractors against the stabilized events: one for candidate facts (subject, predicate, content, confidence), one for an episode summary (only if the run completed a coherent task). Each extracted candidate runs through the promotion gate, where most get rejected. High rejection rates are what you want here.

Retrieve by type: orchestration and per-type strategy

Resolve a request into the right slice of memory. Not "top-k from the vector store," but a parallel fan-out across the right stores with the right strategy per type. Policy and preference lookups, fact hybrid search, episode semantic search, and a recent-trace summary fan out in parallel, either across stores, or as a single unified retrieval query when the data is co-located in one engine, then assemble into a token-budgeted prompt with reserved slots for policy and preferences (always included) and ranked slots for facts, episodes, and recent activity (filled until the budget runs out).

A memory manager that treats every retrieval as a vector query will be wrong about half the time, in ways that look like "the agent feels off" rather than "the agent threw an error." The strategy that actually works depends on the type:

Type	Strategy
Policy	Exact match: WHERE tenant_id = ? AND policy_key = ? AND effective_until IS NULL
Preference	Exact match: WHERE user_id = ? AND tenant_id = ? (full set every turn)
Fact	Hybrid: lexical + vector, fused, filtered by tenant/user scope
Episodic	Hybrid: lexical + vector over summary, fused, with optional task_type filter
Trace	Replay by run_id; vector search only for forensic queries

Decide what enters the context window

The most underappreciated job. Even with the right memory in hand, the manager still has to choose what gets surfaced on this turn and in what order. It enforces the token budget while doing it. When the budget is exceeded, the manager decides what to compact or drop entirely.

The pattern that holds up:

Reserve fixed slots for policy and preference (they're cheap and they always apply)
Bound retrieval payloads (top-k with a max-token cap per item)
Order recent context last (recency near the answer position helps)
Compact rather than truncate when over budget (drop the least-recent episode, keep all preferences)

The assembler walks the reserved slots first (policy, preferences) and adds them unconditionally, then walks the ranked slots (facts, episodes, recent) and adds items in order until the token budget is exhausted. Anything that doesn't fit gets dropped rather than truncated mid-record.

None of these are vector store operations. They're coordination logic. The vector store is one component; the memory manager runs the loop.

What supports privacy and data security in memory systems

Memory systems amplify privacy stakes. A logged conversation is one privacy boundary. A fact extracted from a hundred conversations and promoted into durable memory is something else. Right-to-forget under GDPR becomes nontrivial the moment "the thing that knows about the user" is a derived artifact rather than a raw record. The EU AI Act's high-risk obligations were originally scheduled for August 2026 and now look likely to be deferred to December 2027 under the Digital Omnibus revisions that reached political agreement in May 2026. Either way, the direction is the same: a higher bar on data governance, audit, and human oversight for systems classified as high-risk.

Five supports that matter, in rough order of how often they fail in production.

Scope as a structural primitive. Every memory record carries scope columns (tenant_id, user_id, agent_id). Retrieval filters by scope as hard predicates, before ranking, never as a soft post-filter. The earlier WRONG/RIGHT example covers this. Scope is a hard access boundary, and resist the temptation to invent new scopes for relevance signals like "this repo only" or "this project only." A repo restriction is a filter; it belongs in metadata or a tag column rather than the access-control system.

Provenance on every durable record. Where it came from, which run promoted it, under which policy version. Without provenance you can't honor a deletion request. You can't audit a decision later either, and you can't prove isolation when a regulator asks. Every promotion writes the source run, the source turn, the promoting policy version, and the timestamp.

Tombstones with cascading invalidation. Deletion in the canonical store has to cascade into every derived projection. Otherwise the embedding index keeps surfacing artifacts that have been removed from canonical truth:

-- GDPR erasure: cascade across all memory types in one transaction
BEGIN;

UPDATE fact_memory
 SET content = '[erased]', content_hash = 'erased', embedding = NULL,
 status = 'revoked'
 WHERE user_id = :user_id
 AND tenant_id = :tenant_id;

UPDATE episodic_memory
 SET summary = '[erased]', embedding = NULL,
 status = 'revoked'
 WHERE user_id = :user_id
 AND tenant_id = :tenant_id;

DELETE FROM preference_memory
 WHERE user_id = :user_id
 AND tenant_id = :tenant_id;

DELETE FROM trace_memory
 WHERE user_id = :user_id
 AND tenant_id = :tenant_id;

INSERT INTO deletion_events (user_id, scope, deleted_at, reason)
VALUES (:user_id, 'all', SYSTIMESTAMP, 'gdpr_erasure');

COMMIT;

In a federated architecture that's a coordination problem across four systems, with a window during which deletion is partial. With everything in one engine inside one transaction, it's atomic.

Retention as a property of the record. TTL, retention policy ID, and sensitivity classification live alongside the record. The manager enforces them at write and read time so application code never has to. A nightly job sweeps expired records:

DELETE FROM fact_memory WHERE expires_at < CURRENT_TIMESTAMP;
DELETE FROM episodic_memory WHERE expires_at < CURRENT_TIMESTAMP;
DELETE FROM trace_memory
 WHERE created_at < CURRENT_TIMESTAMP - INTERVAL '90 days'
 AND retention_class = 'short';

Tenant-scoped encryption domains. Logical isolation is a software invariant; encryption isolation is a cryptographic one. Both matter. Encryption reinforces scope; it doesn't substitute for it.

Anti-pattern: memory that never forgets. No TTL, no decay, no eviction. Six months in, the store becomes a liability rather than an asset because the irrelevant outweighs the useful and retrieval gets noisier over time. Aggressive forgetting policies are some of the best behaviors you can add.

Privacy gets routed before any durable write occurs, or it doesn't get enforced at all. You can't bolt it on after the architecture is settled.

From file retrieval to reusable memory

A useful way to see what we've built up so far is to walk through what happens when a stack of notes turns into agent memory.

A user dumps a folder of meeting notes, customer reports, and design docs into the system. Classical RAG indexes them, runs nearest-neighbor search at query time, and stuffs the top matches into the prompt. The notes serve the chat. Useful, limited, exactly what RAG is for.

A memory system does something different. The notes still get indexed, but indexing is only the starting point. As the user and the agent work over those notes, three kinds of distillation happen in the background:

Stable preferences get extracted into preference memory ("this user always wants exec summaries first"). They survive every session and feed every prompt.
Durable assertions get extracted into fact memory ("Acme runs production in us-east-1"). They get attached to the entities they're about, with provenance back to the source notes, and become available to any agent that should see them.
Completed tasks get distilled into episodic memory ("migrated Acme's webhook to v2, here's how"). They become the shape the next similar task can reuse instead of re-deriving.

The same memory layer now serves a chat, an autonomous agent operating on the user's behalf, and a human searching their own notes. None of them needs a separate store. The notes were the raw material; the memory layer is the durable, reusable artifact distilled from them. Local files were the input. They're not the end state.

This is what the brief at the top of this article meant by ingest once, distill, retrieve the right slice, reuse over time. The win isn't better prompt packing. The win is lower context overhead and better continuity, because the system is no longer reconstructing the same context from scratch every turn.

Where the memory layer actually lives

Co-located versus split memory architectures for AI agents

By this point we've designed the typed memory model. We've worked through the storage tradeoffs. We've built two retrieval paths into the manager and pinned down what the promotion gate has to enforce. The remaining question is what to put it all on.

The accidental architecture most teams arrive at splits the memory layer along the axis that hurts most. Relational data goes in Postgres. Hybrid retrieval goes in Elasticsearch, OpenSearch, or a dedicated vector store. Traces end up in S3 or a time-series database. Each of these is a strong choice for the job it was built for, and many production stacks use them well. The problem is what happens when context has to cross between them.

Every meaningful retrieval becomes a join across systems:

“Find facts relevant to this query, filtered by the policies that apply to this tenant, scoped to users this agent is allowed to see, joined with the user's current preferences.”

That join crosses a trust boundary. It also crosses a transactional boundary and a latency boundary. Every time context crosses one of those, you re-introduce the consistency problem you were trying to avoid. This is the polyglot persistence trap: each component excels at a specific function, but you end up running multiple databases with multiple security models and multiple backup strategies, all coordinated by glue code that becomes its own failure surface.

Hybrid retrieval as a feature is largely a solved problem. Elasticsearch, OpenSearch, LanceDB, Pinecone, Weaviate, and the vector extensions to Postgres all support some version of it, and the production-quality gap between them is narrower than vendor marketing suggests. If the memory layer were just embeddings and full-text search, the architecture question would be boring.

Memory is more than retrieval. The capability that actually matters is hybrid retrieval co-located with the relational data that governs it. A single query has to enforce policy, resolve user preferences, apply access controls, and rank by semantic similarity, all under one query plan, one transactional boundary, one security model.

Oracle AI Database is a strong fit for this pattern in production today. It gives you the things every memory system needs in a single converged engine:

Durable storage. Canonical truth lives in real tables with real constraints. The vector index is a cache for retrieval, never the system of record.
Relational precision. Policies and preferences are exact-match lookups; the database has done that for forty years.
Vector retrieval. Embeddings live in VECTOR columns alongside the rows they describe, queryable through VECTOR_DISTANCE and VECTOR_EMBEDDING in the same SQL statement as the structured filters.
JSON and metadata handling. JSON columns and JSON_VALUE give you flexible schemas where you need them, without introducing a separate document store.
Governed access patterns. Tenant isolation lives in the data plane. So do row-level security, audit logging, and encryption. None of those are application-code concerns.
A path from local workflow to production. The same SQL works on a developer's laptop with Oracle Database Free as it does in a regulated multi-region production deployment. The memory layer doesn't need to be reimplemented when it grows up.

ACID transactions are the reason all of this collapses into one place rather than four. When a fact promotion supersedes an old fact, the same transaction can invalidate the cached projection and write the deletion event. Either all of it lands or none of it does. The same query that ranks facts by similarity can enforce tenant isolation, apply active-policy thresholds, and respect a user's personalization preference, inside one transaction boundary, with one security model.

In practice, that's the difference between this:

-- Retrieve facts relevant to this query, scoped to the user's tenant,
-- filtered by active policy, ranked by semantic similarity. One query plan.
SELECT f.fact_id, f.content, f.source_run_id, f.created_at
FROM fact_memory f
JOIN preference_memory p
 ON p.user_id = :current_user
 AND p.tenant_id = f.tenant_id
JOIN policy_memory t
 ON t.tenant_id = f.tenant_id
 AND t.policy_key = 'fact_retrieval'
 AND t.effective_until IS NULL
WHERE f.tenant_id = :current_tenant
 AND f.status = 'active'
 AND f.superseded_by IS NULL
 AND (f.expires_at IS NULL OR f.expires_at > SYSTIMESTAMP)
 AND JSON_VALUE(p.pref_value, '$.allow_personalization') = 'true'
 AND f.confidence >= JSON_VALUE(t.policy_value, '$.min_confidence')
ORDER BY VECTOR_DISTANCE(
 f.embedding,
 VECTOR_EMBEDDING(ALL_MINILM_L12_V2 USING :query AS DATA),
 COSINE
 )
FETCH FIRST 5 ROWS ONLY;

And four round trips across three stores with a consistency model you have to defend yourself.

Oracle has also recently introduced Oracle AI Agent Memory, an SDK on top of the same database that gives you a working memory system out of the box: typed records (guideline, preference, fact, memory, message), scope columns (thread_id, user_id, agent_id), exact and similarity search APIs, and managed schema policies. It is the most direct implementation of the architecture this article has been building toward, and starting from it skips a lot of the reinvention that teams otherwise pay for in the first year of a memory system. Everything in this article still applies on top of it. The SDK handles the substrate; the design choices that make memory good for your product are still yours to make.

You'll still want specialized stores for some things. Trace memory at high volume has different durability and cost characteristics than your operational database. Blob storage for large documents, attachments, and tool outputs belongs somewhere built for that. A converged platform doesn't eliminate every other store. What it eliminates is the most expensive join: the one between the relational data that governs what the agent is allowed to do and the vector data that shapes what the agent actually does.

When those two are co-located, the architectural problem shifts from "how do I keep these stores in sync" to "how do I design good memory policies," which is the problem that actually matters.

How do you add memory to an AI agent

If you have a working RAG pipeline today and want to evolve it into a memory system, the steps are sequential. Each one builds on the last, and none of them are optional if the system is going into production.

Step 1. Type your memory

Inventory what your current system stores. Label each as policy, preference, fact, episodic, or trace. You'll discover that most of it is sitting in the same vector store and that some of it shouldn't be there at all.

A practical exercise: take ten retrieval results from your current system and label them. If half come back as "trace memory I'm doing semantic search over," you've found the reason your agent feels noisy.

Step 2. Scope every record

Tag every memory record with scope columns: tenant_id, user_id, agent_id. Make these required fields. Make scope a hard predicate at every retrieval site so a missing filter becomes a query error rather than a leak.

ALTER TABLE fact_memory ADD COLUMN tenant_id VARCHAR(64) NOT NULL DEFAULT 'default';
ALTER TABLE fact_memory ADD COLUMN user_id VARCHAR(64);
ALTER TABLE fact_memory ADD COLUMN agent_id VARCHAR(64);
CREATE INDEX idx_fact_scope ON fact_memory (tenant_id, user_id, agent_id);

Anti-pattern: scope as relevance signal. Inventing new scopes (team_scope, repo_scope, project_scope) for what are actually filters. Scope is access control; filters are metadata. Mixing them turns retrieval into a permissions audit and makes the access boundary harder to reason about. If you find yourself adding a fifth or sixth scope dimension, what you actually need is a metadata column.

Step 3. Separate truth from acceleration

Move authoritative state into structured columns with provenance and retention. Every fact gets a row in fact_memory with the canonical content, provenance, status, and retention. The embedding column is a projection of that content; if the embedding model changes tomorrow, you re-embed from the structured rows. You should be able to drop the embedding column and rebuild it from the rows without losing any information.

Anti-pattern: the vector store is the system of record. If a piece of data exists only in the embedding index, you've lost provenance, replayability, and deletion guarantees. The vector index is acceleration; the structured row is truth. You can re-derive embeddings from rows, but never the other direction.

Step 4. Build a promotion gate

Before anything writes to durable memory, it has to pass through a gate that decides scope, type, provenance, retention, and verification. Promotion is a database write. Treat it like one.

The simplest viable gate is a function with explicit rules per type, plus the computed-status rule from earlier.

The rules per type, in plain English. Preferences accept candidates with a confidence floor (0.5 is reasonable) and a non-empty pref_key. Production systems often tighten this with explicit confirmation or repeated observation across multiple turns. Facts need both a source run ID for provenance and a confidence above a higher threshold (0.7 holds up well). Episodes promote on task completion. Policy never auto-promotes; it requires explicit authoring. Status is then computed from scope and type, never accepted from the caller.

Tune the thresholds against your own data. The first version will be wrong; that's expected. The point is having a gate at all.

Anti-pattern: promotion on user thumbs-up. The strongest user signal is also the rarest and most biased. Systems that promote only on explicit positive feedback learn what makes users click the button, which is a different thing than what makes the agent good. Treat thumbs-up as one signal among several and combine it with confidence thresholds, repeated observation, and source provenance.

Step 5. Reassemble per turn

Stop accumulating the prompt across turns. Reassemble it on every turn from policy, preferences, retrieved facts, retrieved episodes, and a short summary of recent activity. The transcript is source material; the prompt is a reconstruction.

You'll know this step is working when the prompt token count stops growing with conversation length.

Step 6. Instrument the loop

Capture trace envelopes for every run: what was retrieved, what was promoted, what the model used, what it cost. Without this you can't evaluate, replay, or debug.

Minimum trace envelope per turn:

{
 "run_id": "run_a1b2c3",
 "turn_index": 7,
 "tenant_id": "acme",
 "user_id": "jane@acme.com",
 "retrieval": {
 "facts_returned": ["fact_001", "fact_042"],
 "episodes_returned": ["ep_017"],
 "preferences_applied": ["response_format", "verbosity"],
 "retrieval_mode": "hybrid"
 },
 "model_call": {
 "model": "llm_model_id",
 "input_tokens": 4231,
 "output_tokens": 412
 },
 "promotions": [
 {
 "type": "fact",
 "fact_id": "fact_103",
 "confidence": 0.82,
 "status": "provisional",
 "outcome": "written | deduplicated | rejected | superseded",
 "reason": "rejection reason or superseded:old_id"
 }
 ]
}

These six steps aren't separable. You can't meaningfully build the promotion gate before you have typed memory, and you can't reassemble per turn before you've separated truth from acceleration. The order matters.

A final note on what counts as memory

A lot of current memory talk is retrieval plus larger prompts with better branding. If a system only needs semantic lookup over documents, basic RAG is enough; calling it memory doesn't make it more useful. If a system has to maintain continuity across sessions, recall structured information accurately, govern reuse against real privacy boundaries, and learn what to retain over time, then it's a memory system design problem. A vector store on its own won't cover it.

"Add memory" undersells what's actually happening. You're moving from a stateless API call to a stateful loop with a managed memory layer underneath it. Done well, the model layer becomes interchangeable. The memory layer becomes the thing nobody else can copy, because what's stored in it reflects choices that only your team could have made.

Models are shared. The memory system is yours.

FAQs

Why isn't a vector database the same as a memory system?

A vector database helps find similar information, but a memory system also decides what to remember, how to organize it, how to retrieve it, and how to maintain continuity across sessions.

What kinds of memory do AI agents need?

The article identifies five types: policy memory (rules), preference memory (user settings), fact memory (durable knowledge), episodic memory (completed tasks), and trace memory (execution history).

How do memory systems avoid huge prompts?

Instead of continually adding conversation history, they rebuild prompts each turn using relevant memories, summaries, policies, and preferences while keeping raw transcripts stored separately.

What's the biggest change when moving from RAG to memory?

The system gains a durable write path, allowing it to store validated information from interactions and reuse it later, creating long-term continuity rather than one-time retrieval.

Oracle AI Agent Memory: A Governed, Unified Memory Core for Enterprise AI Agents

Wojtek Pluta — Mon, 04 May 2026 08:57:04 +0000

This article is syndicated from the original post on blogs.oracle.com. Read the canonical version there for the latest updates.

Recently, Oracle introduced the Oracle AI Agent Memory Python package, a model and framework-agnostic memory solution that gives enterprise AI teams a governed memory core on Oracle AI Database: short-term threads with summaries and context cards, long-term durable memories with vector search, automatic LLM-based memory extraction, and the governance and isolation production agents require.

Oracle Agent Memory: a unified agent memory layer built on Oracle AI Database.

Oracle AI Agent Memory is available now via PyPI as oracleagentmemory and documented in the Oracle Help Center. It is designed to replace the patchwork memory stack most production agents inherit with a single governed memory substrate built on Oracle AI Database.

This is the difference between an agent that is memory-augmented, given a vector store to consult, and one that is memory-aware, responsible for reading from and writing to its own governed, durable state with one enterprise-grade backend.

"Agent memory has shifted from a research curiosity to a production requirement in under two years. Teams shipping serious agentic systems need a backend that handles vectors, structured data, and transactional consistency in one place, not three stitched together. Oracle AI Database is one of the few platforms that delivers all of that natively, which is why we built Hindsight to run on it as a first-class backend."
— Chris Latimer, Co-Founder & CEO of Vectorize

Why Agent Memory, Why Now

The four types of agent memory: working, semantic, episodic, and procedural.

Most agent implementations treat memory as a bolt-on. A vector store consulted at retrieval time, a chat history table glued on beside it, and whatever hand-written extraction logic the team can maintain.

That stack holds together for a demo. It falls apart from the moment an enterprise AI team asks questions that actually matter. Who owns the memory? Where is it governed? How do we isolate tenants? How do we audit what the agent learned, and how do we forget it on request?

Context windows have grown over the years, but no context window is large enough to hold the full state of a long-running agent: weeks of user preferences, accumulated domain knowledge, prior tool outcomes, evolving task state, and the reasoning history that makes each decision defensible.

Agents need memory for the same reasons people do: to hold an active state while working on a problem, to retain facts learned over time, to recall specific past experiences, and to encode behavioral rules and procedures.

A practical taxonomy for agent memory commonly used in agent design covers four types:

Working memory is the active state the agent is reasoning over right now, the running conversation and the scratchpad the model sees at inference time.
Semantic memory is the durable facts and knowledge the agent accumulates about users, entities, and the world: preferences, canonical definitions, structured reference data.
Episodic memory is specific past experiences the agent can recall, what happened on a prior session, what the user asked three weeks ago, how a similar task resolved last time.
Procedural memory is the behavioral rules, guidelines, and learned procedures that shape how the agent acts, how to handle customers, which tools to prefer, what not to do.

These are not four different systems. They are four access patterns over the same underlying state, which is what makes a unified memory core the right architectural answer rather than four bolted-together services.

Oracle AI Database as the Memory Core

Reference architecture for Oracle AI Agent Memory on Oracle AI Database.

Oracle AI Database combines vector similarity search, relational querying, and graph-aware data access in one governed engine, enabling semantic recall alongside precise transactional and relationship-centric retrieval. Combined with Oracle's operational story, backups, replication, high availability, encryption, fine-grained access control, and audit, teams get a path from notebook to regulated production without swapping storage layers, rewriting compliance reviews, or stitching together bespoke isolation logic along the way.

Memory engineering, as a discipline, demands substrate choices that hold up under the access patterns a real enterprise agent actually has: concurrent writes, per-user and per-tenant scoping, full audit, and semantic retrieval at scale.

“Enterprise agents need an agent memory solution with robust security guarantees, strong governance controls, sophisticated workload isolation, as well as deep integration within the enterprise data platform. Oracle AI Agent Memory greatly simplifies building agent memory solutions by consolidating what are usually multiple separate and fragmented services, within the converged database architecture that customers already trust for their most critical data.”
— Tirthankar Lahiri, SVP, Mission-Critical Data and AI Engines, Oracle Database

Production agent memory carries two loads. Developers wire the stack together: vector store, chat log, extraction scripts, isolation logic, governance per piece. Agents reason over the fragments, deciding what to retrieve from where and fitting the relevant world into a finite context window each turn.

Oracle AI Agent Memory lifts both. One governed client replaces the four-service stack, with one set of credentials, one compliance review, and one backup story. Working, semantic, episodic, and procedural memory share one substrate and one retrieval surface, so the model reasons over a coherent view of its state. Summarization and scoped retrieval put the right subset into context at the right moment, freeing the model to spend its reasoning budget on the task rather than memory bookkeeping.

Automatic LLM-based extraction turns conversation into durable memories without hand-rolled prompt chains. Multi-tenant isolation is enforced at the store layer, so a single schema can host multiple deployments without cross-tenant leakage. And because the SDK is framework-agnostic, integrating with LangGraph, Claude Agent SDK, OpenAI Agent SDK, WayFlow, and custom harnesses, teams aren't locked into a single runtime to get the substrate.

Key Benefits For AI Workloads

Configuration: gpt-5.5, reasoning effort xhigh, nomic-embed-v1.5 embeddings, local HNSW index, top-K = 200. X-axis truncated; all categories scored above 88%.

Oracle AI Agent Memory is built for the operational realities of running AI agents in production.

Production-grade recall on long-horizon memory benchmarks. On LongMemEval, the standard academic benchmark for long-context agent memory, Oracle AI Agent Memory scores 93.8% (469 of 500), with the strongest results on the categories that matter most for production agents: 100% on single-session assistant recall, 96% on temporal reasoning, and 95% on knowledge-update tasks. Multi-session recall, the hardest category in the benchmark, lands at 88%. Configuration: OpenAI gpt-5.5 (reasoning effort xhigh), nomic-embed-text-v1.5 embeddings, local HNSW index, top-K = 200.

Bounded per-turn cost as sessions extend. Periodic thread summarization, durable memory extraction, and prompt-time message compaction keep the working context bounded as conversations grow. In an 80-turn scripted conversation, Oracle AI Agent Memory held per-request input around 1,300 tokens for the full run while a flat-history baseline grew linearly past 13,900 — roughly 9.5× more tokens per request by the final turn, and a much steeper bill across the full conversation. Teams shipping long-running agents trade a linear-in-history cost curve for a flat one.

80-turn ChromAtlas-ND scripted conversation · gpt-5.4 (raw OpenAI client, no framework). Token estimate: chars / 4 (notebook convention)

Better answers than a flat-history baseline. A flat-history agent has the entire verbatim conversation in its prompt — every fact ever mentioned, in order. By rights it should be hard to beat on recall. Across the same 80-turn conversation, evaluated by an impartial gpt-5.4 judge on accuracy, completeness, relevance, and coherence, Oracle AI Agent Memory won 48 turns to flat history's 13, with 19 ties: 3.7× more wins despite the baseline's information advantage. A retrieved context card focuses the model on what matters; a sprawling transcript dilutes attention across noise.

80-turn ChromAtlas-ND scripted conversation; judge: gpt-5.4; scored on accuracy, completeness, relevance, coherence

Cost is a tunable knob, not a fixed value. The summarization trigger controls how aggressively the package compacts thread context, and it moves the cost-fidelity trade-off directly. In an 8-query demo conversation (five runs per threshold), a 10,000-token trigger landed at a mean of 121,268 total tokens, about 60% under the 306,823-token flat-history baseline. As the trigger rises, the package compacts less often and preserves more raw context per turn; by a 50–70k trigger, mean total tokens approach or exceed the baseline, and run-to-run variance widens. Teams pick the threshold that matches their answer-quality requirements and lock in the cost envelope they want, rather than accepting whatever curve a fragmented stack produces.

Memory Agent Efficiency vs Summarization Threshold on Demo Conversation. Num queries = 8; num runs per threshold = 5.

One backend, every Python runtime. LangGraph, the Claude Agent SDK, the OpenAI Agents SDK, WayFlow, and custom Python harnesses all instantiate the same OracleAgentMemory client and read and write the same Oracle Database store. Teams running more than one framework no longer rebuild memory per runtime, and migrations between frameworks no longer mean migrating memory.

Primitives for audit and erasure on a single substrate. Every record carries user, agent, thread, and timestamp scoping fields, and the SDK exposes search, list, and per-record delete operations across memories, threads, and messages, so callers can locate records for a subject and remove them on request. Oracle Database's native auditing covers the storage layer underneath. Compliance reviews land on a single substrate (one database with audit, retention, and access controls already in the data plane) rather than four services with four reviews.

One vendor relationship for production agent memory. A single Oracle AI Database instance carries vector search, structured state, JSON document retrieval, transactional consistency, and database-native audit. No second vector database to license, no third service to monitor and scale, no fourth backup pipeline to maintain.

Who Oracle AI Agent Memory Is For

Oracle AI Agent Memory is designed for:

AI Developers and engineers building production agents who need durable short-term and long-term memory in one place, with enterprise security and isolation.
Teams already running Oracle AI Database who want their agents to write to the same governed backend as the rest of the business
Technical leaders evaluating Oracle AI Database for agent memory infrastructure at scale, with compliance and audit requirements

Getting Started

Install the Oracle AI Agent Memory package:

pip install oracleagentmemory

A minimal end-to-end loop in Python looks like this:

from oracleagentmemory import AgentMemory

memory = AgentMemory.from_connection(
    connection_string="...",
    user_id="user_123",
)

# Add conversation turns to a short-term thread
thread_id = memory.create_thread(user_id="user_123")
memory.add_messages(thread_id, messages=[
    {"role": "user", "content": "I prefer vegan meals."},
    {"role": "assistant", "content": "Noted."},
])

# Extract durable long-term memories from the thread
memory.extract_memories(thread_id)

# Scoped search over long-term memory, enforced per-user
results = memory.search(
    user_id="user_123",
    query="dietary preferences",
    limit=5,
)

Code samples are illustrative; the final API surface is documented in the Oracle Help Center.

The quickstart notebook and framework how-to guides are available in the Oracle AI Developer Hub, and the full API reference is available in the Oracle Help Center.

Oracle AI Agent Memory is the first release of a broader commitment to a governed memory substrate enterprise agents need. Memory engineering is still an emerging discipline. The infrastructure behind it should not be.

16 Ways to Make a Small Language Model Think Bigger

Wojtek Pluta — Tue, 21 Apr 2026 07:56:58 +0000

This article is syndicated from the original post on blogs.oracle.com. Read the canonical version there for the latest updates.

All of the code in this article is available in the Oracle AI Developer Hub. The repository is part of Oracle’s open-source AI collection and serves as the reference implementation for everything covered here.

You can install it with pip install agent-reasoning, browse the 16 agent classes, run the TUI, or integrate it directly into an existing Ollama pipeline as a zero-change replacement client. If you find it useful, a GitHub star goes a long way.

Key Takeaways

Small language models struggle with complex reasoning on their own, but agent-based architectures (like Tree of Thoughts or Self-Consistency) can significantly improve their performance.
The agent-reasoning framework adds 16 research-backed reasoning strategies to any Ollama model using a simple +strategy tag—no code changes required.
Different strategies suit different tasks: CoT works well overall, ReAct excels with external data, and branching methods improve accuracy at the cost of speed.
Much of modern AI progress comes from orchestration (prompting, search, control flow), not just larger models.

Generally, a 270M parameter LLM (as of today, April 2026) struggles with even basic multi-step reasoning. Ask a model like gemma3:270m to solve the classic water jug problem, and it will often return a confidently incorrect answer—much like other small language models (SLMs) of similar size and training.

However, take that same model and wrap it inside a Tree of Thoughts (ToT) agent, running a breadth-first search (BFS) with three levels and weighted branches, and it can reliably solve the puzzle. The improvement comes from the architecture: the agent distributes the reasoning process across structured exploration steps, compensating for the limitations of a single LLM call.

This is where things get interesting. Much of the progress in applied AI isn't coming from bigger models alone, but from engineers rethinking how to orchestrate them—layering search, memory, and control flow on top of a standard LLM call to unlock new capabilities.

This is the fundamental idea behind agent-reasoning: sixteen cognitive architectures—each backed by peer-reviewed research—can be applied to any Ollama-served model via a simple +Strategy tag appended to the model name. Call gemma3:270m+tot instead of gemma3:270m, and the interceptor handles everything else.

We’ll talk about the different ways to invoke these reasoning strategies through the project.

What You’ll Learn

How the ReasoningInterceptor intercepts model names, removes the +Strategy tag, and directs traffic to one of 16 agent classes
How 16 strategies divide into four families: sequential, branching, reflective, and meta —each representing a different reasoning approach and set of trade-offs
What each major strategy accomplishes in practice, focusing on implementation rather than theory
Which type of problem each strategy is best suited for, based on benchmark results from March 2026

The Interception Layer

Key insight: The ReasoningInterceptor is an interchangeable drop-in client for Ollama that analyzes the model name for a +Strategy tag and directs traffic to one of 16 cognitive agent classes while making no modifications to your pre-existing code.

Everything relies on a single template: add +Strategy to any Ollama model name.

Using ReasoningInterceptor as a drop-in replacement client

The image below illustrates the entire routing process from start to finish. The interceptor acts as a middleman between your code and Ollama, removes the +Strategy tag, and sends traffic to the correct agent class.

Illustrating how the interceptor separates the base model from the Strategy tag

agent_map contains over fifty-five aliases mapped to sixteen agent classes. For example, cot, chain_of_thought, and CoT all map to CotAgent, while mcts and monte_carlo map to MCTSAgent. Because the interceptor is a drop-in client for Ollama—supporting the same .generate() and .chat() APIs— existing LangChain pipelines, web UIs, and scripts can automatically gain reasoning capabilities by changing a single string in the model name.

Additionally, the interceptor can be used as a network proxy. Instead of pointing an Ollama compatible application at http://localhost:11434, direct it to http://localhost:8080 instead. Using a model name like gemma3:270m+CoT, the gateway will apply reasoning transparently.

Family 1: Sequential Strategies

Key insight: Sequential Strategies process problems in a linear chain, where each step feeds into the next. In benchmarks, CoT achieved 88.7% average accuracy, compared to 81.3% for standard generation on the same model and weights.

Each of the sixteen strategies fall into one of four families. The diagram below illustrates how they are grouped.

Categorization of the four strategy families

Sequential strategies are designed for high-speed processing with minimal latency. They are ideal for problems with discrete, sequential steps.

Chain of Thought (CoT)

Paper: Wei et al. (2022), “Chain-of-Thought Prompting Elicits Reasoning in Large Language Models”

Chain of Thought (CoT) is a prompting strategy in which the model generates intermediate reasoning steps before producing a final response. As noted in the original paper: prompting a model to produce these intermediate steps can significantly improve accuracy.

For example, standard prompting on GSM8K achieves 66.7% accuracy. With CoT prompting, this increases to 73.3%— a 10% relative improvement achieved through simple prompt design alone.

The following graphic illustrates how CoT chains appear in practice: a sequence of numbered steps, each building on the previous one.

CoT in operation

In terms of implementation within CotAgent, the query is wrapped in a structured prompt:

Structured prompting enforces step-by-step reasoning in CoTAgent

Benchmark result for qwen3.5:9b (9.7B): CoT achieves 88.7% average accuracy, across GSM8K (math), MMLU (logic), and ARC-Challenge (reasoning), compared to 81.3% for standard generation. This seven-point gain in performance is attributable solely to structural prompts. Identical weights and temperatures were applied to both models.

Recommended usage: Math word problems; logic puzzles; any multi-step reasoning task where the individual steps are sequential and do not have branches.

Decomposed Prompting

Paper: Khot et al. (2022), “Decomposed Prompting: A Modular Approach for Solving Complex Tasks”

Decomposed prompting is an architectural module that splits large problems into smaller sub-problems. Each sub-problem is handled independently while carrying forward accumulated context from earlier steps. Once all sub-problems are processed, their outputs are synthesized into a final result. DecomposedAgent follows a three-phase process—decomposition, execution and synthesis—and propagating context throughout so that each step can build on prior results.

Recommended usage: Planning problems; trip itinerary generation; any problem where the ultimate answer consists of multiple distinguishable parts that may be individually addressed.

Note: Decomposed prompting achieved only 38.5% average accuracy in benchmark testing. This result requires context. GSM8K primarily evaluates arithmetic reasoning, where decomposing a problem like “what is 47 × 13 + 9?” introduces overhead without improving the model's ability to compute the answer.

Decomposition is more effective for problems with genuinely separable components (trip planning, multi-section reports etc.), where each part benefits from focused attention. These strengths are not captured by the benchmark, and the results reflect that mismatch.

Least-to-Most Prompting

Paper: Zhou et al. (2022), “Least-to-Most Prompting Enables Complex Reasoning in Large Language Models”

Least-to-most prompting is a strategy that orders sub-questions from simplest to most complex, establishing prerequisite knowledge before tackling harder steps. Unlike decomposed prompting which generates arbitrary sub-problems, it enforces a deliberate progression where each step builds on the last. Knowledge is accumulated iteratively until the model reaches the final question.

Recommended usage: Questions with genuine prerequisites — e.g., “what is x?” before determining “how does x relate to y?”; educational style explanation sequences (“concept ladder”); tasks that require establishing foundational concepts before addressing more complex components.

Family 2: Branching Strategies

Key insight: Branching strategies explore multiple reasoning paths simultaneously and choose the best path. ToT scored 76.7% on GSM8K math, compared to 66.7% on GSM8K math with standard generation.

More LLM calls mean higher latency— but often better answers on hard problems. Take this into consideration when running all branching strategies.

Tree of Thoughts (ToT)

Paper: Yao et al. (2023), “Tree of Thoughts: Deliberate Problem Solving with Large Language Models”

ToT is a search-based methodology that evaluates numerous possible reasoning paths concurrently, selecting the best performing path as determined by evaluation metrics such as distance traveled or quality of intermediate solutions etc.

Similar to chess engines, ToT applies BFS through an expanding tree of possible solutions. The core idea is straightforward: generate multiple partial solutions, evaluate them, prune weaker candidates, and continue exploring the most promising branches.

Below is an illustration of how ToT generates and eliminates branches: green nodes represent surviving branches, while red nodes indicate those that have been eliminated. The final answer is derived from the highest scoring leaf node.

A key design decision is how branches are evaluated. Should the same model handle both generation and scoring, or should a stronger model be introduced as a judge? In these benchmarks, the same model was used for both roles, but this is an area worth experimenting with, depending on your accuracy and latency constraints.

Generating candidate branches at each level

ToTAgent implements this as configurable by depth (default=3) and width (default=2 branches). At every level, the agent generates a set of candidate next steps, evaluates them using a scoring function, prunes low-scoring options, and expands the remaining candidates into the next level.

Tot achieved 76.7% accuracy—a 10% percent improvement over standard generation on GSM8K math problems. This performance comes at a cost: additional LLM calls are required at each step to evaluate candidate paths and their intermediate result, making it roughly 5-8x slower than CoT equivalent queries.

Recommended usage: Logic puzzles with multiple solution paths; strategic decision problems; tasks where multiple approaches can be explored and compared.

Self-Consistency (Majority Voting)

Paper: Wang et al. (2022), “Self-Consistency Improves Chain of Thought Reasoning in Language Models”.

Self-Consistency is a sampling method that generates multiple independent reasoning traces and selects a final answer through majority voting. Unlike standard prompting, it relies on sampling k diverse traces at a higher temperature to encourage variation. Each trace produces a candidate answer, and the most frequently occurring answer is selected as the final output.

The image below illustrates how both Self-Consistency and Monte Carlo Tree Search (MCTS) sample multiple reasoning paths, but differ fundamentally in how those paths are evaluated—majority voting versus UCB1-based exploration-exploitation balancing.

Self-Consistency vs MCTS comparison

ConsistencyAgent uses k=5 samples at temperature of 0.7 by default. It extracts final answers using regex-based pattern matching and selects the most frequent result via counter.most_common().

Self-Consistency matches CoT on both MMLU (96.7%) and GSM8K (76.7%). Its advantage lies in reliability rather than raw accuracy: majority voting across independent reasoning traces reduces the risk of single-trace errors propagating to the final answer.

Recommended usage: Factual question answering; multiple-choice style questions; problems where arriving at the correct answer via diverse reasoning paths is more important than inspecting a single reasoning trace.

Family 3: Reflective Strategies

Self-Reflection

Paper: Shinn et al. (2023), “Reflexion: Language Agents with Verbal Reinforcement Learning” — arXiv:2303.11366

Self-Reflection is a draft-critique-refine loop in which the model generates an initial answer, critiques it for errors, and then revises it. The Reflexion paper showed that this iterative process can meaningfully improve output quality, even without any gradient updates.

The image below shows all 3 reflective strategies side by side: Self-Reflection, Debate, and Refinement Loop.

Reflective strategies comparison

SelfReflectionAgent runs a draft-critique-refine loop for up to 5 iterations, with early termination when the critique returns “CORRECT” in under 20 characters. If the critique is satisfied on an early pass, subsequent iterations are skipped. This approach helps keeps latency low for queries the model answers correctly on the initial pass.

Recommended usage: Creative writing, high-stakes technical explanations, anything where “good enough on the first try” is insufficient.

Adversarial Debate

Paper: Irving et al. (2018), “AI Safety via Debate”

Irving proposed debate as a mechanism for improving AI safety. Two agents present opposing arguments, and a judge (either a human or another LLM) evaluates their merits. The underlying premise is that that identifying flaw in weak arguments is often easier than constructing strong ones.

DebateAgent conducts multiple rounds of PRO and CON arguments, with a judge evaluating each exchange. Following all rounds, the strongest arguments from both sides are synthesized into a final answer that balances competing perspectives. Context is carried forward between rounds, enabling incremental refinement rather than redundant arguments.

Recommended usage: Controversial or ambiguous subjects; policy analysis; ethics and any subject matter requiring a balanced perspective.

Refinement Loop

Paper: Madaan et al. (2023), “Self-Refine: Iterative Refinement with Self-Feedback”

This paper describes a refinement loop similar to self-reflection, but instead of relying on a human-style critique to guide revisions, it uses a machine-based evaluation system with quantifiable quality metrics. These metrics determine whether further refinement is necessary. The loop terminates when a predefined quality metric is reached (> 0.9 by default) or when the maximum number of iterations is exceeded.

The five-stage complex refinement pipeline consists of sequential stages, each focused on a distinct type of critique: technical accuracy, structure, depth, examples, and polish.

Each stage targets a distinct aspect of quality, ensuring the model focuses exclusively on improving that dimension rather than attempting to optimize everything at once.

Recommended usage: Highly technical writing; documentation; blog posts, a scenario where production-quality output is required rather than simply a first draft.

Family 4: Cross-Domain and Meta Strategies

Key insight: Cross-domain strategies enable sharing knowledge among disciplines, while meta-strategies automatically route queries to the most appropriate reasoning technique without requiring manual selection.

Analogy-Based Reasoning

Paper: Gentner (1983), “Structure Mapping: A Theoretical Framework for Analogy”, Cognitive Science

Gentner's structure-mapping theory proposes that analogical reasoning operates by identifying structural correspondences across domains, rather than relying on surface-level similarity. The AnalogicalAgent builds on this idea through three phases: (1) identify the underlying structure independent of domain specifics, (2) generate analogous solutions from different domains that share that structure, (3) select the most effective analogy and apply its solution approach.

This process reduces reliance on memorized patterns. By focusing on underlying structure, the model learns why a solution works, rather than simply recalling what worked before.

Recommended usage: Solving problems that are structurally similar to prior ones, even if they differ superficially; transferring knowledge across domains; explaining complex concepts through analogy.

Socratic Questioning

Paper: Paul & Elder (2007), “The Art of Socratic Questioning”

The Socratic Method: Do not answer the question directly. Instead, ask follow-up questions that reduce ambiguity in the solution space.

SocraticAgent repeatedly asks questions and receives model responses, continuing until it reaches a limit of five question-response exchanges. It then synthesizes the collected information into a final answer. A deduplication or normalization step helps prevent repeated queries that differ only in wording.

Recommended for: Philosophy; ethics; deep technical knowledge; any field requiring the model to “know” something as opposed to merely answering it.

ReAct (Reason + Act)

Paper: Yao et al. (2022), “ReAct: Synergizing Reasoning and Acting in Language Models”

ReAct is a conceptual framework that interweaves reasoning steps with tool invocations, allowing the model to ground its thinking in external information. In practice, the model decides what action to take, calls a tool such as a web search engine, examines the result, updates its reasoning, and repeats the cycle until it reaches a satisfactory answer. Current tools include web scraping, accessing Wikipedia via an API call, and a calculator interface, with mock-ups available for off-line execution scenarios.

Using ReAct achieved 70.0% accuracy on ARC-Challenge (Science Reasoning). While not the highest on this particular benchmark, it enabled tool use for the LLM and allowed it to search for required information on the Internet.

Recommended usage: Fact-checking; current events queries; mathematical calculations; tasks where access to grounded, external information is important.

Auto Router: MetaReasoningAgent

Key insight: A single LLM invocation allows MetaReasoningAgent to classify each input into one of eleven categories and route it to the most appropriate strategy, without human intervention.

All sixteen strategies depend on selecting the appropriate strategy for a given task. By removing this requirement, MetaReasoningAgent eliminates the need for manual selection.

The diagram below shows how each category maps to its corresponding strategy.

MetaReasoningAgent classification diagram

MetaReasoningAgent instantiates the selected strategy class and passes control to it, along with all event objects for visualization.

To use this capability, specify a model such as gemma3:270m+meta or gemma3:270m+auto.

In practice, routing is generally intuitive: math problems are directed to CoT, logic puzzles to ToT, philosophical questions to Socratic Questioning, and controversial topics to Adversarial Debate.

The trade-off is reduced control over strategy-specific hyperparameters in exchange for automatic routing aligned with the problem type.

What Strategy Should You Pick? Benchmark Results (March 2026)

Key insight: CoT performs best on average (88.7%) across diverse tasks. ReAct excels when tool use is available (70.0% on ARC-Challenge). ToT and Self-Consistency tie on GSM8K math at 76.7%.

These results are based on 4,200 evaluations across 11 strategies using qwen3.5:9b, collected as of March 2026. All 16 strategies are implemented and production-ready. However, the benchmarks shown below focus on the 11 that produce a single extractable answer. The remaining five are generation-focused and not suited to multiple-choice evaluation.

The heat map and bar chart below provide a complete view of the results.

Benchmark results heatmap and bar chart

The short version: CoT wins on average across diverse tasks. Self-Consistency and ToT beat it on specific math benchmarks. ReAct dominates on factual/science tasks. Self-Reflection and Refinement Loop are not well captured by these benchmarks, as they primarily improve generation quality rather than multiple-choice accuracy.

For most queries, start with +cot. If you’re solving logic puzzles or planning problems, try +tot. If you need factually grounded responses, use +react. If you need polished, high-quality output rather than a quick answer, use +refinement. When in doubt, +meta will route the query automatically.

In my experience building agent-reasoning, the most surprising finding is how much prompt structure alone can improve performance. For example, qwen3.5:9b improves from 81.3% to 88.7% average accuracy simply by prompting it to produce numbered reasoning steps.

As of March 2026, all 16 strategies are production-ready and have been evaluated across 4,200 benchmark runs.

You can find the repository here. Install with pip install agent-reasoning or uv add agent-reasoning. The commands to get started:

Getting started commands

The TUI provides a 16-agent sidebar, live streaming, and a step-through debugger. Arena mode runs all 16 agents simultaneously on the same query in a 4×4 grid.

If this is useful, a GitHub star is always appreciated.

Frequently Asked Questions

Do I need to modify my existing code to use agent-reasoning?

No. The interceptor is a drop-in replacement for the Ollama client. Just change the model name string by appending +strategy (e.g., gemma3:270m+cot) and the interceptor handles everything else. Existing LangChain pipelines, web UIs, and scripts work without any other changes.

Which strategy should I start with?

Start with +cot (Chain of Thought). It scored the highest average accuracy (88.7%) across our benchmarks and adds minimal latency. If you are unsure, use +meta and let the auto-router pick the best strategy for you.

Why were only 11 of the 16 strategies benchmarked?

The benchmarks (GSM8K, MMLU, ARC-Challenge) measure multiple-choice accuracy, which works well for strategies that produce a single extractable answer. The remaining five strategies are generation-focused (e.g., Refinement Loop, MCTS) and their strengths in output quality are not captured by multiple-choice evaluations. All 16 strategies are fully implemented and production-ready.

Can I use this with models other than Ollama-served models?

Currently the interceptor targets the Ollama API. Since it exposes the same .generate() and .chat() endpoints, any Ollama-compatible client works out of the box. Support for additional inference backends is on the roadmap.

How much slower are branching strategies compared to CoT?

ToT is roughly 5-8x slower than CoT because it generates and evaluates multiple candidate branches at each level. Self-Consistency (k=5 samples) adds similar overhead. For latency-sensitive applications, stick with sequential strategies (CoT, Least-to-Most) and reserve branching strategies for problems where accuracy matters more than speed.

Created by Nacho Martinez, Data Scientist at Oracle. Find Nacho on GitHub and LinkedIn, or visit the Oracle AI Developer page for more resources.

Agent Memory: Why Your AI Has Amnesia and How to Fix It

Wojtek Pluta — Fri, 17 Apr 2026 09:08:10 +0000

Key Takeaways

Today's AI agents forget everything between conversations. Every interaction starts from zero, with no recall of who you are or what you've discussed before.
Agent memory isn't about bigger context windows. It's about a persistent, evolving state that works across sessions.
The field has converged on four memory types (working, procedural, semantic, episodic) that map directly to how human memory works.
Building agent memory at enterprise scale is fundamentally a database problem. You need vectors, graphs, relational data, and ACID transactions working together.

What Is Agent Memory and Why Does Your AI Agent Need It?

You've spent weeks building an AI customer service agent. It handles complaints, processes refunds, even cracks the occasional joke when the moment's right. A customer calls back the next day, and your agent has no idea who they are. The conversation from yesterday? Gone. The preference they mentioned twice last week? Never happened. Every single interaction starts from scratch.

This isn't a bug in your code. It's a fundamental design problem in how we build AI agents today.

LangChain put it well: 'Imagine if you had a coworker who never remembered what you told them, forcing you to keep repeating that information'. In the coworker scenario, that’s frustrating, and for AI applications, forgetfulness, that’s a dealbreaker.

At Oracle, we've been deep in this problem as we continue to provide support to customers building AI applications. And here's what we've found: the solution isn't bigger context windows or more verbose prompts. It's a proper memory infrastructure. The kind that databases have been providing for decades.

Agent memory is the composition of system components and infrastructure layer that gives AI agents a persistent, evolving state across conversations and sessions. It enables agents to store, retrieve, update, and forget information over time: learning user preferences, retaining context from past interactions, and adapting behavior based on accumulated experience. Without it, every interaction starts from zero.

This article breaks down what agent memory actually is, how it works under the hood, the frameworks shaping the field, and guidance on how to build it for production. Whether you're prototyping your first agent or scaling one to thousands of users, this is the foundation you need to get right.

Why Bigger Context Windows Aren't the Answer

The rapid expansion of context windows, now ranging from hundreds of thousands to millions of tokens, has created a convincing illusion across the industry: that with this much capacity available, the memory problem is effectively solved and retrieval-based mechanisms are behind us. That assumption is wrong.

The industry calls it 'the illusion of memory'. Stuffing more tokens into a prompt isn't memory. It's a bigger Post-it note: more space to scribble on, but it still goes in the bin when the conversation ends. Memory means the notes survive. Here's why that distinction matters:

Context windows degrade before they fill up. Most models break well before their advertised limits. A model claiming 200K tokens typically becomes unreliable around 130K, with sudden performance drops rather than gradual degradation.

There's no sense of importance. Context windows treat every token equally. Your name gets the same weight as a throwaway comment from three weeks ago. There's no prioritisation, no salience, no relevance filtering.

Nothing persists. Close the session and it's all gone. Every conversation starts from zero.

The cost scales linearly. Maintaining full context across a long agent lifetime gets expensive fast. You're paying per token, and most of those tokens are irrelevant noise.

Memory is not only about storing chat history or passing more tokens into the context window. It's about building a persistent state stored in an external system, that evolves and informs every interaction the agent has, even weeks or months apart.

Another misconception to address early on is that RAG (retrieval augmented generation) is agent memory. RAG brings external knowledge into the prompt at inference time. It's great for grounding responses with facts from documents. But RAG is fundamentally stateless. It has no awareness of previous interactions, user identity, or how the current query relates to past conversations. Memory brings continuity. Put simply: RAG helps an agent answer better. Memory helps it learn and adapt. You need both.

The Concept: A Mental Model for Agent Memory

Let me give you a framework that makes all of this click. It maps directly to how your own brain works.

In 2023, researchers at Princeton published the CoALA framework (Cognitive Architectures for Language Agents). It defines four types of memory, drawn from cognitive science and the SOAR architecture of the 1980s. Every major framework in the field builds on this taxonomy, and it answers a fundamental question: what options are available for adding memory to an AI agent?

Memory Type	Human Equivalent	What It Does in an Agent	Example Implementation
Working Memory	Your brain's scratch pad: holding what you're actively thinking about	Current conversation context, retrieved data, intermediate reasoning	Conversation buffers, sliding windows, rolling summaries, scratchpads
Procedural Memory	Muscle memory: knowing how to ride a bike without thinking	System prompts, agent code, decision logic	Prompt templates, tool definitions, agent configs
Semantic Memory	General knowledge: facts and concepts accumulated over your lifetime	User preferences, extracted facts, knowledge bases	Vector stores with similarity search
Episodic Memory	Autobiographical memory: recalling specific experiences from your past	Past action sequences, conversation logs, few-shot examples	Timestamped logs with metadata filtering

Think of it this way. When you're in a meeting, your working memory holds what's being discussed right now. Your procedural memory knows how to take notes and when to speak up. Your semantic memory reminds you that Sarah's team prefers Slack over email. Your episodic memory recalls that the last time you proposed this feature, the VP shut it down because of budget constraints.

An agent needs all four types working together. Most agents today only have working memory: whatever fits in the current context window. That's like trying to do your job using nothing but a whiteboard that gets wiped clean every evening.

Lilian Weng's influential formula captures the big picture simply:

Agent = LLM + Memory + Planning + Tool Use.

Her short-term memory maps to CoALA's working memory. Her long-term memory encompasses the other three types.

LangChain adds a practical layer with two approaches to memory updates:

Hot path memory: the agent explicitly decides to remember something before responding. This is what ChatGPT does. It adds latency but ensures immediate memory updates.
Background memory: a separate process extracts and stores memories during or after the conversation. No latency hit, but memories aren't available straight away.

The key insight: memory is application-specific. What a coding agent remembers about a user is very different from what a research agent might store.

Letta (formerly MemGPT) takes a different angle entirely, borrowing from operating systems. Treat the context window like RAM and external storage like a disk. The agent pages data between these tiers, creating a 'virtual context' that feels unlimited. The agent manages its own memory using tools: it decides what to remember, what to update, and what to archive.

The distinction between programmatic memory (developer decides what to store) and agentic memory (the agent itself decides) matters. The field is moving towards the latter. Agents that manage their own memory adapt to individual users without requiring developer intervention for each new use case. The decision as to which memory operations are programmatic and agent triggered isn’t always as clear cut, and we’ve seen various approaches work well in certain use cases and domains. In a future post, we will go into the common patterns and design principles of memory engineering.

Referring back to the customer service agent from the start of this article. Customer service is the most common use case for agents in production (26.5% of deployments, per LangChain's 2025 industry survey), and it demands all four memory types working together. Episodic memory recalls past tickets and interactions. Semantic memory stores customer preferences and account details. Working memory tracks the live conversation. Procedural memory encodes resolution workflows and escalation rules. All four memory types enable the chatbot to perform well on continuous tasks and adapt to new information.

The Landscape: Frameworks and Open-Source Libraries

What are the commonly used libraries and open-source projects for agent memory? The ecosystem has matured quickly. Here are the projects shaping how developers build agent memory today.

Project	What It Does	Open Source
LangChain / LangMem / LangGraph	Agent orchestration with built-in memory abstractions. Hot path and background memory. LangMem SDK handles extraction and consolidation.	Yes
Letta (MemGPT)	Stateful agent platform with OS-inspired memory hierarchy. Agents self-edit their own memory via tool calls.	Yes
Zep / Graphiti	Temporal knowledge graphs for relationship-aware memory. Bi-temporal modelling with sub-200ms retrieval.	Yes (Graphiti)
Mem0	Self-improving memory layer with vector and graph architecture. Automatic memory extraction and conflict resolution.	Yes
langchain-oracledb	Official LangChain integration for Oracle Database. Vector stores, hybrid search, and embeddings with enterprise-grade security.	Yes

The orchestration library matters, but at scale, the storage backend matters more. Most of these frameworks are database-agnostic by design. The question isn't which framework to use. It's what database sits underneath it.

The Deep Dive: How Agent Memory Actually Works

What are the common storage options for agent memory? Production systems today use three paradigms working together. You need to understand all three.

Vector stores for semantic memory

This is the most common approach. You take text, convert it to embeddings (typically 128 to 2,048 dimensions depending on embedding model utilised), and store them in a vector database. Retrieval works through vector search, against vectors that are indexed using HNSW (hierarchical navigable small world); typically we find the memories (embeddings in database) that are semantically closest to the current query.

It's fast and simple but limited. Vector search captures semantic similarity well, yet misses structural relationships.

Knowledge graphs for relationship memory

Vector search can tell you that a user mentioned coffee. But it can't tell you that they prefer a specific shop, ordered last Tuesday, and always get oat milk. That chain of connections (person, preference, place, time, detail) is a graph problem.

Knowledge graphs store facts as entities and relationships, with edges capturing how they connect. Add bi-temporal modelling (tracking both when events happened and when the system learned about them) and you can ask not just 'what do we know?' but 'what did we know at any point in time?'

Frameworks like Zep's Graphiti implement this pattern, achieving 94.8% accuracy on the Deep Memory Retrieval benchmark. Oracle Database supports property graphs natively through SQL/PGQ, so graph queries run inside the same engine as your vector search and relational data.

Structured databases for factual memory

Relational databases store the structured data: user profiles, access controls, session metadata, audit logs. As Cognee puts it: 'Vectors deliver high-recall semantic candidates (what feels similar), while graphs provide the structure to trace relationships across entities and time (how things relate).' Relational tables anchor both with the transactional guarantees that production systems demand.

Why does a converged database change the equation?

Most teams stitch this together with separate databases: Pinecone for vectors, Neo4j for graphs, Postgres for relational data. Three security models, three failure modes, no shared transaction boundaries. If one write fails, your agent's memory is in an inconsistent state.

Oracle's converged database runs all three paradigms natively inside a single engine:

AI Vector Search for embedding storage and similarity retrieval
SQL/PGQ for property graph queries across entity relationships
Relational tables for structured data, metadata, and audit trails
JSON Document Store for flexible, schema-free memory objects

All four share the same ACID transaction boundary and the same security model. Row-level security, encryption, and access controls apply uniformly across every data type. One engine, one transaction, one security policy: the three paradigms above become three views of the same underlying data.

The Four Memory Operations

Every memory system runs on four core operations:

ADD: Store a completely new fact
UPDATE: Modify an existing memory when new information complements or corrects it
DELETE: Remove a memory when new information contradicts it
SKIP: Do nothing when information is a repeat or irrelevant

Modern memory systems delegate these decisions to the LLM itself rather than using brittle if/else logic. The extraction phase ingests context sources (the latest exchange, a rolling summary, recent messages) and uses the LLM to extract candidate memories. The update phase compares each new fact against the most similar entries in the vector database, using conflict detection to determine whether to add, merge, update, or skip.

Retrieval: how agents recall

Due to the heterogenous nature of data that agents encounter, production systems combine multiple retrieval approaches:

Semantic search: vector similarity (cosine distance) for meaning-based matching
Temporal search: bi-temporal models enable point-in-time queries ('What did the user prefer last March?')
Graph traversal: multi-hop queries across knowledge graph edges for complex reasoning
Hybrid retrieval: combining keyword (full-text) and semantic (vector) search in a single query, which is critical for retrieving specific facts like names, dates, or project codes alongside conceptually related memories

Forgetting: the underrated operation

Effective forgetting can be implemented with decay functions applied to vector relevance scores: by analysing the results of vector search, old and unreferenced embeddings naturally fade from the agent's attention, imitating biological human memory decay patterns. In a database, this is straightforward. A recency-weighted scoring function multiplies semantic similarity by an exponential decay factor based on time since last access. The result: memories that haven't been recalled recently lose salience gradually, just like human recall.

Some systems take a different approach entirely. Old facts are invalidated but never discarded, preserving historical accuracy for audit trails. The right strategy depends on your use case, but both are fundamentally database operations.

The Enterprise Reality: What Changes at Scale

Here's where the gap between demo and production becomes a chasm.

KPMG's Pulse Survey of 130 C-suite leaders (all at companies with over $1B revenue) found that 65% cite agentic system complexity as the top barrier for two consecutive quarters. Agent deployment has more than doubled, from 11% in Q1 2025 to 26% in Q4 2025, but that still means three quarters of large enterprises haven't deployed. McKinsey puts it even more starkly: only 1% of leaders describe their companies as 'mature' in AI deployment.

The problems that surface at scale are database problems. They've been database problems all along.

Security and isolation. Memory must be scoped per user, per team, per organisation. Memory poisoning is a real attack vector: adversaries can inject malicious information into an agent's memory to corrupt future decision-making. You need row-level security, not just namespace-level isolation.

Multi-tenancy. Agents serving multiple organisations need complete data isolation. Most vector-only databases offer namespace-level separation. That's not the same as the row-level security that regulated industries require. Oracle's native PDB/CDB architecture provides inherent multi-tenant isolation.

Compliance is getting complex. GDPR's right to be forgotten applies to explicit agent memory stores. But the EU AI Act (fully applicable from August 2026) requires 10-year audit trails for high-risk AI systems. Think about that tension: you need to delete personal data on request while maintaining a decade of audit history. That requires architectural sophistication that most startups are only beginning to address.

ACID transactions matter. Agent memory operations often touch multiple data types simultaneously. Updating a vector embedding, modifying a graph relationship, and changing relational metadata must all succeed or all fail. Without atomicity, partial memory updates leave your agent in an inconsistent state.

These aren't theoretical concerns. They're the reasons three quarters of enterprises are still stuck at the pilot stage.

The Implementation: Building Agent Memory with LangChain and Oracle

Let's get practical. We'll use LangChain as our orchestration framework and Oracle Database as the memory backend, using the langchain-oracledb package. Here's how quickly you can go from zero to a working memory system.

pip install langchain-oracledb oracledb langchain-core

Connect and create a vector store

This is all it takes to set up a production-ready vector store backed by Oracle:

import oracledb
from langchain_oracledb.vectorstores import OracleVS

# Create a connection pool (production-ready)
pool = oracledb.create_pool(
 user="agent_user", password="password",
 dsn="hostname:port/service",
 min=2, max=10, increment=1
)

# Initialise vector store for semantic memory
semantic_memory = OracleVS(
 client=pool.acquire(),
 embedding_function=embeddings, # any LangChain-compatible embeddings
 table_name="AGENT_SEMANTIC_MEMORY",
 distance_strategy=DistanceStrategy.COSINE,
)

That's your semantic memory store. Oracle handles the vector indexing, ACID transactions, and security natively. No separate vector database needed.

Store and retrieve a memory

The core pattern is simple: write memories with metadata, retrieve them with similarity search.

# Store a memory
semantic_memory.add_texts(
 texts=["User prefers dark mode and concise responses."],
 metadatas=[{"user_id": "user_123", "memory_type": "preference"}]
)

# Retrieve relevant memories
results = semantic_memory.similarity_search(
 "What are this user's preferences?",
 k=5,
 filter={"user_id": "user_123"}
)
for doc in results:
 print(doc.page_content)

From here, you can create separate vector stores for each memory type (semantic, episodic, procedural) under the same Oracle instance, all sharing the same security policies and transaction guarantees.

Go deeper: the full memory engineering notebook

The snippets above show the building blocks, but a production agent memory system needs considerably more. We've published a complete, runnable notebook in the Oracle AI Developer Hub that implements the full architecture discussed in this post. This notebook builds a complete Memory Manager with six distinct memory types, each backed by Oracle:

Memory Type	Purpose	Storage
Conversational	Chat history per thread	SQL Table
Knowledge Base	Searchable documents and facts	SQL Table + Vector Enabled
Workflow	Learned action patterns	SQL Table + Vector Enabled
Toolbox	Dynamic tool definitions with semantic retrieval	SQL Table + Vector Enabled
Entity	People, places, systems extracted from context	SQL Table + Vector Enabled
Summary	Compressed context for long conversations	SQL Table + Vector Enabled

It also covers context engineering (monitoring context window usage, auto-summarisation at thresholds, just-in-time retrieval), semantic tool discovery (scaling to hundreds of tools while only passing the relevant ones to the LLM), and a complete agent loop that ties everything together.

Run the notebook: oracle-devrel/oracle-ai-developer-hub

The Perspective: Where This Is Heading

Here's what I think is coming, and where I'm still working things out.

Sleep-time computation will change the game. The idea is simple: agents that 'think' during idle time (reorganising, consolidating, refining their memories) perform better and cost less at query time. OpenAI's internal data agent already runs this pattern in production. Their engineering team describes a daily offline pipeline that aggregates table usage, human annotations, and code-derived enrichment into a single normalised representation, then converts it into embeddings for retrieval. At query time, the agent pulls only the most relevant context rather than scanning raw metadata.

Letta's research puts numbers to it: agents using this approach achieve 18% accuracy gains and 2.5x cost reduction per query. We're going to see a clear separation between 'thinking agents' that run in the background and 'serving agents' that handle real-time interactions. That's a pattern databases have supported forever: batch processing alongside real-time queries.

Memory will extend naive RAG implementations. The spectrum is already shifting: traditional RAG to agentic RAG to full memory systems. VentureBeat predicts that contextual memory will surpass RAG for agentic AI in 2026. I think that's right. RAG retrieves documents. Memory understands context. The agents that win will do both, but memory will be the differentiator.

The convergent database will become non-negotiable. Agent memory needs vectors, graphs, relational data, and temporal context working together. Stitching together separate databases for each type creates brittle systems with security gaps and consistency problems. I'm still figuring out exactly how fast this consolidation will happen, but the direction is clear.

One open question remains, and that is the pace at which enterprises will transition from pilot to production deployment. At the moment the technology is at a clear stage of maturity and architectural design patterns are proven and battle tested. On the other hand, organisational readiness, encompassing governance, infrastructure modernisation, and cross-functional alignment, is a fundamentally different challenge.

What is clear: agent memory is, at its foundation, a database problem. And building databases for mission-critical workloads is what Oracle has been doing for nearly five decades.

Frequently Asked Questions

What are the main types of agent memory used in AI systems?

The field has converged on four types, drawn from cognitive science: working memory (current conversation context), procedural memory (system prompts and decision logic), semantic memory (accumulated facts and user preferences), and episodic memory (past interaction logs and experiences). Every major framework builds on this taxonomy, first formalised in the CoALA framework from Princeton in 2023.

What options are available for adding memory to an AI agent?

Two broad approaches exist. Programmatic memory is where the developer defines what gets stored and retrieved. Agentic memory is where the agent itself decides what to remember, update, and forget using tool calls. Frameworks like Letta (formerly MemGPT) and LangChain's LangMem SDK support both patterns. The field is moving towards agentic memory, where agents manage their own state without developer intervention for each new use case.

What are common agent memory storage options?

Production systems typically combine three paradigms: vector stores for meaning-based retrieval (storing embeddings and querying by cosine similarity), knowledge graphs for relationship-aware retrieval (entities, edges, and bi-temporal modelling), and structured relational databases for transactional data like user profiles, access controls, and audit logs. Most teams stitch these together with separate databases, though converged databases like Oracle can run all three natively in a single engine.

What techniques allow AI agents to forget or selectively erase memory?

The most common approach uses decay functions applied to vector relevance scores: a recency-weighted scoring function multiplies semantic similarity by an exponential decay factor based on time since last access. Memories that haven't been recalled recently lose salience gradually, mimicking biological memory decay. An alternative approach invalidates old facts without discarding them, preserving historical accuracy for audit trails while removing them from active retrieval.

What are the differences between short-term and long-term agent memory?

Short-term memory (also called working memory) is the current context window: whatever the agent is actively reasoning about in this conversation. It's fast but volatile; close the session and it's gone. Long-term memory encompasses everything that persists across sessions: semantic memory (facts and preferences), episodic memory (past interactions), and procedural memory (learned behaviours and decision logic). Long-term memory requires external storage and retrieval infrastructure.

What are commonly used libraries for agent memory?

The ecosystem includes LangChain/LangMem (hot path and background memory with extraction and consolidation), Letta/MemGPT (OS-inspired memory hierarchy where agents self-edit memory via tool calls), Zep/Graphiti (temporal knowledge graphs with sub-200ms retrieval), Mem0 (self-improving memory with automatic conflict resolution), and langchain-oracledb (Oracle Database integration for vector stores, hybrid search, and embeddings with enterprise-grade security).

How do I store and query vector embeddings?

The core pattern is straightforward: convert text into embeddings (typically 128 to 2,048 dimensions), store them in a vector-capable database, and retrieve them using cosine similarity search. With langchain-oracledb and Oracle Database, you initialise a vector store, add texts with metadata (such as user ID and memory type), then query with similarity_search() filtered by metadata. Oracle handles vector indexing, ACID transactions, and security natively.

Which databases offer vector search capabilities for enterprises?

Several databases now support vector search, but enterprise requirements go beyond basic similarity queries. You need ACID transactions, row-level security, multi-tenancy, and compliance features alongside your vector operations. Oracle Database provides native AI Vector Search within its converged architecture, meaning vector queries run in the same engine as relational tables, property graphs (SQL/PGQ), and JSON document stores, all sharing a single transaction boundary and security model.

What Is the AI Agent Loop? The Core Architecture Behind Autonomous AI Systems

Wojtek Pluta — Fri, 17 Apr 2026 09:05:55 +0000

Key Takeaways

The architectural difference between a chatbot and an AI agent is one pattern: the agent loop. It’s an LLM invoking tools inside an iterative cycle, repeating until the task is complete or a stopping condition is reached.
A chatbot responds in a single pass. An agent persists, adapts, and acts across multiple steps: perceiving its environment, reasoning over available options, executing an action, and observing the result before deciding what comes next.
Every major AI company (OpenAI, Anthropic, Google, Microsoft, Meta) has converged on this same core pattern, despite building very different products around it.
Building agent loops for production requires engineering for two constraints: cost, where agents consume approximately 4x more tokens than standard chat interactions and up to 15x in multi-agent systems, and observability, the ability to trace every reasoning step, tool call, and decision across an iterative execution cycle.

What Is the AI Agent Loop and Why Should You Care?

The five-stage agent loop: Perceive, Reason, Plan, Act, Observe

You have built a chatbot. It works. Users ask a question, it generates a response, and the interaction is complete. Then someone asks it to do something that requires more than one step.

‘Find me the three cheapest flights to Tokyo next month, check if my loyalty points cover any of them, and book the best option’. The chatbot has no mechanism to proceed. It generates a response and stops. It can answer questions about flights. It can explain how loyalty points work. It cannot execute the workflow. The interaction is stateless. Each prompt is processed in isolation, with no persistent context, no access to intermediate results, and no ability to chain decisions across steps.

This is not a limitation of the model. Chat-GPT, Claude, and Gemini are all capable of reasoning through multi-step problems. The limitation is architectural. A chatbot is built to respond. An agent is built to act.

The difference is one while loop.

What is the Agent Loop?

The AI agent loop is the iterative execution cycle at the core of every agentic AI system. At each iteration, the agent assembles context from available inputs, invokes an LLM to reason and select an action, executes that action, observes the outcome, and feeds the observation back into the next iteration. This process repeats until the task is complete or a defined stopping condition is reached.

Across the engineering teams Oracle works with building AI applications, one architectural pattern consistently separates working prototypes from production-grade systems: the agent loop. It’s the architecture that transforms a language model from a text generation system into one that can take actions, adapt to results, and complete multi-step tasks autonomously.

This article examines the agent loop architecture: what it is, how it works, why every major AI company has converged on the same core pattern, and what is required to build one that holds up in production.

All code in this article is available as a runnable companion notebook in the Oracle AI Developer Hub on GitHub. Follow along step by step or execute the full implementation end to end.

Why Single-Pass Responses Hit a Wall

Single-pass chatbot vs. iterative agent loop: one response versus continuous execution until task completion

The standard chatbot interaction follows a simple pattern: user sends message, model generates response, done. One input, one output, no state between turns. It works brilliantly for question-answering, summarisation, and creative writing. It falls apart from the moment you need the model to do something in the real world.

A single-pass response has three fundamental constraints:

It cannot iterate on results. A single-pass system can execute a tool call within a turn, but it has no mechanism to evaluate whether that action succeeded, adapt based on the outcome, or chain a subsequent decision from the result. There is no feedback loop.
It cannot recover from failure. Without iterative execution, a failed tool call, an empty result set, or an ambiguous API response cannot trigger a revised strategy. The model has no visibility into downstream outcomes.
It cannot decompose dependent tasks. Real-world workflows require gathering information, making decisions based on that information, executing actions, and handling the consequences of those actions. Each step depends on the result of the previous one. That is a loop, not a straight line.

Russell and Norvig defined an agent back in 1995 as 'anything that can be viewed as perceiving its environment through sensors and acting upon that environment through actuators.' That definition is 30 years old and it still holds. The key word is acting. Not responding. Acting.

The ReAct framework from Princeton and Google Research (Yao et al., 2022) made this practical for LLMs by interleaving reasoning with action in a single prompt-driven loop. The results demonstrated that models perform significantly better when they can reason, act, observe, and reason again: a 34% improvement on ALFWorld and 10% on WebShop. Single-pass responses are not just architecturally limiting. They leave measurable performance on the table.

The Agent Loop: A Mental Model

The five-stage agent loop: Perceive, Reason, Plan, Act, Observe

The agent loop operates across five stages that repeat until the task is complete or a stopping condition is met:

Perceive: The agent receives input. This could be a user message, an API response, an error, or the result of its last action.
Reason: The LLM processes everything in context and decides what to do next.
Plan: For complex tasks, the agent decomposes the objective into discrete subtasks before execution. Simpler workflows proceed directly to the Act stage without a dedicated planning step.
Act: The agent executes something: a tool call, an API request, a database query, a code execution.
Observe: The agent examines the result. Did it work? Is the task complete? Does the plan need adjusting?

Then, it loops back to step 1.

In pseudocode, the complete pattern reduces to six lines:

while not done:
   response = call_llm(messages)
   if response has tool_calls:
      results = execute_tools(response.tool_calls)
      messages.append(results)
   else:
      done = True
      return response

This execution pattern underpins every autonomous AI system currently in production. It is the foundation on which every major AI organisation has built its agentic architecture. Anthropic's engineering guidance describes the pattern plainly: agents are often just LLMs using tools based on environmental feedback in a loop.

When the Agent Loop Is Not the Right Architecture

The agent loop is not the appropriate architecture for every use case. Before building an agentic system, validate that the workflow requires iterative execution.

Agent loops are well-suited to tasks where the number of required steps cannot be predicted in advance, where the agent must adapt based on intermediate results, and where the cost of latency is acceptable relative to the value of task completion.

Workflows that follow a fixed, predictable sequence of steps are better served by deterministic pipelines. Single-step tasks that require one LLM call and one tool invocation do not benefit from the overhead of an agent loop. Tasks where latency is the primary constraint should be evaluated carefully, as each loop iteration adds LLM call latency.

The principle from both OpenAI and Anthropic's published guidance is consistent: start with the simplest architecture that solves the problem. Introduce the agent loop only when iterative reasoning and adaptive tool use are required.

How Every Major AI Company Converged on the Same Architecture

Six major AI organisations, one underlying architecture: LLM plus tools in a loop

Despite differences in SDK design, nomenclature, and architectural philosophy, every major AI organisation has converged on the same underlying execution pattern. The table below maps each implementation against the five stages of the core loop:

Company	What they call it	Core pattern	Key contribution
OpenAI	Agent Loop	Tool-calling loop via Codex SDK	Code-first approach; anti-declarative-graph philosophy
Anthropic	Agent loop	Augmented LLM + tools in loop	Simplicity-first design; workflows vs. agents distinction
Google	Orchestration layer	ReAct (Thought-Action-Observation)	Invented Chain-of-Thought and co-created ReAct
Microsoft	Think-Act-Learn	Conversation-driven loop	Dual-loop ledger planning (Magentic-One)
Meta	Agent loop	ReAct via Llama Stack	Open-source building blocks; security-first ('Rule of Two')
LangChain	Agent executor / StateGraph	Tool-calling state machine	Graph-based orchestration; middleware hooks for control

The implementations differ in naming conventions, SDK design, and architectural philosophy. The execution pattern is identical.

Lilian Weng's formula captures it simply: Agent = LLM + Memory + Planning + Tool Use. The agent loop is the runtime that ties those four components together.

How the Agent Loop Actually Works

Three iterations, three tool calls, one complete response.

The canonical pattern is ReAct: reasoning interleaved with acting. The model does not simply select a tool. It reasons about why that tool is appropriate, executes the call, processes the result, and reasons again.

To illustrate how the loop executes in practice, consider the following task: identify the most cited paper on agent memory published in 2026 and summarise its key findings.

Iteration 1 (Reason → Act → Observe):

The agent reasons that it needs to search for papers on agent memory from 2026 and selects the search tool. It calls the search API with relevant keywords. The result returns 15 papers with citation counts.

Iteration 2:

The agent identifies the top result with 340 citations and calls a document retrieval tool to access the full abstract and key sections.

Iteration 3:

The agent determines that sufficient information has been gathered, generates the summary, and exits the loop.

Three iterations. Three tool calls. One complete answer that no single-pass chatbot could have produced.

Tool integration: the universal pattern

Across every provider, tool integration follows the same structure. Tools are defined with a name, description, and JSON Schema parameters. The model decides whether to call a tool and with what arguments. The system executes the function and returns results as a tool message. The model processes results and decides whether to continue looping or return a final response.

Tools in an agent loop can be classified into three categories. Data tools retrieve context, such as database queries, vector search, or document retrieval. Action tools perform operations with side effects, such as writing records, calling external APIs, or executing code. Orchestration tools invoke other agents as callable sub-modules, enabling multi-agent coordination within a single workflow. Clear classification of tools at design time reduces ambiguous model behaviour at runtime.

Anthropic's Model Context Protocol (MCP) has emerged as a leading open standard for how agents discover and connect to external tools, with adoption across OpenAI, Google, Microsoft, and the broader ecosystem.

Beyond the basic loop

The core ReAct loop handles most use cases, but the pattern extends in two important directions.

Plan-and-execute separates planning from execution. Instead of invoking the LLM at every step, a planner generates a full task breakdown upfront, an executor works through each subtask, and a re-planner adjusts when execution diverges from the plan. LangChain's LLMCompiler implementation streams a directed acyclic graph of tasks with explicit dependency tracking, enabling parallel execution. The original paper (Kim et al., ICML 2024) reports a 3.6x speedup over sequential ReAct-style execution. At production scale, where each LLM call carries a direct cost, the architectural decision to plan upfront rather than reason at every step has measurable financial implications.

Multi-agent orchestration distributes work across specialised agents. Anthropic's Claude Research system uses an orchestrator-worker pattern where a lead agent spawns sub-agents to explore different threads in parallel. Their multi-agent system outperformed a single-agent setup by 90.2% on internal research evaluations. Microsoft's Magentic-One takes it further with a dual-loop system: an outer loop for strategic planning and an inner loop for step-by-step execution, with the ability to reset the entire strategy when progress stalls.

These are powerful extensions, but the advice from every company is the same: start with the simplest loop that works. Only add complexity when you can measure the improvement.

The Enterprise Reality: Cost and Observability

Token cost scaling from standard chat (1x) to single agent (4x) to multi-agent (15x), with corresponding production requirements

Agent loops that perform well in controlled environments frequently expose new failure modes at production scale. The two constraints that dominate enterprise deployments are cost and observability.

Cost scales with iteration

Every loop iteration is an LLM call. Anthropic's internal data shows that agents consume roughly 4x more tokens than standard chat. Multi-agent systems push that to approximately 15x. At thousands of agent sessions per day, token costs compound with every loop iteration. Without cost controls embedded at the architecture level, this becomes a significant operational constraint.

The mitigation strategies are architectural. Plan-and-execute patterns reduce the number of LLM calls by planning upfront rather than reasoning at every step. Caching commonly retrieved tool results avoids redundant work. Setting token and cost budgets per agent run prevents runaway spending. These controls must be designed into the system from the start, not added retroactively.

Observability: knowing what your agent did and why

A standard chat interaction produces a single response from a single LLM call. An agent running 15 iterations, calling 8 different tools, and branching across multiple reasoning paths produces a complex execution trace. When a failure occurs, diagnosing it requires structured visibility into every stage of that trace: what the model reasoned, which tool it invoked, what arguments it passed, what the result was, and how the model interpreted that result before the next iteration.

Production agent systems need structured logging at every stage of the loop: what the model reasoned, which tool it called, what arguments it passed, what came back, and how it interpreted the result. Microsoft's AutoGen 0.4 builds on OpenTelemetry for this. LangChain's middleware hooks (before_model, after_model, modify_model_request) let you intercept and inspect every iteration.

Stopping conditions are the other critical piece. Without them, agents can loop indefinitely, burning tokens and producing increasingly incoherent results. Every production system needs maximum iteration limits, no-progress detection (exiting when repeated iterations produce no new information), and token/cost budgets as hard guardrails.

The following scenario illustrates the consequence of deploying an agent loop without hard stopping conditions:

An agent is deployed to scrape a website and summarise the data. The target website updates its structure, causing the scraping tool to return an empty result. The agent lacks a hard stopping condition, and its prompt instructs it to retry until data is retrieved. It enters a runaway loop, calling the broken tool 400 times in five minutes and consuming thousands of tokens before hitting a platform rate limit. A maximum iteration limit of three cycles would have prevented the failure entirely.

Building an Agent Loop with LangChain and Oracle

Before selecting a framework or writing code, address the following implementation requirements. These apply regardless of which orchestration library is used:

Identify tools and schema: What actions can the agent take, and what exact parameters do those tools need?
Choose state representation: How will you store the conversation history and intermediate tool results?
Define stopping criteria: What are the hard limits (iterations, tokens, budget) that will force the loop to terminate?
Establish logging and telemetry: How will you track each reasoning step, tool call, and result?
Select a memory layer: Where will you store persistent knowledge (like vector embeddings or user preferences) across sessions?

Here is one concrete way to implement that checklist using LangChain and Oracle.

import oracledb
from langchain.agents import create_agent
from langchain_core.tools import tool
from langchain_core.messages import AIMessage, ToolMessage

# Connect to Oracle AI Database
pool = oracledb.create_pool(
    user="agent_user", password="password",
    dsn="hostname:port/service",
    min=2, max=10, increment=1,
)

# Define tools the agent can use
@tool
def calculate(expression: str) -> str:
    """Evaluate a mathematical expression and return the numeric result."""
    ...

@tool
def convert_units(value: float, from_unit: str, to_unit: str) -> str:
    """Convert a numeric value from one unit to another."""
    ...

@tool
def timezone_convert(time_str: str, from_city: str, to_city: str) -> str:
    """Convert a local time from one city's timezone to another."""
    ...

# Create the agent -- returns a compiled StateGraph that runs the loop
agent = create_agent(
    model=llm,
    tools=[calculate, convert_units, timezone_convert],
    system_prompt="You are a precise reasoning assistant. Use tools for all calculations.",
)

QUESTION = (
    "A flight from London to New York JFK covers 5,570 km. "
    "The aircraft cruises at 900 km/h. "
    "The flight departs London at 14:00 local time. "
    "How long is the flight in hours and minutes, "
    "and what local time does it arrive in New York?"
)

# Stream the loop live -- each chunk shows one stage of the agent's reasoning
for chunk in agent.stream(
    {"messages": [("human", QUESTION)]},
    stream_mode="values",
):
    last_msg = chunk["messages"][-1]
    if isinstance(last_msg, AIMessage) and last_msg.tool_calls:
        for call in last_msg.tool_calls:
            print(f"[ACT] \u2192 {call['name']}({call['args']})")
    elif isinstance(last_msg, ToolMessage):
        print(f"[OBSERVE] \u2190 {last_msg.content}")
    elif isinstance(last_msg, AIMessage) and last_msg.content:
        print(f"\nAnswer: {last_msg.content}")

The implementation above is a working agent loop. The compiled agent graph manages the while loop internally, invoking the LLM, evaluating tool calls, executing them, appending results to the message state, and repeating until the model returns a final response without further tool calls or the recursion limit is reached.

Oracle AI Database provides the storage backend for the tools the agent calls. Vector search for semantic retrieval, relational tables for structured data, and ACID transactions ensuring that every tool call either fully succeeds or fully rolls back. No partial state. No corrupted memory.

We've published a complete, runnable notebook that implements a full agent loop architecture with LangChain and Oracle AI Database in the Oracle AI Developer Hub.

Run the notebook → oracle-devrel/oracle-ai-developer-hub

Where This Is Heading

Three structural shifts are emerging in how production agent systems are designed and operated.

The core loop architecture is stable. The active area of development is the infrastructure built around it: context management, multi-loop coordination, and decision auditability. The while loop itself is not changing. What is evolving is how context is managed within it, how multiple loops are coordinated together, and how the loop's decisions are made auditable and controllable.

Agent middleware is emerging as the standard abstraction layer for production systems. LangChain's recent work on middleware hooks (intercepting the loop at before_model, after_model, and modify_model_request) suggests a future where developers don't modify the loop itself but layer behaviour on top of it: summarisation, PII redaction, human-in-the-loop approval, dynamic model switching. It's the same pattern that made web frameworks powerful: don't change the request-response cycle, add middleware to it.

Cost-per-task will replace cost-per-token as the primary efficiency metric. Token usage is an input measure. The metric that reflects actual business value is the total cost required to complete a task end to end, including LLM calls, tool executions, and any human escalations triggered by agent failures.

An agent that consumes 15x more tokens but resolves a customer issue without human escalation is cheaper than a chatbot that consumes fewer tokens but requires human intervention to complete the task.

The primary open question in production agent deployment is the pace at which observability tooling will mature. Debugging a 20-iteration agent run currently requires piecing together structured logs, tool call traces, and LLM reasoning outputs across multiple systems. The industry needs better tooling for tracing, replaying, and interpreting agent decisions. The building blocks exist in OpenTelemetry, structured logging, and middleware hooks. The developer experience remains the unsolved problem.

The agent loop is the foundational pattern for any AI system that needs to do more than generate a response. It is the architectural starting point for production-grade autonomous AI.

Frequently Asked Questions

What is an AI agent loop?

The AI agent loop is an iterative architecture where a large language model repeatedly reasons about a task, takes an action (typically a tool call), observes the result, and decides what to do next. The cycle continues until the task is complete or a stopping condition is met. In its simplest form, it's an LLM calling tools inside a while loop. This pattern, formalised in the ReAct framework (2022), is the core architecture behind every major autonomous AI system shipping today.

What is the architectural difference between an AI agent and a chatbot?

A chatbot generates a single response to a single input. It answers questions but cannot execute multi-step actions or adapt based on intermediate results. An AI agent uses the agent loop to iteratively reason, act, and observe, handling complex tasks that require multiple steps, tool interactions, and course corrections. The architectural difference is simple: a chatbot is one LLM call; an agent is an LLM calling tools in a loop until the job is done.

How does the ReAct framework work?

ReAct (Reasoning + Acting) interleaves reasoning traces with tool actions in a prompt-driven loop. At each step, the model generates a 'thought' explaining its reasoning, takes an 'action' by calling a tool, and receives an 'observation' with the result. This cycle repeats until the task is complete. The key innovation is that reasoning and acting reinforce each other: the model reasons about what to do (reason to act) and uses action results to inform further reasoning (act to reason).

What are common patterns for multi-agent orchestration?

Three patterns dominate. The manager pattern uses a central agent that delegates subtasks to specialised sub-agents via tool calls (used by OpenAI's Agents SDK). The orchestrator-worker pattern has a lead agent spawning workers for parallel exploration (used by Anthropic's Claude Research). The handoff pattern treats agents as peers that transfer control to one another based on specialisation. Most production systems start with a single agent loop and only move to multi-agent orchestration when task complexity genuinely demands it.

How do you prevent an AI agent from running forever?

Production agent loops use multiple stopping conditions layered together. Maximum iteration limits cap the number of loop cycles (for example, max_iterations=10). Token and cost budgets set hard spending limits per agent run. No-progress detection exits the loop when repeated iterations produce no new information. Goal-achievement checks evaluate whether the task objective has been met. Microsoft's Magentic-One adds a dual-loop approach where the outer loop can reset the entire strategy when the inner loop stalls, preventing the agent from spinning on a failed approach.

Building ONNX Embedding Workflows in Oracle AI Database with Python

Wojtek Pluta — Fri, 17 Apr 2026 08:44:35 +0000

A practical guide to importing an ONNX embedding model, generating embeddings, and running semantic search in Oracle AI Database

Companion notebook: ONNX In-Database Embeddings with Oracle AI Database 26ai

Key Takeaways

Oracle AI Database can load and register an augmented ONNX embedding model with DBMS_VECTOR.LOAD_ONNX_MODEL().
VECTOR_EMBEDDING() lets SQL generate embeddings directly inside Oracle AI Database.
Embeddings can be stored natively in VECTOR columns.
VECTOR_DISTANCE() enables semantic search directly in SQL.
LangChain can build on the same Oracle-native workflow without moving embeddings or retrieval outside the database (LangChain Oracle vector store integration).

Why This Matters

In many embedding pipelines, source data resides in a relational database, the model runs somewhere else as an external service, and the vectors are stored in a separate vector database. While this architecture can work well, it introduces additional data movement, infrastructure, and operational complexity.

Oracle AI Database supports a more consolidated approach. You can load an ONNX embedding model directly into the database, invoke it, store the generated embeddings in native VECTOR columns, and perform semantic search in the same database.

This article walks through that end-to-end workflow using an ONNX model: loading it into Oracle AI Database, validating that it is registered correctly, generating embeddings with SQL, storing them in a native vector column, and querying them using semantic similarity. It also demonstrates how the same architecture can be used with LangChain, without changing where embedding and retrieval occur.

What You'll Learn

How to load an augmented ONNX model with Oracle AI Database.
How to generate embeddings directly in SQL with VECTOR_EMBEDDING().
How to run semantic search with VECTOR_DISTANCE() in Oracle AI Database and through LangChain.

Architecture Overview

This workflow keeps model execution, vector storage, and semantic retrieval inside Oracle AI Database. An augmented ONNX model is exposed through an Oracle directory object, loaded with DBMS_VECTOR.LOAD_ONNX_MODEL(), invoked with VECTOR_EMBEDDING(), and queried with VECTOR_DISTANCE(). The model artifact can come either from a local or container-mounted path or directly from Oracle Cloud Object Storage using DBMS_VECTOR.LOAD_ONNX_MODEL_CLOUD(). LangChain can build on the same Oracle-native execution path through OracleEmbeddings and OracleVS.

Prerequisites

Python 3.10+
Oracle AI Database 26ai running in a container
Dependencies such as oracledb, python-dotenv, pandas, numpy, langchain, langchain-community, and langchain-oracledb
For cloud loading: an Oracle Cloud Object Storage bucket and model URI, or a PAR URL
If not using a PAR URL, an Object Storage credential created with DBMS_CLOUD.CREATE_CREDENTIAL

In the notebook, those packages are installed up front:

import subprocess
import sys

result = subprocess.run(
    [sys.executable, "-m", "pip", "install", "-q",
     "oracledb", "python-dotenv", "pandas", "numpy",
     "langchain", "langchain-core", "langchain-community", "langchain-oracledb"],
    capture_output=True, text=True
)
print("Packages installed." if result.returncode == 0 else f"Install failed: {result.stderr}")

The example also assumes Oracle AI Database 26ai is running in a container, with a mounted directory for ONNX model files. That mounted directory becomes important later, because Oracle accesses the model through a database directory object rather than through ad hoc file access.

Step-by-Step Guide

Step 1: Understand why Oracle requires an augmented ONNX model

One of the most important details in this workflow is that Oracle needs an augmented ONNX model, not just a standard transformer export.

For VECTOR_EMBEDDING() to accept raw text directly, tokenization and related preprocessing need to be included inside the ONNX graph itself. That is what allows Oracle to take a normal text string and produce an embedding without relying on external preprocessing in Python.

In the notebook, the model used is an augmented version of all-MiniLM-L12-v2:

MODEL_NAME = "all_MiniLM_L12_v2"
ONNX_FILE = "all_MiniLM_L12_v2.onnx"

Without that augmented packaging, the flow would no longer be fully Oracle-native, because preprocessing would have to happen outside the database first.

Step 2: Prepare an ONNX model for Oracle AI Database

Before the model can be used in SQL, Oracle needs controlled access to the ONNX file through a database directory object. This is a database-managed reference to a filesystem location, which means access to the model artifact is handled through Oracle privileges rather than through direct filesystem assumptions.

The notebook includes a one-time admin setup that creates the user, grants privileges, and registers the ONNX model directory. At runtime, the important pieces are:

a database user with the required privileges
permission to load mining models
a registered Oracle directory such as ONNX_DIR
access to the ONNX file from inside the container

A simplified version of the directory setup looks like this:

CREATE OR REPLACE DIRECTORY ONNX_DIR AS '/opt/oracle/onnx_models';
GRANT READ, WRITE ON DIRECTORY ONNX_DIR TO my_user;

This matters because the model import is not treated as an ad hoc file operation. The file is exposed to Oracle through a controlled database object, which is much more aligned with enterprise governance expectations.

Figure 1. An augmented ONNX model is exposed through an Oracle directory object, loaded with DBMS_VECTOR.LOAD_ONNX_MODEL(), registered in Oracle, and invoked from SQL.

Step 2b: Cloud option - load ONNX from Oracle Object Storage

Oracle also supports loading ONNX models from Oracle Cloud Object Storage with DBMS_VECTOR.LOAD_ONNX_MODEL_CLOUD(). This is a documented alternative to the local directory workflow used in the companion notebook.

Per Oracle documentation, use a credential for standard Object Storage URIs, and pass credential => NULL for pre-authenticated request (PAR) URLs.

-- Option A: regular Object Storage URI (credential required)
EXECUTE DBMS_VECTOR.LOAD_ONNX_MODEL_CLOUD(
  model_name => 'ALL_MINILM_L12_V2',
  credential => 'OBJ_STORE_CRED',
  uri        => 'https://objectstorage.<region>.oraclecloud.com/n/<namespace>/b/<bucket>/o/all_MiniLM_L12_v2.onnx',
  metadata   => JSON('{
    "function":"embedding",
    "embeddingOutput":"embedding",
    "input":{"input":["DATA"]}
  }')
);

-- Option B: PAR URL (credential must be NULL)
EXECUTE DBMS_VECTOR.LOAD_ONNX_MODEL_CLOUD(
  model_name => 'ALL_MINILM_L12_V2',
  credential => NULL,
  uri        => 'https://objectstorage.<region>.oraclecloud.com/p/<par-token>/n/<namespace>/b/<bucket>/o/all_MiniLM_L12_v2.onnx'
);

Note: According to Oracle documentation, metadata is optional for models prepared with Oracle's Python utility defaults, model names must follow Oracle naming rules, and the ONNX file size limit for cloud loading is 2 GB.

Step 2c: Multi-cloud note (AWS/GCP/Google Drive)

DBMS_VECTOR.LOAD_ONNX_MODEL_CLOUD() is documented for Oracle Cloud Object Storage. If your model artifact is hosted in AWS S3, Google Cloud Storage, or Google Drive, use a portable two-step pattern: download the ONNX file to a database-accessible local path, then load it with DBMS_VECTOR.LOAD_ONNX_MODEL().

This keeps embedding generation and semantic retrieval Oracle-native while allowing model artifact hosting outside OCI.

import os
import requests

model_url = os.environ["MODEL_SIGNED_URL"]  # S3 pre-signed URL / GCS signed URL / Drive direct URL
target_path = "/opt/oracle/onnx_models/all_MiniLM_L12_v2.onnx"

resp = requests.get(model_url, stream=True, timeout=180)
resp.raise_for_status()

with open(target_path, "wb") as f:
    for chunk in resp.iter_content(chunk_size=1024 * 1024):
        if chunk:
            f.write(chunk)

print(f"Model downloaded to {target_path}")

BEGIN
  DBMS_VECTOR.LOAD_ONNX_MODEL(
    directory  => 'ONNX_DIR',
    file_name  => 'all_MiniLM_L12_v2.onnx',
    model_name => 'ALL_MINILM_L12_V2'
  );
END;
/

Step 3: Connect to Oracle AI Database from Python

The notebook connects to Oracle AI Database using python-oracledb in Thin mode, so no Oracle Client libraries are required:

import oracledb

conn = oracledb.connect(...)
print("Connected to Oracle AI Database")

That same connection is then reused across the SQL examples and the LangChain integration later in the notebook.

To keep the notebook readable, it defines a small helper function for executing SQL and optionally returning results as a pandas DataFrame:

import pandas as pd

def run_sql(sql, params=None, fetch=False, many=False, data=None):
    """Execute SQL against Oracle Database."""
    with conn.cursor() as cur:
        if many and data:
            cur.executemany(sql, data)
        elif params:
            cur.execute(sql, params)
        else:
            cur.execute(sql)

        if fetch:
            cols = [c[0] for c in cur.description]
            return pd.DataFrame(cur.fetchall(), columns=cols)

    conn.commit()

Step 4: Load an ONNX embedding model into Oracle AI Database

The notebook does not assume the ONNX model is already present. If the file is missing, it downloads the official pre-built augmented model and places it in the model directory used by Oracle.

Once the model file is available, either through an Oracle directory object or a cloud URI, it can be imported with DBMS_VECTOR.LOAD_ONNX_MODEL() or DBMS_VECTOR.LOAD_ONNX_MODEL_CLOUD().

A simplified version of the local directory call looks like this:

BEGIN
  DBMS_VECTOR.LOAD_ONNX_MODEL(
    directory  => 'ONNX_DIR',
    file_name  => 'all_MiniLM_L12_v2.onnx',
    model_name => 'ALL_MINILM_L12_V2',
    metadata   => JSON('{
      "function":"embedding",
      "embeddingOutput":"embedding",
      "input":{"input":["DATA"]}
    }')
  );
END;
/

This is the point where the model becomes more than a file. Oracle registers it, stores the associated metadata, and exposes it as a named object that SQL can invoke directly.

The metadata is especially important. It defines how Oracle maps the SQL input text into the model graph and identifies which output node should be used as the embedding vector. In the notebook, the workflow also checks whether the model already exists before reloading it. This makes reruns safer and ensures the workflow remains idempotent.

model_check = run_sql(
    "SELECT COUNT(*) AS cnt FROM USER_MINING_MODELS WHERE MODEL_NAME = UPPER(:model_name)",
    params={"model_name": MODEL_NAME},
    fetch=True
)

Expected output: the model check confirms whether the ONNX model is already registered, so reruns stay idempotent.

Step 5: Verify that Oracle registered the model correctly

After the import, the next step is to validate that Oracle recognizes the model.

The notebook queries the model catalog to verify that the ONNX model has been loaded successfully:

SELECT model_name, mining_function, algorithm
FROM user_mining_models
WHERE model_name = 'ALL_MINILM_L12_V2';

This is a small but important part of the workflow. It confirms that the model is visible to Oracle as a registered object and is ready to be used by the vector functions that come next.

Expected output: the query returns the registered ONNX model from USER_MINING_MODELS.

Step 6: Generate embeddings in SQL with VECTOR_EMBEDDING()

Once the model is registered, Oracle can use it directly through VECTOR_EMBEDDING().

The notebook first tests this with a simple text input to confirm that the model works and that the returned vector has the expected size.

SELECT VECTOR_EMBEDDING(
         ALL_MINILM_L12_V2
         USING 'Oracle Database supports vector search.' AS DATA
       ) AS embedding
FROM dual;

This is one of the most important parts of the article. Embedding generation is no longer a separate service call. It becomes a SQL operation:

the application does not need to call an external embedding API
the database can generate embeddings internally
the semantic representation stays close to the data it describes

Expected output: Oracle returns a 384-dimensional embedding for the supplied text.

Step 7: Store embeddings in a native VECTOR column

After validating embedding generation, the notebook creates a table where the source text and its embedding live together.

CREATE TABLE onnx_docs (
  id        NUMBER GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
  category  VARCHAR2(100),
  doc_text  CLOB,
  embedding VECTOR(384, FLOAT32)
);

This is an important design choice. The vector is not stored as an opaque blob or external payload. It is stored in Oracle's native VECTOR type, which means it becomes part of the same database model as the relational data.

vectors stay linked to the exact rows they describe
access control applies consistently
backups and retention policies stay unified
the application does not need to coordinate data across multiple storage systems

The notebook inserts demo content and generates the embedding directly in the same SQL statement:

INSERT INTO onnx_docs (category, doc_text, embedding)
VALUES (
  'database',
  'Oracle AI Database supports in-database vector search and semantic retrieval.',
  VECTOR_EMBEDDING(
    ALL_MINILM_L12_V2
    USING 'Oracle AI Database supports in-database vector search and semantic retrieval.' AS DATA
  )
);

The semantic representation is created at the same time as the row is written, inside the same transactional boundary.

Figure 2. Embedding generation happens at insert time inside Oracle AI Database, where document text is embedded with VECTOR_EMBEDDING() and stored together with the row in a VECTOR column.

Before moving into retrieval, the notebook inspects the inserted rows:

SELECT id, category, DBMS_LOB.SUBSTR(doc_text, 120, 1) AS preview
FROM onnx_docs
ORDER BY id;

Step 8: Run semantic search in SQL and LangChain

Once embeddings are stored, semantic retrieval is handled entirely inside Oracle. The notebook uses VECTOR_DISTANCE() together with VECTOR_EMBEDDING() so that the query text is embedded on the fly and compared against the stored vectors:

SELECT
  id,
  category,
  DBMS_LOB.SUBSTR(doc_text, 200, 1) AS doc_preview,
  VECTOR_DISTANCE(
    embedding,
    VECTOR_EMBEDDING(ALL_MINILM_L12_V2 USING 'How does Oracle support semantic search?' AS DATA),
    COSINE
  ) AS distance
FROM onnx_docs
ORDER BY distance
FETCH FIRST 3 ROWS ONLY;

The user query is embedded directly within Oracle, where it is compared against stored document vectors. The results are then ranked by similarity, and the closest semantic matches are returned through SQL.

The notebook explicitly explains how to interpret the output: the smaller the cosine distance, the more semantically similar the document is to the query.

The notebook also runs several queries to validate that semantic ranking remains meaningful across different phrasings:

test_queries = [
    "Which Oracle feature helps semantic retrieval?",
    "Can I store embeddings in the database?",
    "How does LangChain work with Oracle vectors?",
    "Why are ONNX models useful here?"
]

Figure 3. At query time, Oracle embeds the input text, compares it with stored vectors using VECTOR_DISTANCE(), and returns the nearest semantic matches directly through SQL.

The notebook then adds an optional framework layer using LangChain:

from langchain_oracledb.embeddings import OracleEmbeddings
from langchain_oracledb.vectorstores.oraclevs import OracleVS

With OracleEmbeddings, the application can use Oracle's registered in-database embedding model:

oracle_embedder = OracleEmbeddings(
    conn=conn,
    params={"provider": "database", "model": MODEL_NAME}
)

The notebook also validates that the LangChain embedding call returns a vector of the expected size:

lc_embedding = oracle_embedder.embed_query(
    "Oracle AI Database performs semantic search using vectors."
)

print(f"Embedding dimension: {len(lc_embedding)}")
print(f"First 5 values: {lc_embedding[:5]}")

The notebook then uses OracleVS, a LangChain-compatible vector store backed by Oracle AI Vector Search.

from langchain_core.documents import Document
from langchain_oracledb.vectorstores.oraclevs import OracleVS
from langchain_community.vectorstores.utils import DistanceStrategy

langchain_docs = [
    Document(page_content="Oracle AI Database supports vector storage and semantic search."),
    Document(page_content="An ONNX embedding model can be loaded directly into Oracle."),
    Document(page_content="LangChain can use OracleVS to query Oracle AI Vector Search."),
    Document(page_content="Using in-database embeddings can reduce architectural complexity."),
]

vector_store = OracleVS.from_documents(
    documents=langchain_docs,
    embedding=oracle_embedder,
    client=conn,
    table_name="LC_ONNX_DEMO",
    distance_strategy=DistanceStrategy.COSINE
)

The notebook also runs a similarity query through the LangChain abstraction:

results = vector_store.similarity_search(
    "How can Oracle Database help with semantic retrieval?",
    k=3
)

for i, doc in enumerate(results, start=1):
    print(f"{i}. {doc.page_content}")

Validation & Troubleshooting

Validate that the model appears in USER_MINING_MODELS after DBMS_VECTOR.LOAD_ONNX_MODEL() or DBMS_VECTOR.LOAD_ONNX_MODEL_CLOUD().
Confirm that VECTOR_EMBEDDING() returns a 384-dimensional embedding for the loaded model.
If semantic ranking looks off, verify that the same model is used for both stored document embeddings and query embeddings.
If using cloud loading, verify URI or PAR validity, bucket path, region, and credential privileges.
When rerunning the notebook, check whether the model and demo tables already exist to avoid duplicate object errors.

Frequently Asked Questions

Why load the model into Oracle instead of calling an external API?

Because Oracle can generate embeddings directly in SQL, which reduces external dependencies and keeps data and inference inside the same system boundary.

Why does the model need to be augmented?

Because Oracle must be able to accept raw text input directly. That requires tokenization and preprocessing logic to already be included in the ONNX graph.

What does VECTOR_EMBEDDING() do?

It invokes the registered model inside Oracle and returns the embedding vector for the input text.

What does the VECTOR column store?

It stores the numeric embedding representation produced by the model. In this example, the vectors are 384-dimensional FLOAT32 values.

How is semantic similarity computed?

This workflow uses VECTOR_DISTANCE() with cosine distance to compare the stored document vectors with the embedded query.

Can the model be reused by multiple applications?

Yes. Once registered and granted appropriately, the model can be invoked by any application that has access to the Oracle environment.

Can I load the model from cloud storage instead of a local mounted directory?

Yes. Oracle AI Database supports DBMS_VECTOR.LOAD_ONNX_MODEL_CLOUD() for models in Oracle Cloud Object Storage, with either a credential or a PAR URL.

Does LangChain move embeddings outside Oracle?

No. LangChain provides a higher-level interface, but the model execution and vector search still run in Oracle.

Does this replace a separate vector database?

For many use cases, yes. Oracle provides native vector storage and vector search directly in the database.

Vector Embeddings: How They Work, Where to Store Them, and Best Practices

Wojtek Pluta — Thu, 16 Apr 2026 15:51:46 +0000

Key Takeaways

Vector embeddings convert unstructured data into numeric representations that power semantic search, recommendations, and multimodal analytics beyond keywords.
Embedding success isn’t just about the model—it also depends on a data platform that can meet requirements for scale, low latency, security, and governance, including vector indexing/ANN search, access controls, encryption, and monitoring.
Oracle AI Database unifies native vector types and similarity search, enterprise-grade security, and integrated vector, structured, and unstructured data—so teams can build RAG, search, and analytics without piecing together multiple systems.

Semantic similarity search over vector space - Oracle Help Center

Introduction to Vector Embeddings

Vector embeddings have changed the way we interact with unstructured data such as text, images, audio, and code. By transforming this data into high-dimensional numeric vectors, we can use embeddings to process the semantic meaning and relationships within the data.

We can look at embeddings as task or domain-specific representations of vectors. The geometric relationships among them represent meaningful similarities between concepts in semantic space. The efficient storage and querying of vector embeddings enables capabilities such as semantic search, recommendations, and advanced analytics; and bridges the gap between unstructured and structured information.

What are Vector Embeddings? A Definition and Their Role

Vector embeddings are mathematical representations of objects—such as words, sentences, images, or audio—encoded as dense, high-dimensional vectors. Each vector encapsulates features that capture semantic meaning, context, or structure of the data. For example, similar words or images will have embeddings positioned closely in the vector space, enabling similarity-based operations. This allows for similar “things” to be grouped together under a distance metric.

The adoption of vector embeddings underpins many cutting-edge technologies:

Retrieval-augmented generation (RAG): Enhances large language models by retrieving relevant context using embedding similarity.
Semantic search: Finds documents with similar context, not just matching keywords.
Recommendations: Suggests products or content by comparing user or item embeddings.
Deduplication and anomaly detection: Identifies near-duplicates or outliers based on embedding distances.
Multimodal analytics: Links information across text, image, audio, and other domains.

This ability to bridge structured and unstructured data makes embeddings indispensable across modern data architectures.

How to Create Embeddings? Some Tools That Can Help

A variety of tools can encode text, images, and code as vector embeddings, enabling similarity search, retrieval workflows (including RAG), and other ML tasks:

OpenAI – provides hosted embedding APIs backed by task-optimized models, accessible with REST interfaces.
Hugging Face – offers a large catalog of pre-trained multimodal embedding models and libraries (such as the Transformers library), plus community benchmarks.
Oracle AI Database – provides a native vector memory store in Oracle Database, enabling storage, indexing (e.g., IVF/flat/HNSW), and retrieval of vector embeddings alongside relational data with SQL and PL/SQL integration; supports hybrid search (vector + metadata filters), enterprise-grade security, and governance for RAG and semantic search workloads
TensorFlow – enables building and serving custom embedding models using Keras, enabling easy integration into training pipelines.
PyTorch – provides flexible primitives to fine-tune or implement embedding models, and deploy them via TorchScript.

Benefits of Working With Vector Embeddings

The following are just a few of the benefits vector embeddings have brought to today's AI tech stack:

Vector embeddings are currently the best way to transform complex data into numerical units that reflect meaning, similarity and enable clustering and retrieval beyond keyword matching.
The limitations of keyword methods were particularly visible in areas such as synonym handling, typos, and paraphrasing, and are now absent in modern-day LLMs relying on vector embeddings.
Embeddings support multilingual and cross-modal experiences by aligning meaning across languages and modalities.

Other approaches, such as sparse lexical retrieval and symbolic/ontology-based methods, can be effective, but dense vector embeddings are often a better fit when you need semantic similarity matching (for example, paraphrases and synonyms) rather than exact keyword overlap.

Challenges in Working With Vector Embeddings

The following are some of the potential challenges you may face in working with vector embeddings, and potential ways to mitigate them:

Storage Volume and High Dimensionality

Storage challenges include:

Large embedding volumes: Billions of vectors require scalable storage and efficient indexing.
High dimensionality: Embeddings of 128, 512, or 1024+ dimensions need specialized data structures and optimized storage formats.

Performance and Latency Bottlenecks

Performance factors include:

Indexing and search speed: ANN techniques improve latency, but very large datasets demand optimized infrastructure.
Batch insertion and streaming: Efficiently handling ongoing ingestion of new embeddings.

Distributed System Complexities and Operational Overhead

At scale, sharding, replication, and consistency management become complex. Automated scaling, monitoring, and failover are desirable for production systems.

Cost Factors

Vector embeddings may affect operational cost:

Compute and storage requirements: High-dimensional data and fast search consume substantial resources.
Operational overhead: Consider cost of infrastructure, team expertise, and maintenance.

Encryption at Rest and in Transit

Securing embeddings is crucial as they can encode sensitive information:

Encryption at rest: Protects stored vectors using strong industry-standard algorithms.
Encryption in transit: Ensures vectors remain confidential when transmitted between systems or users.

Oracle AI Database enforces encryption by default and integrates with enterprise key management solutions.

Access Control and Authentication

Control who can access, modify, or query embeddings:

Granular permissions: Define user roles and table-level permissions.
Integration with SSO and identity providers: Streamlines enterprise authentication.
Audit trails: Track access and changes for compliance.

Data Sanitization and Monitoring

Reduce risk by implementing:

Sanitization: Remove or obfuscate sensitive or personal information in embeddings before storage.
Monitoring and anomaly detection: Detect unusual access patterns or potential misuse.

Advanced Cryptographic Techniques

For highly sensitive embeddings:

Homomorphic encryption or secure multi-party computation: Enables computation and search on encrypted embeddings, minimizing exposure.

Common Vector Embedding Use Cases

Embeddings open up a wide array of practical use cases:

Enterprise search and information retrieval: Improved accuracy and relevance in document and knowledge base searches.
Personalization and recommendation engines: Enhanced user experiences by surfacing relevant content or products.
Fraud and anomaly detection: Early identification of unusual patterns using embedding distances.
Data deduplication and clustering: Streamlined datasets and improved analytics through intelligent grouping.
Multimodal retrieval and analytics: Unified analysis over diverse data types, fostering deeper insights.

Storing Vector Embeddings and the Oracle Advantage

The following are a few key points related to the storage of vector embeddings, and how Oracle AI Database's native vector store capabilities can streamline and strengthen your stack with its native vector store capabilities.

Specialized Vector Databases

Dedicated vector databases are built for storing, indexing, and searching embeddings efficiently. These databases excel at large-scale similarity search with features such as:

High-dimensional indexing: Specialized data structures to support billion-scale embeddings.
Approximate search capabilities: Fast, scalable similarity queries using Approximate Nearest Neighbor (ANN) techniques.
RESTful APIs and SDKs: Developer-friendly interfaces for ingestion and search.

Popular examples include Pinecone, Weaviate, Milvus, and Vespa. Specialized databases are ideal for workloads with large volumes of embeddings and demanding similarity search requirements.

SQL/NoSQL Databases with Vector Support

Traditional databases are evolving to meet AI's demands by adding native vector data types and search capabilities:

SQL databases: PostgreSQL (with pgvector), Oracle AI Database, and others support vector columns and similarity search via extensions or built-in features.
NoSQL databases: MongoDB and Redis now offer basic vector search features, often using plugins or modules.

This integration enables seamless blending of embeddings with structured business data, supporting hybrid query scenarios.

Oracle AI Database Approach

From Oracle's viewpoint, AI databases must natively support vector data types, efficient similarity queries, and enterprise security for integrating embeddings across applications. Oracle AI Database is designed to address these needs at scale.

The Oracle AI Database offers a unified approach allowing developers to:

Store embeddings alongside structured and unstructured data.
Run similarity queries directly using SQL and specialized vector search operators.
Integrate with Oracle's rich security, high availability, and scalability features.
Combine vector search, filtering, ranking, and analytical queries in a single stack.

Example Procedures - Using Vector Embeddings in Oracle AI Database

The following examples are intentionally minimal and illustrative. They highlight how Oracle AI Database supports native vector storage and SQL-based similarity search.

CREATE TABLE documents (

 id NUMBER,

 content CLOB,

 embedding VECTOR

);

This example shows a minimal table definition using Oracle AI Database’s native VECTOR data type. In practice, embeddings are stored alongside structured or unstructured application data in the same database.

SELECT id, content

FROM documents

ORDER BY VECTOR_DISTANCE(embedding, :query_vector, COSINE)

FETCH FIRST 5 ROWS ONLY;

This example illustrates SQL-based similarity search in Oracle AI Database. The :query_vector placeholder represents the embedding generated from user input by an embedding model (inside or outside the database) and is used to rank the nearest matches.

Hybrid query pattern (semantic + relational filtering)

SELECT id, content

FROM documents

WHERE content IS NOT NULL

ORDER BY VECTOR_DISTANCE(embedding, :query_vector, COSINE)

FETCH FIRST 5 ROWS ONLY;

This hybrid pattern combines standard SQL filtering with semantic ranking in a single query. It is useful when semantic search must also respect metadata constraints, access controls, or business rules. This streamlines workflows and facilitates embedding-driven applications without moving data across siloed systems.

Using Oracle Autonomous AI Database in conjunction with langchain-oracledb, for example, we can simply generate embeddings, store, and interact with vectors directly from within the database – requiring no additional investment in another separate vector database.

Querying and Searching for Stored Vector Embeddings

The following are a few of the things you should keep in mind if your work involves querying and searching for stored vector embeddings:

Approximate Nearest Neighbor (ANN) Algorithms and Data Structures

Searching for similar embeddings at scale requires efficient algorithms:

ANN Techniques: Rather than exact search, algorithms like HNSW (Hierarchical Navigable Small World), IVF (Inverted File Index), and PQ (Product Quantization) yield fast, near-accurate results.
Data Structures: Use trees (KD-Tree, Ball Tree), graphs (HNSW), or hash-based indices (LSH) to organize and retrieve vectors efficiently.

ANN can deliver millisecond-latency searches over millions or billions of embeddings, making it essential for operational AI applications.

High-level retrieval workflow (generalized)

At a high level, semantic retrieval follows a simple and reusable pattern that applies across vector databases, frameworks, and application stacks:

Convert user input into a query embedding.
Compare it against stored embeddings.
Rank results by similarity.
Apply filters and business rules as needed.

This high-level workflow is framework- and language-agnostic. While the underlying implementation differs across platforms and tools, the conceptual flow remains the same for the most vector search and RAG-style applications.

Popular Libraries

Several tools make it easier to store, and search embeddings:

Vector search libraries: FAISS (Facebook AI Similarity Search), Annoy (Spotify), NMSLIB, ScaNN.

These libraries power both stand-alone vector stores and integrations within general-purpose databases.

How to Choose the Right Similarity Metrics

Selecting the right similarity metric is critical for effective search:

Cosine similarity: Measures the angle between vectors; ideal for text and semantic similarity.
Euclidean distance: Useful for geometric or spatial data.
Dot product: Common in deep learning models; efficient for high-dimensional comparisons.

Your choice depends on the nature of your data and the specifics of your application.

Oracle AI Database Capabilities

Oracle’s AI Database combines native vector capabilities, enterprise security, and proven scalability, making it a robust choice for organizations seeking a unified solution for traditional data and AI-enabled workloads.

Native vector data types and indexing: Supports efficient storage and retrieval of high-dimensional vectors.
Integrated similarity search: Enables querying and filtering based on vector proximity.
Enterprise-grade security: Encryption at rest, robust access controls, and activity monitoring.
Hybrid queries: Seamless combination of structured, unstructured, and vector data in complex analytical tasks.
High scalability: Handles massive volumes of embeddings without performance degradation.

Best Practices for Working With Vector Embeddings

The following are a few of the best practices for using vector embeddings to power semantic search, personalized recommendations, multimodal analytics (including anomaly detection), and domain-specific insights across enterprise applications.

Semantic Search and Information Retrieval

Semantic search with embeddings offers better context and intent recognition than keyword search. Querying an embedding retrieves documents or objects with similar meanings—crucial for legal, healthcare, customer support, and research applications.

Recommendation Systems and Personalization

Compare user and item embeddings to power personalized recommendations. This increases engagement, retention, and value in e-commerce, media, and B2B applications.

Multimodal Search and Anomaly Detection

Combine embeddings across text, image, and audio for multimodal analytics or use distance-based thresholds to flag anomalies and outliers in fraud prevention or system monitoring.

Domain-Specific Analytics

Specialized embeddings can be trained for particular industries—finance, healthcare, retail—and stored/retrieved for advanced analytics, predictions, or compliance monitoring.

How to Select Appropriate Tools and Architectures

Match your use case to the data platform (dedicated vector database vs. extended relational/NoSQL).
If you want both, Oracle AI Database is a good option.
Factor in scale, integration needs, security requirements, and budget.
Leverage proven libraries and frameworks to speed up development.

Security and Scalability Considerations

Encrypt embeddings, control access, and monitor usage.
Choose solutions that scale with data growth and user demand.
Balance security, performance, and cost based on enterprise requirements.

Architectural Patterns

Hybrid architecture: Combine vector storage/search with structured data in a unified database like Oracle AI Database.
Microservices: Separate ingestion, search, and analytics as independently scaling components if needed.
Cloud-native solutions: Consider managed vector databases for elasticity and reduced operational burden.

Tooling Reminders

Use specialized libraries (FAISS, Annoy, HNSWLib) for local development, prototyping, or custom solutions.
For production or enterprise use, rely on databases with native vector support and robust security, such as Oracle AI Database.

Frequently Asked Questions (FAQ)

What are vector embeddings and why do they matter?

Vector embeddings are dense, high-dimensional numeric representations of objects like text, images, audio, or code. They place semantically similar items near each other in a continuous space, enabling tasks like semantic search, recommendations, RAG, deduplication, and anomaly detection. Compared with keyword or symbolic methods, embeddings better capture meaning, handle synonyms/paraphrases, and are robust across languages and modalities.

What are the main challenges in storing and querying embeddings at scale?

Volume and dimensionality: Billions of vectors, often 128–1024+ dimensions
Performance: Fast indexing and low-latency search, efficient batch/stream ingestion
Distributed ops: Sharding, replication, consistency, monitoring, and failover
Cost: Compute, storage, and operational overhead
Security: Encryption at rest/in transit, access control, auditing, data sanitization, and advanced cryptographic techniques for sensitive data

Where should I store embeddings: a dedicated vector database or a database with vector support?

Two common patterns:

Specialized vector databases (e.g., Pinecone, Weaviate, Milvus, Vespa) for high-scale, low-latency similarity search with ANN, SDKs, and REST APIs.
SQL/NoSQL databases with vector support (e.g., Oracle AI Database, PostgreSQL with pgvector, MongoDB, Redis) for blending vectors with structured data and enabling hybrid queries. Your choice should consider scale, integration with existing data, security, cost, and operational complexity.

What does Oracle AI Database provide for embeddings?

Oracle AI Database offers native vector types and indexing, integrated similarity search in SQL, enterprise-grade security (encryption, granular access control, auditing), and high scalability. It supports hybrid analytical queries across structured, unstructured, and vector data. With Oracle Autonomous AI Database and libraries like langchain-oracledb, teams can generate, store, and query embeddings within one platform—avoiding data silos and extra operational overhead. Encrypt data, enforce access controls, and monitor usage to meet enterprise requirements.

Conclusion

Storing and querying vector embeddings is a critical enabler for next-generation AI and data applications. By leveraging the right databases, libraries, and best practices, organizations and engineers can unlock new value from unstructured content, while maintaining performance, scalability, and security.

Resources

Agent Memory: A Free Short Course on Building Memory-Aware Agents

Wojtek Pluta — Thu, 16 Apr 2026 12:13:17 +0000

Oracle and DeepLearning.AI have launched Agent Memory: Building Memory-Aware Agents, a free short course on DeepLearning.AI that teaches developers how to architect memory systems that give agents persistence, continuity, and the ability to learn over time.

"Memory turns a stateless LLM into an agent that learns over time. How to architect agentic memory is one of the most debated topics in AI right now. This course gives AI developers and engineers a comprehensive view of the most common memory patterns."

Andrew Ng, Founder, DeepLearning.AI

Most agents forget. Each new session starts from zero, accumulated context from previous interactions is discarded, and the agent has no mechanism to learn from what it has already done. As a result, AI developers often rely on workarounds: cramming everything into the context window, reloading conversation logs, or bolting on ad-hoc retrieval.

These approaches can work, but they don't provide a clear mental model for how information should live inside an agentic system boundary. This course treats memory as a first-class citizen in AI agents, and is built around that memory-first perspective.

"For the past few years, we have focused on prompt and context engineering to get the best results from a single LLM call. But engineering the right context for agents that need to work over days or weeks needs an effective memory system. This course takes that memory-first approach to building agents."

Richmond Alake, AI Developer Experience Director, Oracle

Beyond Prompt Engineering

You’ve heard about prompt engineering. You've probably heard about context engineering. This course introduces the next layer: memory engineering, treating long-term memory as first-class infrastructure that is external to the model, persistent, and structured.

The course covers the full memory stack across five hands-on modules, built on LangChain, Tavily, and Oracle AI Database:

Why AI Agents Need Memory: Explore failure modes of stateless agents and the memory-first architecture used throughout the course.
Constructing the Memory Manager: Design persistent memory stores across memory types, model memory data for efficient retrieval, and implement a manager that orchestrates read, write, and retrieval operations during agent execution.
Scaling Agent Tool Use with Semantic Tool Memory: Treat tools as procedural memory, index them in a vector store, and retrieve only contextually relevant tools at inference time using semantic search.
Memory Operations: Extraction, Consolidation, and Self-Updating Memory: Build LLM-powered pipelines that extract structured facts from raw interactions, consolidate episodic memory into semantic memory, and implement write-back loops that let an agent autonomously update and resolve conflicts in its own knowledge base.
Memory-Aware Agent: Assemble a stateful agent that initializes from long-term memory at startup, checkpoints intermediate reasoning states during execution, and persists learned context across sessions.

"The patterns we cover here are not theoretical. AI developers and engineers will walk through real implementations: building memory stores, wiring up extraction pipelines, and handling contradictions in memory. You leave with working code you can adapt for your own production agents."

Nacho Martinez, AI Developer Advocate, Oracle

Oracle AI Database as the Agent Memory Core

Oracle AI Database serves as the unified agent memory core throughout the course. Instead of treating a database as a passive store, the course demonstrates how Oracle AI Database functions as the active retrieval and persistence layer that makes each memory pattern work in production.

Oracle AI Database brings key retrieval strategies into a single engine, including vector search for semantic similarity and unstructured knowledge retrieval, graph traversal for relationship-aware reasoning across connected entities, and relational queries for structured, transactional memory that demands precision and consistency. This helps reduce complexity by avoiding separate systems for different data types.

The memory patterns taught in this course, such as semantic tool memory, self-updating memory, and memory consolidation, are the same patterns used to build production-grade agentic systems on Oracle AI Database. This course puts that architecture directly in the hands of AI developers and engineers.

Who This Course Is For

Agent Memory: Building Memory-Aware Agents is designed for:

AI developers and engineers building or evaluating agentic systems who need production-grade memory architecture
ML engineers integrating LLMs into multi-turn or multi-session workflows
Developers working with LangChain, LangGraph, or Tavily who want durable, structured memory
Technical leaders assessing Oracle AI Database for agent infrastructure at scale

Availability

Agent Memory: Building Memory-Aware Agents is available now on DeepLearning.AI. The course is free to access and requires no prior Oracle experience. Developers can enroll in the course.

About Oracle AI Database

Oracle AI Database is a converged database platform built for AI workloads. It provides native vector search, graph traversal, relational retrieval, and the persistence infrastructure required for production agent memory systems in a single database engine. This removes the fragmented infrastructure that can become a bottleneck for AI innovation. Oracle AI Database is used by developers and enterprises as the unified memory core for AI agents to build and deploy intelligent, secure, memory-aware systems.

A Practical Guide to Choosing the Right Memory Substrate for Your AI Agents

Wojtek Pluta — Thu, 16 Apr 2026 12:11:25 +0000

Key Takeaways

Don't conflate interface with substrate. Filesystems win as an interface (LLMs already know how to use them); databases win as a substrate (concurrency, auditability, semantic search).
For prototypes, files are hard to beat. Simple, transparent, debuggable—a folder of markdown gets you surprisingly far when iteration speed matters most.
Shared state demands a database. Concurrent filesystem writes can silently corrupt data. If multiple agents or users touch the same memory, start with database guarantees.
Semantic retrieval beats keyword search at scale. Grep performance degrades on paraphrases and synonyms. Vector search finds content by meaning, this is critical once your knowledge base grows.
Avoid polyglot persistence. Running separate systems for vectors, documents, and transactions means four failure modes. Oracle AI Database simplifies your memory architecture.

AI developers are watching agent engineering evolve in real time, with leading teams openly sharing what works. One principle keeps showing up from the front lines: build within the LLM’s constraints.

In practice, two constraints dominate:

LLMs are stateless across sessions (no durable memory unless you bring it back in).
Context windows are bounded (and performance can degrade as you stuff more tokens in).

So “just add more context” isn’t a reliable strategy due to the quadratic cost of attention mechanisms and the degradation of reasoning capabilities as context fills up. The winning pattern is external memory + disciplined retrieval: store state outside the prompt (artifacts, decisions, tool outputs), then pull back only what matters for the current loop.

There’s also a useful upside: because models are trained on internet-era developer workflows, they’re unusually competent with developer-native interfaces: repos, folders, markdown, logs, and CLI-style interactions. That’s why filesystems keep showing up in modern agent stacks.

This is where the debate heats up: “files are all you need” for agent memory. Most arguments collapse because they treat interface, storage, and deployment as the same decision. They aren’t.

Filesystems are winning as an interface because models already know how to list directories, grep for patterns, read ranges, and write artifacts. Databases are winning as a substrate because once memory must be shared, audited, queried, and made reliable under concurrency, you either adopt database guarantees or painfully reinvent them.

In this piece, we give a systematic comparison of filesystems and databases for agent memory: where each approach shines, where it breaks down, and a decision framework for choosing the right foundation as you move from prototype to production.

Our aim is to educate AI developers on various approaches to agent memory, backed by performance guidance and working code.

All code presented in this article can be found here.

Understanding Agent Memory and Its Importance

Let’s take the common use case of building a Research Assistant with Agentic capabilities.

You build a Research Assistant agent that performs brilliantly in a demo; in the current execution, it can search arXiv, summarize papers, and draft a clean answer in a single run. Then you come back the next morning, start from a clean run, and then prompt the agent: “Continue from where we left off, and also compare Paper A to Paper B.” The agent responds as if it has never met you because LLMs are inherently stateless. Unless you send prior context back in, the model has no durable awareness of what happened in previous turns or previous sessions.

Once you move beyond single-turn Q&A into long-horizon tasks, deep research, multi-step workflows, and multi-agent coordination, you need a way to preserve continuity when the context window truncates, sessions restart, or multiple workers act on shared state. This takes us into the realm of leveraging systems of record for agents and introduces the concept of Agent Memory.

The Stateless LLM Problem

Why your Research Assistant forgets everything between sessions?

What is Agent Memory?

Agent memory is the set of system components and techniques that enable an AI agent to store, recall, and update information over time so it can adapt to new inputs and maintain continuity across long-horizon tasks. Core components typically include the language and embedding model, information retrieval mechanisms, and a persistent storage layer such as a database.

Types of Agent Memory

Types of Agent Memory

In practical systems, agent memory is usually classified into two distinct forms:

Short-term memory (working memory): whatever is currently inside the context window.
Long-term memory: a persistent state that survives beyond a single call or session (facts, artifacts, plans, prior decisions, tool outputs).

Concepts and techniques associated with agent memory all come together within the agent loop and the agent harness, as demonstrated in this notebook and explained later in this article.

Agent Loop and Agent Harness

The agent loop is the iterative execution cycle in which an LLM receives instructions from the environment and decides whether to generate a response or make a tool call based on its internal reasoning about the input provided in the current loop. This process repeats until the LLM produces a final output or an exit criterion is met. At a high level, the following operations are present within the agent loop:

Assemble context (user request + relevant memory + tool json schemas).
Call the model (plan, decide next action).
Take actions (tools, search, code execution, database queries).
Observe results (tool outputs, errors, intermediate artifacts).
Update memory (write transcripts, store artifacts, summarize, index).
Repeat until the task completes or hands control back to the user.

Anthropic’s guidance on long-running agents directly points to this: they describe harness practices that help agents quickly re-understand the state of work when starting with a fresh context window, including maintaining explicit progress artifacts.

The agent harness is the surrounding runtime and rules that make the loop reliable: how you wire tools, where you write artifacts, how you log/trace behavior, how you manage memory, and how you prevent the agent from drowning in context.

To complete the picture, the discipline of context engineering is heavily involved in the agent loop and aspect of the agent harness itself. Context engineering is the systematic design and curation of the content placed in an LLM’s context window so that the model receives high-signal tokens and produces the intended, reliable output within a fixed budget.

In this piece, we implement context engineering as a set of repeatable techniques inside the agent harness:

Context retrieval and selection: Pull only what is relevant (via grep for filesystem memory, via vector similarity and SQL filters for database memory).
Progressive disclosure: Start small (snippets, tails, line ranges) and expand only when needed.
Context offloading: Write large tool outputs and artifacts outside the prompt, then reload selectively.
Context reduction: Summarize or compact information when you approach a degradation threshold, then store the summary in durable memory so you can rehydrate later.

The concepts and explanations above set us up for the rest of the comparison we introduce in this piece. Now that we have the “why” and the moving parts (stateless models, the agent loop, the agent harness, and memory), we can evaluate the two dominant substrates teams are using today to make memory real: the filesystem and the database.

Filesystem-first Agentic Research Assistant

A filesystem-based memory architecture is not “the agent remembers everything forever”. It is the agent that can persist state and artifacts outside the context window and then pull them back selectively when needed. This aligns with two of the earlier-mentioned LLM constraints: a limited context window and statelessness.

In our Research Assistant, the filesystem becomes the memory substrate. Rather than injecting a large number of tools and extensive documentation into the LLM's context window (which would inflate the token count and trigger early summarization), we store them on disk and let the agent search and selectively read what it needs. This matches with what the Applied AI team at Cursor calls “Dynamic Context Discovery”: write large output to files, then let the agent tail and read ranges as required.

Our FSAgent and demo is using valid filesystem-OS related operations (such as tail and cat to read the contents of files; but that this is a very "simplified" approach, with a limited number of operations for demonstration purposes, and the capabilities offered in the file system can be optimized (with other commands and implementations).

On the other hand, it's a great start for people to get familiarized with tool access and how file system memory is achieved.

Semantic memory (durable knowledge): papers and reference docs saved as markdown.
Episodic memory (experience): conversation transcripts + tool outputs per session/run.
Procedural memory (how to work): “rules” / instructions files (e.g., CLAUDE.md / AGENTS.md) that shape behavior across sessions.

What does this look like in tooling?

Before we jump into the code, here’s the minimal tool surface we provide to the agent in the table below. Notice the pattern: instead of inventing specialized “memory APIs,” we expose a small set of filesystem primitives and let the agent compose them (very Unix).

Tool	What it does	Output
arxiv_search_candidates(query, k=5)	Searches arXiv and returns a JSON list of candidate papers with IDs, titles, authors, and abstracts.	JSON string of paper candidates
fetch_and_save_paper(arxiv_id)	Fetches full paper text (PDF → text) and saves to `semantic/knowledge_base/<id>.md`. Avoids routing full content through the LLM.	File path
read_file(path)	Reads a file from disk and returns its contents in full (use sparingly).	Full file contents
tail_file(path, n_lines=80)	Reads the last N lines of a file (first step for large files).	Last N lines
read_file_range(path, start_line, end_line)	Reads a line range to “zoom in” without loading everything.	Selected line range
grep_files(pattern, root_dir, file_glob)	Grep-like search across files to find relevant passages quickly.	Matches with file path + line number
list_papers()	Lists all locally saved papers in `semantic/knowledge_base/`.	List of filenames
conversation_to_file(run_id, messages)	Appends conversation entries to one transcript file per run in `episodic/conversations/`.	File path
summarise_conversation_to_file(run_id, messages)	Saves full transcript, then writes a compact summary to `episodic/summaries/`.	Dict with transcript + summary paths
monitor_context_window(messages)	Estimates current context usage (tokens used/remaining).	Dict with token stats

This design directly reflects what the AI ecosystem is converging on: a filesystem and a handful of core tools, rather than an explosion of bespoke tools.

Progressive reading (read, tail, range)

The first memory principle implementation is simple: don’t load large files unless you must. Filesystems are excellent at sequential read/write and work naturally with tools like grep and log-style access. This makes them a strong fit for append-only transcript and artifact storage.

That’s why we implement three reading tools:

Read everything (rare),
Read the end (common for logs/transcripts)
Read a slice (common for zooming into a match)

The tools below were implemented in Python and converted into objects callable by a langchain agent using the @tool decorator from the langchain agent module.

First is the read_file tool, the “load it all” option. This tool is useful when the file is small, or you truly need the full artifact, but it’s intentionally not the default because it can expand the context window.

@tool
def read_file(path: str) -> str:
 p = Path(path)
 if not p.exists():
 return f"File not found: {path}"
 return p.read_text(encoding="utf-8")

The tail_file function is the first step for large files. It grabs the end of a log/transcript to quickly see the latest or most relevant portion before deciding whether to read more.

@tool
def tail_file(path: str, n_lines: int = 80) -> str:
 p = Path(path)
 if not p.exists():
 return f"File not found: {path}"
 lines = p.read_text(encoding="utf-8").splitlines()
 return "\n".join(lines[-max(1, n_lines):])

The read_file_range function is seen as the surgical tool that is used once you’ve located the right region (often via grep or after a tail), pulls in just the exact line span you need, so the agent stays token-efficient and grounded.

@tool
def read_file_range(path: str, start_line: int, end_line: int) -> str:
 p = Path(path)
 if not p.exists():
 return f"File not found: {path}"
 lines = p.read_text(encoding="utf-8").splitlines()
 start = max(0, start_line)
 end = min(len(lines), end_line)
 if start >= end:
 return f"Empty range: {start_line}:{end_line} (file has {len(lines)} lines)"
 return "\n".join(lines[start:end])

Again, this is essentially dynamic context discovery in a microcosm: load a small view first, then expand only when needed.

Grep-style search (find first, read second)

A filesystem-based agent should quickly find relevant material and pull only the exact slices it needs. This is why grep is such a recurring theme in the agent tooling conversation: it gives the model a fast way to locate relevant regions before spending tokens to pull content.

Here’s a simple grep-like tool that returns line-numbered hits so the agent can immediately jump to read_file_range:

@tool
def grep_files(
pattern: str,
root_dir: str = "semantic",
file_glob: str = "**/*.md",
max_matches: int = 200,
ignore_case: bool = True,
) -> str:

root = Path(root_dir)
if not root.exists():
return f"Directory not found: {root_dir}"

flags = re.IGNORECASE if ignore_case else 0
try:
rx = re.compile(pattern, flags)
except re.error as e:
return f"Invalid regex pattern: {e}"

matches = []
for fp in root.glob(file_glob):
if not fp.is_file():
continue
try:
with open(fp, 'r', encoding='utf-8', errors='ignore') as f:
for i, line in enumerate(f, start=1):
if rx.search(line):
matches.append(f"{fp.as_posix()}:{i}: {line.strip()}")
if len(matches) >= max_matches:
return "\n".join(matches) + "\n\n[TRUNCATED: max_matches reached]"
except Exception:
continue

if not matches:
return "No matches found."
return "\n".join(matches)

One subtle but important detail in our grep_files implementation is how we read files. Rather than loading entire files into memory with read_text().splitlines(), we iterate lazily with for line in open(fp), which streams one line at a time and keeps memory usage constant regardless of file size.

This aligns with the "find first, read second" philosophy: locate what you need without loading everything upfront. For readers interested in maximum performance, the full notebook also includes a grep_files_os_based variant that shells out to ripgrep or grep, leveraging OS-level optimizations like memory-mapped I/O and SIMD instructions. In practice, this pattern (“search first, then read a range”) is one reason filesystem agents can feel surprisingly strong on focused corpora: the agent iteratively narrows the context instead of relying on a single-shot retrieval query.

Tool outputs as files: keeping big JSON out of the prompt

One of the fastest ways to blow up your context window is to return large JSON payloads from tools. Cursor’s approach is to write these results to files and let the agent inspect them on demand (often starting with tail).

That’s exactly why our folder structure includes a tool_outputs/<session_id>/ directory: it acts like an “evidence locker” for everything the agent did, without forcing those payloads into the current context.

{
 "ts_utc": "2026-01-27T12:41:12.135396+00:00",
 "tool": "arxiv_search_candidates",
 "input": "{'query': 'memgpt'}",
 "output": "content='[\\n {\\n \"arxiv_id\": \"2310.08560v2\",\\n \"entry_id\": \"http://arxiv.org/abs/2310.08560v2\",\\n \"title\": \"MemGPT: Towards LLMs as Operating Systems\",\\n \"authors\": \"Charles Packer, Sarah Wooders, Kevin Lin, Vivian Fang, Shishir G. Patil, Ion Stoica, Joseph E. Gonzalez\",\\n \"published\": \"2024-02-12\",\\n \"abstract\": ...msPnaMxOl8Pa'"
}

Putting it together: the agent toolset

Before we create the agent, we bundle the tools into a small, composable toolbox. This matches the broader trend: agents often perform better with a smaller tool surface, less choice paralysis (aka context confusion), fewer weird and overlapping tool schemas, and more reliance on proven filesystem workflows.

FS_TOOLS = [
 arxiv_search_candidates, # search arXiv for relevant research papers
 fetch_and_save_paper, # fetch paper text (PDF->text) and save to semantic/knowledge_base/<id>.md
 read_file, # read a file in full (use sparingly)
 tail_file, # read end of file first
 read_file_range, # read a specific line range
 conversation_to_file, # append conversation entries to episodic memory
 summarise_conversation_to_file, # save transcript + compact summary
 monitor_context_window, # estimate token usage
 list_papers, # list saved papers
 grep_files # grep-like search over files
]

The “filesystem-first” system prompt: policy beats cleverness

Filesystem tools alone aren’t enough, you also need a reading policy that keeps the agent's token usage efficient and grounded. This is the same reason CLAUDE.md, AGENTS.md, and SKILLS.md matter: they’re procedural memory that is applied consistently across sessions.

Key policies we encode below:

Store big artifacts on disk (papers, tool outputs, transcripts).
Prefer grep + range reads over full reads.
Use tail first for large files and logs.
Be explicit about what you actually read (grounding).

Below is the implementation of an agent using the langchain framework.

fs_agent = create_agent(
 model=f"openai:{os.getenv('OPENAI_MODEL', 'gpt-4o-mini')}",
 tools=FS_TOOLS,
 system_prompt=(
 "You are a conversational research ingestion agent.\n\n"
 "Core behavior:\n"
 "- When asked to find a paper: use arxiv_search_candidates, pick the best arxiv_id, "
 "then call fetch_and_save_paper to store the full text in semantic/knowledge_base/.\n"
 "- Papers/knowledge base live in semantic/knowledge_base/.\n"
 "- Conversations (transcripts) live in episodic/conversations/ (one file per run).\n"
 "- Summaries live in episodic/summaries/.\n"
 "- Conversation may be summarised externally; respect summary + transcript references.\n"
 ),
)

What the memory footprint looks like on disk

After running the agent, you end up with a directory layout that makes the agent’s “memory” tangible and inspectable. In your example, the agent produces:

episodic/conversations/fsagent_session_0010.md — the session transcript (episodic memory)
episodic/tool_outputs/fsagent_session_0010/*.json — tool results saved as files (evidence + replay)
semantic/knowledge_base/*.md — saved papers (semantic memory)

That is exactly the point of filesystem-first memory: the model doesn’t “remember” by magically retaining state; it “remembers” because it can re-open, search, and selectively read its prior artifacts.

This is also why so many teams keep rediscovering the same pattern: files are a simple abstraction, and agents are surprisingly good at using them.

Advantages of File Systems In AI Agents

In the previous section, we showed what a filesystem‑first memory harness looks like in practice: the agent writes durable artifacts (papers, tool outputs, transcripts) to disk, then “remembers” by searching and selectively reading only the parts it needs.

This approach works because it directly addresses two core constraints of LLMs: limited context windows and inherent statelessness. Once those constraints are handled, it becomes clear why file systems so often become the default interface for early agent systems.

Pretraining‑native interface: LLMs have ingested massive amounts of repos, docs, logs, and README‑driven workflows, so folders and files are a familiar operating surface.
Simple primitives, strong composition: A small action set (list/read/write/search) composes into sophisticated behavior without needing schemas, migrations, or query planning.
Token efficiency via progressive disclosure: Retrieve via search, then load a small slice (snippets, line ranges) instead of dumping entire documents into the prompt.
Natural home for artifacts and evidence: Transcripts, intermediate results, cached documents, and tool outputs fit cleanly as files and remain human‑inspectable.
Debuggable by default: You can open the directory and see exactly what the agent saved, what tools returned, and what the agent could have referenced.
Portability: A folder is easy to copy, zip, diff, version, and replay elsewhere, great for demos, reproducibility, and handoffs.
Low operational overhead: For PoCs and MVPs, you get persistence and structure without provisioning extra infrastructure.

In practice, filesystem memory excels when the workload is artifact‑heavy (research notes, paper dumps, transcripts), when you want a clear audit trail, and when iteration speed matters more than sophisticated retrieval. It also encourages good agent hygiene: write outputs down, cite sources, and load only what you need.

Disadvantages of Filesystems In AI Agents

But, unfortunately, it doesn’t end there. The same strengths that make files attractive, simplicity, relatively low cost, and fast implementation, can quickly become bottlenecks once you promote these systems into production, where they are expected to behave like a shared, reliable memory platform.

As soon as an agent moves beyond single-user prototypes into real-world scenarios, where concurrent reads and writes are the norm and robustness under load is non-negotiable, filesystems start to show their limits.

Weak concurrency guarantees by default: Multiple processes can overwrite or interleave writes unless you implement locking correctly. Even then, locking semantics vary across platforms and network filesystems.
No ACID transactions: You don’t get atomic multi-step updates, isolation between writers, or durable commit semantics without building them. Partial writes and mid-operation failures can leave memory in inconsistent states.
Search quality is usually brittle: Keyword/grep-style retrieval misses meaning, synonyms, and paraphrases.
Scaling becomes “death by a thousand files”: Directory bloat, fragmented artifacts, and expensive scans make performance degrade as memory grows, especially if you rely on repeated full-folder searches.
Indexing is DIY: The moment you want fast retrieval, deduplication, ranking, or recency weighting, you end up maintaining your own indexes and metadata stores (which, being honest here…is basically a database).
Metadata and schema drift: Agents inevitably accumulate extra fields (source URLs, timestamps, embeddings, tags). Keeping those consistent across files is harder than enforcing constraints in tables.
Poor multi-user / multi-agent coordination: Shared memory across agents means shared state. Without a central coordinator, you’ll hit race conditions, inconsistent views, and an unclear “source of truth.”
Harder auditing at scale: Files are human-readable, but reconstructing “what happened” across many runs and threads becomes messy without structured logs, timestamps, and queryable history.
Security and access control are coarse: Permissions are filesystem-level, not row-level. It’s hard to enforce “agent A can read X but not Y” without duplicating data or adding an auth layer.

The core pattern is that filesystem memory stays attractive until you need correctness under concurrency, semantic retrieval, or structured guarantees. At that point, you either accept the limitations (and keep the agent single-user/single-process) or you adopt a database.

Database For Agent Memory

By this point, most AI developers can see why filesystem first agent implementations are having a moment. It is a familiar interface, easy to prototype with, and our agents can “remember” by writing artifacts to disk and reloading them later via search plus selective reads. For a single developer on a laptop, that is often enough. But once we move beyond “it works on my laptop” and start supporting developers who ship to thousands or millions of users, memory stops being a folder of helpful files and becomes a shared system that has to behave predictably under load.

Databases were created for the exact moment when “a pile of files” stops being good enough because too many people and processes are touching the same data. One of the most-cited origin stories of the database dates to the Apollo era. IBM, alongside partners, built what became IMS to manage complex operational data for the program, and early versions were installed in 1968 at the Rockwell Space Division, supporting NASA. The point was not simply storage. It was coordination, correctness, and the ability to trust shared data while many activities were happening simultaneously.

That same production reality is what pushes agent memory toward databases today.

When agent memory must handle concurrent reads and writes, preserve an auditable history of what happened, support fast retrieval across many sessions, and enforce consistent updates, we want database guarantees rather than best-effort file conventions.

Oracle has been solving these exact problems since 1979, when we shipped the first commercial SQL database. The goal then was the same as now: make shared state reliable, portable, and trustworthy under load.

On that note, allow us to show how this can work in practice.

Database-first Research Assistant

In the filesystem first section, our Research Assistant “remembered” by writing artifacts to disk and reloading them later using cheap search plus selective reads. That is a great starting point. But when we want memory that is shared, queryable, and reliable under concurrent use, we need a different foundation.

In this iteration of our agent, we keep the same user experience and the same high-level job. Search arXiv, ingest papers, answer follow-up questions, and maintain continuity across sessions. The difference is that memory now lives in the Oracle AI Database, where we can make it durable, indexed, filterable, and safe for concurrent reads and writes. We also achieve a clean separation between two memory surfaces: structured history in SQL tables and semantic recall via vector search.

The result is what we call a MemAgent, an agent whose memory is not a folder of artifacts, but a queryable system. It is designed to support multi-threaded sessions, store full conversational history, store tool logs for debugging and auditing, and store a semantic knowledge base that can be searched by meaning rather than keywords.

Available tools for MemAgent

Before we wire up the agent loop, we need to define the tool surface that MemAgent can use to reason, retrieve, and persist knowledge. The design goal here is similar to the filesystem-first approach: keep the toolset small and composable, but shift the memory substrate from files to the database. Instead of grepping folders and reading line ranges, MemAgent uses vector similarity search to retrieve semantically relevant context, and it persists what it learns in a way that is queryable and reliable across sessions.

In practice, that means two things.

First, ingestion tools do not just “fetch” content; they also chunk and embed it so it becomes searchable later.
Second, retrieval tools are meaning-based rather than keyword-based, so the agent can find relevant passages even when the user paraphrases, uses synonyms, or asks higher-level conceptual questions.

The table below summarizes the minimal set of tools we expose to MemAgent and where each tool stores its outputs.

Tool	What it does
arxiv_search_candidates(query, k)	Searches arXiv for candidate papers
fetch_and_save_paper_to_kb_db(arxiv_id)	Fetches paper, chunks text, stores embeddings
search_knowledge_base(query, k)	Semantic search over stored papers
store_to_knowledge_base(text, metadata)	Manually store text with metadata

FSAgent and MemAgent can look similar from the outside because both can ingest papers, answer questions, and maintain continuity. The difference is what powers that continuity and how retrieval works when the system grows.

FSAgent relies on the operating system as its memory surface, which is great for iteration speed and human inspectability, but it typically relies on keyword-style discovery and file traversal. MemAgent treats memory as a database concern, which adds setup overhead, but unlocks indexed retrieval, stronger guarantees under concurrency, and richer ways to query and filter what the agent has learned.

Aspect	FSAgent (Filesystem)	MemAgent (Database)
Search	Keyword and grep	Semantic similarity
Persistence	Markdown files	SQL tables + vector indexes
Scalability	Directory traversal	Indexed queries
Query Language	Paths and regex	SQL + vector similarity
Setup Complexity	Minimal	Requires database runtime

Creating data stores with LangChain and Oracle AI Database

Before we start defining tables and vector stores, it is worth being explicit about the stack we are using and why. In this implementation, we are not building a bespoke agent framework from scratch.

We use LangChain as the LLM framework to abstract the agent loop, tool calling, and message handling, then pair it with a model provider for reasoning and generation, and with Oracle AI Database as the unified memory core that stores both structured history and semantic embeddings.

This separation is important because it mirrors how production agent systems are typically built. The agent logic evolves quickly, the model can be swapped, and the memory layer must remain reliable and queryable.

Think of this as the agent stack. Each layer has a clear job, and together they create an agent that is both practical to build and robust enough to scale.

Model provider (OpenAI): generates reasoning, responses, and tool decisions.
LLM framework (LangChain): provides the agent abstraction, tool wiring, and runtime orchestration.
Unified memory core (Oracle AI Database): stores durable conversational memory in SQL and semantic memory in vector indexes.

With that stack in place, the first step is simply to connect to the Oracle Database and initialize an embedding model. The database connection serves as the foundation for all memory operations, and the embedding model enables us to store and retrieve knowledge semantically through the vector store layer.

def connect_oracle(user, password, dsn="127.0.0.1:1521/FREEPDB1", program="langchain_oracledb_demo"):
 return oracledb.connect(user=user, password=password, dsn=dsn, program=program)

database_connection = connect_oracle(
 user="VECTOR",
 password="VectorPwd_2025",
 dsn="127.0.0.1:1521/FREEPDB1",
 program="devrel.content.filesystem_vs_dbs",
)

print("Using user:", database_connection.username)

embedding_model = HuggingFaceEmbeddings(
 model_name="sentence-transformers/paraphrase-mpnet-base-v2"
)

Next, we define the database schema to store our agent’s memory and prepare a clean slate for the demo. We separate memory into distinct tables so each type can be managed, indexed, and queried appropriately.

Installing the Oracle Database integration in the LangChain ecosystem is straightforward. You can add it to your environment with a single pip command:

pip install -U langchain-oracledb

Conversational history and logs are naturally tabular, while semantic and summary memory are stored in vector-backed tables through OracleVS. For reproducibility, we drop any existing tables from previous runs, making the notebook deterministic and avoiding confusing results when you re-run the walkthrough.

from langchain_oracledb.vectorstores import OracleVS
from langchain_oracledb.vectorstores.oraclevs import create_index
from langchain_community.vectorstores.utils import DistanceStrategy

CONVERSATIONAL_TABLE = "CONVERSATIONAL_MEMORY"
KNOWLEDGE_BASE_TABLE = "SEMANTIC_MEMORY"
LOGS_TABLE = "LOGS_MEMORY"
SUMMARY_TABLE = "SUMMARY_MEMORY"

ALL_TABLES = [
 CONVERSATIONAL_TABLE,
 KNOWLEDGE_BASE_TABLE,
 LOGS_TABLE,
 SUMMARY_TABLE
]

for table in ALL_TABLES:
 try:
 with database_connection.cursor() as cur:
 cur.execute(f"DROP TABLE {table} PURGE")
 except Exception as e:
 if "ORA-00942" in str(e):
 print(f" - {table} (not exists)")
 else:
 print(f" [FAIL] {table}: {e}")

database_connection.commit()

Create the vector stores and HNSW indexes

For this section, it is worth explaining what a “vector store” actually is in the context of agents. A vector store is a storage system that persists embeddings alongside metadata and supports similarity search, so the agent can retrieve items by meaning rather than keywords.

Instead of asking “which file contains this exact phrase”, the agent asks “which chunks are semantically closest to my question” and pulls back the best matches.

Under the hood, that usually means an approximate nearest neighbor index, because scanning every vector becomes prohibitively expensive as your knowledge base grows. HNSW is one of the most common indexing approaches for this style of retrieval.

In the code below, we create two vector stores using the langchain_oracledb module OracleVS, one for the knowledge base and one for summaries, both using cosine distance.

Second, it builds HNSW indexes so similarity search stays fast as memory grows, which is exactly what you want once your Research Assistant starts ingesting many papers and running over long-lived threads.

knowledge_base_vs = OracleVS(
 client=database_connection,
 embedding_function=embedding_model,
 table_name=KNOWLEDGE_BASE_TABLE,
 distance_strategy=DistanceStrategy.COSINE,
)

summary_vs = OracleVS(
 client=database_connection,
 embedding_function=embedding_model,
 table_name=SUMMARY_TABLE,
 distance_strategy=DistanceStrategy.COSINE,
)

def safe_create_index(conn, vs, idx_name):
 try:
 create_index(
 client=conn,
 vector_store=vs,
 params={"idx_name": idx_name, "idx_type": "HNSW"}
 )
 print(f" Created index: {idx_name}")
 except Exception as e:
 if "ORA-00955" in str(e):
 print(f" [SKIP] Index already exists: {idx_name}")
 else:
 raise

print("Creating vector indexes...")
safe_create_index(database_connection, knowledge_base_vs, "kb_hnsw_cosine_idx")
safe_create_index(database_connection, summary_vs, "summary_hnsw_cosine_idx")
print("All indexes created!")

Memory Manager

In the code below, we create a custom Memory manager. The Memory manager is the abstraction layer that turns raw database operations into “agent memory behaviours”. This is the part that makes the database-first agent easy to reason about.

SQL methods store and load conversational history by thread_id
Vector methods store and retrieve semantic memory by similarity search
Summary methods store compressed context and let us rotate the working set when we approach context limits

from langchain.tools import tool
from typing import List, Dict

class MemoryManager:
 """
 A simplified memory manager for AI agents using Oracle AI Database.
 """

 def __init__(self, conn, conversation_table: str, knowledge_base_vs, summary_vs, tool_log_table):
 self.conn = conn
 self.conversation_table = conversation_table
 self.knowledge_base_vs = knowledge_base_vs
 self.summary_vs = summary_vs
 self.tool_log_table = tool_log_table

 def write_conversational_memory(self, content: str, role: str, thread_id: str) -> str:
 thread_id = str(thread_id)
 with self.conn.cursor() as cur:
 id_var = cur.var(str)
 cur.execute(f"""
 INSERT INTO {self.conversation_table} (thread_id, role, content, metadata, timestamp)
 VALUES (:thread_id, :role, :content, :metadata, CURRENT_TIMESTAMP)
 RETURNING id INTO :id
 """, {"thread_id": thread_id, "role": role, "content": content, "metadata": "{}", "id": id_var})
 record_id = id_var.getvalue()[0] if id_var.getvalue() else None
 self.conn.commit()
 return record_id

 def load_conversational_history(self, thread_id: str, limit: int = 50) -> List[Dict[str, str]]:
 thread_id = str(thread_id)
 with self.conn.cursor() as cur:
 cur.execute(f"""
 SELECT role, content FROM {self.conversation_table}
 WHERE thread_id = :thread_id AND summary_id IS NULL
 ORDER BY timestamp ASC
 FETCH FIRST :limit ROWS ONLY
 """, {"thread_id": thread_id, "limit": limit})
 results = cur.fetchall()

 return [{"role": str(role), "content": content.read() if hasattr(content, 'read') else str(content)} for role, content in results]

 def mark_as_summarized(self, thread_id: str, summary_id: str):
 thread_id = str(thread_id)
 with self.conn.cursor() as cur:
 cur.execute(f"""
 UPDATE {self.conversation_table}
 SET summary_id = :summary_id
 WHERE thread_id = :thread_id AND summary_id IS NULL
 """, {"summary_id": summary_id, "thread_id": thread_id})
 self.conn.commit()
 print(f" Marked messages as summarized (summary_id: {summary_id})")

 def write_knowledge_base(self, text: str, metadata_json: str = "{}"):
 metadata = json.loads(metadata_json)
 self.knowledge_base_vs.add_texts([text], [metadata])

 def read_knowledge_base(self, query: str, k: int = 5) -> str:
 results = self.knowledge_base_vs.similarity_search(query, k=k)
 content = "\n".join([doc.page_content for doc in results])
 return f"""## Knowledge Base Memory: This are general information that is relevant to the question
### How to use: Use the knowledge base as background information that can help answer the question

{content}"""

 def write_summary(self, summary_id: str, full_content: str, summary: str, description: str):
 self.summary_vs.add_texts(
 [f"{summary_id}: {description}"],
 [{"id": summary_id, "full_content": full_content, "summary": summary, "description": description}]
 )
 return summary_id

 def read_summary_memory(self, summary_id: str) -> str:
 results = self.summary_vs.similarity_search(
 summary_id,
 k=5,
 filter={"id": summary_id}
 )
 if not results:
 return f"Summary {summary_id} not found."
 doc = results[0]
 return doc.metadata.get('summary', 'No summary content.')

 def read_summary_context(self, query: str = "", k: int = 5) -> str:
 results = self.summary_vs.similarity_search(query or "summary", k=k)
 if not results:
 return "## Summary Memory\nNo summaries available."

 lines = ["## Summary Memory", "Use expand_summary(id) to get full content:"]
 for doc in results:
 sid = doc.metadata.get('id', '?')
 desc = doc.metadata.get('description', 'No description')
 lines.append(f" - [ID: {sid}] {desc}")
 return "\n".join(lines)

Then we instantiate it:

memory_manager = MemoryManager(
 conn=database_connection,
 conversation_table=CONVERSATION_HISTORY_TABLE,
 knowledge_base_vs=knowledge_base_vs,
 tool_log_table=TOOL_LOG_TABLE,
 summary_vs=summary_vs
)

Creating the tools and agent

The database-first agent follows a simple, production-friendly pattern.

Persists every conversation turn as structured rows, including user and assistant messages with thread or run IDs and timestamps, so sessions are recoverable, traceable, and consistent across restarts.
Persists long-term knowledge in a vector-enabled store by chunking documents, generating embeddings, and storing them with metadata, so retrieval is semantic, ranked, and fast as the corpus grows.
Persists tool activity as first-class records that capture the tool name, inputs, outputs, status, errors, and key metadata, so agent behavior is inspectable, reproducible, and auditable.

On top of that, the agent actively manages context: it tracks token usage and periodically rolls older dialogue and intermediate state into durable summaries (and/or “memory” tables), so the working prompt stays small while the full history remains available on demand.

Ingest papers into the knowledge base vector store

This is the database-first equivalent of “fetch and save paper”. Instead of writing markdown files, we do three steps:

Load paper text from arXiv
Chunk it to respect the embedding model limits
Store chunks with metadata in the vector store, which gives us fast semantic search later

from datetime import datetime, timezone
from langchain_core.tools import tool
from langchain_community.document_loaders import ArxivLoader
from langchain_text_splitters import RecursiveCharacterTextSplitter

@tool
def fetch_and_save_paper_to_kb_db(
 arxiv_id: str,
 chunk_size: int = 1500,
 chunk_overlap: int = 200,
) -> str:
 loader = ArxivLoader(
 query=arxiv_id,
 load_max_docs=1,
 doc_content_chars_max=None,
 )
 docs = loader.load()
 if not docs:
 return f"No documents found for arXiv id: {arxiv_id}"

 doc = docs[0]

 title = (
 doc.metadata.get("Title")
 or doc.metadata.get("title")
 or f"arXiv {arxiv_id}"
 )

 entry_id = doc.metadata.get("Entry ID") or doc.metadata.get("entry_id") or ""
 published = doc.metadata.get("Published") or doc.metadata.get("published") or ""
 authors = doc.metadata.get("Authors") or doc.metadata.get("authors") or ""

 full_text = doc.page_content or ""
 if not full_text.strip():
 return f"Loaded arXiv {arxiv_id} but extracted empty text (PDF parsing issue)."

 splitter = RecursiveCharacterTextSplitter(
 chunk_size=chunk_size,
 chunk_overlap=chunk_overlap,
 )
 chunks = splitter.split_text(full_text)

 ts_utc = datetime.now(timezone.utc).isoformat()
 metadatas = []
 for i in range(len(chunks)):
 metadatas.append(
 {
 "source": "arxiv",
 "arxiv_id": arxiv_id,
 "title": title,
 "entry_id": entry_id,
 "published": str(published),
 "authors": str(authors),
 "chunk_id": i,
 "num_chunks": len(chunks),
 "ingested_ts_utc": ts_utc,
 }
 )

 knowledge_base_vs.add_texts(chunks, metadatas)

 return (
 f"Saved arXiv {arxiv_id} to {KNOWLEDGE_BASE_TABLE}: "
 f"{len(chunks)} chunks (title: {title})."
 )

We create two more tools below:

search_knowledge_base(query, k=5): Runs a semantic similarity search over the database-backed knowledge base and returns the top k most relevant chunks, so the agent can retrieve context by meaning rather than exact keywords.
store_to_knowledge_base(text, metadata_json="{}"): Stores a new piece of text into the knowledge base and attaches metadata (as JSON), which gets embedded and indexed so it becomes searchable in future queries.

import os
from langchain.tools import tool

@tool
def search_knowledge_base(query: str, k: int = 5) -> str:
 return memory_manager.read_knowledge_base(query, k)

@tool
def store_to_knowledge_base(text: str, metadata_json: str = "{}") -> str:
 memory_manager.write_knowledge_base(text, metadata_json)
 return "Successfully stored text to knowledge base."

Now we build the LangChain agent using the database-first tools.

from langchain.agents import create_agent

MEM_AGENT = create_agent(
 model=f"openai:{os.getenv('OPENAI_MODEL', 'gpt-4o-mini')}",
 tools=[search_knowledge_base, store_to_knowledge_base, arxiv_search_candidates, fetch_and_save_paper_to_kb_db],
)

Result Comparison: FSAgent vs MemAgent: End-to-End Benchmark (Latency + Quality)

At this point, the difference between a filesystem agent and a database-backed agent should feel less like a philosophical debate and more like an engineering trade-off. Both approaches can “remember” in the sense that they can persist state, retrieve context, and answer follow-up questions. The real test is what happens when you leave the tidy laptop demo and hit production realities: larger corpora, fuzzier queries, and concurrent workloads.

To make that concrete, we ran an end-to-end benchmark and measured the full agent loop per query—retrieval, context assembly, tool calls, model invocations, and the final answer—across three scenarios:

Small-corpus retrieval: a tight, keyword-friendly dataset to validate baseline retrieval and answer synthesis with minimal context.
Large-corpus retrieval: a larger dataset with more paraphrase variability to stress retrieval quality and context efficiency at scale.
Concurrent write integrity: a multi-worker stress test to evaluate correctness under simultaneous reads/writes (integrity, race conditions, throughput).

FSAgent vs MemAgent: End-to-End Benchmark (Latency + Quality)

From the result shown in the image above, two conclusions immediately stand out: latency and answer quality.

In our run, MemAgent generally finished faster end-to-end than FSAgent. That might sound counterintuitive if you assume “database equals overhead,” and sometimes it does.

But the agent loop is not dominated by raw storage primitives. It is dominated by how quickly you can find the right information and how little unnecessary context you force into the model, also known as context engineering. Semantic retrieval tends to return fewer, more relevant chunks (subject to tuning of the retrieval pipelines), which means less scanning, less paging through files, and fewer tokens burned on irrelevant text.

In this particular run, both agents produced similar-quality answers. That is not surprising. When the questions are retrieval-friendly and the corpus is small enough, both approaches can find the right passages. FSAgent gets there through keyword search and careful reading. MemAgent gets there through similarity search over embedded chunks. Different roads, similar destination.

And I think it’s worth zooming in on one nuance here. When the information to traverse is minimal in terms of character length and the query is keyword-friendly, the retrieval quality of both agents tends to converge. At that scale, “search” is barely a problem, so the dominant factor becomes the model’s ability to read and synthesise, not the retrieval substrate. The gap only starts to widen when the corpus grows, the wording becomes fuzzier, and the system must retrieve reliably under real-world constraints such as noise, paraphrases, and concurrency. Which it eventually does.

About the “LLM-as-a-Judge” metric

We also scored answers using an LLM-as-a-judge prompt. It is a pragmatic way to get directional feedback when you do not have labeled ground truth, but it is not a silver bullet. Judges can be sensitive to prompt phrasing, can over-reward fluency, and can miss subtle grounding failures.

If you are building this for production, treat LLM judging as a starting signal, not the finish line. The more reliable approach is a mix of:

Reference-based evaluation when you have ground truth, such as rubric grading, exact match, or F1-style scoring.
Retrieval-aware evaluation when context matters, such as context precision and recall, answer faithfulness, and groundedness. Tracing plus evaluation tooling so you can connect failures to the specific retrievals, tool calls, and context assembly decisions that caused them.

Even with a lightweight judge, the directional story remains consistent. As retrieval becomes more difficult and the system becomes busier, database-backed memory tends to perform better.

Large Corpus Benchmark: Why the gap widens as data grows

The large-corpus test is designed to stress the exact weakness of keyword-first memory. We intentionally made the search problem harder by growing the corpus and making the queries less “exact match.”

FSAgent with a concatenated corpus:
When you merge many papers into large markdown files, FSAgent becomes dependent on grep-style discovery followed by paging the right sections into the context window. It can work, but it gets brittle as the corpus grows:

If the user paraphrases or uses synonyms, exact keyword matches can fail.
If the keyword is too common, you get too many hits, and the agent has to sift through them manually.
When uncertain, the agent often loads larger slices “just in case,” which increases token count, latency, and the risk of context dilution.

MemAgent with chunked, embedded memory:
Chunking plus embeddings makes retrieval more forgiving and more stable:

The user does not need to match the source phrasing exactly.
The agent can fetch a small set of high-similarity chunks, keeping context tight.
Indexed retrieval remains predictable as memory grows, rather than requiring repeated scans of files.

The narrative takeaway is simple. Filesystems feel great when the corpus is small and the queries are keyword-friendly. As the corpus grows and the questions get fuzzier, semantic retrieval becomes the differentiator, and database-backed memory becomes the more dependable default.

The quality gap widens with scale. On a handful of documents, grep can brute-force its way to a reasonable answer: the agent finds a keyword match, pulls surrounding context, and responds.

But scatter the same information across hundreds of files, and keyword search starts missing the forest for the trees. It returns too many shallow hits or none when the user's phrasing doesn't match the source text verbatim. Semantic search, by contrast, surfaces conceptually relevant chunks even when the vocabulary differs. The result isn't just faster retrieval, it's more coherent answers with fewer hallucinated gaps. This is evident in our LLM judge evaluation on the large corpus benchmark, where FSAgent achieved a score of 29.7% while MemAgent reached 87.1%.

Concurrency Test: What production teaches you very quickly

We find that the real breaking point for filesystem memory is rarely retrieval. It is concurrency.

We ran three versions of the same workload under concurrent writes:

Filesystem without locking, where multiple workers append to the same file.
Filesystem with locking, where writes are guarded by file locks.
Oracle AI Database with transactions, where multiple workers write rows under ACID guarantees.

Then we measured two things:

Integrity, meaning, did we get the expected number of entries with no corruption?
Execution time, meaning how long the batch took end-to-end.

What we observed maps to what many teams discover the hard way.

Naive filesystem writes can be fast and still be wrong. Without locking, concurrent writes conflict with each other. You might get good throughput and still lose memory entries. If your agent’s “memory” is used for downstream reasoning, silent loss is not a performance issue. It is a correctness failure.

Locking fixes integrity, but now correctness is your job. With explicit locking, you can make filesystem writes safe. But you inherit the complexity. Lock scope, lock contention, platform differences, network filesystem behavior, and failure recovery all become part of your agent engineering work.

Databases make correctness the default. Transactions and isolation are exactly what databases were designed for. Yes, there is overhead. But the key difference is that you are not bolting correctness on after a production incident. You start with a system whose job is to protect the shared state.

And of course, you can take the file-locking approach, add atomic writes, build a write-ahead log, introduce retry and recovery logic, maintain indexes for fast lookups, and standardise metadata so you can query it reliably.

Eventually, though, you will realise you have not “avoided” a database at all.

You have just rebuilt one, only with fewer guarantees and more edge cases to own.

Conclusion: Is there a happy medium for AI Developers?

This isn’t a religious war between “files” and “databases.” It’s a question of what you’re optimizing for—and which failure modes you’re willing to own. If you’re building single-user or single-writer prototypes, filesystem memory is a great default. It’s simple, transparent, and fast to iterate on. You can open a folder and see exactly what the agent saved, diff it, version it, and replay it with nothing more than a text editor.

If you’re building multi-user agents, background workers, or anything you plan to ship at scale, a database-backed memory store is a safer foundation at that stage. At that stage, concurrency, integrity, governance, access control, and auditability matter more than raw simplicity. A practical compromise is a hybrid design: keep file-like ergonomics for artifacts and developer workflows, but store durable memory in a database that can enforce correctness.

And if you insist on filesystem-only memory in production, treat locking, atomic writes, recovery, indexing, and metadata discipline as first-class engineering work. Because the moment you do that seriously, you’re no longer “just using files”—you’re rebuilding a database.

One last trap worth calling out: polyglot persistence.

Many AI stacks drift into an anti-pattern: a vector DB for embeddings, a NoSQL DB for JSON, a graph DB for relationships, and a relational DB for transactions. Each product is “best at its one thing,” until you realize you’re operating four databases, four security models, four backup strategies, four scaling profiles, and four cascading failure points.

Coordination becomes the tax. You end up building glue code and sync pipelines just to make the system feel unified to the agent. This is why converged approaches matter in agent systems: production memory isn’t only about storing vectors—it’s about storing operational history, artifacts, metadata, and semantics under one consistent set of guarantees.

For AI Developers, your application acts as an integration layer for multiple storage engines, each with different access patterns and operational semantics. You end up building glue code, sync pipelines, and reconciliation logic just to make the system feel unified to the agent.

Of course, production data is inherently heterogeneous. You will inevitably deal with structured, semi-structured, unstructured text, embeddings, JSON documents, and relationship-heavy data.

The point is not that “one model wins”.

The point is that when you understand the fundamentals of data management, reliability, indexing, governance, and queryability, you want a platform that can store and retrieve these forms without turning your AI infrastructure into a collection of loosely coordinated subsystems.

This is the philosophy behind Oracle’s converged database approach, which is designed to support multiple data types and workloads natively within a single engine. In the world of agents, that becomes a practical advantage because we can use Oracle as the unified memory core for both operational memory (SQL tables for history and logs) and semantic memory (vector search for retrieval).

Frequently Asked Questions

What is AI Agent memory? AI agent memory is the set of system components and techniques that enable an AI agent to store, recall, and update information over time. Because LLMs are inherently stateless—they have no built-in ability to remember previous sessions—agent memory provides the persistence layer that allows agents to maintain continuity across conversations, learn from past interactions, and adapt to user preferences.
Should I use a filesystem or a database for an AI agent's memory? It depends on your use case. Filesystems excel at single-user prototypes, artifact-heavy workflows, and rapid iteration—they're simple, transparent, and align with how LLMs naturally operate. Databases become essential when you need concurrent access, ACID transactions, semantic retrieval, or shared state across multiple agents or users. Many production systems use a hybrid approach: file-like interfaces for agent interaction, with database guarantees underneath.
How do I build an AI agent with long-term memory? Start by separating memory types: working memory (current context), semantic memory (knowledge base), episodic memory (interaction history), and procedural memory (behavioral rules). Implement storage: a filesystem for prototypes and a database for production. Add retrieval tools that the agent can call. Build a summarization to compress the old context. Test with multi-session scenarios where the agent must recall information from previous conversations.
What are semantic, episodic, and procedural memory in AI agents? These terms, borrowed from cognitive science, describe different types of agent memory. Semantic memory stores durable knowledge and facts (like saved documents or reference materials). Episodic memory captures experiences and interaction history (conversation transcripts, tool outputs). Procedural memory encodes how the agent should behave—instructions, rules, files like CLAUDE.md, and learned workflows that shape behavior across sessions.
What is the best database for AI applications? The best database depends on your requirements. For AI agent memory specifically, you need: vector search capability for semantic retrieval, SQL or structured queries for history and metadata, ACID transactions if multiple agents share state, and scalability as your memory corpus grows. Converged databases that combine these capabilities—like Oracle AI Database—reduce operational complexity versus running separate specialized systems.

How I Added Memory to an AI Agent Using Spring AI and Oracle AI Database

Wojtek Pluta — Tue, 14 Apr 2026 18:17:32 +0000

Practical guide with a sample app for adding episodic, semantic, and procedural memory to an AI agent using Spring AI and a single Oracle AI Database instance.

This post shows how to build three types of persistent memory — episodic (chat history), semantic (domain knowledge via hybrid search), and procedural (tool calls) — using Spring AI and a single Oracle AI Database instance. Here's the code: GitHub repo

Key Takeaways

LLMs forget everything between sessions. Episodic, semantic, and procedural memory fix that — chat history, domain knowledge retrieval, and actionable tool calls, all persisted in the database.
One database handles it all. Oracle AI Database stores chat history, runs hybrid vector search, and hosts the application tables — no need to bolt on a separate vector database or search engine.
Hybrid search beats pure vector search. Combining dense embeddings with keyword matching (fused via Reciprocal Rank Fusion) means the agent finds documents by meaning and by exact terms like order IDs.
Embeddings stay in the database. A loaded ONNX model computes embeddings on insert — no external embedding API calls, no extra infrastructure.
Agent memory doesn't have to be complicated. Two advisors, six tools backed by real database tables, one database, and the LLM stops forgetting.

Why This Matters

Every LLM has the same problem: it forgets everything the moment the conversation ends, sometimes even during long conversations. Spend twenty minutes explaining your project setup, your constraints, your preferences — and it nails the answer. Close the tab, open a new session, and it greets you like a stranger. All that context, gone.

If you want to build an AI agent — one that remembers context, understands your domain, and can take action — you need to give it memory. Practical memory: capturing what users say, retrieving learned facts and executing real workflows backed by database queries.

This post walks through a proof of concept that does exactly that. Three types of memory, one database, and minimal code.

What You'll Learn

How to implement episodic, semantic, and procedural memory for an AI agent using Spring AI advisors and @Tool methods
How to use Oracle AI Database Hybrid Vector Indexes (vector and keyword search fused with Reciprocal Rank Fusion) for semantic retrieval
How to compute embeddings in-database with a loaded ONNX model — no external embedding API calls
How to wire it all together with one database, one connection pool, and minimal configuration

Architecture Overview

System architecture for a memory-enabled AI assistant using Streamlit, Spring Boot, Oracle AI Database 26ai, Ollama, and @Tool methods.

The agent runs on Spring Boot with Spring AI, with Ollama handling local chat inference (qwen2.5). Oracle AI Database 26ai stores all three memory types: a relational table for chat history (episodic), a hybrid vector index for domain knowledge retrieval (semantic), and application tables queried by @Tool methods (procedural). Embeddings are computed in-database by a loaded ONNX model (all-MiniLM-L12-v2), eliminating the need for external embedding API calls. A Streamlit frontend provides a simple web UI.

Both advisors and all six tools run on every request. The agent simultaneously remembers what you said, retrieves relevant knowledge, and executes tasks — all from a single Oracle Database instance. No second database. One connection pool, one set of credentials, one system to monitor.

Prerequisites

Java 21
Gradle 8.14
Oracle AI Database 26ai (container or instance)
Ollama with the qwen2.5 model pulled
Python 3.x with Streamlit (optional, for the web UI)
The ONNX model file (all_MiniLM_L12_v2.onnx) for in-database embeddings

Step-by-Step Guide

Step 1: Set Up the Oracle AI Database and Hybrid Vector Index

Start an Oracle AI Database instance, then run the one-time setup script to load the ONNX embedding model and create the hybrid vector index. This enables in-database embeddings and combined vector and keyword search.

-- Load the ONNX model for in-database embeddings
BEGIN
  DBMS_VECTOR.LOAD_ONNX_MODEL(
    directory  => 'DM_DUMP',
    file_name  => 'all_MiniLM_L12_v2.onnx',
    model_name => 'ALL_MINILM_L12_V2'
  );
END;
/

-- Create a hybrid index: vector similarity + Oracle Text keyword search
CREATE HYBRID VECTOR INDEX POLICY_HYBRID_IDX
ON POLICY_DOCS(content)
PARAMETERS('MODEL ALL_MINILM_L12_V2 VECTOR_IDXTYPE HNSW');

Once the index is created, embeddings are computed automatically on insert — no external embedding API calls required.

Step 2: Define Procedural Memory with @Tool Methods

Procedural memory is implemented as @Tool-annotated methods in a Spring component. These methods execute real database queries via JPA, which the LLM can call when it decides a task requires action, not just an answer. The @Tool description tells the LLM when to use each method, and @ToolParam defines the inputs.

@Tool(description = "Look up the status of a customer order by its order ID. " +
        "Returns the current status including shipping information.")
public String lookupOrderStatus(
        @ToolParam(description = "The order ID to look up, e.g. ORD-1001") String orderId) {
    // Fetches order from DB via JPA, returns formatted status string
}

@Tool(description = "Initiate a product return for a given order. " +
        "Validates the order exists, checks that it is in DELIVERED status, " +
        "and verifies the return is within the 30-day return window.")
public String initiateReturn(
        @ToolParam(description = "The order ID to return") String orderId,
        @ToolParam(description = "The reason for the return") String reason) {
    // Validates order exists, checks DELIVERED status and 30-day window, updates status via JPA
}

The full class has six tools: getCurrentDateTime, listOrders, lookupOrderStatus, initiateReturn, escalateToSupport, and listSupportTickets. The LLM decides when to act; the Java methods define how.

Step 3: Wire the Controller with Advisors and Tools

The controller builds a single ChatClient with two advisors and six tools. MessageChatMemoryAdvisor handles episodic memory by loading the last 100 messages for the current conversation from a relational table and persisting each new exchange. RetrievalAugmentationAdvisor, with a custom OracleHybridDocumentRetriever, handles semantic memory by calling DBMS_HYBRID_VECTOR.SEARCH to run vector and keyword search in parallel, fused with Reciprocal Rank Fusion (RRF). The tools are registered via .defaultTools(agentTools).

@RestController
@RequestMapping("/api/v1/agent")
public class AgentController {

    public AgentController(ChatClient.Builder builder,
                           JdbcChatMemoryRepository chatMemoryRepository,
                           JdbcTemplate jdbcTemplate,
                           AgentTools agentTools) {
        // Builds a ChatClient with:
        //   - MessageChatMemoryAdvisor (episodic: last 100 messages per conversation)
        //   - RetrievalAugmentationAdvisor + OracleHybridDocumentRetriever (semantic: hybrid search)
        //   - AgentTools via .defaultTools() (procedural: 6 @Tool methods)
        //   - System prompt defining the agent persona and tool usage rules
    }

    @PostMapping("/chat")
    public ResponseEntity<String> chat(
            @RequestBody String message,
            @RequestHeader("X-Conversation-Id") String conversationId) {
        // Sends message to ChatClient with conversation ID, returns LLM response
    }

    @PostMapping("/knowledge")
    public ResponseEntity<String> addKnowledge(@RequestBody String content) {
        // Inserts text into POLICY_DOCS table via JDBC (hybrid index handles embedding)
    }
}

All three memory types run on every request. The agent simultaneously remembers what you said, retrieves relevant knowledge, and executes tasks.

Step 4: Implement the Hybrid Document Retriever

The custom OracleHybridDocumentRetriever implements Spring AI's DocumentRetriever interface and calls DBMS_HYBRID_VECTOR.SEARCH via JDBC. It passes a JSON parameter specifying the hybrid index, the RRF scorer, and a keyword match clause, bypassing OracleVectorStore entirely for retrieval.

Why hybrid instead of pure vector search? Dense embeddings capture meaning — a query about "return policy" can match documents about refunds and exchanges. But they're weaker on exact terms: a query for "ORD-1001" performs poorly because embeddings encode semantics, not keywords. Hybrid search addresses both: the vector side captures meaning, the keyword side handles exact matches, and RRF merges the result sets by rank position.

Step 5: Run the Application

Start the Oracle DB container, install Ollama, pull the chat model, run the Spring Boot backend with the local profile, and optionally start the Streamlit UI.

The assistant recalls the customer’s name, prior issue, and order details to continue support without repeating context.

Optionally, quick test with cURL:

curl -X POST http://localhost:8080/api/v1/agent/chat \
  -H "Content-Type: text/plain" \
  -H "X-Conversation-Id: test-1" \
  -d "What orders do I have?"

The agent will remember your name and details, or use procedural memory (the listOrders tool) to query the database and return the demo orders. Try "What is your return policy?" to see semantic memory (hybrid search over policy documents) in action. Then type "My name is Victor" followed later by "What's my name?" to test episodic memory.

Frequently Asked Questions

Why does the agent need three types of memory instead of just chat history?
Chat history (episodic memory) only covers what was said in the conversation. Semantic memory lets the agent retrieve domain knowledge — like return policies or shipping rules — that was never mentioned in chat. Procedural memory lets it take actions, such as looking up an order or initiating a return, by calling tool methods backed by real database queries.

Why use hybrid search instead of plain vector similarity?
Pure vector search matches by meaning, which works well for natural-language questions but struggles with exact terms like product codes or order IDs. Hybrid search runs vector and keyword search in parallel and merges the results by rank position (Reciprocal Rank Fusion), so the agent finds relevant documents whether the match is semantic, lexical, or both.

Do I need a separate vector database to build this?
No. Oracle AI Database 26ai supports relational tables, hybrid vector indexes, and full-text search in a single instance. The POC uses one connection pool and one set of credentials for chat history, vector retrieval, and all application data.

How are the embeddings generated?
An ONNX model (all-MiniLM-L12-v2) is loaded directly into Oracle AI Database. Embeddings are computed automatically whenever a row is inserted into the indexed table — no external API calls and no separate embedding service required.

What are the limitations?
This is a proof of concept. There's no authentication, no rate limiting, and no streaming responses. It demonstrates the architecture and approach — production use would require hardening those areas.

Next Steps

Author

Victor Martin Alvarez – Senior Principal Product Manager, Oracle AI DatabaseBuilding AI-powered applications with Oracle AI Database and Spring AI.LinkedIn

Build an Ultra-Lightweight, Local-First AI Assistant with Persistent Memory

Wojtek Pluta — Tue, 14 Apr 2026 14:16:59 +0000

Key Takeaways

PicoOraClaw is a lightweight, offline AI assistant with local inference via Ollama.
Oracle AI Database stores sessions, memories, transcripts, prompts, and state with durable ACID-backed persistence.
Semantic recall happens in the database using ONNX embeddings and vector search, removing the need for an external embedding API.
The same project runs locally for development and can move to Oracle Cloud Infrastructure (OCI) when you need a managed deployment.

Local-First AI Assistant with Built-In Memory

If you want to build an AI assistant that runs locally, retains meaningful context, and can move to the cloud without rearchitecting the stack, PicoOraClaw is a strong starting point. It pairs a lightweight Go runtime with local inference via Ollama and uses Oracle AI Database as the persistent memory layer.

This matters for developers building edge AI systems, private assistants, or local-first prototypes. Instead of stitching together separate services for storage, embeddings, and retrieval, you can keep memory, state, and semantic recall within Oracle AI Database while still running a lightweight local runtime.

What is PicoOraClaw?

PicoOraClaw is a fork of PicoClaw that keeps the runtime lightweight, uses Ollama as the default inference backend, and adds Oracle AI Database for persistent memory and state.

PicoClaw is an independent open-source project initiated by Sipeed, written entirely in Go from scratch - not a fork of OpenClaw, NanoBot, or any other project.

The result is a developer-friendly architecture for assistants that retain meaningful context and retrieve it semantically, rather than relying on keyword matching. PicoOraClaw targets use cases such as edge AI, IoT, private assistants, and local-first developer workflows, where a small footprint and persistent context matter more than a cloud-only approach. See the PicoOraClaw repository.

Features

Lightweight Go runtime for local and edge-friendly assistant workflows
Oracle AI Database-backed memory, state, and semantic recall
Ollama as the default local inference backend
Support for multiple LLM providers including OpenRouter, Anthropic, OpenAI, Gemini, DeepSeek, Groq, and Zhipu
Default: Oracle AI Database Free with Oracle AI Vector Search for semantic memory
Optional Autonomous AI Database path for managed cloud deployment
Graceful file-based fallback when Oracle is unavailable

Why Choose PicoOraClaw vs. Standard PicoClaw?

If you're already familiar with PicoClaw, PicoOraClaw adds a more complete memory layer for developers who need durable context and semantic recall.

Oracle AI Database as the persistent backend for memories, sessions, transcripts, state, notes, prompts, and configuration
In-database ONNX embeddings and vector search for semantic memory using VECTOR_EMBEDDING() and VECTOR_DISTANCE()
Ollama as the default local LLM backend with no cloud dependency
One-click OCI deployment with Oracle AI Database Free, Ollama, and the PicoOraClaw gateway
Optional OCI Generative AI integration through the included oci-genai proxy
oracle-inspect CLI support for inspecting what the assistant stores without writing SQL

Features of PicoOraClaw

What PicoOraClaw enables:

Unified Memory Core - PicoOraClaw uses Oracle AI Database to store sessions, transcripts, notes, prompts, configuration, and long-term memories in a single persistent system. The database is the memory substrate for long-running, context-aware assistant behavior.
Build Fast with Modern APIs - Get started locally with a lightweight runtime, Ollama for local inference, and Oracle AI Database Free for semantic memory.
A Robust Scaling Path - Start locally, keep the same overall architecture, and move to OCI later when you need a managed environment.

Installation - Quick Start (in 5 minutes!)

For the fastest path to a working setup, use the PicoOraClaw one-command installer. It clones, configures, and runs the application in a single step:

curl -fsSL https://raw.githubusercontent.com/oracle-devrel/oracle-ai-developer-hub/refs/heads/main/apps/picooraclaw/install.sh | bash

To control the workspace path, clone the Oracle DevRel repository directly and build from the PicoOraClaw app directory.

Follow the steps below:

Prerequisites

Go 1.24+
Ollama
Docker (for Oracle Database Free)

Step 1: Build

Clone the Oracle DevRel repository, navigate to the PicoOraClaw app folder, and build the binary:

git clone https://github.com/oracle-devrel/oracle-ai-developer-hub.git
cd oracle-ai-developer-hub/apps/picooraclaw
make build

Step 2: Initialize

Initialize the application so it creates the local configuration and working directories:

./build/picooraclaw onboard

Step 3: Start Ollama and pull a model

Ollama is the default and recommended LLM backend for private local inference with no API keys and no cloud dependency.

# Install Ollama if needed: https://ollama.com/download
ollama pull qwen3:latest

Step 4: Configure for Ollama

Edit ~/.picooraclaw/config.json so PicoOraClaw points at your Ollama instance and model:

{
  "agents": {
    "defaults": {
      "provider": "ollama",
      "model": "qwen3:latest",
      "max_tokens": 8192,
      "temperature": 0.7
    }
  },
  "providers": {
    "ollama": {
      "api_key": "",
      "api_base": "http://localhost:11434/v1"
    }
  }
}

Step 5: Test semantic memory

Once the binary, config, and model are ready, start the assistant and test local conversations:

# One-shot
./build/picooraclaw agent -m "Hello!"

# Interactive mode
./build/picooraclaw agent

At this stage, you have a working local AI assistant with no cloud dependency.

The default LLM backend is Ollama, with an optional alternative for using OCI-hosted models. See oci-genai/README.md for related documentation.

The oci-genai module provides OCI Generative AI as an optional backend for PicoOraClaw. It runs a local OpenAI-compatible proxy that authenticates with OCI using your ~/.oci/config credentials and forwards requests to the OCI GenAI inference endpoint.

Deploying to Oracle Cloud (one-click procedure)

Click here to deploy to Oracle Cloud

This deployment provisions:

an OCI Compute instance
Ollama with a model preloaded for CPU inference
Oracle AI Database Free by default, with an optional Autonomous AI Database path
the PicoOraClaw gateway as a systemd service

You can start locally, keep the same overall architecture, and move to OCI when you need a managed environment.

After deployment, use these commands to verify setup, start chatting, and check gateway health:

# Check setup progress
ssh opc@<public_ip> -t 'tail -f /var/log/picooraclaw-setup.log'

# Start chatting
ssh opc@<public_ip> -t picooraclaw agent

# Check gateway health
curl http://<public_ip>:18790/health

Adding Oracle AI Vector Search

Oracle AI Database provides persistent storage, semantic memory and recall, and crash-safe ACID transactions, with an optional file-based storage mode.

Simply run the setup script:

./scripts/setup-oracle.sh [optional-password]

This script performs the following steps:

Pulls and starts the Oracle AI Database Free container
Waits for the database to be ready
Creates the picooraclaw database user with the required grants
Patches ~/.picooraclaw/config.json with the Oracle connection settings
Runs picooraclaw setup-oracle to initialize the schema and load the ONNX embedding model

This step gives the assistant durable semantic memory. Instead of relying on local files or ephemeral process state, PicoOraClaw persists and retrieves meaning-based context directly through Oracle AI Vector Search.

Expected output when setup is complete:

── Step 4/4: Schema + ONNX model ─────────────────────────────────────────
  Running picooraclaw setup-oracle...
✓ Connected to Oracle AI Database
✓ Schema initialized (8 tables with PICO_ prefix)
✓ ONNX model 'ALL_MINILM_L12_V2' already loaded
✓ VECTOR_EMBEDDING() test passed
✓ Prompts seeded from workspace

════════════════════════════════════════════════════════
  Oracle AI Database setup complete!
  Test with:
    ./build/picooraclaw agent -m "Remember that I love Go"
    ./build/picooraclaw agent -m "What language do I like?"
    ./build/picooraclaw oracle-inspect
════════════════════════════════════════════════════════

Test semantic memory

Use the following commands to test semantic memory:

# Store a fact
./build/picooraclaw agent -m "Remember that my favorite language is Go"

# Recall by meaning (not keywords)
./build/picooraclaw agent -m "What programming language do I prefer?"

The second command finds the stored memory via cosine similarity on 384-dimensional vectors rather than exact keyword matching.

Inspecting Oracle Data with oracle-inspect

A useful operational feature is oracle-inspect, a CLI tool that lets you inspect stored data without writing SQL.

picooraclaw oracle-inspect [table] [options]

These are the tables:

memories, sessions, transcripts, state, notes, prompts, config, meta

These are the options:

-n <limit> max rows (default 20), -s <text> semantic search (memories only)

To list all memories:

./build/picooraclaw oracle-inspect memories

You can also perform semantic search over memories:

./build/picooraclaw oracle-inspect memories -s "what does the user like to program in"

This is a meaningful developer benefit. Oracle-backed memory is inspectable, debuggable, and operationally visible. You can understand what the assistant stores without building a separate admin layer.

Overview dashboard

Run the following command to view an overview dashboard of stored data:

./build/picooraclaw oracle-inspect

Running the command with no arguments gives you a summary view across tables, recent memory entries, transcripts, sessions, state, notes, prompts, and schema metadata.

=============================================================
  PicoOraClaw Oracle AI Database Inspector
=============================================================

  Table                  Rows
  ─────────────────────  ────
  Memories                  20  ████████████████████
  Sessions                   4  ████
  Transcripts                6  ██████
  State                      8  ████████
  Daily Notes                3  ███
  Prompts                    4  ████
  Config                     2  ██
  Meta                       1  █
  ─────────────────────  ────
  Total                     48

  Tip: Run 'picooraclaw oracle-inspect <table>' for details
       Run 'picooraclaw oracle-inspect memories -s "query"' for semantic search

List all memories

Run the following command to list all stored memories:

./build/picooraclaw oracle-inspect memories

All Memories
─────────────────────────────────────────────────────────

ID: faffd019  Vector: yes
Created: 2026-02-19 04:13  Importance: 0.9  Category: preference  Accessed: 0x
Content: User prefers Oracle Database as the primary database. They work at Oracle
and prefer Oracle AI Vector Search for embeddings.

ID: 0e39036f  Vector: yes
Created: 2026-02-19 04:13  Importance: 0.8  Category: preference  Accessed: 0x
Content: Go is the user's primary programming language. They use Go 1.24 and target
embedded Linux devices (RISC-V, ARM64, x86_64).

Semantic search over memories

The following example shows how to perform semantic search over stored memories:

./build/picooraclaw oracle-inspect memories -s "what does the user like to program in"

Semantic Search: "what does the user like to program in"
─────────────────────────────────────────────────────────

[ 61.3% match]  ID: 383ff5d3
Created: 2026-02-16 06:13  Importance: 0.7  Category: preference  Accessed: 0x
Content: I prefer Python and Go for programming

[ 60.7% match]  ID: 0e74a94c
Created: 2026-02-18 02:20  Importance: 0.7  Category: preference  Accessed: 0x
Content: my favorite programming language is Go

For deeper inspection of sessions, transcripts, notes, config, prompts, and schema metadata, see the PicoOraClaw app in the Oracle DevRel repository.

Inspect sessions

You can inspect stored chat sessions using the following command:

./build/picooraclaw oracle-inspect sessions

Chat Sessions
─────────────────────────────────────────────────────────

Session: discord:dev-channel
Created: 2026-02-19 04:13  Updated: 2026-02-19 04:13  Messages size: 673 bytes

Session: cli:default
Created: 2026-02-16 06:12  Updated: 2026-02-18 06:07  Messages size: 2848 bytes

Inspect agent state

Inspect the agent's stored state:

./build/picooraclaw oracle-inspect state

Agent State (Key-Value)
─────────────────────────────────────────────────────────
agent_mode                     = interactive
last_channel                   = cli
last_model                     = gpt-4o-mini
total_conversations            = 42
user_name                      = jasperan

How Oracle Storage Works

The remember tool stores text along with a vector embedding using VECTOR_EMBEDDING(ALL_MINILM_L12_V2 USING :text AS DATA). The recall tool then uses VECTOR_DISTANCE() for cosine similarity search.

With Oracle-backed storage in place, PicoOraClaw supports the following LLM providers:

Ollama
OpenRouter
Zhipu
Anthropic
OpenAI
Gemini
DeepSeek
Groq

PicoOraClaw also supports OCI Generative AI as an optional LLM backend for enterprise models via the included oci-genai proxy.

CLI Reference

The following commands cover the core PicoOraClaw workflows:

picooraclaw onboard - initialize config and workspace
picooraclaw agent -m "..." - one-shot chat
picooraclaw agent - interactive chat mode
picooraclaw setup-oracle - initialize Oracle schema and ONNX model
picooraclaw oracle-inspect - inspect data stored in Oracle AI Database
picooraclaw oracle-inspect memories -s "query" - semantic search over stored memories
picooraclaw gateway - start the long-running service with channels enabled

Conclusion

PicoOraClaw is more than a lightweight assistant runtime. Combined with Oracle AI Database, it becomes a practical pattern for building assistants that retain context, retrieve facts semantically, and scale from local development to OCI without rearchitecting.

Start small, stay local, add durable semantic memory with Oracle AI Vector Search, and keep a clear path to a managed deployment model when you need it.

PicoOraClaw GitHub repository

Frequently Asked Questions (FAQs)

What hardware do I need to run PicoOraClaw?

PicoOraClaw runs on resource-constrained environments including x86_64, ARM64, and RISC-V platforms with a very small footprint. See the project repository for exact requirements.

How does PicoOraClaw remember information?

PicoOraClaw stores memories, sessions, and related state in Oracle AI Database. It uses in-database ONNX embeddings and vector search to retrieve memory by meaning rather than exact keyword matches.

Do I need an external embedding API?

No, the Oracle-backed memory flow uses in-database embeddings.

Can I run PicoOraClaw fully offline?

Yes. Ollama as the default backend enables fully local inference, making PicoOraClaw suitable for offline or privacy-sensitive workflows.

Can I deploy PicoOraClaw to Oracle Cloud?

Yes. The OCI deployment path provisions compute, Oracle AI Database Free, Ollama, and the PicoOraClaw gateway as a systemd service, with an optional Autonomous AI Database path. Deploy here.

Which LLM providers are supported?

Ollama (default), OpenRouter, Zhipu, Anthropic, OpenAI, Gemini, DeepSeek, Groq, and optional OCI Generative AI integration through the included proxy. See the PicoOraClaw repository for details.