DEV Community: Joshua Nwachinemere

A Practical Agent Memory Architecture: What to Keep, Retrieve, Revise, and Forget

Joshua Nwachinemere — Mon, 03 Aug 2026 19:45:18 +0000

A Practical Agent Memory Architecture: What to Keep, Retrieve, Revise, and Forget

A bigger context window enlarges what you can store. It does not decide what your agent should actually use. This is a build checklist for the part that does.

Here is the mental model to start from. Context is the working set, the few things the model reasons over this turn. Memory is the policy that decides what reaches that working set from a much larger store. A million-token window makes the store bigger. It does not hire a librarian, and it does not tell the forklift which box to fetch before the battery dies. A warehouse does not become a library just because you keep adding shelves.

Before the checklist, two evidence points explain why the window alone is not enough.

Why the window is not the answer

Position matters inside one prompt. The Lost in the Middle study found a U-shaped pattern in controlled question-answering and key-value retrieval on 2023-era models: information near the beginning or end was used more reliably than information in the middle. Read it as a demonstrated failure mode, not a fixed score. Synthetic stress tests go further. RULER evaluated 17 models on 13 synthetic tasks and found only about half of the models claiming at least 32K context held its satisfactory threshold at 32K, where satisfactory is RULER's own 85.6% definition. NoLiMa, a 2025 preprint, stripped literal keyword overlap so a match needed latent association; under its v1 evaluation, GPT-4o fell from 99.3% at short context to 69.7% at 32K. Long context is still improving: on RULER's own tables some later models held strong performance through 128K. The takeaway is narrow: usable context can be smaller than nominal context.

Long histories are a separate problem. LongMemEval uses 500 questions across five long-term memory abilities: extraction, multi-session reasoning, temporal reasoning, knowledge updates and abstention. At roughly the 115K-token setting, tested long-context models dropped 30% to 60%. The window held the history; holding it was not enough. Crucially, the authors split memory into indexing, retrieval and reading, and report that even with perfect-recall memory, structured reading such as Chain-of-Note improved question-answering by up to 10 absolute points across three tested LLMs. Retrieving the right record is only half the job.

Different tests expose different gaps between advertised context capacity and reliable use. Lost in the Middle varies position, RULER adds harder tracing and aggregation tasks, and LongMemEval evaluates memory across long interaction histories. Results are study-specific and do not share one scale. Sources: Liu et al. (TACL 2024), Hsieh et al. (COLM 2024), and Wu et al. (ICLR 2025).

The build checklist

1. Separate three layers, not one transcript

Do not fuse raw history, durable memory and working context into "just save the transcript." Split them:

Raw event history: an auditable archive, subject to retention and erasure rules.
Curated memory: small, typed, principal-scoped and governed, with source-event references so consolidation can be unwound.
Working context: assembled from the curated layer, not replayed directly from untrusted raw events.

This separation is an engineering synthesis across LongMemEval, MemGPT, Anthropic's context-engineering guidance, LangChain's memory docs and the OpenAI Agents SDK sessions model, not a single-source rule.

Raw events can remain auditable within retention limits while principal-scoped curated memory decides what the model sees now. Admission, retrieval, revision and forgetting are separate control points.

2. Implement five decisions as separate control points

Every event runs a gauntlet you can test and instrument independently:

Admit. Is this worth remembering, or transient noise for the raw log only?
Retrieve. Given the current task, which stored items are relevant?
Consolidate. Can related events collapse into one summary while keeping pointers to source events?
Revise. Has a higher-trust fact superseded an older one, with provenance for each version?
Forget. Should an item leave active memory because it expired, was disproved or became unsafe, or must an authorized erasure remove it across raw and derived stores?

3. Attach metadata to every curated record

Each record carries timestamps, source-event references, confidence, conflict status, expiry and deletion state. User-visible memories stay inspectable and correctable. This is what makes revision provenance-aware and consolidation reversible.

4. Treat forgetting as quality control, not loss

Hoarding every tool output is not photographic memory. It is a garage with a token bill, and it buries the few records that matter under thousands that do not. Forgetting relies on separation, not silent destruction: staleness, disproof or low relevance should deactivate a curated entry while preserving whatever raw evidence retention allows. Authorized user erasure and legally required deletion are different operations and must propagate through raw records, indexes and consolidated memories rather than leaving a derived copy behind. This draws on the forgetting logic in MemoryBank, whose Ebbinghaus-inspired mechanism is a reasonable design choice rather than a universal decay law, plus compaction guidance from Anthropic.

5. Harden the write path against poisoning

Once memory is read back as trusted context, it is a security boundary. AgentPoison, a preprint, poisoned memory and retrieval stores across three controlled agents so malicious records surface later as retrieved demonstrations. MINJA, a preprint under a controlled threat model, showed an attacker using only ordinary queries could inject malicious records. These are experimental threat models, not deployment prevalence, but the mechanism holds: if any utterance can become durable truth, any user can write to your model's future context. OWASP's data and model poisoning guidance recommends provenance tracking, versioning, anomaly detection, sandboxing untrusted data and validation against trusted sources. Mapped onto memory:

Put a write gate at the trust boundary; keep the admission classifier inside it, since a malicious input may inflate its own importance.
Records carry provenance; updates are reversible.
Content from untrusted turns cannot silently promote itself into durable, high-confidence memory.
Partition curated memory and retrieval by principal so one user's state cannot surface in another's working context.

6. Test each stage separately

Evaluate indexing, retrieval and reading as distinct stages rather than one answer-accuracy number, because a system can retrieve perfectly and still read wrong. Write explicit tests for the write path and the retrieval path. Track stale-memory, conflict and isolation errors, not just correctness. Retrieval and compression add their own failure modes: a retriever can drop the one detail that mattered, a summary can go stale, and an agent controlling its own paging can cascade one bad decision into several.

Prior art you can read as this checklist

Generative Agents, evaluated in a twenty-five-agent sandbox for believable behavior, scored a memory stream on relevance, recency and importance, then wrote back reflections and plans. MemGPT, a preprint, paged information between in-context memory and external storage using function calls. A-MEM, a preprint, creates structured notes, links memories and lets a new memory update an existing one, which is revision as a first-class operation. Mem0, a vendor-authored preprint benchmarked on LoCoMo with LLM-as-judge scoring, reported a 26% relative quality improvement over OpenAI's memory system, plus 91% lower p95 latency and over 90% token savings versus a full-context baseline. Those are author-reported numbers on selected models, not a production guarantee, but the direction is clear: a small curated memory beat a full-context baseline on cost. LangChain, the OpenAI Agents SDK, OpenAI and Anthropic compaction, and Letta all encode the same archive-versus-working-set split under different vocabularies.

When to skip most of this

Do not build a memory platform on day one. For short, bounded tasks, a larger context window plus compaction, or no durable memory at all, is often simpler and safer than a full persistent-memory stack. This is engineering judgment supported by RULER's evidence that long context increasingly works, Anthropic's smallest-high-signal-context guidance, and documented compaction that shrinks history without a separate system. Persistence creates provenance, deletion and poisoning obligations a scoped session never incurs. Start with a retention-scoped raw log and a recent-turn window. Add typed memories only when a repeated, observed failure justifies each type. Give users a way to inspect, correct and erase what the system believes about them.

The best memory system is not the one that remembers the most. It is the one that puts the right record on the workbench at the right moment, can point to where it came from, and knows when to stop bringing it up. A bigger warehouse buys you room. The librarian is still the job.

A Practical Checklist for MCP Tool Contracts (Because the Connection Is the Easy Part)

Joshua Nwachinemere — Sat, 01 Aug 2026 13:17:14 +0000

A Practical Checklist for MCP Tool Contracts (Because the Connection Is the Easy Part)

A universal socket is useful, but an agent still needs a legible contract for what each tool does, when it is safe to call, and how to prove the effect.

Picture two tools on a model's menu. One is called lookup. The other is called send. Both are valid under the protocol, and both connect cleanly. Neither one tells the model whether lookup can quietly pull a customer's full email history, or whether send drafts a message for review versus firing it to an external inbox the instant it is called.

The connection succeeded, but the decision surface failed. MCP gives that surface a standard shape, but it does not make the decision legible by itself. If you build agent tools, that gap is where your work actually starts, and this piece is meant to be run as a checklist against your own surfaces.

What the connection gets you for free

Give the protocol its due. When Anthropic introduced MCP in late 2024, the stated purpose was to replace one-off, custom connectors with a single open standard for wiring AI systems to tools and data. That was a real problem, and MCP is a real answer to it. Instead of rebuilding integration plumbing for every model and every data source, you get a common boundary.

The current tools specification makes that boundary concrete. Tools are model-controlled, and each one is represented with metadata: a name, a description, an input schema, an optional output schema, and optional behavior annotations. Discovery, invocation, and the shape of the exchange are standardized. That is genuine leverage.

But notice what standardization does and does not buy you. With a conforming application, a tool can be discovered and called through a standard interface. It does not guarantee that the tool can be understood, chosen correctly, or safely invoked. Transport compatibility is not semantic clarity, and it is not host policy. The same specification is careful about this line. It recommends that a human remain able to deny tool invocations, and it warns that behavior annotations must be treated as untrusted unless they come from a server you already trust. In other words, the protocol itself tells you that its metadata is a starting point, not a safety guarantee. Everything below is the part you have to build.

The contract, part by part

When people say a tool is "wired up," they usually mean the call works. The more useful question is whether the tool exposes a legible contract. Treat that contract as one mechanism with several moving parts, not a checklist to pad out a config file.

Caption: An MCP socket carries the request. Tool meaning, host permission, and proof of effect are designed downstream.

An intent-revealing name lets the model separate this tool from its nearest look-alike before it reads anything else. A bounded description states what the tool does and, critically, what it does not do. A constrained input schema can make invalid or underspecified calls harder to express, reducing the room for a dangerous or ambiguous argument. A structured output gives the caller evidence of what actually happened, rather than a friendly sentence that sounds like success. Explicit side effects and preconditions say what changes the moment the tool runs and what must be true first. And failure semantics tell the model how to recognize and recover when the call does not go as planned.

Compare lookup(query) against two better-specified tools. A read-only customer search that promises no writes and returns structured records is a fundamentally different object from a tool that immediately transmits an external email. Under a vague surface, both can hide behind the same generic verb. Under a real contract, the model can tell them apart before it commits to an action. The parts of the contract are not decoration. They are the information the model uses to choose.

A small experiment you can reason about

I wanted to see how much the wording of a tool surface changes a model's first move, so I ran a small local fixture for this article. I want to be precise about how small it is, because it is directional evidence and nothing more.

I wrote ten synthetic CRM and email requests. I showed each request to the same model, GPT-5.6 Sol, under two conditions. The vague condition used generic names, one-line descriptions, and permissive schemas. The contract-rich condition bundled intent-revealing names, side-effect and precondition descriptions, and constrained schemas. Each condition received six observations per case across two independent runs. No tool was ever actually executed. I used an OpenAI-compatible function-calling adapter rather than a live MCP server, so there was no backend, no authorization layer, no latency, and no recovery path in the loop.

Caption: In two runs of ten synthetic prompts, the vague surface produced 49 expected first actions out of 60; the contract-rich surface produced 60 out of 60. One model was tested, the treatment changed several interface dimensions together, and no tool ran.

The vague surface produced the expected first action in 49 of 60 observations, or 81.7 percent. The contract-rich surface produced it in 60 of 60, or 100 percent. Two categories of prompt drove the gap. On the six prompts that asked for an action the tools did not actually support, an email audit, the vague surface failed all six by reaching for a generic lookup instead of declining or clarifying. On the six prompts that explicitly asked to send an email, the vague surface handled only one as expected. Better metadata closed both gaps in this fixture.

I will not oversell it. This is one model, ten hand-authored prompts, and provider-default temperature. The treatment bundled names, descriptions, and schemas together, so it cannot isolate which of the three did the work. It is a fixture rather than a benchmark, and it shows a direction rather than a coefficient.

Why the checklist matters beyond one fixture

Two recent preprints show why the boundary matters beyond my toy setup.

DynamicMCPBench, a preprint submitted in July 2026, evaluates 24 models across 121 servers and 750 tasks in 15 categories, using effect-scored traces and a strict pass^3 rule that requires three successes. The strongest agents solved only about half of the tasks. Thirty-one percent were solved by no model at all. Accuracy fell from 39 percent on the shortest tool chains to 13 percent on the longest. Connectivity, in other words, is not completion. In this benchmark, longer tool chains were less reliable. The authors also make a point worth stealing: an effect-scored trace tells you more about whether a real action occurred than a polished final answer does.

The second preprint, an empirical study of 1,723 GitHub MCP applications submitted the same month, looks at how these systems are built rather than how they score. It reports that 90.8 percent log at least part of execution and 77.2 percent expose enable and disable controls, but only 37.2 percent implement a blocking approval gate before a tool executes. Treat that as a GitHub sample analyzed with an LLM-assisted classification pipeline, not a census of production deployments. Still, it says the host-side controls that actually govern safety are heterogeneous. The protocol permits a human veto. Most inspected projects did not enforce one before execution.

Put the two together. The tool interface gives the model its decision surface, and host policy governs what happens when a choice becomes an action. Neither comes for free with a working connection.

The checklist: five questions for your next tool review

Run this against any tool surface before you ship it. It is short on purpose.

Could another tool plausibly match the same request? If yes, your names and descriptions are doing too little.
Can the schema represent an underspecified or dangerous call too easily? Tighten the inputs until the wrong call is hard to express.
Does the model know what changes immediately? Side effects and preconditions belong in the contract, not in tribal knowledge.
Can the result prove the effect in structured data? A confident sentence is not evidence that an action occurred.
What does the host enforce even if the model makes the wrong choice? Authorization, approval, and traceability live outside the model, and better wording cannot substitute for them.

The plug fits. Now the engineering starts.

MCP made the plug fit, and that was worth doing. But a socket only moves the hard part downstream. The real work begins when you decide what may flow through it, how a model can tell one tool from another, and how your host proves that an action happened the way it was supposed to.

Here is my question for you: what is the most dangerously ambiguous tool contract you have shipped or inherited, and what finally made you tighten it?

A Field Guide to Language Design: Who Pays for Every Abstraction

Joshua Nwachinemere — Wed, 29 Jul 2026 08:49:26 +0000

A Field Guide to Language Design: Who Pays for Every Abstraction

Here is a mental model you can apply to any language, current or future, in about thirty seconds. It has saved me from a lot of framework holy wars.

Every language is a burden-transfer machine. It takes some recurring chore off your hands, moves that chore into a compiler, a runtime, or a toolchain, and charges you a fee for the service. The fee is usually invisible on day one and very visible on day 900. Once you start reading languages this way, the endless "which language is best" thread stops mattering and a much better question takes its place: best at moving which burden, and who gets stuck with the bill?

This post walks the history through that lens. Not a ranking. A checklist you can reuse.

Warmup: "first language" is an underspecified query

Developers argue about the first programming language the way we argue about tabs versus spaces, and for the same reason: the question is underspecified. There is no single defensible answer. It depends on what you mean.

First published algorithm meant for a machine
First high-level language designed on paper
First one actually implemented
First true compiler
First high-level language a lot of people trusted for real work

Five milestones, often years apart. Collapse them into one trivia answer and you delete the distinctions that make the story readable. Treat "first" like a version field and always name the lifecycle stage.

Ada Lovelace's 1843 Note G laid out a tabulated procedure for computing Bernoulli numbers on Babbage's never-built Analytical Engine (Note G). It is an early worked example of an algorithm expressed for a machine. It is not a programming language, and the "first programmer" label is contested because Babbage drafted earlier programs. The narrow, safe claim: people wrote out mechanical computations before hardware existed to run them.

The burden-transfer loop, made concrete

A language removes a recurring burden by introducing a mechanism and assigning work to a compiler, runtime, or toolchain. The payoff comes with a price that often becomes the next language-design problem.

The loop has five stops: programmer burden, language mechanism, system responsibility, practical payoff, new price. A dashed arrow runs from the new price back to burden, because today's fee is tomorrow's problem to solve. Three concrete passes through the loop:

Assembly. Burden: numeric opcode and address bookkeeping. Mechanism: mnemonics and labels plus an assembler. Payoff: readability. Price: still one-to-one with the hardware and machine-specific. Readable is not portable, and readable is not high-level.
Garbage collection. Burden: manual object reclamation. Mechanism: a runtime collector. Payoff: fewer lifetime errors. Price: pauses and memory overhead.
Ownership and borrowing. Burden: proving memory access is safe. Mechanism: compile-time checks. Payoff: safety without a GC. Price: a learning curve and real constraints on how you write code.

Same loop, three different eras. Hold it in your head for the rest of the post.

FORTRAN shipped trust, not syntax

When electronic machines arrived you programmed in raw machine code. Symbolic assembly was the first real trade, but it did not change the shape of the work; you still thought in registers and memory layout. The machine went quiet, not away.

Sort the early candidates by explicit definition and the fight evaporates. Plankalkul (Zuse, 1942 to 1945) had branches, loops, and structured data, so it is a strong first designed high-level language, but it went essentially unimplemented and was published too late to influence its era (Computer History Museum timeline). Short Code (about 1950) is probably the earliest implemented high-level notation, interpreted and slow. Glennie's Autocode (about 1952) is a candidate first compiled language almost nobody used. "Compiler" traces to Grace Hopper's A-0, which behaved more like a loader or linker (Hopper, "The Education of a Computer").

FORTRAN settled the practical question by shipping a working compiler in 1957 and earning wide adoption for real scientific work (Backus et al., "The FORTRAN Automatic Coding System"). The win was not nicer math notation, it was trust. Assembly programmers would not give up control unless generated code ran nearly as fast as hand-written assembly, so the real achievement was optimization good enough to make that trade tolerable at scale. Instruction selection moved from the programmer to the compiler. That is the first big burden transfer with a real fee attached, and everything after it rhymes.

Branching, not a ladder

Once translation was trusted, different kinds of work pulled languages apart.

Programming languages branched around different pressures rather than replacing one another in a single sequence. Dates identify the milestone shown. Era bands are grouped for readability and are not a uniform time scale.

COBOL (CODASYL, around 1960, from FLOW-MATIC, COMTRAN, FACT) split data description from procedure to cut business records loose from any one machine (COBOL history). Its first spec and first running programs are different dates, and Hopper influenced it without writing it alone. Fee: verbosity, in exchange for readability and portability.
Lisp (McCarthy, around 1960) bundled recursion, symbolic expressions, eval, code-as-data, and garbage collection (McCarthy, "Recursive Functions of Symbolic Expressions"). It took manual memory management off the programmer for unpredictably sized structures. McCarthy intended M-expressions; everyone wrote the machine-facing S-expressions. The surviving language was not quite the designed one.
ALGOL 58 then 60 gave the field block structure, lexical scope, recursion, and BNF (ALGOL 60). No standard I/O, diverging implementations, never a commercial heavyweight. It became the language designers used to talk to each other.

Four answers to "what should code look like" now coexisted: math, English-like records, symbolic lists, structured algorithms. None displaced the others.

When the burden became your own head

By the late 1960s the question was whether a person could still hold a large program in mind. Dijkstra's 1968 letter, titled by Wirth, argued unrestricted jumps make execution too hard to reason about (Dijkstra, "Go To Statement Considered Harmful"). He did not prove the underlying result, and "never use goto" flattens what he said. The burden was cognitive, and the language fenced you in so you could keep the whole thing in mind.

Objects came from simulation. Simula (Dahl and Nygaard) introduced classes, objects, subclasses, virtual procedures, and coroutines (Dahl and Nygaard, Simula); it invented the core OO mechanisms without wide adoption. Smalltalk (Xerox PARC, 1970s) made message-passing objects the whole model, inside a live image-based environment on a bytecode VM with GC (Kay, "The Early History of Smalltalk"). It popularized and stretched OO rather than inventing the class.

C ran the other way. From BCPL and B, Ritchie kept the machine close so Unix could leave assembly without losing control (Ritchie, "The Development of the C Language"). Not a nicer assembly: near enough to the hardware to write an OS, yet one source could move across architectures with a recompile. Fee: manual memory management, weak safety guarantees, and undefined behavior behind ordinary-looking code. Decades of security work descend from that bargain.

"Better" stops meaning "better overall"

Through the 1970s "better" quietly became "better fit for one model of computation."

SQL. Codd's 1970 relational model separated logical data from physical storage (Codd, "A Relational Model of Data"); SEQUEL/SQL let you state the result instead of the traversal (Chamberlin and Boyce, "SEQUEL"). Navigation became declaration and the engine took over traversal. Caveat: SQL is not a clean realization of the relational model, with duplicates and its own NULL semantics.
Prolog (Colmerauer and Roussel, early 1970s) let you state facts and rules while unification and backtracking searched (Colmerauer and Roussel, "The Birth of Prolog"). Fee: predictability, and real Prolog smuggles non-logical control back in.
ML (Milner, late 1970s) proved static typing need not mean annotating everything; Hindley-Milner inference computes principal types, catching errors before runtime (Milner, "A Theory of Type Polymorphism in Programming"). Fee: some expressive forms are boxed in and errors can be opaque.
Haskell (committee, from the late 1980s) consolidated lazy-functional research into one pure language with type classes and later monadic I/O (A History of Haskell). It consolidated more than it replaced, and laziness complicates performance reasoning.
C++ (Stroustrup) layered classes and later generics onto C to refuse the choice between abstraction and control (Stroustrup, "A History of C++: 1979-1991"). It invented neither OO nor generics, and inherited C's unsafe model plus growing complexity.

Progress is plural from here on. Each language is better for one model of computation and beside the point for the rest.

Ceremony gets cut

More people, cheaper machines, then a network. A run of languages cut ceremony.

Perl (Wall, late 1980s) fit between shell and C with Unix text idioms, regexes, and associative arrays (perlhist). Flexibility was appeal and tax both.
Python (van Rossum, 1991) drew from ABC, Modula-3, Unix, and C for indentation-based blocks, high-level built-ins, dynamic execution, and easy C extension (van Rossum, "A Brief Timeline of Python"). It popularized readable scripting; dynamic typing, interpreter speed, and the GIL stay implementation-sensitive.
Ruby (Matsumoto, mid-1990s) bet on programmer happiness and expressive design (About Ruby).

All three shed ceremony and took on less predictable behavior at scale.

Java chased portability at the runtime layer: architecture-neutral bytecode on a JVM with GC, verification and sandboxing, and a big standard library (The Java Language Environment). The managed-runtime idea predated it in Smalltalk, Lisp, and p-code systems; Java's feat was scale. The work moved into the VM; the cost was low-level control.

JavaScript is constraints carving a language. Eich wrote the first version at Netscape in May 1995 under schedule pressure, splicing Scheme-like functions, Self-like prototypes, and a Java-like surface (Wirfs-Brock and Eich, "JavaScript: The First 20 Years"). The ten-day story is the first prototype, not decades of ECMAScript work, and backward compatibility froze early quirks. PHP embedded server-side code in the page (PHP history); Lerdorf's 1995 tools are separate from the later PHP 3 language rewrite, and the organic route left consistency and security costs. C#/.NET and the ECMA CLI combined an intermediate language, JIT, GC, metadata, and a common type system for multi-language runtime sharing (ECMA-335); the VM/GC model was inherited and the early ecosystem was Windows-centric. None knocked out C or C++. They moved application development into new runtime homes.

Modern set: group by pressure, not release year

Go targeted large server software: slow compiles, fiddly deploys, awkward concurrency (Go). Fast builds, static binaries, GC, goroutines and channels from CSP, deliberately small. Better for cloud and server work, a constraint elsewhere.
Rust aimed at memory safety and data-race prevention with no GC (Rust). Ownership, borrowing, lifetimes, and Send/Sync reject broad classes of memory errors and data races at compile time while keeping native performance and zero-cost abstractions. The most ambitious handoff in the set: obligations moved from programmer discipline or a runtime into the type system and compiler. Fee: learning curve, ongoing borrow-checker effort, and it neither kills every bug nor removes unsafe.
TypeScript adds structural, gradual static types that erase back to JavaScript, catching mistakes and driving tooling while staying JS-compatible (TypeScript). The type system is unsound on purpose and promises nothing at runtime, a deliberate adoption trade.
Swift and Kotlin are ecosystem successors. Swift addressed Objective-C with optionals, value semantics, ARC, and later concurrency safety (Swift); Kotlin addressed Java with null safety, coroutines, concision, and full JVM interop (Kotlin). Their advantage is bound to the Apple and JVM or Android ecosystems.
Julia attacked the two-language problem with LLVM JIT and multiple dispatch so high-level numerical code approaches low-level speed without a second implementation language (Julia). Fee: startup and compile latency, smaller ecosystem, and "as fast as C" oversells it.
Elixir made Erlang/OTP concurrency approachable on the BEAM (Elixir); Gleam added a sound static ML-style type system on the same runtime, compiling to BEAM and JavaScript (Gleam). Gleam's verified edge is compile-time type soundness, not better fault tolerance, since both inherit BEAM strengths.
Zig favors explicitness over guarantees: comptime, explicit allocators, C interop, cross-compilation, little hidden behavior (Zig). Pre-1.0, with nothing like Rust's compile-time memory-safety guarantee.

Same shape every time: name a cost, add machinery, accept a new one.

"Newest language in 2026" is a trick question

No registry names one globally latest language, and every attempt hides a buried criterion. Announcement, first public compiler, stable 1.0, latest build, and production use point at different endpoints.

Cleanest recent stable milestone in the cohort I looked at: Gleam, 1.0 on 2024-03-04, stable 1.x through 2026 (gleam-lang/gleam). Repo dates to 2016, so newest stable release is not the same event as invention.
Most recent public introduction: Rue, official "Hello, World!" dated 2025-12-21, exploring memory safety without a GC and ergonomics beyond Rust and Zig via a native compiler pipeline (Rue, "Hello, World!"). Its README says early-stage, changing, not production-ready (rue-language/rue).
Active pre-1.0: Mojo hit a 1.0.0b2 beta on 2026-06-18 with a closed compiler (Mojo release); MoonBit was 0.9 in April 2026 with vendor-microbenchmark performance numbers (moonbitlang/moonbit-docs).
Design signals with loud caveats: Carbon, an experimental C++ successor its docs call not ready (carbon-language/carbon-lang); Roc, pure functional platform/application separation (Roc); Hylo, mutable value semantics at 0.0 (hylo-lang/hylo); Verse, in production inside Unreal Editor for Fortnite but proprietary and platform-bound (Verse language reference).

Every one describes an intention, not a proven advantage. None is "the latest" unless a qualifier is doing the heavy lifting.

The reusable test

The through-line holds as the branches multiply. Assembly turned numeric bookkeeping into symbols. FORTRAN pushed instruction selection into the compiler. Lisp and the managed runtimes handed memory and portability to collectors and VMs. SQL gave data navigation to the query engine. Rust relocated memory-safety proof into the type system and compiler. The work always left the programmer and settled somewhere else, and always cost something: control, predictability, runtime independence, or the hours of learning new rules.

So next time a language shows up in your feed, skip the ranking and run the loop. What burden does it lift? What machinery shoulders it now? What is the bill: performance, control, complexity, or a runtime you now depend on? A language earns its keep by making that trade pay off for some real kind of work. The tree keeps branching because the trades keep being worth it, each for a different problem, and never for free.

Transformers Beyond ChatGPT

Joshua Nwachinemere — Tue, 28 Jul 2026 11:32:22 +0000

Chatbots made transformers famous. Their deeper contribution is a reusable way to model context across almost any data that can be represented as tokens.

Ask most people what a transformer does and they will picture a chat window. Type a question, receive a paragraph. The association makes sense: ChatGPT put the architecture in front of a mass audience.

It also hides most of the story.

A transformer might inspect a medical scan, combine views from several car cameras, choose the next movement of a robot arm, compare years of satellite observations, or examine a stream of server events for behavior that does not belong. In none of those cases is conversation the point.

Language was simply where transformers became impossible to ignore. Their broader appeal comes from something more basic: they are good at learning relationships among pieces of information in context.

A token does not have to be a word

The original 2017 paper, “Attention Is All You Need”, introduced the Transformer for sequence transduction. Its central move was to let the model weigh relationships among tokens directly through attention rather than process a sequence one step at a time with recurrence.

In a language model, a token might be a word or part of one. Elsewhere, it can be an image patch, a moment in a sensor stream, an item a customer clicked, a robot action, or a group of satellite spectral bands.

That translation is the key. Once a problem can be represented as tokens, attention can learn which parts should influence one another. A suspicious login may matter more when viewed beside earlier events. A small region in a scan may make more sense when compared with distant anatomy. A road object seen now may be easier to place when the system remembers where it appeared a fraction of a second ago.

The transformer is usually the contextual middle, not the whole machine. Domain-specific encoders prepare the input. Task-specific heads turn internal representations into segmentations, actions, ranked items, structures, or anomaly scores. Training data, objectives, safety checks, and human review still determine whether the result is useful.

The reusable middle learns relationships across context. The input encoding, output head, training objective, and validation remain domain-specific.

Eight domains, eight meanings of context

The architecture travels because the token changes. A token can be an image patch, action step, pair representation, event, item, or time window.

Medical imaging: connecting anatomy across a scan

Many medical-imaging tasks require both local precision and a wider view. A model segmenting an organ must preserve boundaries measured in pixels while also understanding how that region relates to the rest of the image.

TransUNet is a useful example because it is a hybrid rather than a victory lap for one architecture. Its transformer encoder captures broader context, while a U-Net-style decoder restores fine spatial detail for segmentation. The design recognizes that global relationships and local structure solve different parts of the problem.

This does not mean a transformer has replaced the radiologist, or even that it should replace convolutional networks. A benchmark result says little by itself about performance across hospitals, scanners, patient populations, or clinical workflows. The practical value lies in giving an imaging model another way to connect distant regions without discarding the machinery that already handles local detail well.

Satellite imagery: seeing across bands and seasons

A normal photograph records visible light at one moment. Satellite imagery can include repeated observations over time and measurements in spectral bands that human eyes cannot see. A useful model may need to compare one location across seasons, cloud conditions, and wavelengths.

SatMAE adapts masked autoencoding to this setting. It divides satellite images into patches, masks some during pretraining, adds temporal information, and represents multispectral data as groups of bands with their own positional encodings. The model learns by reconstructing what was hidden, then transfers those representations to tasks such as land-cover classification and semantic segmentation.

The important detail is not that satellite images have been made to look like sentences. They have not. Their spatial, temporal, and spectral structure has been converted into a form in which relationships can be learned together.

Robotics: turning perception into action

A robot has a more awkward job than a chatbot. It must connect an instruction to what its cameras see, decide what matters, and produce an action before the world changes.

Google’s RT-1 treated robot control as a token-sequence problem. The system combined visual observations and task instructions, then generated discretized actions for a real robot. RT-2 pushed the idea further by representing actions as tokens inside vision-language-action models co-trained on web and robotic data. Knowledge learned from images and text could then help with some forms of robotic generalization.

That last sentence needs restraint. A robot action may be encoded as a token, but a dropped cup remains stubbornly physical. Latency, hardware limits, unfamiliar objects, poor camera angles, and safety constraints do not disappear because the policy inherited a transformer. These systems show a way to share representations across perception, instructions, and control. They do not turn an internet-trained model into a universally competent machine.

Autonomous driving: building a view no camera captured

A vehicle receives several partial views of a scene. A car at the edge of one camera may appear in another. An object briefly hidden behind a truck may still matter. The system needs a stable spatial model rather than six unrelated images.

BEVFormer uses spatiotemporal transformers to construct a bird’s-eye-view representation from multiple cameras. Spatial cross-attention gathers relevant features across camera views. Temporal self-attention brings in information from earlier bird’s-eye-view representations.

This is a transformer inside a perception stack, not an entire self-driving system. It does not plan a route, negotiate with a cyclist, or certify the vehicle as safe. Its contribution is narrower and more defensible: attention offers a way to combine views and history into a shared spatial frame.

Protein folding: attention meets geometry

Proteins begin as sequences of amino acids, but their biological behavior depends heavily on the three-dimensional structures into which those sequences fold. That makes protein structure prediction look temptingly like another sequence problem, until geometry enters and ruins the simplicity.

AlphaFold2 uses attention-based processing in its Evoformer, which exchanges information between multiple-sequence representations and pairwise relationships. Those representations feed a larger system built around evolutionary, physical, and geometric constraints.

Calling AlphaFold2 “a transformer that folds proteins” is convenient but incomplete. It is not a plain language transformer pointed at amino-acid letters. Its attention mechanisms sit inside an architecture designed specifically for molecular structure. That distinction explains both the power of the transformer idea and the limits of copying it unchanged. The reusable mechanism travels; the surrounding science has to come with it.

Cybersecurity: reading behavior before it becomes an incident

Security teams work with sequences everywhere: authentication events, process launches, network flows, system logs, and threat reports. The useful question is often not whether one event looks strange in isolation, but whether it fits what came before and after.

LogBERT learns patterns from normal log sequences and uses deviations to identify possible anomalies without requiring a fully labeled collection of attacks. SecureBERT takes a different route. It adapts BERT-style pretraining and tokenization to cybersecurity text such as threat intelligence, where terms and relationships differ from general English.

This domain exposes a weakness in the “transformers beyond language” slogan: some valuable non-chat applications still process language. What changes is the job. The model is not trying to hold a conversation; it is extracting security-relevant context or comparing machine events.

Deployment is harder than a clean benchmark. Log formats change. Normal behavior drifts. Attackers deliberately probe the model’s assumptions. A detector that raises too many false alarms becomes expensive background noise.

Recommendation systems: context is a history, not a prompt

A recommendation engine can treat a person’s interactions as an ordered sequence. The last item matters, but so might a purchase from months ago or a recurring pattern that appears only on weekends.

SASRec uses self-attention to identify relevant actions in a user’s history when predicting the next item. BERT4Rec uses bidirectional context and masked-item training, learning to reconstruct hidden items from the interactions around them. A later replicability study found that BERT4Rec’s reported advantage over SASRec was not consistent across the literature and could not be reproduced from the released implementation without substantially longer training.

This is close to language modeling in form but different in consequence. The tokens are products, songs, videos, or actions. The output shapes what the user sees next. Attention can connect distant interactions, but it does not reveal why a person chose something. Nor does it solve feedback loops, popularity bias, stale preferences, or the temptation to optimize engagement at the expense of usefulness.

Anomaly detection: learning what does not fit

Anomaly detection asks a deceptively difficult question: what counts as unusual when normal behavior is complex and constantly moving?

The Anomaly Transformer compares relationships across a time series using a purpose-built anomaly-attention mechanism. Its central intuition is that abnormal points may form different associations with the rest of the sequence than normal points do. That association discrepancy becomes part of the anomaly score.

The approach shows why transformers appeal to monitoring systems. A spike may be harmless at noon and alarming at midnight. A sensor value may look normal alone but strange beside the pattern around it. Context changes the meaning.

Still, transformers do not automatically win every temporal problem. In a 2023 study of time-series forecasting, simple linear models outperformed several transformer-based forecasters across nine datasets. Forecasting is not the same as anomaly detection, but the warning travels well: a fashionable architecture is not a substitute for a strong baseline.

The architecture travels; the validation does not

Across these examples, the shared pattern is easy to see:

Raw data becomes tokens. Attention builds contextual representations. A domain-specific component turns those representations into a useful output.

Everything around that pattern changes. Medical models need clinical validation. Robot policies need hardware-aware safety. Satellite systems must handle sensor and geographic shifts. Driving perception has strict latency and reliability constraints. Protein models must respect geometry and experimental evidence. Security tools face adversaries. Recommenders alter the behavior they measure. Anomaly detectors live or die by thresholds and false positives.

Many of the strongest systems are hybrids because the world does not care about architectural purity. Convolutions remain good at local image structure. Geometric modules remain necessary for molecules. Control systems need more than next-token prediction. Sometimes a linear model is enough, which is less glamorous but wonderfully cheap.

ChatGPT did not make transformers universal. It made one use of them visible.

The better question is no longer, “Can a transformer chat about this problem?” Ask instead: What are the tokens? Which relationships carry useful context? What output must the system produce? And what evidence would show that it works outside the benchmark?

That is where the transformer story gets much larger than the chat box.

Where Does a Vision Model Keep Its Knowledge? A Developer's History of Computer Vision

Joshua Nwachinemere — Mon, 27 Jul 2026 19:46:40 +0000

In 2001, Paul Viola and Michael Jones described a face detector that ran at 15 frames per second on 384×288 images using a 700 MHz Pentium III (Viola & Jones, CVPR 2001). Roughly a quarter-century later, Meta's 2025 SAM 3 takes the phrase "yellow school bus" and returns masks for the object instances it matches to that concept across an image or video (Carion et al., SAM 3, 2025).

If you build things, the interesting gap between those two systems is not faster chips. It is where the visual knowledge is stored, and who or what put it there. That single question — where does the model keep what it knows? — is the cleanest way I know to hold the whole history in your head.

For a long time, the intelligence lived in the head of the engineer who designed the features. Then it moved into convolutional filters learned from labeled examples. Then into deeper architectures and scaling rules, into transformer attention, and finally into large pretrained models that connect pixels to regions, language, prompts, and video. Each move changed the hard part of the job. None of them retired the old problems of data quality, geometry, evaluation, and knowing when a system is about to be wrong.

Era 1: engineers wrote the features down by hand

Many classical pipelines turned pixels into human-designed signals — edges, corners, gradients, contrast patterns, local descriptors — before any matching or classification happened. Most of the design work lived in feature extraction, not in the final decision rule.

The roots go back further than the deep-learning timeline usually admits. Marr and Hildreth modeled edges as zero-crossings after Laplacian-of-Gaussian filtering in 1980 (Marr & Hildreth, 1980); Canny formalized edge detection in 1986 (Canny, 1986); Harris and Stephens gave us a corner detector in 1988 (Harris & Stephens, 1988). And convolution-flavored hierarchies predate 2012 by decades: Fukushima's 1980 neocognitron used alternating shift-tolerant S- and C-cell stages (Fukushima, 1980). It anticipated ideas central to CNNs, though it was not a modern backprop-trained network.

There is a debugging lesson buried in this era that still applies. When a hand-designed feature extractor discarded information the task needed, a downstream classifier working only from that representation could not get it back. So engineers had to reason about invariance up front — scale, rotation, lighting, occlusion — because nothing downstream would rediscover what the front end threw away.

Between 2001 and 2006, four methods turned that philosophy into practical systems:

Viola–Jones (2001) paired an integral image, AdaBoost feature selection, and a rejection cascade so most of the image failed cheap tests early (paper). The 15 fps figure is a historical result on historical hardware, not a modern benchmark.
SIFT (2004) built keypoints designed for scale and rotation invariance and robustness to viewpoint, noise, lighting, clutter, and partial occlusion (Lowe, IJCV 2004). The paper claims robust and partial invariance, not perfect invariance. SIFT first appeared in a 1999 ICCV paper; the 2004 journal article is the mature four-stage version.
HOG (2005) described local shape with dense, locally normalized gradient-orientation histograms, evaluated on pedestrian detection (Dalal & Triggs, CVPR 2005).
SURF (2006) chased SIFT's invariance goals with speed as an explicit target (Bay et al., ECCV 2006). It is not universally better than SIFT; the two trade off by workload.

Calling these "primitive AI" gets it backwards. They were the moment the field learned to state its assumptions out loud.

Era 2: CNNs changed who designed the representation

The convenient legend is that CNNs arrived in 2012 and swept the old methods away. Both halves need qualifying.

CNNs already worked long before 2012 — LeCun and colleagues described gradient-based learning with convolutional features for document recognition in 1998 (LeCun et al., 1998). And it helps to keep three ImageNet milestones separate, because they get merged constantly: the dataset paper (5,247 synsets, 3.2M images) appeared in 2009 (Deng et al., CVPR 2009); the annual ILSVRC challenge began in 2010 (Russakovsky et al.); and the result that turned heads landed in 2012.

That result was AlexNet: 15.3% top-5 test error versus 26.2% for the runner-up (Krizhevsky, Sutskever & Hinton, NeurIPS 2012). Two cautions ride along with that number. It is a competition-system result on one benchmark, not a value to line up against later scores from other tasks and protocols. And the win was a convergence — a large-capacity CNN, a GPU implementation, ReLUs, dropout, and 1.2 million labeled images — not the invention of convolution.

The lasting consequence was a change in job description. Instead of hand-designing a feature hierarchy, practitioners increasingly specify data, objectives, architecture, and optimization, and let the hierarchy emerge through training. Representation design did not vanish. It moved.

Once a network produced rich features, people asked what else those features could do. "What's in this picture?" is one question; here is a compact taxonomy of the tasks that followed, each with a primary source that produced that output type:

Classification assigns an image-level category (AlexNet).
Detection predicts object categories with locations (YOLO).
Semantic segmentation assigns classes at pixel level (U-Net).
Instance segmentation separates masks for individual object instances (Mask R-CNN).

The detection lineage is a clean case study in removing duplicated work. R-CNN transferred CNN features into localization but ran the network many times over overlapping regions (Girshick et al., 2014). Fast R-CNN computed one image-level feature map and pooled regions from it (Fast R-CNN, 2015) — though its headline inference time excluded external proposal generation. Faster R-CNN learned the proposals on the shared features (Ren et al., 2015). Three papers, one direction: stop recomputing the same thing. YOLO took the other route, regressing straight from the full image to boxes and class probabilities in a single pass, trading some localization precision on small or clustered objects for speed (Redmon et al., 2015/2016). That was the original YOLO; later versions changed substantially, so don't read the first paper's numbers as describing anything that shipped afterward.

Era 3: depth broke, and then it didn't

If a few conv layers were good, more should be better. Reality disagreed.

VGG argued for depth from discipline — stacks of small 3×3 convolutions whose representations transferred well (Simonyan & Zisserman, 2015). Inception widened under a compute budget with parallel multi-scale branches and 1×1 projections (Szegedy et al., 2015); its famous 6.67% top-5 came from an ensemble of seven models with 144 crops per image, not one model on one crop.

Then ResNet named the real problem. In its experiments, deeper plain networks had higher training error than shallower ones — the degradation problem, which the authors explicitly distinguish from overfitting. Residual learning fixed it by having each block learn a residual relative to its input, with an identity shortcut carrying the input forward (He et al., 2016). Worth getting right, because the shorthand does damage: the paper does not claim "skip connections solved vanishing gradients." It notes normalization and initialization had already helped convergence, and frames the contribution as making very deep networks (up to 152 layers) optimizable at all. EfficientNet later treated accuracy and efficiency as a joint problem, balancing depth, width, and resolution with one compound coefficient (Tan & Le, ICML 2019) — the balance is the idea, and its reported numbers are tied to the authors' specific setup, not a current leaderboard.

Era 4: transformers, language, and promptable segmentation

DETR reframed detection as direct set prediction: a transformer encoder-decoder with learned object queries and a set-matching loss, which removed several hand-designed components like anchors and non-maximum suppression (Carion et al., 2020). It kept a convolutional backbone and had slow-convergence and small-object limits; the contribution was the formulation. ViT then fed raw image patches as tokens to a near-standard transformer (Dosovitskiy et al., 2021). Its strongest results depended heavily on large-scale pretraining — the paper says so plainly. ViT did not prove attention is inherently smarter than convolution, and it did not retire CNNs; it offered different priors and a different scaling path that paid off where that path fit.

Then the label space stopped being a fixed list. CLIP aligned images and text from 400 million web pairs in a shared embedding space, enabling classification against categories described in words (Radford et al., 2021). "Zero-shot" here means no task-specific gradient update for new categories — not learning from nothing. The same paper documents prompt sensitivity, uneven fine-grained performance, and inherited web bias. CLIP is an alignment and embedding model, not a guarantee of correct visual reasoning. DINOv2 is a useful counterweight: general-purpose visual features from self-supervised ViT training on curated data, no captions at all (Oquab et al., 2023). And Flamingo (Alayrac et al., 2022) and LLaVA (Liu et al., 2023) connected vision encoders to language models so you could prompt and instruct them in plain language.

Here is the part to tattoo on your deployment checklist: fluent multimodal answers are not proof of grounded perception. A language interface can hide uncertainty rather than remove it, so an answer still has to be checked against the underlying visual task and evidence. Articulate output is not verification.

Segmentation changed its contract in parallel. The original Segment Anything introduced promptable masks from points, boxes, or a rough mask — not unrestricted text — trained on SA-1B, reported as 1B+ masks over 11M licensed, privacy-respecting images (Kirillov et al., ICCV 2023). SAM 2 added streaming memory to track prompted objects through video (Ravi et al., 2024). SAM 3 added concept prompts — noun phrases and visual exemplars — unifying detection, segmentation, and tracking, with a data engine built around 4 million unique concept labels (Carion et al., 2025). As of this article's research cutoff, the most recent official step is SAM 3.1: Meta's repository records a March 27, 2026 release adding shared-memory joint multi-object tracking, intended to improve speed without sacrificing accuracy (release note; Meta blog). Two things are true at once: it is an official project-team release, and it is a vendor claim rather than an independent evaluation.

The caution generalizes across the whole family. A mask is a claim about which pixels belong together. It is not, by itself, a verified statement about what the object is or what it means in the scene, and video tracking stays workload-dependent as scenes and visibility change. Segmentation solved a real problem. It did not quietly solve perception.

The takeaway: the bottleneck keeps moving

Read across the timeline and one pattern holds. The hardest part of building a vision system keeps relocating:

Classical: inventing robust features by hand.
Deep learning: getting enough labeled data and GPU compute to train large networks.
Deeper networks: optimizing depth and efficiency so models trained at all and ran in budget.
Foundation models: pretraining reusable representations and promptable interfaces that many tasks share.

And one methodological warning has to travel with all of it: the accuracy and speed numbers across these papers are not points on a single curve. ImageNet top-5 error, VOC mAP, COCO mask AP, segmentation IoU, and fps measure different tasks on different datasets with different protocols and hardware, sometimes single models and sometimes big ensembles. This is a synthesis across the cited primary papers, not a result from any one of them. Lining them up as one rising line tells a satisfying story and a dishonest one.

The deepest continuity is the one that's easy to miss: modern systems are full of hand-designed choices. They just live in the data curation, the training objectives, the augmentation, the prompt design, the architecture, and the evaluation protocol rather than in a gradient operator. The engineer who once wrote down what an edge was now writes down what counts as a good training example, which concepts the data engine should cover, and which benchmark actually reflects the deployment. The judgment did not leave. It changed address — which, if you maintain these systems, is either reassuring or ominous depending on the day.

Sources link inline to primary papers, official repositories, and project pages. Benchmark figures are historical and specific to their original tasks, datasets, and hardware, and are not directly comparable across eras. A longer technical-history edition with the full milestone timeline is linked at the canonical URL above.

The Handoff Problem: Why Multi-Model AI Pipelines Quietly Corrupt Good Research

Joshua Nwachinemere — Mon, 27 Jul 2026 05:59:47 +0000

Consider a hypothetical but representative handoff. A research model returns a careful finding: an association observed in one dataset, in a controlled setting, with limited generalization beyond the domain tested. Correlation, not cause. A preprint, not a settled result.

Then a writing model receives a compressed summary and produces a clean sentence: Studies show that X causes Y.

Nothing threw an error. The JSON parsed. The next stage ran. The prose reads well. And yet the sentence is now wrong in a way the original was not. The handoff succeeded syntactically and failed epistemically.

That gap is the subject here. Not model selection, not prompt engineering, not which vendor has the smartest reasoner this quarter. The quieter problem is broken claim lineage: evidence crosses a model boundary as persuasive prose instead of as inspectable structure.

If you build multi-model pipelines, this is a reproducibility problem. So let me put the fix on the table before the diagnosis.

The transfer contract, up front

If the interface between stages is where value leaks, then the interface deserves a schema. The practical version is a claim-level record with five fields:

claim: "..."
evidence: "source URL + exact support"
scope: "population, task, model, jurisdiction, date"
uncertainty: "fact / estimate / interpretation / prediction"
citation: "canonical URL and metadata"

Add counterevidence, status, or requires_review where it helps. The point is a discipline, not this exact YAML: this five-field schema is my evidence-informed proposal, not a standardized or experimentally validated format. What the evidence supports is the underlying idea — that structured transfer preserves more than free-form prose in controlled settings — not this particular set of keys.

The rule that makes it work is a division of labor. The writing model may change language and structure freely, because rephrasing is the entire reason it exists. What it may not do is change the semantics of the evidence. Any new factual claim that appears in the draft but not in the ledger routes back to research before it can stay.

Why teams build these pipelines anyway

It would be easy, and wrong, to conclude that multi-model systems are a mistake. They are attractive for concrete reasons: you can run genuinely parallel exploration, assign specialist models to specialist tasks, give each stage its own context window and tools, and spend more total inference effort on a hard problem than any single pass would allow.

Anthropic's engineering team described a production research system built around a lead researcher delegating to subagents, each acting as an information filter that compresses findings back into the lead's context. They report internal gains on breadth-first research tasks, where multiple independent search directions can run at once. That is a real advantage, and worth naming before we start finding fault.

It comes with a bill. Anthropic reports that its multi-agent system used about 15× as many tokens as ordinary chats; it also found that tightly coupled tasks were a poor fit and that vague delegation produced coordination failures. These are Anthropic's internal results on its own system and data, not an industry benchmark. But the shape of the trade-off is instructive: specialization buys you parallelism and depth, and charges you in cost and coordination risk.

The question is not whether to specialize. It is how to specialize without letting the truth mutate at the seams.

The handoff is an information bottleneck

Here is a useful lens, drawn from a decision-theoretic analysis of delegated systems. In a technical preprint, Ao, Gao, and Simchi-Levi model multi-agent planning as a communication network and argue that, absent new exogenous signals, additional stages merely reorganize the same underlying evidence. A relay of models is bounded by what a single centralized decision maker with the same information could achieve. Adding stages does not add facts.

That reframing separates three things pipeline design tends to conflate:

New evidence entering the system — a retrieval call, a tool result, a fresh source. This genuinely expands what the system knows.
The same evidence being rewritten — a summary of a summary. This changes the wording, not the knowledge.
Verification that actually checks against evidence — comparing a claim to its source, rather than to its own fluency.

Most "add another agent" instincts fall into the second bucket while feeling like the first. If every stage sees a transformed version of the same source material, the central design problem is not intelligence but preservation: what survives compression, and what quietly falls out.

The same authors report that free-form relay conditions in their controlled experiments lost decision-relevant information, while a more structured, posterior-style message format preserved more of it. These are controlled multiple-choice tasks, and the result rests on the paper's formal assumptions, so treat it as a design lens rather than a measured guarantee about deployed systems. As a lens, it holds up: the interface between stages does more work than the stages themselves. That is exactly why the YAML above is worth the trouble.

How research mutates without anyone lying

None of this requires a model to hallucinate. The failures are mostly ordinary transfer losses. Drawing on the edge-level categories proposed in AgentAsk, a preprint on inter-agent message errors, plus some editorial synthesis of my own, here is a working taxonomy you can audit:

Data gap. A necessary fact or caveat never crosses the boundary. (AgentAsk's term.)
Signal corruption. A number, qualifier, or citation changes during transfer. (AgentAsk's term.)
Referential drift. "This study" or "they" no longer points to the intended source. (AgentAsk's term.)
Scope inflation. A narrow observation becomes a general claim. (My synthesis, not AgentAsk's.)
Verification theater. A reviewer checks fluency or format instead of evidence alignment. (My synthesis.)

The first three are AgentAsk's; the last two are my editorial labels for patterns visible in writing pipelines. Scope inflation is the case that opened this article: a qualified finding becomes unqualified, and the prose gets cleaner as it becomes less faithful.

The prose can become cleaner while the claim becomes less faithful. A structured record makes the losses visible.

These transfer failures are not the whole story, but they are consistent with broader trace evidence. The MAST taxonomy of multi-agent failures, accepted to the NeurIPS 2025 Datasets and Benchmarks Track, analyzed 1,642 annotated execution traces across seven frameworks and identified fourteen failure modes spanning system design, inter-agent misalignment, and task verification. Among the inter-agent modes: failure to clarify, information withholding, ignored input, and reasoning-action mismatch. Among the verification modes: incomplete and incorrect verification.

Two honest caveats. Much of the large-scale annotation was produced by a calibrated LLM annotator (human inter-annotator agreement was κ=0.88, with the LLM annotator calibrated to κ=0.77 against humans), and the reported framework failure rates use different benchmarks and are explicitly not directly comparable. The percentages are dataset-specific, not universal failure probabilities. What matters is the recurrence of failures that a smarter base model alone cannot address because they are rooted in system design, coordination, and verification.

More reasoning is not monotonically safer

A tempting fix is to let models exchange more of their reasoning at each step, on the theory that more deliberation converges on truth. Sometimes it does; the effect is not reliable.

A preprint on runtime monitoring of error propagation measured what happens when models exchange reasoning traces on quiz subsets across cybersecurity, machine learning, and networking. Paired reasoning often improved accuracy, but not everywhere. In one networking condition, one model's accuracy decreased after the exchange, and in some settings the additional reasoning introduced more errors than it corrected. This is a six-page preprint on multiple-choice subsets, so don't extract a universal rate from it. The defensible point: extra reasoning is another input, not a truth guarantee.

What seems to matter more is how uncertainty travels. DebUnc, a preprint on communicating uncertainty in agent debate, found that an attention-based method for incorporating uncertainty was the only tested approach that consistently matched or exceeded standard debate across its benchmarks, and that the benefit grew as uncertainty estimates improved. The specific method requires open-source model access and code modification, which rules it out for many proprietary APIs. Still, the direction is worth keeping: uncertainty is not noise to be smoothed away before the next stage. It is data the next stage needs — and it belongs in the uncertainty field of the contract.

The machinery already exists

Structured transfer is a design choice you already have the machinery for, though that machinery alone proves nothing. The OpenAI Agents SDK handoff documentation (observed at v0.18.3) exposes an input_type for structured handoff arguments and input_filter/history controls for what the receiving agent sees, and it explicitly distinguishes model-generated metadata from application state. Those are mechanisms for controlling the handoff. They are not evidence that any particular handoff design improves factual accuracy. You still have to decide what crosses the boundary; the SDK just gives you the door.

Specialize the models, preserve claim lineage, and verify the public result anonymously.

Clarify at the edge, audit after the draft

Two control loops help, at different times.

The first is clarification at the boundary. AgentAsk added a learned clarifier at inter-agent message edges without redesigning the orchestration, and reported framework-level averages improving by roughly 3.29 to 3.45 percentage points across five benchmarks and four frameworks, with a GPT-5 clarifier reaching roughly 4.40 to 4.69 points. These are the authors' preprint results on benchmark tasks — math, QA, code — not open-ended research writing, and a clarifier cannot fix an intrinsic model error. The transferable habit is simple: when a handoff is ambiguous or incomplete, ask before assuming.

The second loop runs after the draft exists — a dedicated evidence audit that treats the finished prose as a suspect rather than a product:

map every material sentence back to a ledger entry;
flag any sentence with no supporting entry;
verify that each citation points to the exact claim it is attached to;
restore dropped caveats and scope;
escalate anything unresolved to a human.

This is a production pattern, not a novel idea. Anthropic describes a dedicated CitationAgent that maps report claims to source locations, and an evaluation rubric that checks factual accuracy, citation accuracy, completeness, source quality, and tool efficiency — that is, whether claims align with sources, not merely whether the writing is good. Human testers also caught source-selection failures that the automated evaluation missed, a practical reason to keep a person in the loop even when the automated checks look thorough.

What not to do

Do not hand the next model only a persuasive summary. Persuasiveness is exactly the property that hides missing evidence.
Do not equate consensus with verification. Agreement between agents is not confirmation against a source.
Do not treat an LLM judge as the sole source of truth. MAST itself relied on a calibrated LLM annotator only after establishing human agreement first.
Do not add agents that only restate the same evidence. That is more cost for the same information.
Do not publish from the drafting stage.

One dry note on the consensus point: adding a reviewer that reads only the summary is less defense in depth and more a second opinion from the same rumor.

Implementation checklist

[ ] Pick one boundary between two models and inspect the artifact that actually crosses it.
[ ] Replace that informal summary with a claim-level record carrying claim, evidence, scope, uncertainty, and citation.
[ ] Let the writing model rephrase freely; block it from changing evidence semantics.
[ ] Route any draft claim absent from the ledger back to research.
[ ] Add a clarification step at ambiguous or incomplete handoffs.
[ ] Run a post-draft evidence audit: map sentences to entries, flag unsupported ones, verify citation alignment, restore caveats and scope.
[ ] Keep a human in the loop for source-selection failures automation misses.
[ ] Never publish from the drafting stage.

References

Cemri, M., Pan, M. Z., Yang, S., et al. Why Do Multi-Agent LLM Systems Fail? NeurIPS 2025 Datasets and Benchmarks Track; arXiv v3, 2025-10-26. https://arxiv.org/abs/2503.13657v3
Anthropic Engineering. How we built our multi-agent research system. 2025-06-13. Vendor-authored engineering report; internal evaluations. https://www.anthropic.com/engineering/multi-agent-research-system
OpenAI. Handoffs — OpenAI Agents SDK. openai-agents-python v0.18.3, accessed 2026-07-26. https://openai.github.io/openai-agents-python/handoffs/
Lin, B., et al. AgentAsk: Multi-Agent Systems Need to Ask. arXiv preprint v2, 2026-01-19. https://arxiv.org/abs/2510.07593v2
Ao, R., Gao, S., Simchi-Levi, D. On the Reliability Limits of LLM-Based Multi-Agent Planning. arXiv preprint v1, 2026-03-27. https://arxiv.org/abs/2603.26993v1
Sakib, S. K., Das, A. B. Preventing Error Propagation in Multi-Agent AI through Runtime Monitoring. arXiv preprint v1, 2026-06-27. https://arxiv.org/abs/2606.29026v1
Yoffe, L., Amayuelas, A., Wang, W. Y. DebUnc: Improving Large Language Model Agent Communication With Uncertainty Metrics. arXiv preprint v2, 2025-02-22. https://arxiv.org/abs/2407.06426v2

The Hidden Complexity of RAG

Joshua Nwachinemere — Sat, 25 Jul 2026 23:45:47 +0000

TL;DR

A production RAG system is not one model call wrapped around vector search. It has two pipelines—knowledge preparation and runtime retrieval—plus cross-cutting evaluation, authorization, observability, freshness, and cost controls.

If the final answer is wrong, inspect the stages separately. The root cause may be a stale source, damaged parsing, a bad chunk boundary, missed retrieval, an incorrect filter, weak reranking, noisy context, or generator overreach.

A basic retrieval-augmented generation demo can fit on a whiteboard:

Load → chunk → embed → retrieve → generate

It is a useful abstraction. It is also where many bad production decisions begin.

Imagine an employee asks:

Can contractors expense international travel in 2026?

The application retrieves a policy paragraph and the model returns a confident answer with a citation. The response looks good. It may still be wrong because:

the indexed policy expired last year;
the PDF parser separated a table from its heading;
the chunk omitted the sentence that applies specifically to contractors;
dense search missed the exact policy code;
a permissions filter exposed an internal exception;
the retrieved passages contradicted one another;
the answer added a claim that no source supported.

All of those failures can produce the same visible symptom: a fluent wrong answer.

That is the hidden complexity of RAG. The model is only the final stage in a much larger system. Production RAG is a knowledge supply chain, a search engine, a context-construction layer, an evaluation program, and a security boundary.

The difficult part is not making the pipeline run. It is knowing whether the right evidence survived every step.

RAG is better understood as two pipelines—knowledge preparation and runtime retrieval—surrounded by evaluation, security, and operational controls.

RAG starts before retrieval

Before a user asks a question, an offline pipeline has already made decisions that constrain every possible answer.

It decided which sources to trust. It parsed files. It removed or preserved structure. It split content into chunks, attached metadata, generated embeddings, and wrote records into an index. It also needs to update or delete those records when the source changes.

This is not clerical preprocessing. It is part of the model’s effective knowledge system.

Parsing is an information-loss problem

A parser can extract every visible word from a document and still destroy its meaning.

Consider a policy PDF with:

section headings that define scope;
footnotes that list exceptions;
a table whose column headers appear on another page;
scanned signatures or handwritten dates;
repeated headers and navigation text;
diagrams with labels that do not appear in the text layer.

A plain-text extractor may flatten that structure into a plausible-looking stream of words. The pipeline succeeds technically, but the relationship between those words is gone.

The fix is not always a better prompt. It may require layout-aware extraction, OCR, table handling, media-specific parsing, or a review queue for documents that fail quality checks.

The practical rule is simple:

If the ingestion pipeline loses the evidence, the retriever cannot recover it later.

Chunking is a retrieval policy

Chunk size is often treated like a configuration value to copy from a tutorial. In practice, it determines what the system can retrieve as one unit.

Chunks that are too small can separate a claim from its qualifier, title, date, or exception. Chunks that are too large can mix several topics, weaken retrieval precision, and waste the context budget.

There is no universally correct number of tokens. The right strategy depends on document structure, query patterns, retrieval method, and what counts as sufficient evidence.

Useful approaches include:

preserving headings and section paths;
keeping tables with their titles and column labels;
storing parent-child relationships;
adding controlled overlap where ideas cross boundaries;
attaching source, version, date, author, and access metadata;
using structure-aware or semantic splitting when fixed windows lose meaning.

Microsoft’s current RAG design guidance treats chunking as an experimental phase that should be evaluated against representative documents and queries, not chosen by folklore.

Chunk boundaries determine which facts, qualifiers, metadata, and table labels can be retrieved together.

Freshness requires a deletion story

Adding documents is easy. Synchronizing reality is harder.

A production knowledge pipeline needs to answer:

How quickly should a changed source appear in search?
How are renamed and duplicated documents detected?
What happens when a page is deleted?
Can an old embedding survive after its source has been revoked?
Which source wins when two versions conflict?
Can an answer show the version and effective date it used?

Without stable document identifiers, versioning, change detection, and deletion propagation, a RAG system can become a polished interface to stale information.

Temporal retrieval is harder than simply preferring the newest document. A question may refer to when an event happened, when a source was published, or when a rule was valid. The 2024 TempRALM study is useful evidence that semantic relevance and temporal relevance need separate treatment—but its reported gains come from specific temporal benchmarks, not a universal production recipe.

Retrieval is a ranking pipeline, not a database lookup

After ingestion, the online system has a different job: translate a user’s language into a ranked set of useful evidence.

That involves more than nearest-neighbor search.

Dense retrieval is useful, not magical

Embedding search is good at matching related meanings even when a query and a document use different words. It can still miss exact identifiers, product names, acronyms, dates, and rare terms.

If the user asks about policy TRV-2026-07, lexical search may recognize the exact string more reliably than a semantic vector alone. If the user says “the overseas contractor expense rule,” dense retrieval may be more helpful.

This is why hybrid retrieval remains important. Current Azure AI Search documentation describes running full-text and vector queries in parallel and merging their results with Reciprocal Rank Fusion. The larger lesson is vendor-independent: lexical and semantic signals fail differently, so combining them can be stronger than assuming one replaces the other.

Filters are part of relevance—and security

Metadata filters narrow the candidate set by attributes such as region, product, language, document type, effective date, or tenant.

They can improve quality, but they can also remove the correct answer when metadata is missing or wrong. The timing of a filter—before or after vector search—can also affect recall and latency.

Current Azure vector-filter guidance makes that tradeoff explicit: filter mode, selectivity, index structure, and candidate count can change both recall and performance. This is implementation-specific guidance, but the general lesson travels well—filters belong in retrieval evaluation, not only in application logic.

Permission filters are more serious. They must ensure that content the user cannot access does not enter the retrieved candidate set. Telling the model “do not reveal confidential information” is not access control.

If an unauthorized chunk reaches the prompt, the security failure has already happened.

Reranking is a separate model decision

A first-stage retriever is optimized to find plausible candidates quickly. A reranker spends more computation comparing the query with a smaller candidate set and reorders those results.

That can improve relevance, but it introduces new choices:

How many candidates enter the reranker?
Which fields does it see?
How much latency and cost does it add?
Does it work for the domain and language?
Does improved ranking actually improve final answers?

Current semantic-ranking documentation makes those constraints concrete: reranking operates over an initial ranked set and has input limits. “Add a reranker” is therefore not a free quality upgrade. It is another stage to benchmark.

Query rewriting can help—or quietly change the question

Real questions are messy. They contain typos, pronouns, missing context, and several requests in one sentence.

A query layer may expand synonyms, rewrite the question, decompose it into subqueries, or use conversation history. This can improve coverage for multi-part and multi-hop questions. It can also drift from the user’s intent, retrieve too broadly, or multiply latency and cost.

Current agentic retrieval guidance describes decomposition, parallel subqueries, reranking, and source tracking. It also states that agentic retrieval adds latency, and some capabilities remain preview features.

Agentic RAG is not simply “advanced RAG.” It is a trade: more adaptive retrieval in exchange for more planning, nondeterminism, cost, failure modes, and observability work.

Good retrieval narrows a broad, authorized candidate set into a small evidence set while measuring the tradeoffs at every transition.

Context construction decides what the model is allowed to see

Retrieval produces candidates. Those candidates should not automatically become the prompt.

A context-construction layer may need to:

remove duplicates and near-duplicates;
keep source diversity;
attach neighboring or parent sections;
preserve titles, dates, and provenance;
detect conflicting versions;
fit evidence within a token budget;
order passages intentionally;
create stable citation anchors;
exclude low-confidence or unauthorized material.

This matters because more context is not always better.

The “Lost in the Middle” study found that language models can use relevant information less reliably depending on where it appears in a long input. Separately, an EMNLP 2024 comparison of RAG and long-context models found a real quality-cost tradeoff rather than a universal winner.

A larger context window does not remove the need to select, organize, and evaluate evidence. It merely raises the amount of information the model can potentially process—and the amount of noise you can send it.

Generation cannot repair missing evidence

Once the context reaches the language model, prompting still matters. The model should know what task it is performing, what sources it may use, what output structure is required, and when it must abstain.

But prompting cannot make absent evidence appear.

Useful generation controls include:

require claims to be supported by supplied evidence;
return stable citation identifiers;
distinguish quoted facts from inference;
state when sources conflict;
abstain when evidence is insufficient;
validate required fields and allowed values;
verify that cited passages actually support the answer;
route consequential outputs to human review.

None of these is a perfect “hallucination detector.” Groundedness checks and LLM judges can also make mistakes. For high-impact tasks, deterministic rules, source verification, and human review still matter.

The same fluent wrong answer can begin in the source, parser, chunker, index, retriever, context builder, or generator.

Evaluation must tell you where the failure happened

A single end-to-end score is not enough.

Suppose the correct answer is absent from the response. Several explanations are possible:

The source never contained it.
The parser dropped it.
The chunker separated it from necessary context.
The retriever failed to find it.
A filter removed it.
The reranker pushed it down.
Context assembly excluded it.
The model ignored or misread it.
The evaluator marked a correct answer as wrong.

If evaluation only inspects the final text, the team may tune the prompt for a retrieval failure or replace the embedding model for a parsing failure.

Recent work reflects this need for diagnosis. The 2024 RAGChecker paper separates retrieval and generation diagnostics. MIRAGE, published in Findings of NAACL 2025, evaluates dimensions including noise vulnerability, context acceptability, context insensitivity, and context misinterpretation.

A useful evaluation program works at several levels.

Ingestion

Were all expected files processed?
Were headings, tables, dates, and relationships preserved?
Is metadata accurate?
Did updates and deletions propagate?

Retrieval

Does the candidate set contain the evidence?
How often does the correct passage appear in the top k?
Are exact terms and semantic paraphrases both handled?
Do filters preserve relevant authorized results?
Does reranking improve measured relevance?

Metrics such as Recall@k, Precision@k, mean reciprocal rank, and nDCG can help, depending on the task and labels. None is sufficient by itself.

Context and generation

Does the assembled context cover the required evidence?
Is it redundant, noisy, or contradictory?
Are answer claims supported by the context?
Are citations correct?
Does the system abstain when the corpus cannot answer?

System behavior

Task success with real users
Latency percentiles rather than averages alone
Cost per successful answer
Freshness and deletion service levels
Permission failures and security events
Failure rates by document type, language, tenant, and query class

The test set matters as much as the metric. It should include answerable, unanswerable, ambiguous, exact-identifier, multi-hop, conflicting-source, freshness-sensitive, permission-sensitive, and adversarial questions.

End-to-end scores show whether the system failed; stage-level contracts, versions, and traces help explain where and why.

LLM-based evaluators are useful for scaling review, but they are models—not ground truth. Calibrate them against human labels, monitor disagreement, and keep deterministic checks where possible.

Security is not a filter you add at the end

RAG connects a model to data that may be private, user-generated, or externally controlled. That makes retrieval a security boundary.

Two risks are especially easy to underestimate.

Indirect prompt injection

A retrieved document can contain instructions aimed at the model rather than information for the user. Those instructions may be hidden in webpages, files, tickets, emails, or poisoned knowledge-base content.

OWASP’s current prompt-injection guidance explicitly covers indirect injection from external sources. Retrieved text should therefore be treated as untrusted data, not trusted system instructions.

Embedding and index weaknesses

An attacker—or a simple ingestion bug—can introduce misleading records, manipulate similarity, leak sensitive relationships, or break tenant isolation. OWASP’s vector and embedding guidance discusses poisoning, unauthorized access, and data leakage in RAG-related systems.

The risk is not merely theoretical. PoisonedRAG, published at USENIX Security 2025, demonstrates targeted knowledge-base poisoning under specific experimental threat models. Its attack rates should not be generalized to every corpus or retriever, but the study establishes that retrieved knowledge can be an attack surface—not just a quality problem.

Practical controls include:

source allowlists and provenance;
retrieval-time authorization;
tenant isolation;
sanitization and content classification;
audit logs for sources and citations;
permission-aware caches;
index update and deletion controls;
adversarial evaluation;
output validation and human approval for consequential actions.

Then comes the operational tax

A production request may involve authentication, policy checks, query rewriting, several searches, embedding calls, reranking, context assembly, generation, citation validation, and logging.

Each stage adds latency and a new way to fail.

The system needs more than an overall timer. Trace each stage so a slow answer can be attributed to the search service, a model call, a retry, an oversized candidate set, or an external source.

The same applies to cost. Track the contribution from:

parsing and OCR;
embedding new or changed content;
storage and indexing;
query embeddings;
multiple retrieval calls;
reranking;
generation tokens;
evaluation and monitoring.

Caching can help, but a cache key that ignores tenant, permissions, source version, or model configuration can return stale or unauthorized information.

Version the components that affect behavior: parser, chunker, embedding model, index schema, retrieval settings, reranker, prompt, generator, and evaluation dataset. Otherwise, a quality regression becomes an archaeological investigation.

A diagnostic matrix for wrong answers

Symptom	First place to inspect	Useful evidence
Correct passage never appears	Parsing, chunking, retrieval	Parse samples, chunk boundaries, Recall@k
Exact code/name is missed	Lexical and hybrid retrieval	BM25 baseline, hybrid result set
Relevant passage ranks too low	Candidate depth and reranker	Before/after rankings, nDCG or MRR
Wrong tenant or restricted text appears	Authorization and filters	ACL evaluation, retrieval trace, cache key
Answer ignores strong evidence	Context order and generation	Final context, citation support, model trace
Old answer persists after update	Sync and deletion pipeline	Source version, index version, deletion log
Quality drops after a release	Component versioning	Dataset run by parser/index/prompt/model version

A practical build order

The safest way to build RAG is not to add every advanced technique. It is to add complexity only when evidence identifies a failure it can solve.

Define the task. Specify authoritative sources, users, permissions, freshness requirements, and what a successful answer means.
Build the evaluation set early. Use representative documents and questions, including questions the corpus cannot answer.
Make ingestion observable. Inspect parsing, chunk boundaries, metadata, versions, and deletions before tuning retrieval.
Establish retrieval baselines. Test lexical and dense retrieval, then measure whether hybrid retrieval improves the actual query set.
Add reranking only after measurement. Record quality, latency, and cost changes.
Engineer context deliberately. Handle duplicates, source conflicts, ordering, citations, and abstention.
Evaluate components and the whole system. A good retriever can still feed a model that overreaches; a grounded generator cannot use evidence it never receives.
Test security and failure behavior. Include permission boundaries, poisoned content, timeouts, partial failures, and stale data.
Introduce agentic retrieval selectively. Use it for demonstrated query classes that need decomposition, iteration, or dynamic source selection.
Re-run evaluation after every material change. A new parser, embedding model, index, prompt, or generator changes the system.

The real mental model

RAG is often presented as a way to give an LLM access to your data. That description hides the engineering responsibility.

A better definition is:

RAG is a controlled process for turning changing, permissioned sources into evidence that a generative model can use—and for measuring whether the resulting answer remains supported.

The hard part is not the connection between the model and the database. It is maintaining a trustworthy path from source to evidence to answer, while being able to explain exactly where that path failed.

20 Most Important AI Concepts Explained in Just 20 Minutes

Joshua Nwachinemere — Sat, 25 Jul 2026 21:42:37 +0000

A plain-English guide to the ideas behind ChatGPT, recommendation systems, AI agents, and the models changing how software is built.

Artificial intelligence can feel harder than it is because the same conversation often mixes computer science, statistics, product design, and science fiction.

One person says "AI" and means ChatGPT. Another means a fraud detector. A researcher may be talking about a training method, while a startup founder is describing an API wrapped in a polished interface.

You do not need a mathematics degree to separate these ideas. You need a map.

The 20 concepts below form that map. They explain how modern AI systems learn, how tools such as ChatGPT generate answers, why those answers can be wrong, and what engineers do to make AI useful in the real world.

Read one section per minute. By the end, most AI conversations should sound much less mysterious.

The quick map

Foundations: AI, machine learning, deep learning, neural networks, training, and inference
Ways models learn: supervised, unsupervised, self-supervised, and reinforcement learning
Modern model stack: foundation models, generative AI, LLMs, transformers, tokens, and context windows
Building real applications: embeddings, prompting, RAG, fine-tuning, multimodal AI, agents, evaluation, and guardrails

The concepts are easier to remember when you see how they connect.

1. Artificial intelligence

Artificial intelligence is the broad field of building machines that perform tasks we associate with human intelligence.

Those tasks include recognizing speech, understanding language, spotting patterns, planning actions, making predictions, and generating images or text. The definition is intentionally broad. A chess engine, a spam filter, and a voice assistant can all be called AI even though they work in very different ways.

AI does not automatically mean consciousness or human-like reasoning. Most systems are narrow: they perform a bounded task under specific conditions. A model that detects tumors in scans cannot drive a car unless someone builds and trains a separate system for driving.

Think of AI as the name of the entire field. The remaining concepts describe different ways of building systems inside it.

2. Machine learning

Traditional software follows rules written by a programmer. Machine learning takes a different route: the programmer gives the computer examples, and an algorithm learns a pattern from them.

Suppose you want to identify fraudulent card transactions. Writing a complete rulebook is nearly impossible because fraud changes constantly. With machine learning, you provide historical transactions and indicate which were fraudulent. The model studies relationships among amount, location, timing, device, and other signals. It then estimates the likelihood that a new transaction is suspicious.

The result is not a database of memorized answers. It is a mathematical function shaped by data.

Machine learning works best when useful patterns exist in the examples. If the data is incomplete, biased, mislabeled, or unrelated to the problem, the model learns the wrong lesson with impressive efficiency.

3. Deep learning

Deep learning is a branch of machine learning built around neural networks with many processing layers.

Earlier machine-learning systems often depended on people to choose the important features. An engineer building an image classifier might manually describe edges, shapes, and colors. A deep-learning model can learn useful representations directly from raw pixels. Early layers may respond to edges, later layers to textures and parts, and deeper layers to larger visual patterns.

This ability to learn representations helped drive major progress in computer vision, speech recognition, translation, and generative AI. It also comes at a cost. Deep models usually require more data, computing power, and energy than simpler methods. Their internal decisions can be difficult to interpret.

Deep learning is powerful, but it is not always the right answer. A small, well-structured dataset may be handled better by a simpler model that is cheaper and easier to explain.

4. Neural networks

A neural network is a collection of connected mathematical units arranged in layers. Each connection has a number called a weight.

An input enters the network. The layers transform it, using the weights, until the network produces an output. During learning, the system compares that output with the desired result and adjusts the weights to reduce the error.

Imagine a network deciding whether an email is spam. Words, links, sender information, and formatting become numerical inputs. The network combines those signals through its layers and returns a probability. A result of 0.97 might mean the model estimates a 97% chance that the message is spam.

The word "neural" comes from a loose analogy with biological neurons, but modern neural networks are mathematical systems, not digital brains. The analogy is useful historically and often misleading technically.

5. Training and inference

Every deployed model has two distinct phases: training and inference.

Training is when the model learns. It processes examples, measures its errors, and updates its internal weights. Training a large model can involve enormous datasets, thousands of specialized chips, and weeks or months of computation.

Inference is when the trained model is used. You provide a new input and the model produces a prediction or response. Each time you send a prompt to a chatbot, ask a phone to transcribe audio, or use an AI image generator, you are running inference.

The distinction matters because the engineering problems differ. Training focuses on data quality, optimization, compute, and reproducibility. Inference focuses on speed, cost, memory, reliability, and handling many users at once.

When a chatbot learns something from the current conversation, it usually has not retrained its weights. It is using the conversation as temporary context during inference.

6. Supervised learning

Supervised learning trains a model with examples that include correct answers, called labels.

A dataset might contain house details paired with sale prices, medical images paired with diagnoses, or customer messages paired with categories such as billing, cancellation, and technical support. The model learns a mapping from each input to its label.

Two common supervised-learning tasks are classification and regression. Classification chooses a category: spam or not spam, cat or dog, approved or rejected. Regression predicts a number: tomorrow's energy demand, a delivery time, or a home's likely price.

The hard part is often not choosing the algorithm. It is collecting reliable labels. Human annotators disagree. Historical decisions may encode discrimination. Labels can become outdated as the world changes.

A supervised model can reproduce those problems because it learns what happened in the dataset, not what should have happened.

7. Unsupervised and self-supervised learning

Unsupervised learning looks for structure in data that has no human-provided labels.

A retailer might group customers by purchasing behavior without deciding the groups in advance. A security system might flag activity that looks different from normal traffic. Common techniques include clustering, dimensionality reduction, and anomaly detection.

Self-supervised learning is closely related but creates a learning task from the data itself. A language model can hide part of a sentence and learn to predict it, or predict the next token from everything that came before. An image model can hide sections of an image and learn to reconstruct them.

This matters because unlabeled data is abundant. The internet contains far more text, images, video, and audio than humans could manually label.

Much of modern generative AI starts with self-supervised learning at scale, followed by additional training that makes the model more useful for particular tasks.

8. Reinforcement learning

Reinforcement learning trains an agent through actions and rewards.

The agent observes an environment, chooses an action, receives feedback, and updates its strategy. A game-playing agent may receive a positive reward for winning. A warehouse robot may be rewarded for moving items efficiently while avoiding collisions.

Unlike supervised learning, the correct action is not necessarily supplied for every situation. The agent must explore and discover which sequences of actions lead to better outcomes. This creates a difficult tradeoff between trying what already works and testing something new.

Rewards must be designed carefully. An agent will optimize the measurable objective, even when that objective is a poor substitute for what people intended. Give a cleaning robot points for collecting dirt and it might learn that spilling dirt creates more opportunities to score.

That failure pattern is called reward hacking. It is one reason reinforcement learning in the real world needs constraints and monitoring.

9. Foundation models

A foundation model is trained on broad data and can be adapted to many tasks.

Instead of building one model for summarization, another for translation, and another for question answering, developers can start with a general model and adapt it through prompts, retrieval, fine-tuning, or additional tools.

Large language models are one type of foundation model. Others work primarily with images, audio, video, or several forms of data together.

The word "foundation" does not mean the model is complete or universally intelligent. It means other applications can be built on top of it. A foundation model is usually one component in a larger system that also contains instructions, business rules, databases, APIs, user interfaces, and safety controls.

This reuse makes powerful capabilities accessible, but it also concentrates risk. A weakness inherited from one foundation model can affect many products that depend on it.

10. Generative AI

Generative AI creates new content by learning patterns from existing data.

Depending on the model, the output may be text, code, images, speech, music, video, or structured data. The system does not normally retrieve a finished answer from a giant archive. It generates an output step by step according to patterns encoded in its parameters and the information supplied at inference time.

"New" does not mean wholly original. Generated content can resemble training examples, repeat common clichés, or reproduce biases found in the underlying data. Models may also memorize and reveal fragments in some circumstances.

Generative systems are probabilistic. Give the same prompt twice and you may receive different results. Settings such as temperature alter how conservative or varied the output is.

This variability is useful for brainstorming and creative work. It is a liability when a system must return the same verified answer every time.

11. Large language models

A large language model, or LLM, is trained to model patterns in language, usually by predicting the next token in a sequence.

Given "The capital of France is", the model assigns probabilities to possible continuations. "Paris" receives a high probability because the training process has shaped the model's weights around patterns found across enormous amounts of text.

Next-token prediction sounds simple, but doing it at scale forces a model to learn grammar, style, facts, code patterns, and some relationships among concepts. This produces surprisingly general behavior: summarization, translation, drafting, question answering, and programming assistance can emerge from the same base model.

An LLM still does not consult a guaranteed internal database before answering. It produces a plausible continuation based on learned patterns and current context. Fluency is therefore evidence of language skill, not proof of factual accuracy.

12. Transformers and attention

Most modern LLMs use an architecture called the transformer, introduced in the 2017 paper "Attention Is All You Need."

Its central mechanism, attention, lets the model weigh relationships among different parts of an input. In the sentence "The trophy would not fit in the suitcase because it was too large," attention helps the model connect "it" with "trophy" rather than "suitcase."

Transformers process many positions in parallel during training, unlike older sequence models that handled text mainly one step at a time. This made them easier to scale on modern hardware.

Attention is not human concentration. It is a numerical operation that calculates which representations should influence one another and by how much.

The same architecture now appears beyond text. Transformers process images, audio, biological sequences, and combinations of several data types.

13. Tokens and context windows

Language models do not read text as words. They process tokens.

A token may be a whole word, part of a word, punctuation, or a short character sequence. "Unbelievable" might be represented as one token or split into pieces, depending on the tokenizer. Code, uncommon names, and some languages may require more tokens than ordinary English prose.

The context window is the amount of tokenized information a model can consider in one request. It may include your prompt, previous messages, retrieved documents, tool results, and the model's response.

A larger context window allows a system to handle longer documents and conversations, but it does not guarantee perfect recall. Important details can still be overlooked, drowned in irrelevant material, or contradicted elsewhere in the prompt.

Tokens also affect cost and speed because model providers usually measure usage by the number of input and output tokens processed.

14. Embeddings and vector search

An embedding is a list of numbers that represents the meaning or other useful properties of an item.

Text with similar meaning tends to have nearby embeddings even when it uses different words. "How do I reset my password?" and "I cannot log in because I forgot my credentials" may be close in embedding space.

Vector search compares these numerical representations to find similar items. It powers semantic search, recommendation systems, duplicate detection, clustering, and document retrieval.

A vector database stores embeddings and searches them efficiently. Despite the name, it does not understand meaning by itself. The usefulness of the results depends on the embedding model, the source data, how documents were divided, metadata filters, and the definition of similarity.

Embeddings turn fuzzy questions about meaning into geometry that software can compute, but the geometry remains an approximation.

15. Prompting

A prompt is the input that tells a generative model what to do and provides the context it should use.

Good prompts reduce ambiguity. They state the task, relevant background, constraints, desired format, and sometimes examples. "Summarize this" is weaker than "Summarize this incident report for an engineering manager in five bullets. Separate confirmed facts from unresolved questions."

Prompt engineering is useful, but it is not magic. A prompt cannot guarantee facts the model does not have, enforce permissions, or replace deterministic validation. If an application requires valid JSON, the system should parse and validate the output. If a user is not allowed to refund a payment, access control should block the action regardless of what the prompt says.

Prompts guide model behavior. Software controls must enforce the rules that matter.

16. Retrieval-augmented generation

Retrieval-augmented generation, usually shortened to RAG, gives a model relevant information before it answers.

A typical RAG system splits documents into searchable chunks, converts them into embeddings, retrieves the chunks closest to a user's question, and adds them to the model's context. The model then writes an answer using that material.

This helps when the required information is private, specialized, or newer than the model's training data. A support assistant can search product manuals. An internal tool can answer questions from company policies without retraining the underlying model every time a policy changes.

RAG does not automatically eliminate incorrect answers. Retrieval may return the wrong passage, miss an important document, or surface outdated material. The model can misunderstand good evidence.

Reliable RAG systems test retrieval and answer quality separately, preserve citations, track document versions, and allow the model to say when the evidence is insufficient.

17. Fine-tuning

Fine-tuning continues training an existing model on a smaller, targeted dataset.

It can teach a model a consistent output format, specialized vocabulary, a particular style, or better behavior on a narrow task. Because the starting model already knows broad patterns, fine-tuning usually requires far less data and compute than training from scratch.

Fine-tuning is often confused with adding knowledge. It can help a model internalize recurring patterns, but it is a poor way to maintain frequently changing facts. Updating a policy document in a RAG index is much easier than retraining a model whenever the policy changes.

A useful rule is:

Use prompting when instructions are enough.
Use retrieval when the model needs external or changing knowledge.
Use fine-tuning when behavior must change consistently across many examples.

Real systems often combine all three.

18. Multimodal AI

A multimodal model can process or generate more than one type of data, such as text, images, audio, or video.

A multimodal assistant might inspect a chart, discuss it in text, listen to a spoken follow-up question, and answer aloud. A document-processing system can combine page layout, printed words, handwriting, and tables rather than treating a PDF as plain text.

The difficult part is connecting representations across modalities. The model must learn that pixels showing a red traffic light, the phrase "red light," and the sound of someone saying those words refer to related ideas.

Multimodal does not mean equally capable in every mode. A model may describe photographs well but struggle with tiny labels, spatial measurements, long videos, or noisy audio. Each modality needs separate evaluation because a strong language score says little about visual accuracy.

19. AI agents and tool use

An AI agent combines a model with tools, state, and a loop for deciding what to do next.

A plain chatbot returns text. An agent may search the web, read files, query a database, run code, call an API, inspect the result, and continue until it reaches a goal.

The model usually acts as the planner, while ordinary software performs the actions. This distinction matters. The model can suggest "send the refund," but a payment API with real credentials is what moves the money.

Agents become risky when a probabilistic model controls consequential tools without tight boundaries. A production agent needs least-privilege permissions, input validation, timeouts, duplicate-action protection, logs, spending limits, and human approval for irreversible steps.

The impressive part of an agent demo is often its autonomy. The useful part of a deployed agent is controlled autonomy: enough freedom to finish the job, but not enough to create a new one.

20. Evaluation, hallucinations, and guardrails

A hallucination is an output that sounds plausible but is unsupported or false. Language models are especially prone to this because their basic objective rewards likely continuations, not verified truth.

Evaluation measures how well an AI system performs. A useful evaluation set reflects the real task, includes difficult and failure cases, and scores what users care about. That may mean factual accuracy, retrieval quality, latency, cost, bias, safety, or whether an agent completed an action without harmful side effects.

Guardrails are the controls around the model: schema validation, content filters, permission checks, source requirements, rate limits, human approval, and monitoring. They reduce risk but do not turn an unreliable model into a guaranteed one.

The application, not just the model, must be evaluated. A strong model can fail inside a poor retrieval pipeline. A weaker model can perform well when given clean data, narrow tools, and clear checks.

Treat model output as untrusted until the system has verified everything that matters.

How the pieces fit together

In production, the model is one component inside a controlled software system.

Consider an internal support assistant.

A foundation model provides the language capability. The application converts company documents into embeddings and stores them for vector search. When an employee asks a question, a RAG pipeline retrieves relevant policy passages and places them inside the model's context window. A prompt tells the model to answer only from those passages and include citations.

If the assistant needs to check an order, an agent tool calls the order database with read-only permission. The final response passes through validation and logging. Evaluations measure whether retrieval found the correct policy, whether the answer matched the evidence, and whether the tool respected access rules.

No single concept makes that system work. The value comes from how the parts are connected.

A mental model worth keeping

AI is the broad field. Machine learning learns patterns from data. Deep learning uses layered neural networks. Foundation models provide general capabilities. Transformers and attention power many of today's language systems. Tokens and context define what a model can process at once. Embeddings and retrieval connect models to external information. Fine-tuning changes recurring behavior. Agents connect models to actions. Evaluation and guardrails determine whether the complete system is trustworthy enough to use.

You do not need to memorize every term. Remember the boundaries:

A fluent answer is not necessarily a factual answer.
A large context window is not perfect memory.
Retrieval is not guaranteed understanding.
Fine-tuning is not a live database.
An agent is not safe merely because its instructions say "be careful."
A model demo is not the same as a reliable product.

Once those distinctions are clear, AI becomes easier to reason about. It stops looking like one mysterious technology and starts looking like what it is: a collection of models, data pipelines, software controls, and human decisions.