DEV Community: Rost

Configure Git User Name and Email Address

Rost — Tue, 08 Sep 2026 10:48:32 +0000

After clonning the git repository, do the local repository configuration, particularly set user name and email address.

This page is part of Developer Tools: The Complete Guide to Modern Development Workflows.

When you doing first commit to newly clonned repo via command line like

git commit -m "test commit msg"

you might receive a sencible message and git do what expected:

[master 3506e11] test commit msg
 Committer: megaadmin <megaadmin@myhost>
Your name and email address were configured automatically based
on your username and hostname. Please check that they are accurate.
You can suppress this message by setting them explicitly
...

but if you use vs code, you might get a message like

How

Basically what we need to do is to run git with configure command:

git config user.email "root@microsoft.com"
git config user.name "bg"

Just replace email and username with yours.

Useful links

About hosting local Gitea server: Choosing free on-prem git server - Gitea is the winner!
For SSL setup with Gitea: Gitea SSL with Apache as reverse proxy
Once your Gitea server is running: Backup and Restore Gitea server
GIT Cheatsheet: Most useful GIT commands
For branching strategies: Gitflow Explained: Steps, Alternatives, Pros, and Cons

What Comes After LLMs? Mamba, Diffusion & World Models

Rost — Tue, 08 Sep 2026 10:48:05 +0000

AI hype follows a rhythm: roughly every three years, the architecture everyone is betting on gets replaced by something newer. The next shift is already forming in research labs.

What follows is a tour of the strongest candidates to replace the transformer, and of the evidence that the shift is already under way. The pattern repeats: a dominant architecture, a wall it cannot scale past, and a new design that routes around that wall.

This article walks through the four limitations closing in on transformers, the three alternative architectures, and what the hybrid systems already in production tell us. The goal is a practical read for engineers deciding what to architect for 2027 and beyond — not a prediction market, just the evidence as it stands in 2026.

The pattern: AI's three-year breakthrough cycle

Every few years the center of gravity of AI shifts:

flowchart LR
    A["~2017<br/>Transformers replace<br/>RNNs and CNNs"] --> B["2020-2022<br/>Diffusion models take<br/>image generation"]
    B --> C["2022<br/>LLMs reach everyone<br/>(GPT-3, ChatGPT)"]
    C --> D["2026-2028<br/>Hybrid post-transformer<br/>systems in production"]

Transformers showed up around 2017 and made RNNs and CNNs look like relics. Diffusion models then took over image generation between 2020 and 2022. And LLMs exploded into everything starting in 2022 with GPT-3 and ChatGPT.

By that pattern, we should be approaching the next shift. It is already happening — just not where you would expect. Not on social media or in press releases, but in research labs.

Yann LeCun spent 2025 arguing that autoregressive LLMs are fundamentally limited. "We need world models, not word predictors," he said at conference after conference. Meanwhile his team published a sequence of papers on Joint Embedding Predictive Architecture (JEPA) — first for images, then video, then language in September 2025.

The supporting evidence from the last twelve months is unusually concrete:

State space models like Mamba handle million-token contexts with linear scaling — something transformers cannot do because of their quadratic attention mechanism.
Google announced Gemini Diffusion, promising text generation up to 10x faster.
NVIDIA shipped Nemotron 3 with Mamba-2 layers built in.
China's Qwen3.5 went live using Gated DeltaNets.
In December 2025, VL-JEPA landed — a vision-language model with half the parameters of its competitors that still performed better.

Why transformers are running out of road

Transformers are incredible at what they do. But four problems are closing in that no amount of scaling can fix.

Quadratic attention breaks at long context

Self-attention means every token talks to every other token — O(n²) complexity. Double your context and you quadruple the compute. At 10 million tokens, the math simply breaks. You can throw more GPUs at it, but you are fighting basic arithmetic.

The training data is running out

LLMs train on internet text, and by 2026 most of it has been scraped. Books, Wikipedia, Reddit, GitHub — all consumed. Synthetic data sounds like a fix until you realize that models trained on AI-generated text degrade over generations: they learn the artifacts and biases of the generation process itself. The scaling laws that powered everything from 2020 to 2024 are hitting diminishing returns.

Energy and cost are the hard ceiling

Frontier models cost tens or hundreds of millions to train now. Serving billions of requests daily burns massive compute. AI data centers need gigawatts of power, and power generation capacity is becoming a bottleneck. Sam Altman admitted in 2024 that compute costs limit everything — and that has not changed.

If the constraint is energy, the answer can come from the model side or the silicon side. My notes on cost optimization for LLM systems and on specialized LLM inference chips cover each side of that tradeoff.

Pattern matching is not understanding

LLMs are pattern matchers at scale, predicting the next token from training data. Exceptional at it, sure — but not reasoning, and not understanding causal relationships in the physical world. If you want AI that can plan, simulate, or genuinely adapt to new environments, token prediction will not get you there.

State space models: linear scaling with a compressed memory

State space models (SSMs) such as Mamba and Gated DeltaNet take a completely different approach to sequences. Instead of attending to all previous tokens, they maintain a compressed internal state that summarizes history — linear O(n) scaling instead of quadratic, with constant memory per token at inference regardless of how long the sequence gets.

The mechanism behind that efficiency is worth understanding, because it explains both the speed and the tradeoff. Classic state space models (the S4 family) were time-invariant: the same fixed transition matrix applied at every step, which is fast but cannot decide that one token matters more than another. Mamba's contribution was to make those transition parameters selective — functions of the current input — so the model can dynamically choose what to keep in its compressed state and what to discard, much like attention's softmax decides what to emphasize, but without materializing an O(n²) score matrix. To keep that selective recurrence fast, Mamba runs it as a hardware-aware parallel scan: an associative operation computed in GPU SRAM rather than a token-by-token loop, which is the same kind of low-level kernel engineering that FlashAttention did for transformers. Mamba-2 pushed this further with "state space duality," showing that a selective SSM and a restricted form of linear attention compute the same underlying operation — letting SSM kernels reuse the matrix-multiplication hardware that GPUs are optimized for.

The real production breakthrough here was Gated DeltaNet, an improvement on Mamba2 that uses a delta rule for better long-context handling. Qwen3.5 adopted it as its core architecture in February 2026 and got strong results on agent tasks and coding benchmarks using only 3 billion active parameters in its hybrid MoE configuration. That is impressive efficiency, and my llama.cpp benchmarks of Qwen 3.6 MTP versus standard decoding on a 16 GB GPU give a sense of how Qwen models behave on consumer hardware.

The tradeoff is structural, not just a training artifact: because an SSM compresses the entire history into a fixed-size state, it can lose fine-grained detail that transformers preserve by keeping every past token individually addressable through attention. If you need exact copying or citation, transformers still win. For most practical workloads — summarization, agentic tool use, long-document reasoning where the gist matters more than verbatim recall — the efficiency gain is worth it.

Diffusion language models: many tokens per forward pass

Diffusion language models apply the same technique that transformed image generation to text. Instead of generating one token at a time, they can produce multiple tokens in parallel during each forward pass.

NVIDIA's Nemotron-Labs-Diffusion, released in May 2026, achieved nearly 6x tokens per forward pass compared to Qwen3-8B while maintaining competitive accuracy across benchmarks including HumanEval, GSM8K, and Math500. It is a tri-mode model that unifies autoregressive, diffusion, and self-speculation decoding. Google's Gemini Diffusion promises even more speedup.

The catch is quality versus speed. Autoregressive models optimize for local coherence at each step — the most probable next token. Diffusion models optimize globally across the entire output, which can improve overall structure but sometimes sacrifices fine-grained detail. For creative writing or brainstorming, diffusion might be better. For precise technical output, autoregressive still wins.

It is worth separating this from inference-time tricks. Speculative decoding already delivers several tokens per forward pass by drafting and verifying, with no change to the output distribution. Diffusion moves parallel token generation from the inference layer into the model architecture itself — a structurally different bet.

World models and JEPA: predicting meaning, not tokens

World models, specifically Yann LeCun's JEPA architecture, take the most radical approach. Instead of predicting the next token, they predict the next latent embedding — a compressed representation of what should come next. That is closer to how human cognition works: we do not predict individual words, we predict meaning.

The architectural difference from both transformers and diffusion models is the missing decoder. A generative model — autoregressive or diffusion — has to reconstruct its target in the original space: exact pixels, exact tokens. That forces the model to spend capacity modeling unpredictable surface noise: the exact phrasing of a sentence, the exact pixel value of a leaf in the wind. JEPA's loss operates entirely inside a learned representation space instead, comparing a predicted embedding against a target embedding produced by a separate, slowly-updated encoder. Nothing is ever decoded back to raw output during training. That lets the predictor discard the noise and keep only the structural, task-relevant part of the signal — which is also, not coincidentally, the part that is useful for planning: an agent can simulate "what happens if I do X" by rolling forward in latent space, without paying the cost of rendering a full image or generating full text for every hypothetical.

VL-JEPA achieved performance comparable to established vision-language models with only 1.6 billion parameters versus 7 billion or more for competitors. It reduces decoding operations by about 2.85x while improving accuracy on visual question answering. More importantly, JEPA architectures are designed for continual learning — they build internal models of how the environment works, not just statistical patterns in text.

The limitation is that JEPA is still maturing for pure language tasks: LLM-JEPA exists but has not matched large autoregressive or diffusion models on standard benchmarks yet, and JEPA is not a drop-in replacement for a chatbot — it is a representation-learning and planning substrate that complements language models more than it competes with them head-on. For embodied AI and robotics, however — where understanding physical causality matters more than generating fluent prose — JEPA might be the right architecture from the start.

What production systems are doing: hybrid architectures

The most interesting developments combine elements from multiple architectures. AI21 Labs' Jamba interleaves transformer attention layers with Mamba state space layers — attention for long-range dependencies, SSMs for efficient local processing. Qwen3.5 combines Gated DeltaNet with Mixture of Experts, activating only the relevant expert networks for each input. Cloud providers are already listing Mamba-based models — Mistral's Codestral Mamba was the first open-source Mamba-2 release — and the cloud LLM providers overview tracks what is available where.

Different tasks have different requirements:

Code generation might benefit from diffusion's parallel token generation.
Long-document analysis needs an SSM's linear scaling.
Creative writing could use transformer attention for coherence.

A single architecture will not solve everything. The same logic that drives multi-model system design — the right mechanism for the right task — is now appearing inside the models themselves.

If you are building AI applications and architecting for 2027 or 2028, do not over-invest in pure transformer solutions. Consider whether SSM or diffusion components might better serve your use case. The ecosystem is shifting, and early adopters will have an advantage.

Two further implications are worth tracking. Qwen3.5's performance with only 3 billion active parameters suggests that specialized, efficient architectures will outperform larger general-purpose models for many tasks — which could democratize AI deployment and make high-performance models accessible on consumer hardware. And diffusion language models could reduce latency by an order of magnitude for certain workloads; applications requiring real-time response — conversational agents, live translation, interactive tools — will benefit most.

Where this is heading: the post-transformer era

Following the three-year cycle, I would expect the first production deployments of non-transformer architectures by 2027–2028. These will likely be hybrid systems that leverage transformer attention where needed but use SSMs or diffusion for efficiency gains.

The consolidation phase we are in now — where labs prove that LLMs can reason and act — is giving way to a harder challenge: making these systems efficient, adaptable, and spatially aware. That demands breakthroughs in continual learning, world models, and architectural efficiency. Each addresses fundamental limitations that pure scaling cannot solve.

The next big thing will not be bigger models. It will be smarter architectures that achieve more with less — models that understand rather than just predict, that adapt rather than just process. That is what comes after LLMs. Not a single new architecture, but a generation of specialized, efficient systems that finally move beyond pattern matching toward genuine understanding.

References

Adaline Labs. "The AI Research Landscape in 2026: From Agentic AI to Embodiment." https://labs.adaline.ai/p/the-ai-research-landscape-in-2026
Aftab, A. "The End of LLMs As We Know Them: Why 2026 Marks the Beginning of AI's Next Architecture Revolution." Medium, 2025. https://medium.com/@aftab001x/the-end-of-llms-as-we-know-them-why-2026-marks-the-beginning-of-ais-next-architecture-revolution-902ee29484f7
Greengard, S. "Beyond LLMs: A Post-Transformer World Emerges." Communications of the ACM, May 2026. https://cacm.acm.org/news/beyond-llms-a-post-transformer-world-emerges
NVIDIA. "Nemotron-Labs-Diffusion: A Tri-Mode Language Model Unifying Autoregressive, Diffusion, and Self-Speculation Decoding." arXiv:2607.05722, 2026. https://arxiv.org/html/2607.05722v1
MarkTechPost. "NVIDIA AI Releases Nemotron-Labs-Diffusion: A Tri-Mode Language Model with 6× Tokens Per Forward Over Qwen3-8B." May 20, 2026. https://www.marktechpost.com/2026/05/20/nvidia-ai-releases-nemotron-labs-diffusion-a-tri-mode-language-model-with-6x-tokens-per-forward-over-qwen3-8b
Chen, D. et al. "VL-JEPA: Joint Embedding Predictive Architecture for Vision-language." arXiv:2512.10942, 2025. https://arxiv.org/pdf/2512.10942
OpenReview. "VL-JEPA: Joint Embedding Predictive Architecture for Vision-language." https://openreview.net/forum?id=tjimrqc2BU
NVIDIA Research. "Gated Delta Networks: Improving Mamba2 with Delta Rule." ICLR 2025. https://research.nvidia.com/publication/2025-04_gated-delta-networks-improving-mamba2-delta-rule
Laon People. "Why Did Qwen3.5 Choose Gated DeltaNet?" February 2026. https://laonpeople.com/en/blog/why-did-qwen3-5-choose-gated-deltanet
Google DeepMind. "Gemini Diffusion." https://deepmind.google/models/gemini-diffusion
Vicentino, C. "Autoregressive vs. Masked Diffusion Language Models: A Controlled Comparison." arXiv:2603.22075, March 2026. https://www.alphaxiv.org/abs/2603.22075
JetBrains AI Blog. "Why Diffusion Models Could Change Developer Workflows in 2026." November 2025. https://blog.jetbrains.com/ai/2025/11/why-diffusion-models-could-change-developer-workflows-in-2026
Spheron Blog. "Mamba-3 and State Space Models on GPU Cloud: Deploy SSM Inference as the Transformer Alternative (2026 Guide)." https://www.spheron.network/blog/mamba-3-state-space-model-gpu-cloud-deployment
Gartner. "Gartner Predicts 40 Percent of Enterprise Apps Will Feature Task-Specific AI Agents by 2026." August 26, 2025. https://www.gartner.com/en/newsroom/press-releases/2025-08-26-gartner-predicts-40-percent-of-enterprise-apps-will-feature-task-specific-ai-agents-by-2026-up-from-less-than-5-percent-in-2025
Reddit r/deeplearning. "Following a 3-year AI breakthrough cycle." https://www.reddit.com/r/deeplearning/comments/1k8gdwg/following_a_3year_ai_breakthrough_cycle
Gu, A., Dao, T. "Mamba: Linear-Time Sequence Modeling with Selective State Spaces." arXiv:2312.00752, 2023. https://doi.org/10.48550/arxiv.2312.00752
Dao, T., Gu, A. "Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality (Mamba-2)." arXiv:2405.21060, 2024. https://arxiv.org/abs/2405.21060
Meta AI Blog. "The first AI model based on Yann LeCun's vision for more human-like AI (I-JEPA)." https://ai.meta.com/blog/yann-lecun-ai-model-i-jepa/
LeCun, Y. "A Path Towards Autonomous Machine Intelligence." Position paper, 2022. https://openreview.net/forum?id=BZ5a1r-kVsf

Agent Skills vs MCP Servers: Decision Framework

Rost — Sun, 06 Sep 2026 01:55:19 +0000

Agent Skills and MCP servers are often presented as competing ways to extend an AI agent. That framing is wrong: a skill teaches the agent how to work, while an MCP server gives it governed access to live capabilities.

The useful question is not "Which standard wins?" It is "Where should this responsibility live?" This guide answers that for hosted assistants such as Hermes Agent and OpenClaw, where context size, long-running connections, credentials, and operational safety matter more than a tidy demo.

This is not an academic comparison. It is a practical decision framework built from real deployment experience with both mechanisms, including the context-cost tradeoffs that only become visible once an agent is running in production. If you are building multi-agent systems, you may also want to read our A2A vs MCP protocol comparison, which covers a different axis of the same problem space.

Agent Skills vs MCP Servers in One Table

Use a skill for procedure, judgment, and reusable operating knowledge. Use an MCP server for authoritative state, protected operations, and a stable capability contract.

Decision signal	Agent Skill	MCP server	Usually both
Static instructions, checklists, or style rules	Best fit	Poor fit	Sometimes
Live tickets, deployments, records, or metrics	No	Best fit	Yes
Credentials or delegated user identity	Avoid	Best fit	Yes
Existing local CLI with safe, narrow commands	Good fit	Optional	Sometimes
Transactional writes or idempotency	Weak fit	Best fit	Yes
Portable procedure across agent hosts	Best fit	Optional	Yes
Shared capability across languages and clients	Limited	Best fit	Yes
Human approval and escalation policy	Best fit	Enforce final check	Best fit
Output format and evidence rubric	Best fit	No	Sometimes

My default is deliberately conservative: start with a skill when the job is local, read-heavy, and procedural. Add an MCP server when the agent crosses a trust boundary, touches changing external state, or needs an operation that should remain correct even when the model is confused.

The Core Distinction: Procedure vs Capability

An Agent Skill is a directory centered on SKILL.md, with optional scripts, references, and assets. The Agent Skills specification defines required metadata and a progressive disclosure model: the host can discover a small name and description first, load the full instructions when relevant, and fetch supporting files only when needed.

That makes a skill a strong home for an incident rubric, a release checklist, a research method, or instructions for using an existing command-line tool. Its central value is encoded procedure: sequence, judgment, constraints, examples, and the definition of a good result. For Hermes-specific authoring details including frontmatter structure and conditional activation, see Hermes Agent Skill Authoring.

MCP solves a different problem. The Model Context Protocol specification gives a client and server a JSON-RPC-based contract for capabilities including tools, resources, and prompts, with standard transports and discovery behavior. Practical implementation guides for MCP servers in Python and MCP servers in Go show how straightforward the integration layer can be once the protocol handles the heavy lifting.

An MCP server is therefore a good boundary around a ticket system, cloud control plane, source-of-truth database, or internal search service. It owns the mechanics of reaching that system and can enforce validation, authorization, timeouts, rate limits, and audit behavior outside the model's prose instructions.

A More Precise Rule

Ask whether a responsibility must remain correct without the model remembering an instruction. If the answer is yes, it belongs in deterministic code or server policy, not only in SKILL.md.

For example, "collect three supporting signals before escalating" is a useful skill instruction. "Reject a status change unless the caller has the incident-manager scope" must be enforced by the service or MCP server, even if the skill repeats the rule.

This is the boundary that matters:

A skill can tell the agent when an action is appropriate.
An MCP tool can make the action available through a typed interface.
The backing service must decide whether the action is actually allowed.

MCP servers can also expose prompts, so the standards overlap at the edges. Still, putting an entire operating procedure in a giant tool description usually produces a brittle capability catalog, while putting a privileged API client in a shell script hidden inside a skill usually produces an avoidable security problem.

When a SKILL.md Is Enough

A skill is enough when the agent already has safe access to everything required and the missing ingredient is know-how. This is common for repository analysis, document transformation, report generation, or a local workflow built on mature CLI commands.

The Data Is Local or Supplied by the User

Suppose an agent must inspect a checked-out repository, run read-only linters, compare configuration files, and produce a migration report. The files are already in the working environment, and the host already exposes filesystem and process tools, so another network service adds little value.

The skill can describe which files to inspect, the command order, failure handling, and the required evidence. A bundled script may normalize output, but the host's existing sandbox and command permissions remain the actual execution boundary.

The Workflow Depends on Judgment

Skills are especially useful when several technically valid actions exist but the organization prefers one operating method. A code-review skill can explain which risks deserve blocking comments, when to request a reproduction, and how to separate correctness issues from taste.

Those rules change as teams learn. Keeping them as version-controlled prose and small references is often clearer than recompiling or redeploying a server for every editorial adjustment.

Portability Matters More Than Central Control

The open Agent Skills format is designed as a portable folder rather than a remote runtime. A well-scoped skill can move between compatible hosts with its instructions, examples, and supporting assets intact, although tool names and sandbox behavior still require host-specific testing.

This portability is useful for Hermes Agent and OpenClaw workflows that share a method but not necessarily the same deployment. Keep host-specific notes in short references instead of forking the core procedure at the first difference. The OpenClaw skills ecosystem guide covers which skills are worth installing and how to gate them safely per agent role.

An Existing CLI Already Provides the Capability

Do not build a server merely to wrap a reliable local command. If a single-user assistant can call a narrow CLI that already handles authentication, structured output, and errors, a skill may be the smaller and more maintainable solution.

The caveat is important: a CLI is not automatically safe because it is local. Avoid broad shell interpolation, prefer structured output, constrain writable targets, and do not treat a skill's suggested tool allowlist as a complete authorization system.

When You Need an MCP Server

Choose MCP when the problem is not merely remembering what to do. An MCP server becomes valuable when the agent needs a durable, typed, and governable bridge to a changing system.

The State Is Live and Authoritative

Customer tickets, deployment status, inventory, billing records, and production metrics can change between two model turns. Copying that state into a skill makes it stale by construction, while asking the model to scrape an interface produces an unstable contract.

An MCP resource or tool can retrieve the current record at execution time. The server can normalize upstream quirks and return a compact result instead of exposing an entire vendor response to the model.

Credentials or User Identity Are Involved

Credentials should not live in SKILL.md, examples, or bundled helper scripts. For remote HTTP deployments, the MCP authorization specification defines an OAuth-based model; for local stdio servers, credentials can be supplied through the process environment or another host-controlled mechanism. See the official MCP authorization guidance.

The deeper reason to use a server is not secret storage alone. A server can map identity to scopes, restrict tenants, redact fields, and record who requested a mutation, while a prose instruction can only ask the model to behave.

Writes Need Transactional Guarantees

Creating an invoice, changing a ticket status, or starting a deployment requires more than a plausible JSON object. The operation may need idempotency keys, optimistic concurrency, server-side validation, and a durable audit trail.

These properties belong below the model. The skill may define the approval policy, but the MCP server should reject an invalid transition and make a retried request safe.

Multiple Agents Need the Same Capability

A shared MCP server can present one contract to several agent hosts, languages, and model providers. That gives platform teams a central place to improve schemas, patch upstream API behavior, and apply access controls without copying integration logic into every skill.

Centralization is not free. The server becomes an operated dependency with versioning, observability, availability, and incident-response obligations, so it should earn its existence with a real boundary rather than architectural enthusiasm.

The Context-Cost Question

Context cost is frequently reduced to the slogan "skills are progressive, tools are always loaded." Real hosts are more nuanced, and the difference should be measured in serialized model input rather than assumed from the extension format.

The Agent Skills documentation describes roughly 100 tokens of discovery metadata per skill, recommends keeping activated instructions below 5,000 tokens, and allows references to load on demand. A simple planning estimate is:

C_skill = discovery metadata + activated instructions + selected references

MCP clients discover tool definitions from servers, but the protocol does not require every discovered schema to appear in every model call. Hosts may filter, defer, cache, or route tools, so the practical estimate is:

C_mcp = tool schemas exposed to this turn + tool results retained in context

The MCP tools specification also notes that stable tool ordering can improve prompt-cache behavior. Caching may reduce repeated processing cost, but it does not make an oversized catalog easier for a model to choose from.

An Illustrative Token Budget

Consider a hosted assistant with 20 installed skills. At the Agent Skills documentation's approximate discovery cost, the compact skill index is around 2,000 tokens; activating a focused triage skill might add another 1,200 tokens and one 600-token reference.

Now compare two MCP designs. A thin ticket server with four concise schemas might serialize to 500-800 tokens, while a broad enterprise server with 35 verbose tools could consume several thousand tokens before any result arrives.

Turn component	Focused design	Broad design
Skill discovery metadata	About 2,000 tokens	About 2,000 tokens
Activated skill and one reference	About 1,800 tokens	About 1,800 tokens
MCP tool catalog exposed to model	500-800 tokens	4,000+ tokens
First tool result	300-700 tokens	1,500+ tokens

These are illustrative planning numbers, not protocol guarantees or benchmarks. Measure the exact prompt generated by your host because schema verbosity, descriptions, routing, result retention, and tokenizer choice can move the total substantially.

The practical conclusion is not "skills are cheap" or "MCP is expensive." It is that progressive disclosure and capability selection are architecture features: keep skill metadata discriminative, activate only relevant instructions, expose the smallest useful tool set, and return projections rather than raw upstream payloads.

The Thin-Server Pattern: MCP Below, Skill Above

The most durable design often combines both mechanisms. Put a small capability boundary in MCP, then place the operating method in a skill that calls it.

Consider a support-incident workflow used from either Hermes Agent or OpenClaw. The agent must read a ticket, gather evidence, classify severity, draft an operator note, and change status only after the required approval.

What the MCP Server Owns

Keep the server interface narrow and literal:

MCP tool	Purpose	Server-side responsibility
`tickets_search`	Find candidate tickets	Tenant filtering, pagination, field projection
`tickets_get`	Read one ticket	Authorization, redaction, current version
`tickets_add_note`	Add an operator note	Input validation, idempotency, audit record
`tickets_change_status`	Apply a valid transition	Scope check, transition rules, concurrency check

The server should not contain a tool called triage_everything with a paragraph-long description and a dozen unrelated flags. Four bounded operations are easier to authorize, test, observe, and reuse.

What the Skill Owns

The skill owns the sequence and judgment. A compact SKILL.md could look like this:

---
name: incident-triage
description: Triage support incidents using ticket evidence and the severity rubric.
---

1. Read the ticket and its current version.
2. Collect at least two independent signals before assigning severity.
3. Separate observed facts from hypotheses in the note.
4. Ask for operator approval before any customer-visible note or status change.
5. Re-read the ticket before a write; stop if its version changed.
6. End with severity, evidence, uncertainty, and recommended next action.

That file is readable, reviewable, and easy to revise when the triage policy changes. A linked reference can hold the severity rubric, while the main instructions remain short enough to activate without dragging an operations handbook into every turn.

How It Runs in Hermes Agent

Hermes Agent's native MCP documentation describes startup discovery, persistent connections, stdio and Streamable HTTP transports, and namespaced MCP tools. Its current configuration also filters the environment for stdio servers and passes explicitly configured variables, which is a useful defense against accidental secret inheritance.

In this design, Hermes discovers the four ticket tools, while the incident skill activates only for relevant requests. The model follows the skill, the MCP server executes bounded operations, and the ticket service remains the final authority.

How It Runs in OpenClaw

OpenClaw's skills documentation follows the Agent Skills structure and builds a compact list of eligible skills for the model. The same incident folder can carry the core procedure, with a short host-specific reference explaining the available ticket tool names.

Do not put the ticket token in the shared skill. OpenClaw explicitly treats shared skills as inputs rather than secret storage, and third-party skills should be reviewed as untrusted code before they are enabled.

Why the Split Survives Change

If the support team revises its severity rubric, update the skill. If the ticket vendor changes authentication or pagination, update the MCP server without rewriting the operating policy.

If a second agent host arrives, it can reuse the same MCP contract and adapt the skill's small host-specific layer. This separation reduces duplicated integration logic without turning every procedural edit into a service deployment.

A Five-Step Decision Framework

The following sequence is more reliable than picking the fashionable extension type first.

1. Identify the Source of Truth

Write down every input and output the workflow touches. Static guidance, repository files, and user-provided documents lean toward a skill; mutable remote records and authoritative systems lean toward MCP.

Not all state justifies a server. A local build artifact is state, but an existing sandboxed CLI may already provide a sufficient boundary.

2. Locate the Trust Boundary

Mark where credentials, tenant identity, privileged data, or irreversible actions appear. If the agent crosses that line, introduce a deterministic enforcement point, typically an MCP server backed by service authorization.

Treat the model and skill as request planners, not policy engines. They may propose a permitted action, but they should not be able to redefine permission by changing their own instructions.

3. Separate Capability From Policy

Name capabilities as narrow verbs with typed inputs: get a ticket, add a note, or change status. Put the conditions for choosing those verbs, the evidence standard, and the preferred sequence in the skill.

Some policy must exist in both layers for different reasons. "Ask the user before deploying" belongs in the skill for interaction quality, while "reject deployment without an approval token" belongs in code for enforcement.

4. Estimate Context and Operating Cost

Capture a real prompt trace and count the skill metadata, activated instructions, tool definitions, and returned data. Then add the non-token cost of an MCP service: deployment, authentication, monitoring, versioning, and on-call ownership.

If a 30-tool catalog supports one workflow, expose a task-specific subset or split the server by coherent capability domain. If a skill repeatedly loads a 200-page reference, create a retrieval step or smaller references instead of congratulating yourself on progressive disclosure.

5. Test the Boundary, Then the Behavior

Test the MCP server as software and the skill as agent behavior. They fail differently, and a single happy-path chat transcript hides both classes of defect.

Layer	Test focus	Example assertion
Skill	Selection and procedure	Activates for incidents but not general support questions
Skill	Judgment	Cites two signals before assigning high severity
MCP server	Contract	Rejects missing fields and malformed identifiers
MCP server	Authorization	Denies cross-tenant reads and under-scoped writes
MCP server	Reliability	A retried note does not create a duplicate
Integrated trace	End-to-end behavior	Requests approval, detects version conflict, and stops safely

For tool safety, the MCP specification recommends input validation, access controls, rate limits, output sanitization, timeouts, confirmations for sensitive operations, and audit logging. Tool annotations are hints, not trusted proof that an operation is read-only or harmless. The A2A and MCP agent security guide covers the broader threat model including prompt injection and tool poisoning.

Security Rules That Do Not Fit in a Slogan

Skills reduce the need for some servers, but they do not remove risk. A skill can include scripts and can persuade an agent to call powerful host tools, so review its instructions and executable files as code, pin trusted versions, and limit the host tools available to the session.

MCP adds another boundary: a local subprocess or remote service with its own dependencies, inputs, outputs, and credentials. Apply least privilege, validate resource audience for remote authorization, use HTTPS, sanitize untrusted content, and keep approval visible for consequential writes.

Most importantly, do not confuse discoverability with authority. A tool appearing in the model's catalog does not mean the current user should be allowed to execute every operation it describes.

Common Anti-Patterns

Hiding a Remote API Client in a Skill

A shell script that reads a static bearer token and calls a production API may work in a demo. It also mixes procedure, credentials, network behavior, and authorization into a package designed to be copied and read by agent hosts.

Move the protected integration behind a narrow server or an existing approved CLI. Keep only the workflow and calling guidance in the skill.

Encoding the Workflow in Tool Descriptions

Tool descriptions should help the model select a capability and fill its schema. They are a poor substitute for a multi-step operating procedure with examples, exceptions, escalation rules, and output conventions.

Long descriptions inflate every turn in which the tool is exposed and make the service contract harder to reuse. Put the procedure in a skill and keep tool semantics precise.

Building an `execute_anything` Tool

A generic shell, SQL, or HTTP proxy collapses many permissions into one difficult-to-audit capability. It shifts validation to the model and makes least privilege mostly fictional.

Expose operations aligned to actual business actions. If expert operators truly need an escape hatch, separate it, restrict it, and require stronger approval and logging.

Publishing a Kitchen-Sink MCP Server

A server with dozens of unrelated tools burdens selection, schema context, permissions, and maintenance. Split by coherent domain or let the host expose a relevant subset for the current task.

Hermes Agent's FastMCP guidance makes a sensible starting recommendation: begin with one to three high-value endpoints and prefer a thin server with clear names and schemas. See the official FastMCP skill documentation.

Treating Tool Hints as Security Policy

An experimental allowed-tools field or a tool's read-only annotation can improve host behavior, but neither replaces sandboxing and server-side authorization. Metadata may be stale, misconfigured, or supplied by an untrusted component.

Use hints to improve the interface. Use code and infrastructure to enforce the boundary.

Using MCP for Static Knowledge

If a procedure or reference changes only with the repository, a remote round trip adds deployment and availability costs without making the information more authoritative. Package concise material with the skill and version it with the workflow.

Introduce a retrieval service only when the corpus is large, access-controlled, independently updated, or genuinely needs search. Architecture should follow the data lifecycle, not the acronym.

Final Decision: Skill, MCP Server, or Both?

Choose an Agent Skill when the hard part is knowing what to do. Choose an MCP server when the hard part is safely reaching something that changes, belongs to another trust domain, or must enforce a contract.

Choose both when a real workflow needs judgment above a protected capability. That is not duplication: the skill makes the agent useful, the server makes the integration governable, and the backing system makes the final decision authoritative.

For most hosted assistants, the best first architecture is modest: one focused skill, a small MCP surface only where live access demands it, and a captured prompt trace to verify the context cost. Add complexity after the boundary is clear, not before.

References

OpenCode CLI in Practice: Workflows, Automation, and Pitfalls

Rost — Tue, 01 Sep 2026 10:29:55 +0000

OpenCode's command-line interface is built for scripting, CI pipelines, and unattended agent runs. This article is a practical guide to using it in daily work.

Behind the CLI sits a system of models, tools, permissions, agents, skills, commands, sessions, MCP servers, and a client/server architecture. The agent can read and edit project files, search repositories, execute shell commands, call external tools, and delegate work to subagents. The same environment runs interactively in the TUI or non-interactively from scripts.

For small tasks, you can install it, connect a model, and start asking questions within minutes. For serious work, the quality of the experience depends heavily on model selection, repository instructions, permission boundaries, context management, and how aggressively you allow the agent to operate. This article focuses on that second stage, from the command line: which use cases pay off, which automation workflows hold up in daily use, and which problems appear after the novelty of an AI-powered terminal wears off. It is part of the AI Developer Tools section of this site.

The observations here were checked against OpenCode 1.18.9 and the current documentation in August 2026. OpenCode changes rapidly, so configuration examples deserve a quick documentation check before being copied into a long-lived team setup. If you have not installed OpenCode yet, the OpenCode quickstart covers installation, verification, and provider connection.

What OpenCode Actually Is: An Agent Environment, Not a Chat Box

OpenCode is an open-source AI coding agent designed around the terminal. A simplified view of what it wires together looks like this:

flowchart TD
    U[Developer] --> T[OpenCode CLI or TUI]

    T --> A[Primary Agent]
    A --> M[Selected LLM]
    A --> F[File Tools]
    A --> S[Shell]
    A --> W[Web Tools]
    A --> L[LSP and Code Intelligence]
    A --> X[MCP Tools]
    A --> C[Skills and Commands]
    A --> G[Subagents]

    F --> R[Repository]
    S --> R
    L --> R
    G --> M

    P[Permission Rules] --> A
    I[AGENTS.md] --> A

The important part is the permission layer between a model's intent and the actions OpenCode can perform. An excellent model with bad permissions can be dangerous. A weak model with perfect permissions is merely slow and annoying. Productive OpenCode use requires getting both sides reasonably right.

The same environment serves both surfaces: the interactive TUI and the non-interactive command line. That is what makes OpenCode scriptable in a way a pure chat interface is not. If you want a deliberately minimal take on the same terminal-agent idea — four default tools, no built-in sandbox, everything else via extensions — the Pi Coding Agent review is a useful contrast.

Setup: Install, Connect, and Why Provider Independence Matters

OpenCode installs in one line — official install script, npm, or Homebrew — and starts with opencode from a repository directory. The OpenCode quickstart covers the full install matrix (Arch, Windows, Docker), verification, and provider connection (/connect and /models), so this article does not repeat it.

You can use OpenCode's own model services or connect supported external providers. OpenCode currently builds much of its provider catalog using Models.dev and supports a wide range of commercial and local model configurations.

Provider independence is one of OpenCode's most useful architectural decisions. Your coding workflow does not need to be permanently coupled to one model vendor: you can use one model for difficult architecture work, another for cheap implementation tasks, and a local model for code that should not leave your environment.

That flexibility is real, but it creates another variable to manage. When OpenCode performs badly, the problem may be the harness, the prompt, the available context, the selected model, or the interaction between all four.

Initialize the Repository with AGENTS.md Before Asking for Code

One of the first commands worth running in a new repository is:

/init

OpenCode analyzes the project and creates an AGENTS.md file. Commit that file.

AGENTS.md is where repository-specific constraints can become durable context for every conversation — interactive or scripted — instead of being repeated by hand. A useful file is short enough to remain relevant but concrete enough to prevent predictable mistakes. For example:

# Repository Instructions

## Architecture

- API handlers live under src/api.
- Business logic belongs under src/services.
- Database access belongs under src/repositories.
- Do not call database clients directly from HTTP handlers.

## Validation

After TypeScript changes run:

```bash
npm run typecheck
npm test
```

After frontend changes also run:

```bash
npm run lint
```

## Constraints

- Do not modify generated files.
- Do not change public API contracts without asking first.
- Do not create database migrations unless explicitly requested.
- Never run deployment commands.

This is less exciting than installing another MCP server, but it usually provides more value. Coding agents fail surprisingly often because they do not know which constraints are important. A short repository contract removes some of that ambiguity before the first tool call.

Work from the Command Line with `opencode run`

The interactive interface receives most of the attention, but OpenCode's non-interactive mode changes the range of useful workflows considerably:

opencode run "Explain the error handling strategy in this package"

You can use OpenCode from shell scripts, CI jobs, Makefiles, task runners, or local automation without manually entering the TUI every time. For example, a diff review as a one-shot command:

opencode run \
  "Review the current git diff for correctness and missing tests. Do not edit files."

Or pipe context straight into a run:

git diff --name-only HEAD~1 |
  opencode run "Inspect the changed files and identify risky behavior changes."

In a Makefile, the same call becomes a target:

.PHONY: review
review:
    opencode run "Review the current git diff for correctness and missing tests. Do not edit files."

Because a scripted run has no human at the prompt, the permission policy you configure is the only guardrail standing between the model and your environment. That is why the permissions section below matters more for automation than for interactive use.

The interesting direction here is not replacing deterministic scripts with an LLM. It is inserting model reasoning into places where traditional shell logic becomes awkward, while keeping deterministic validation around it.

The Best OpenCode CLI Use Cases

OpenCode can attempt almost any programming task, but that does not mean every task should be delegated in the same way. The highest-value workflows tend to have three properties: the desired result is testable, the relevant repository context can be discovered, and incorrect changes are cheap to inspect or revert. Each prompt below works in the TUI or as an argument to opencode run.

1. Repository exploration

OpenCode is excellent at answering questions that would otherwise require a sequence of grep, editor searches, file jumps, and git log commands. For example:

Explain how authentication works in this repository.

Trace a request from the HTTP middleware through token validation,
user loading, authorization, and the final handler.

Do not modify anything.

A good agent will search for entry points, follow references, inspect tests, and return something closer to an architecture walkthrough than a plain text search. This is one of the safest ways to introduce OpenCode into an existing codebase because the agent can provide value without writing code.

2. Small, well-bounded fixes

A narrowly scoped bug is close to the ideal coding-agent task. For example:

The CLI exits with status 0 when config validation fails.

Find the code path responsible, add a regression test, implement
the smallest fix, and run the relevant tests.

Do not refactor unrelated code.

The key phrase is not "fix the bug." It is the constraints surrounding the task. OpenCode works better when success can be demonstrated with a test, a compiler, or an observable command. Vague requirements give the model room to create plausible code rather than demonstrably correct code.

3. Test generation after implementation

Tests are useful agent work because the existing implementation gives OpenCode something concrete to reason about. A productive prompt might be:

Review src/parser.ts and its existing tests.

Identify important edge cases that are currently uncovered.
Add tests only. Do not modify the implementation.

Run the parser test suite when finished.

Separating test generation from implementation is important. If the same agent writes both the feature and the tests in one unconstrained pass, it can accidentally create tests that validate its own interpretation rather than the intended behavior.

4. Mechanical refactoring

OpenCode is very good at repetitive transformations where the desired end state is clear. Examples include:

replacing a deprecated API across a repository;
converting repeated code to a shared helper;
renaming a configuration field;
migrating tests from one assertion pattern to another;
updating imports after moving a package;
replacing an obsolete logging abstraction.

The repository's compiler and tests become the agent's feedback loop. A useful pattern is:

Replace uses of LegacyResult<T> with Result<T, AppError> in
packages/api only.

Preserve runtime behavior.

Work in small batches. After each batch run the package typecheck.
At the end run the package test suite and show me the final git diff
summary.

This is often more reliable than asking for the entire migration in one enormous step.

5. Code review

OpenCode becomes much more useful when review is treated as a separate agent role rather than another prompt sent to the same editing context. You can create a review-oriented agent that cannot edit files. Then give it instructions such as:

Review the current git diff.

Focus on:
- correctness;
- security;
- concurrency;
- missing tests;
- error handling;
- accidental API changes.

Do not summarize files that are unchanged.
Rank findings by severity.

A read-only reviewer is useful even if another coding agent produced the changes. The separation is valuable because implementation and criticism are different tasks: an agent that has just spent several thousand tokens defending one approach is often less skeptical of that approach than a fresh reviewer.

Use Plan and Build Modes as Different Mental Modes

OpenCode provides primary agents and subagents, with built-in workflows including planning and implementation-oriented behavior. Even when the exact agent configuration changes over time, the conceptual separation remains useful.

Planning should answer questions such as:

Which files matter?
What existing patterns should be followed?
What could break?
How will we verify the change?
Is the requested change actually local?

Implementation should happen only after those questions have reasonable answers. For substantial tasks, I prefer a prompt sequence like this:

First investigate the request.

Do not edit files yet.

Return:
1. the relevant files;
2. the current behavior;
3. the proposed change;
4. risks;
5. the exact verification commands.

The same sequence works as a one-shot command:

opencode run "First investigate the request. Do not edit files yet. Return: the relevant files; the current behavior; the proposed change; risks; the exact verification commands."

Then inspect the plan before allowing edits. This feels slower than immediately telling an agent to "implement it," but the expensive failures in agentic coding usually come from incorrect assumptions made before the first edit.

This is a lightweight, per-task version of the same instinct behind spec-driven development: agree on the plan before the agent starts editing. For work that spans multiple files, sessions, or contributors, a written spec earns its overhead in a way a single investigate-first prompt cannot; the GitHub Spec Kit vs Kiro vs Claude Code comparison covers structured SDD workflows that go further than a single prompt.

Subagents Are Useful, but Delegation Is Not Free

OpenCode can invoke specialized subagents automatically or through explicit mentions. For example:

@general find where retry behavior is implemented

You can also define dedicated subagents for jobs such as security review, dependency analysis, frontend testing, or documentation. The same pattern works in other harnesses; the Claude Code subagents guide covers the analogous design on the Anthropic side.

This is powerful because subagent work can remain outside the primary conversation's immediate reasoning path. A primary agent can delegate repository exploration and consume the result instead of filling its own context with every intermediate search. But subagents create three less obvious costs:

They consume tokens. A tree of agents that each re-read the repository can become surprisingly expensive — my Oh My Opencode experience report documents what happens when that overhead is taken to the extreme.
They create policy complexity. Permissions must be considered for the delegated agent, not only for the parent.
Delegation can hide reasoning. When the primary agent says "the subagent found X," you may need to inspect the child session to understand how reliable that conclusion actually is.

For most coding tasks, two or three purposeful agents are more useful than an elaborate fictional software company living inside your terminal.

If the task genuinely needs a full orchestrator delegating to a fixed roster of specialists — parallel background execution, planning and research phases, model routing per role — that is a different product built on top of these primitives, not a bigger prompt. The Oh My Opencode quickstart covers that harness.

Use Custom Agents as Permission Boundaries

OpenCode agents can have separate prompts, models, and permissions. That makes agents useful as security and workflow boundaries, not merely different personalities. For example, a project-local review agent can be defined as a Markdown file:

---
description: "Reviews code without modifying the repository"
mode: subagent
permission:
  edit: deny
  bash:
    "*": ask
    "git diff *": allow
    "git status *": allow
    "git log *": allow
  webfetch: deny
---

Review code for correctness, security, maintainability,
unexpected behavior changes, and missing tests.

Do not modify files.

This is much better than writing "please do not edit anything" in prose. Instructions influence the model. Permissions constrain the tool. Those are not equivalent controls.

Configure OpenCode Permissions on Day One

One of OpenCode's most important practical characteristics is that its normal defaults are permissive. That is convenient during a demo and not necessarily what I want on a workstation containing SSH keys, production credentials, package publishing tokens, Kubernetes contexts, and access to several cloud accounts. For unattended opencode run jobs, the stakes are higher still: nothing stops a run except the policy you configure.

OpenCode's current permission model supports:

allow
ask
deny

Rules can be applied to file access, edits, shell commands, web operations, subagents, skills, external directories, and other tool categories. A conservative starting point might look like this:

{
  "$schema": "https://opencode.ai/config.json",
  "permission": {
    "*": "ask",
    "read": "allow",
    "grep": "allow",
    "glob": "allow",
    "edit": "ask",
    "bash": {
      "*": "ask",
      "git status *": "allow",
      "git diff *": "allow",
      "git log *": "allow",
      "git push *": "deny",
      "rm *": "deny"
    }
  }
}

The exact rules should match your environment. What matters is making the policy intentional rather than discovering the default after the agent has already executed something surprising.

Do not confuse approval prompts with sandboxing

Permission rules are useful, but they are not the same as operating-system isolation. If OpenCode has permission to execute an allowed shell command, that process runs with the access available to your environment. A coding agent can potentially interact with files, network services, environment variables, credentials, sockets, package managers, and other developer tooling.

For sensitive repositories or unattended runs, stronger isolation is worth considering:

flowchart TD
    H[Host Workstation] --> B[Restricted Container or VM]
    B --> O[OpenCode]
    O --> R[Repository Copy]
    O --> T[Build and Test Tools]
    O --> K[Limited Model Credentials]

    P[No Production Credentials] --> B
    N[Restricted Network Access] --> B

Git is rollback. Permissions are policy. A container or VM is isolation. Those are three different layers.

Turn Repetitive Prompts into Custom Commands

If you repeatedly type the same instructions, they should probably stop being chat history and become configuration. OpenCode supports project and global custom slash commands for the interactive interface. For example, create:

.opencode/commands/review.md

with:

---
description: Review the current changes
agent: plan
---

Review the current git diff.

Look for:
- bugs;
- security issues;
- incomplete error handling;
- missing tests;
- accidental API changes.

Do not modify files.

Then use:

/review

This is a small feature with disproportionate value. Reliable coding-agent workflows emerge when good prompts become shared project infrastructure rather than personal clipboard snippets. Other useful project commands might include:

/test-changes
/review
/prepare-pr
/check-migration
/update-docs
/release-check

Use Skills for Reusable Workflows

OpenCode also supports SKILL.md files. Skills are useful when a workflow needs more than a single prompt. A skill can contain detailed operational guidance and supporting files while remaining unloaded until the agent actually needs it. Useful candidates include:

database migration procedures;
release workflows;
incident investigation;
API compatibility reviews;
package publishing;
infrastructure validation;
internal architecture conventions.

For example:

.opencode/skills/database-migration/SKILL.md

A skill description might tell OpenCode when the procedure applies, while the body explains how to inspect schemas, create migrations, validate rollback behavior, and run integration tests. If you already use the equivalent concept in another harness, the Claude Skills guide for developers maps the same design decisions.

The advantage is context discipline. Dumping every organizational rule into AGENTS.md eventually creates a giant system prompt that is expensive and increasingly easy for the model to ignore. Skills let specialized instructions enter context only when needed.

MCP: Useful Until the Tool Catalog Becomes the Problem

OpenCode supports local and remote MCP servers. That can expose issue trackers, documentation systems, browsers, observability platforms, databases, APIs, and other tools to the coding agent. A typical configuration might provide a documentation server:

{
  "$schema": "https://opencode.ai/config.json",
  "mcp": {
    "context7": {
      "type": "remote",
      "url": "https://mcp.context7.com/mcp"
    }
  }
}

MCP is one of those features that is easy to overuse. Every tool the agent must understand consumes attention and frequently context tokens. A setup with fifteen MCP servers may look powerful in a configuration screenshot while making actual model behavior slower, more expensive, and less predictable.

My rule is simple: if a tool is not useful in a normal week of work, it probably should not be enabled globally. Load tools because a workflow requires them, not because the integration exists.

Local Models in OpenCode: A Real Option, Not a Magic Fix

One of OpenCode's strongest use cases is its ability to work with local or self-hosted OpenAI-compatible model endpoints. This is attractive when:

source code must remain local;
API costs are significant;
you already operate GPU infrastructure;
you want to experiment with open models;
internet connectivity is unreliable;
model routing is part of your platform.

A custom provider can point OpenCode to a local endpoint such as LM Studio, llama.cpp via llama-server, vLLM, Ollama-compatible infrastructure, or another OpenAI-compatible server.

This is where expectations matter. A good coding harness cannot compensate completely for a model that is weak at tool use, long-horizon planning, code reasoning, or instruction retention. Community discussions around local OpenCode setups repeatedly converge on the same observation: local models can be excellent for bounded tasks, but the smaller ones often need more supervision. For measured numbers on how specific models actually behave inside OpenCode, see my hands-on LLM comparison for OpenCode.

The practical solution is model routing by task complexity. Use a cheaper or local model for:

repository search;
documentation;
simple tests;
repetitive edits;
formatting changes;
straightforward bug fixes.

Use a stronger model for:

architectural changes;
ambiguous bugs;
cross-package refactors;
concurrency;
security-sensitive code;
difficult migrations.

Provider independence makes this strategy possible. It does not make model quality irrelevant.

A Practical Daily Workflow with OpenCode

My preferred workflow is deliberately conservative. It works the same in the TUI or via opencode run; where a step is a prompt, the command-line variant is shown alongside.

Step 1: Start from a clean Git state

git status

Either commit existing changes or deliberately record what is already modified. An AI coding agent operating inside a dirty working tree makes review much harder because human changes and agent changes become mixed together.

Step 2: Ask for investigation first

Investigate issue #482.

Do not modify files.

Explain:
- the current behavior;
- likely root cause;
- relevant files;
- existing tests;
- proposed fix;
- verification commands.

The same prompt as a one-shot command:

opencode run "Investigate issue #482. Do not modify files. Explain the current behavior, likely root cause, relevant files, existing tests, proposed fix, and verification commands."

If the investigation is wrong, correcting it is cheap.

Step 3: Narrow the implementation

Implement the proposed fix only.

Do not refactor unrelated code.
Add the regression test first.
Run the smallest relevant test suite after the change.

Or non-interactively:

opencode run "Implement the proposed fix only. Do not refactor unrelated code. Add the regression test first. Run the smallest relevant test suite after the change."

The agent now has a smaller decision space.

Step 4: Inspect the diff yourself

git diff --stat
git diff

Do not outsource this step to another model entirely. You are checking not only whether the code looks plausible, but whether the agent changed files it did not need to touch.

Step 5: Run deterministic verification

npm run typecheck
npm test
npm run lint

Use the actual project commands. "OpenCode says the tests pass" is not stronger evidence than your terminal displaying a successful test process.

Step 6: Run an independent review

Ask a read-only agent:

Review the uncommitted diff as if it were a pull request written
by another engineer.

Try to find reasons this implementation is wrong.

Do not edit files.

The command-line variant:

opencode run "Review the uncommitted diff as if it were a pull request written by another engineer. Try to find reasons this implementation is wrong. Do not edit files."

The phrase "try to find reasons this is wrong" is intentional. Models are very good at politely confirming plausible work. Review prompts should encourage falsification rather than applause.

Step 7: Commit only after the diff makes sense

git add -p
git commit

I still prefer interactive staging after agent-generated changes. It forces one final human pass across every hunk that becomes part of the repository history.

Where OpenCode Becomes Frustrating: Common Pitfalls

The interesting limitations are not usually "the AI made a syntax error." Compilers are good at catching those. The difficult problems come from autonomy, context, hidden assumptions, and configuration.

Challenge 1: Model quality dominates the experience

The same OpenCode workflow can feel excellent with one model and almost unusable with another. This makes product reviews difficult because users often attribute model behavior to the agent harness.

If OpenCode repeatedly:

ignores instructions;
rewrites too much code;
calls tools incorrectly;
loops on the same failed action;
loses track of constraints;
invents APIs;

try another model before redesigning the entire OpenCode configuration. The terminal did not suddenly become smarter. The model did.

Challenge 2: Long sessions accumulate bad context

Coding-agent sessions become less trustworthy as incorrect assumptions accumulate. An early mistake such as "this service is stateless" can influence dozens of later decisions even after the relevant files have changed.

OpenCode supports compaction to manage context limits, but compression creates its own tradeoff. A summary necessarily decides which details survive. For major task transitions, starting a fresh session is often cleaner than continuing a heroic 80,000-token conversation. With scripted runs the answer is even simpler: a new opencode run starts with a clean context every time, so long-lived state belongs in files like AGENTS.md, not in a conversation.

Context is not free memory. It is working state, and working state gets stale.

Challenge 3: Permissions can become deceptively complicated

Simple rules are easy:

read -> allow
edit -> ask
git push -> deny

Complex agent hierarchies are harder. Once primary agents can invoke subagents, custom tools, MCP servers, and shell commands, you need to reason about the effective capabilities of the whole workflow rather than one configuration line.

There have also been real community and GitHub discussions about subagent permission behavior and permission inheritance. The lesson is broader than any one bug: test security assumptions with an actual disposable repository — for example, git init /tmp/opencode-perm-test and try to make the agent run a command you denied there. If an action absolutely must not occur, do not rely on prose instructions alone.

Challenge 4: OpenCode changes quickly

OpenCode has shipped at a remarkable pace. That is good for features and less pleasant for documentation longevity. A particularly easy trap in 2026 is finding configuration examples from a different OpenCode generation. Current documentation also contains separate V2 material whose configuration schema differs from the established 1.x syntax. For example, permission concepts may appear under different field names in V2 documentation.

Do not casually combine examples from:

/docs/

and:

/v2/docs/

without checking which runtime you are actually using. When troubleshooting a configuration copied from a blog post, check its publication date before assuming OpenCode is broken.

Challenge 5: The TUI can hide scale

A conversational interface makes a ten-file change feel smaller than a ten-file change. The agent may report:

Implemented the new validation and updated the relevant tests.

That sentence could represent three lines or 600 lines. Keep independent shell tools close:

git status --short
git diff --stat
git diff --name-only
git diff

The coding agent should not be the only interface through which you observe the coding agent.

Challenge 6: MCP can destroy context efficiency

More integrations do not automatically produce a better coding agent. Large MCP servers can expose many tool schemas, each consuming model context and increasing tool-selection complexity. If OpenCode feels oddly indecisive after you installed half the MCP ecosystem, disable most of it and compare. A smaller tool surface often produces better agent behavior.

Challenge 7: Local privacy requires verifying the whole path

Running a local model does not automatically guarantee that every part of a toolchain is local. Community discussions in early 2026 raised privacy questions around OpenCode's auxiliary model behavior and web interface architecture. Some of those claims referred to older versions, some were disputed, and some code paths have since changed.

The durable lesson is not "OpenCode sends everything somewhere." The durable lesson is: if local-only operation is a hard requirement, verify it. Use network inspection, read the current configuration, understand which interface you are using, disable unnecessary remote integrations, and test the exact version you intend to deploy. Security requirements should be validated, not inferred from a product category.

OpenCode vs Claude Code and Codex CLI

OpenCode's strongest differentiator is not necessarily coding quality. The underlying model still contributes heavily to coding quality. Its differentiator is control over the harness.

OpenCode is compelling when you value:

an open-source implementation;
provider flexibility;
terminal-first workflows;
configurable agents;
granular permissions;
local-model support;
MCP;
reusable commands and skills;
client/server architecture.

Claude Code is compelling when you want a tightly integrated Anthropic experience with strong first-party model behavior and increasingly polished built-in agent workflows. Codex CLI is compelling when your preferred models and workflow are already centered on OpenAI's coding stack.

I would not choose among them by counting features. Choose based on which layer you want to own. If you want a vendor to make most workflow decisions, a first-party coding agent can be attractive. If you want the harness to remain replaceable while you experiment with providers and agent configurations, OpenCode makes a stronger argument.

What Reddit and Hacker News Get Right About OpenCode

Community discussions around OpenCode are unusually polarized. Some developers describe it as their favorite coding harness, especially when combined with Codex models, local models, or custom agent setups. Others focus on resource usage, permission behavior, privacy questions, provider authentication changes, or the complexity that appears once a seemingly simple terminal application becomes infrastructure. Both perspectives are reasonable.

OpenCode is not difficult because the command line is difficult. It is difficult because an autonomous coding environment exposes questions developers previously did not need to answer explicitly:

Which model should make this decision?
Which files may it read?
Which commands may it execute?
Which credentials can the process see?
Which tasks should become subagents?
How much context should survive?
Which tools deserve permanent context?
Which result must a human verify?

A polished closed product can make many of those decisions for you. OpenCode makes more of them yours. That is precisely why advanced users like it.

A Starting Configuration I Would Recommend

I would resist building an elaborate setup immediately. Start with:

one strong default model;
one AGENTS.md;
conservative shell and edit permissions;
a read-only review agent;
two or three custom commands;
no MCP servers until a real need appears.

A simple environment is easier to debug. Once a workflow becomes repetitive, promote it into a command, skill, or agent. Once a capability becomes risky, restrict it with permissions. Once context becomes bloated, split the workflow instead of merely buying a larger context window. This incremental approach is less impressive in screenshots and considerably more pleasant to maintain.

Who Should Use OpenCode

OpenCode is a particularly good fit for developers who already live in terminals and want their coding agent to behave like another programmable development tool. I would strongly consider it if you:

work across multiple LLM providers;
want local-model support;
dislike being locked to one AI vendor;
need project-specific agents;
automate development tasks from shell scripts;
want to inspect or modify the coding harness;
already understand Git and normal CLI tooling.

It is less compelling if you primarily want an invisible AI layer inside an IDE. OpenCode also expects more operational judgment than a traditional autocomplete tool. If reviewing diffs, understanding shell commands, and managing Git branches already feel uncomfortable, giving an autonomous process access to those tools will not make the underlying complexity disappear.

Final Verdict

OpenCode is one of the more convincing examples of an AI coding tool becoming developer infrastructure rather than merely a chat interface. Its best features are not flashy. Provider independence, explicit permissions, reusable agents, non-interactive execution, repository instructions, commands, skills, and composable tools make it possible to shape the agent around a real engineering workflow.

The weakness is the mirror image of that strength. OpenCode gives you enough control to create a disciplined coding environment, but it also gives you enough control to create a complicated, expensive, poorly isolated swarm of agents with twenty MCP servers and no clear verification boundary.

I would not optimize OpenCode for maximum autonomy. I would optimize it for short feedback loops. Give it a bounded problem, enough context to understand the problem, permission to perform only the necessary actions, and deterministic commands that can prove whether the result works. That is less magical than asking an agent to build an entire application while you sleep. It is also much closer to how OpenCode becomes genuinely useful.

References

OpenCode documentation: https://opencode.ai/docs/
OpenCode changelog: https://opencode.ai/changelog
OpenCode GitHub repository: https://github.com/anomalyco/opencode
Models.dev provider catalog: https://models.dev

Superpowers Quickstart: Install, Workflow, and Tryout

Rost — Mon, 24 Aug 2026 11:32:26 +0000

Superpowers packages a full spec-driven methodology into installable Claude Skills, enforcing brainstorming, planning, subagent-driven implementation, and strict TDD instead of leaving that structure up to you.

Most Claude Code SDD setups fail the same way: someone writes a solid specify-plan-implement skill once, uses it diligently for a week, then quietly drifts back to unstructured prompting the first time a deadline gets close. Nothing in a hand-rolled skill actually stops that. Superpowers is a direct answer to that discipline problem -- a skills package built by Jesse Vincent and the team at Prime Radiant that treats brainstorming, planning, subagent review, and red-green-refactor TDD as mandatory steps the agent checks before every task, not optional suggestions it can skip under pressure.

What makes it worth a dedicated look rather than a footnote is the install story. Superpowers ships plugin manifests for Claude Code, Cursor, Codex, Gemini CLI, GitHub Copilot CLI, Grok Build CLI, Kimi Code, OpenCode, Pi, Devin CLI, Factory Droid, and Hermes Agent, so the same methodology follows you across harnesses instead of living in one repo's .claude/skills/ folder. This guide covers what it actually is, how to install and verify it on the agent you use, what a first end-to-end run looks like, and where it fits next to other spec-driven development tooling.

What Is Superpowers?

Superpowers describes itself as "a complete software development methodology for your coding agents, built on top of a set of composable skills and some initial instructions that make sure your agent uses them." That last clause is the part that matters. Plenty of skill collections exist; the differentiator here is the bootstrap instruction that forces the agent to check for relevant skills before starting any task, so the workflow activates automatically instead of requiring you to remember a slash command.

The philosophy behind it is stated plainly in the project's own docs:

Test-Driven Development -- write tests first, always
Systematic over ad-hoc -- process over guessing
Complexity reduction -- simplicity as the primary goal
Evidence over claims -- verify before declaring success

In practice, that philosophy shows up as a sequence of skills that hand off to each other. If you have read Claude Skills and SKILL.md for Developers, the mechanics will feel familiar -- Superpowers is a large, opinionated set of exactly that kind of SKILL.md directory, distributed as an installable plugin rather than something you author yourself.

Installing Superpowers Across Coding Agents

Installation is harness-specific, and the project is explicit that you install it separately for each agent you use -- there is no single global install.

Claude Code

Superpowers is distributed through two marketplaces for Claude Code:

# Official Anthropic plugin marketplace
/plugin install superpowers@claude-plugins-official

Or through the dedicated Superpowers marketplace, which also carries a few related plugins:

/plugin marketplace add obra/superpowers-marketplace
/plugin install superpowers@superpowers-marketplace

Cursor

/add-plugin superpowers

You can also search "superpowers" directly in Cursor's plugin marketplace UI.

Other agents

Agent	Install command
Codex App	Plugins sidebar -> Coding section -> install Superpowers
Codex CLI	`/plugins`, search `superpowers`, select Install Plugin
Antigravity	`agy plugin install https://github.com/obra/superpowers`
Devin CLI	`devin plugins install obra/superpowers`
Factory Droid	`droid plugin marketplace add https://github.com/obra/superpowers` then `droid plugin install superpowers@superpowers`
Gemini CLI	`gemini extensions install https://github.com/obra/superpowers`
GitHub Copilot CLI	`copilot plugin marketplace add obra/superpowers-marketplace` then `copilot plugin install superpowers@superpowers-marketplace`
Grok Build CLI	`grok plugin install superpowers@xai-official --trust`
Kimi Code	`/plugins install https://github.com/obra/superpowers`
OpenCode	Fetch and follow `.opencode/INSTALL.md` from the repo (separate install even if already used elsewhere)
Pi	`pi install git:github.com/obra/superpowers`
Hermes Agent	`hermes plugins install obra/superpowers --enable`

A few install details are worth flagging before you run a real task:

Antigravity runs the plugin's session-start hook automatically, so Superpowers is active from the first message; reinstall with the same command to update.
Pi loads the skills through a small extension that injects the using-superpowers bootstrap at session startup and after context compaction, and does not require Pi's compatibility Skill tool since Pi has native skill support.
Hermes Agent has no post-compaction hook -- a very long session that compacts over its first turn can lose the bootstrap, so start a fresh session if skills stop triggering.
OpenCode treats this as a completely separate install path from any other harness you already use, even on the same machine.

Verify the install

Once installed, ask the agent directly rather than trusting a silent success message:

What skills are available?

If Superpowers installed correctly, the agent should list skills such as brainstorming, writing-plans, test-driven-development, and subagent-driven-development among the results. This is the same discovery check recommended for debugging Claude Skills that fail to trigger -- if the skill exists but never activates, the routing description is almost always the problem, not the model.

The Superpowers Workflow: Brainstorm to Shipped Code

Superpowers documents seven skills as "the basic workflow," each one activating at a specific point and handing off to the next:

flowchart LR
  A[brainstorming] --> B[using-git-worktrees]
  B --> C[writing-plans]
  C --> D{subagent-driven-development<br/>or executing-plans}
  D --> E[test-driven-development]
  E --> F[requesting-code-review]
  F --> G[finishing-a-development-branch]

brainstorming activates before any code gets written. It refines a rough idea through questions, explores alternatives, and presents the design in short, reviewable chunks rather than one long document -- then saves the result as a design artifact.
using-git-worktrees activates once you approve the design. It creates an isolated workspace on a new branch, runs project setup, and verifies a clean test baseline before implementation starts.
writing-plans breaks the approved design into bite-sized tasks -- Superpowers targets two to five minutes of work per task -- with exact file paths, complete code where relevant, and explicit verification steps.
subagent-driven-development (fast iteration) or executing-plans (batch execution with human checkpoints) dispatches a fresh subagent per task, with a two-stage review: spec compliance first, then code quality.
test-driven-development enforces strict red-green-refactor -- write a failing test, watch it fail, write the minimal code to pass, watch it pass, commit. Code written before a failing test existed for it gets deleted, not kept.
requesting-code-review activates between tasks, reviewing the diff against the plan and reporting issues by severity; critical issues block progress to the next task.
finishing-a-development-branch activates once all tasks complete. It verifies tests pass, then presents options -- merge, open a PR, keep the branch, or discard -- and cleans up the worktree.

This maps closely onto the tool-neutral five-phase loop in Spec-Driven Development Workflow From Requirements to Code: specify becomes brainstorming, plan becomes writing-plans, tasks and implement fold into subagent-driven-development, and validate is split across TDD and code review. The difference is enforcement -- Superpowers is built so the agent checks for these skills automatically rather than relying on you to invoke each phase.

Trying It Out: A First Task Walkthrough

Do not point Superpowers at your most important feature on the first run. Pick something real but bounded -- a small endpoint, a focused bug fix, a single module refactor -- and start a session in a project that already has a test suite, since TDD enforcement is most visible there.

I want to add rate limiting to our public API endpoints.

With Superpowers installed, expect the session to open with brainstorming questions rather than code: which endpoints, what limit, per-user or per-key, what should happen when the limit is hit. That friction is intentional -- it is the same non-goals discipline covered in Spec-Driven Development Workflow From Requirements to Code, just enforced by the skill rather than left to you to remember. Once you approve the design in chunks, expect a plan with numbered, small tasks before any file changes, then one subagent dispatched per task with a diff you review at each checkpoint.

Two things to watch on a first run:

Confirm TDD is actually enforced. Ask the agent to implement one task and check whether a failing test appears before the implementation. If code lands first, the skill did not trigger for that step, and it is worth re-running the discovery check from the install section.
Watch task granularity. The two-to-five-minute task sizing is aggressive for some codebases. If tasks feel needlessly fragmented for your project, that is worth raising during the writing-plans review checkpoint rather than after the fact.

What's Inside: The Skills Library

Beyond the seven workflow skills, Superpowers ships supporting skills grouped by area:

Category	Skills
Testing	`test-driven-development` (includes testing anti-patterns reference)
Debugging	`systematic-debugging` (4-phase root cause process, includes root-cause-tracing, defense-in-depth, condition-based-waiting), `verification-before-completion`
Collaboration	`brainstorming`, `writing-plans`, `executing-plans`, `dispatching-parallel-agents`, `requesting-code-review`, `receiving-code-review`, `using-git-worktrees`, `finishing-a-development-branch`, `subagent-driven-development`
Meta	`writing-skills` (create new skills with testing methodology), `using-superpowers` (bootstrap and introduction)

dispatching-parallel-agents is worth calling out separately: it covers concurrent subagent workflows, which is the same territory as Claude Code subagents -- isolated context, independent review, and a summary reported back to the main session instead of flooding it with intermediate output.

When Superpowers Fits (and When It Doesn't)

Good fit:

You already like the Claude Code skills approach to Spec-Driven Development vs Vibe Coding but keep sliding back to unstructured prompting because nothing enforces the review gates.
You work across multiple coding agents and want one methodology that travels with you instead of a Claude Code-only skill.
You want strict TDD enforcement without writing the enforcement logic yourself.
Multi-session features where architectural drift and forgotten constraints are a real risk -- the exact failure mode described in Spec-Driven Development vs Vibe Coding.

Weaker fit:

Quick scripts, throwaway prototypes, and one-off automation -- the brainstorming and planning overhead is real, and it is not worth paying on a fifty-line change.
Teams with an existing, well-tuned project-specific SDD skill. Superpowers trades that customization for enforced, opinionated ceremony; you may prefer to keep what already works and borrow only individual skill ideas.
Environments where the strict two-to-five-minute task granularity or mandatory TDD would conflict with an established team process you are not ready to change.

How Superpowers Differs from Other SDD Tools

Superpowers is not a competitor to GitHub Spec Kit or AWS Kiro in the usual sense -- it operates at the Claude Code / cross-agent skills layer rather than as a standalone CLI or IDE. Spec Kit gives you portable markdown artifacts and agent independence; Kiro gives you a guided, integrated IDE experience; a hand-rolled Claude Code skill gives you maximum flexibility with zero enforcement. Superpowers sits between the last two: same skills-based mechanism as a DIY Claude Code workflow, but with mandatory-feeling enforcement and cross-agent plugin distribution that a repo-local skill does not provide.

For the full breakdown of portability, setup friction, review gates, and lock-in across Spec Kit, Kiro, Claude Code skills, OpenSpec, BMAD-METHOD, and Tessl -- including where Superpowers lands in the comparison table and decision framework -- see GitHub Spec Kit vs Kiro vs Claude Code SDD Workflows.

Troubleshooting and Things to Know

Updates are mostly automatic but agent-dependent. How current your install stays depends on the harness -- Antigravity re-runs the session-start hook on reinstall, npm- or Git-based installs need an explicit update command, and marketplace-based installs typically follow the host agent's own plugin update flow.

Telemetry is on by default but easy to disable. The brainstorming skill's optional visual companion loads the Prime Radiant logo from their website, which includes your Superpowers version but no project, prompt, or click data. Set SUPERPOWERS_DISABLE_TELEMETRY to any true value to turn it off. Superpowers also honors Claude Code's own DISABLE_TELEMETRY and CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC opt-outs.

Contributions require the dev branch. If you want to modify or add skills, fork the repository, switch to dev, and follow the writing-skills skill for creating and testing changes -- new skill contributions specifically are generally not accepted, since every skill must work identically across all supported agents.

If a skill does not trigger, treat it like any other Claude Skill problem. Ask "What skills are available?" to confirm discovery, check that you installed for the specific agent you are currently using (not a different one on the same machine), and remember that Hermes sessions need a fresh start after long-session compaction if the bootstrap silently drops.

Conclusion

Superpowers is worth installing when the gap between your SDD intentions and what actually happens in a session is the problem you are trying to solve. It does not introduce a new specify-plan-implement idea -- that loop is already well covered by What Is Spec-Driven Development? -- it makes an existing idea harder to skip, and it does that across more agents than a repo-local skill can reach. Install it on the one harness you use daily first, run a small bounded task through the full brainstorm-to-finish cycle, and decide from there whether the enforced ceremony earns its keep on your actual workload.

Useful Links

Superpowers repository -- source, skills, and plugin manifests for every supported agent
Superpowers marketplace -- Claude Code marketplace for Superpowers and related plugins
GitHub Spec Kit vs Kiro vs Claude Code SDD Workflows -- full tool comparison and decision framework
Claude Skills and SKILL.md for Developers -- the underlying skill mechanics Superpowers builds on
Claude Code Subagents: Setup, Config, and When to Use Them -- isolated-context review, the mechanism behind subagent-driven-development
Spec-Driven Development Workflow From Requirements to Code -- the tool-neutral five-phase process Superpowers automates
What Is Spec-Driven Development? The Spec as Source of Truth -- core SDD concepts and terminology
Claude Code install and config for Ollama, llama.cpp, pricing -- install Claude Code itself before adding Superpowers

Pi Coding Agent Review: Minimal, Hackable AI Coding CLI

Rost — Sun, 23 Aug 2026 10:56:24 +0000

Pi Coding Agent is a minimal, open-source terminal coding harness that ships with four default tools and leaves most of the behaviour to extensions, skills and your own workflow.

Most AI coding agents are becoming bigger products. They add planning modes, subagents, permission layers, IDE integrations, background workers and increasingly elaborate orchestration around the model. Pi takes almost the opposite approach: the default agent starts with four basic tools — read, write, edit and bash — while most of the interesting behaviour is deliberately left to extensions, skills, packages and your own workflow.

This review is aligned with Pi v0.84.2, released on 14 August 2026. Mario Zechner created the project; it moved to Earendil Works in May 2026, which is why current packages use the @earendil-works scope rather than the older @mariozechner names. Pi has been changing quickly, so configuration and extension APIs deserve a version check before you standardise them across a team.

The catch is equally important: Pi gives you considerably more control than many coding agents, and correspondingly more responsibility. It does not provide a built-in security sandbox or the kind of permission system developers may expect after using Claude Code or similar tools. Below I cover the quickstart, architecture, model support, extensions, session model, security implications and where I think Pi fits among modern AI coding tools.

What is Pi Coding Agent?

Pi is best understood as an agent harness rather than a finished, opinionated coding environment. That distinction matters: a coding model answers prompts, while an agent harness decides what context that model sees, which tools it can call, how tool results return to it, how sessions persist and what happens between turns. Pi makes almost all of those layers accessible, which is why it sits at the kit-building end of the AI developer tools field rather than among the batteries-included coding products.

Area	My rating	Why
Architecture	9/10	Small core with unusually clear extension points
Model flexibility	9/10	Broad provider support and easy model switching
Extensibility	10/10	TypeScript extensions can alter tools, events, context and UI
Session handling	9/10	Tree-based sessions make experimentation unusually natural
Initial usability	8/10	Easy to start, but advanced use expects technical confidence
Safety defaults	5/10	No built-in sandbox or comprehensive permission boundary
Team governance	6/10	Possible to build, but much is intentionally not built in

The strongest reason to use Pi is not that it has more features than competing coding agents — it does not. The reason is that Pi exposes more of the agent itself. That makes it particularly attractive to senior developers, platform engineers, AI tooling teams and anyone who has reached the point where the limitations of their coding agent are caused by the harness rather than the underlying model. Pi is less convincing for someone who wants to install an agent, approve a few safe defaults and never think about its architecture again: its minimalism is productive only if you value the control that minimalism creates.

What Pi actually gives you

The default model-facing tool set is intentionally small:

read
write
edit
bash

Additional read-only tools including grep, find and ls are available, and recent Pi releases allow the initial built-in tool selection to be configured.

The default looks sparse, and that is the point. Every additional tool increases the number of decisions the model has to make, expands the system prompt and creates another behavioural surface that may need debugging. Pi instead starts from a capable primitive set and lets you add specialised tools when your workflow actually needs them.

The philosophy extends further than tools. Pi deliberately does not make built-in subagents or a mandatory plan mode central to the product; those behaviours can be implemented through extensions or installed packages instead. That makes Pi less convenient out of the box, but it gives developers more control over how those mechanisms are implemented. If you are used to Claude Code's built-in subagents, the Claude Code subagents guide is a useful reference for what you would be reimplementing. I find that approach slightly refreshing: Pi does not pretend there is one correct way to operate an AI coding agent.

How to install Pi Coding Agent and start a session

Install the current package from the @earendil-works scope. Older tutorials may still show @mariozechner packages from before the May 2026 move; those names are stale for new installs.

The name also collides with oh-my-pi, a community fork of the Oh My Opencode harness. That project is unrelated to this coding agent; the Oh My Opencode review explains the fork if you landed on the wrong Pi.

Install and authenticate

Install Pi globally with npm:

   npm install -g --ignore-scripts @earendil-works/pi-coding-agent

--ignore-scripts disables dependency lifecycle scripts during install. Pi does not need those scripts for a normal npm install, and skipping them reduces a common supply-chain risk.

Verify the installation:

   pi --version

Enter a project and start it:

   cd /path/to/project
   pi

Pi expects a bash-capable environment. On Windows, use WSL or Git Bash rather than cmd.exe.

Pi supports interactive subscription authentication as well as API-key based providers. Inside Pi, the simplest route is:

/login

After authentication, select a model with:

/model

You can also supply provider credentials through environment variables. For example:

export ANTHROPIC_API_KEY="your-api-key"
pi

Pi currently supports a broad collection of model providers, including OpenAI, Anthropic, Google, Azure OpenAI, Amazon Bedrock, NVIDIA NIM, DeepSeek, Mistral, Groq, Cerebras, Cloudflare, xAI, OpenRouter and several others. A llama.cpp router is also supported for locally served models: the llama.cpp quickstart covers serving GGUF models with an OpenAI-compatible API, and the LLM hosting guide maps the surrounding local, self-hosted and cloud runtimes if you want to compare that path with Ollama, vLLM or a hosted provider. Custom providers that speak a supported API can be added through ~/.pi/agent/models.json, and custom APIs or OAuth flows can be wired up with extensions.

This multi-provider design is one of Pi's practical advantages. The model and the coding harness are separate variables: with Pi, you can change the model while keeping essentially the same tooling and session environment, which makes model comparisons considerably more meaningful than comparing completely different coding products.

Start with a controlled first session

I would not make your first Pi prompt "refactor my application". Start by asking it to inspect rather than modify:

pi --tools read,grep,find,ls -p "Inspect this repository. Explain its architecture, identify the main entry points, and list the commands you would run before making a change. Do not modify files."

When I ran that read-only prompt on an existing repository, the useful result was not a clever architecture essay. It was whether the model named real entry points and the commands I would actually run before asking for write access. That gives you a look at how the selected model navigates your tree without immediately granting it a write path through the normal tool set.

For interactive work, I would also create a disposable Git branch first:

git switch -c ai/pi-evaluation
pi

Pi can modify files in the working directory and can execute shell commands through bash, so Git remains one of the simplest practical rollback layers when evaluating it. If you want a test session that does not persist, pi --no-session runs in ephemeral mode.

Project instructions belong in AGENTS.md. A useful starting file might look like this:

# Project Instructions

- Read the existing implementation before modifying files.
- Keep changes narrowly scoped to the requested task.
- Run `npm test` after code changes.
- Run `npm run lint` before declaring the task complete.
- Do not modify database migrations unless explicitly requested.
- Do not access production infrastructure.
- Explain any destructive command before running it.

Pi also understands CLAUDE.md while walking project directories, and AGENTS.override.md can override the normal project instructions for a directory.

Then give it a concrete task:

Read the authentication module and its tests.

Find one maintainability problem that can be fixed without changing public behaviour. Explain the proposed change first, then implement it and run the relevant tests.

This is a better agent evaluation than asking it to generate a new toy application. Existing code forces the agent to discover constraints, preserve behaviour, choose relevant files and verify its work.

Why Pi Coding Agent is an editable harness

Pi's most important architectural idea is that the coding agent itself should remain editable. The extension system uses TypeScript modules that can register tools, subscribe to lifecycle events, intercept tool calls, inject or transform context, add commands and modify the terminal interface. Project-local extensions can also be reloaded with /reload, which makes experimenting with the harness surprisingly immediate.

In simplified form, Pi looks like this:

flowchart TD
    U[Developer] --> P[Pi Agent Harness]
    P --> M[Selected LLM Provider]

    M --> P

    P --> R[read]
    P --> W[write]
    P --> E[edit]
    P --> B[bash]

    X[TypeScript Extensions] --> P
    S[Skills] --> P
    C[Project Context] --> P
    PKG[Pi Packages] --> P

    R --> FS[Project Workspace]
    W --> FS
    E --> FS
    B --> OS[Shell and Toolchain]

The diagram is simple because Pi is trying to keep the control plane simple. That gives you an unusual option: when the agent lacks a capability, you do not necessarily have to wait for the Pi maintainers to add it — you can add the behaviour yourself. An extension can create a new model-callable tool, inspect a proposed bash call and block commands matching your own policy, insert dynamic project information before a turn, alter compaction behaviour or expose a custom command inside the terminal UI. That is a much deeper extension surface than changing a system prompt.

MCP integration is not treated as a mandatory built-in part of Pi's minimal core. It can be added through the same extensibility model, which is consistent with the broader design: specialised integrations live above the small default harness rather than permanently expanding it.

The same harness is also reusable outside the interactive terminal. Pi ships an SDK for embedding the agent session in Node.js applications, and an RPC mode (pi --mode rpc) that speaks JSON over stdin/stdout for non-Node hosts, IDEs and custom UIs.

The four-tool idea is more serious than it looks

I initially thought about coding-agent tools in two groups. The first contains primitives: read a file, modify a file and execute a command. The second contains conveniences: repository search, test runners, issue trackers, browser tools, deployment tools, database helpers, code intelligence and specialised planners. Pi keeps the primitive layer small and allows the convenience layer to remain optional.

There is an architectural advantage to this. A team working on Kubernetes infrastructure can build a very different Pi from a team maintaining a TypeScript monorepo, without both teams carrying the other's tools in the same default agent. This is where Pi stops looking like a stripped-down Claude Code alternative and starts looking like a framework for building your own coding agent.

The price is maintenance. Once you depend on custom extensions, their behaviour becomes part of your developer platform: you need to review them, version them and occasionally update them as Pi evolves. That is the hidden Pi tax — you gain ownership of the agent, and ownership comes with maintenance.

Skills and packages keep customisation modular

Not every custom behaviour needs to become an extension. Pi supports skills that package instructions, scripts and reference material around a capability. Skills use progressive disclosure: Pi can advertise the available skill to the model while loading the full instructions only when the capability is needed. That is useful for workflows such as deployment reviews, database migrations, release preparation or framework-specific code review; instead of permanently injecting pages of instructions into every conversation, the agent can load specialised guidance on demand. If you already build Claude Skills, the mechanics will feel familiar — see Claude Skills and SKILL.md for developers for the SKILL.md layout and trigger tuning.

Pi Packages go further by bundling extensions, skills, prompt templates and themes. Packages can be installed from npm or Git repositories and can be pinned to specific versions:

pi install npm:@foo/pi-tools@1.2.3
pi list

The package model is powerful, but treat packages more like executable developer tooling than harmless prompt collections. A Pi extension can execute code with the permissions of the Pi process, and a skill can also instruct the model to execute code, so installing an unknown package deserves essentially the same suspicion as installing an unknown development dependency.

Session trees in Pi Coding Agent

Many coding assistants treat a conversation as a mostly linear transcript. Pi stores sessions as a tree, which means you can return to an earlier point, try another approach and retain both paths inside the session rather than destroying one history or creating a pile of unrelated chats.

The main commands are straightforward:

/resume
/tree
/fork
/clone
/compact

/tree lets you navigate within the current session tree. /fork creates a new session from an earlier user message, while /clone duplicates the currently active branch into a separate session. From the command line, pi -c continues the most recent session and pi -r browses past ones.

The distinction became obvious the first time I used Pi on a debugging pass rather than a greenfield prompt. I let it follow a caching-layer hypothesis for a failing integration test, then forked from the message where that theory had been chosen and started a second branch on transaction isolation. The failed investigation stayed available for contrast instead of contaminating the new path or forcing a fresh chat, which is a useful representation of how debugging actually works: multiple plausible explanations often exist at the same time.

Pi persists sessions as JSONL and can automatically compact older context as conversations approach model limits; branches can also be summarised when you move between them. I would rank this above several more visible coding-agent features: good context management usually matters more than another button in the interface.

Context engineering is a first-class capability

Pi also makes it possible to customise what reaches the model. Extensions can inject information before turns, alter history and participate in compaction. The default compaction system summarises older material while retaining recent conversation, but the mechanism can be replaced or customised.

For advanced teams, this opens interesting possibilities. A large monorepo could inject ownership information only for files the agent is currently touching. A backend project could surface schema metadata when database files enter the context. A long-running migration agent could maintain structured state outside the conversation rather than repeatedly asking the model to rediscover it. This is what I consider Pi's real long-term strength: coding-agent quality is increasingly constrained by context engineering, tool design and orchestration rather than raw model intelligence, and Pi gives developers unusually direct access to those layers.

Pi Coding Agent security: no built-in sandbox

The most important negative point in this review is simple: Pi does not provide a built-in security sandbox. By default, Pi runs with the permissions of the user and process that started it. Its tools can read and modify accessible files and execute shell commands within that operating-system context.

Pi does have project trust, but project trust is not a sandbox. Project trust controls whether project-local Pi settings, extensions, skills, prompts and similar resources are loaded. Once you are working in a project, it does not create a filesystem, network, process or credential boundary around model-directed tool calls. I actually appreciate that the Pi documentation is explicit about this distinction: a weak pseudo-sandbox can be more dangerous than no sandbox because users may trust a boundary that does not really exist. Pi instead expects meaningful isolation to come from containers, virtual machines, micro-VMs or another policy-controlled runtime.

For disposable personal projects, you may decide that Git and normal operating-system permissions are sufficient. For untrusted repositories, unattended agents, workstations containing production credentials or agents processing externally supplied content, I would use stronger isolation. The Pi documentation describes three patterns:

Pattern	What is isolated	Where credentials live	When to use it
Gondolin extension	Built-in tools and `!` commands inside a local Linux micro-VM; the `pi` process stays on the host	Provider auth can remain on the host	Local isolation when you want the TUI and keys on the host
Plain Docker	The whole `pi` process	API keys enter the container unless you proxy them	Simple local isolation of the entire agent
NVIDIA OpenShell	The whole `pi` process inside a policy-controlled sandbox	The gateway can keep raw model keys outside the sandbox	Local or remote policy over filesystem, network and credentials

The important boundary is not just the source directory. An agent that can run shell commands may potentially interact with SSH configuration, cloud credentials, package registries, local services, browser state and anything else available to the process. A production-quality setup therefore needs to think about filesystem mounts, environment variables, network access and credentials together. My preferred evaluation pattern is:

flowchart TD
    H[Host] --> T[Terminal and Git]
    H --> SB[Sandbox]
    SB --> P[Pi]
    SB --> RC[Repository copy]
    SB --> BT[Build tools]
    SB --> MC[Minimum provider credentials]

Mounting the real repository read/write into a container still allows the agent to modify that repository; containers help only to the extent that their mounts, credentials, capabilities and network access are actually constrained. Gondolin has a related caveat: custom extension tools still run on the host unless they also delegate into the VM.

There is another security surface worth remembering: extensions and packages. A third-party extension is code running inside the agent environment. Review it before installation, pin important dependencies and avoid treating a public package gallery as an implicit security review.

Two practical checks before you run Pi on a machine you care about:

# Confirm the version you actually have
pi --version

# Disable startup network operations (update checks and install telemetry)
export PI_OFFLINE=1
pi

PI_OFFLINE=1 disables all startup network operations, including update checks, package update checks and install/update telemetry. If you want a finer split, PI_SKIP_VERSION_CHECK=1 turns off only the version check, and PI_TELEMETRY=0 disables install/update telemetry while leaving update checks enabled.

This security model is the main reason I would hesitate before giving Pi to a large engineering organisation as an unmanaged default. Pi provides the pieces to build a strong environment, but it intentionally leaves much of that environment to you.

Pi Coding Agent vs Claude Code and OpenCode

Pi makes the most sense when compared by philosophy rather than by feature count. Among terminal agents, OpenHands is the closer sibling: it also runs from the CLI, but it ships sandboxing and a larger built-in workflow rather than asking you to assemble one. Claude Code has increasingly sophisticated built-in permissions, hooks and subagents. OpenCode exposes configurable allow, ask and deny rules plus built-in agent or subagent concepts. Pi starts lower in the stack.

Capability	Pi	Claude Code	OpenCode
Core approach	Minimal extensible harness	Opinionated coding-agent product	Configurable coding agent
Default coding tools	Small primitive set	Larger integrated tool surface	Integrated tool surface
Permission system	Not a core security boundary	Built-in permission modes and rules	Built-in allow, ask and deny rules
Built-in subagents	No, add them through customisation	Isolated subagents with tool allowlists	Built-in primary agents and subagents
Custom lifecycle behaviour	TypeScript extensions and events	Hooks, plugins and `settings.json`	Agent and tool configuration
Session branching	Tree sessions with `/fork` and `/clone`	Linear sessions plus subagent workflows	Session continuity plus built-in agents
Best fit	Developers building their own agent workflow	Developers wanting stronger built-ins	Developers wanting configurable agents with integrated policy

This is why calling Pi a "Claude Code clone" misses the interesting part. If you are happy with Claude Code's workflow, Pi does not automatically improve it: Claude Code gives you more security and agent orchestration behaviour without requiring you to assemble those pieces yourself. Pi becomes interesting when you want to change the assumptions underneath the product. Maybe you want the same harness across Anthropic, OpenAI and local models. Maybe your organisation has its own sandbox. Maybe you need a tool that injects internal architecture metadata before every turn. Maybe you want compaction to produce structured engineering state instead of a generic conversation summary. That is Pi territory.

OpenCode is arguably closer philosophically because it also offers substantial configuration. Even there, Pi feels more like a kit for constructing the harness itself, while OpenCode exposes more ready-made agent and permission concepts.

There is no universal winner here. For many developers, Claude Code's opinionated defaults are an advantage. For others, OpenCode may offer a better middle ground. Pi is the option I would investigate when the sentence "I wish my coding agent worked differently" has become more important than "I wish my coding agent had another feature".

Should you use Pi Coding Agent?

Pi Coding Agent is one of the more technically interesting coding agents because it refuses to hide the harness. The default implementation is deliberately small, but the extension API, skills, packages, provider layer, SDK, RPC mode and session tree give it a much larger effective surface than its four default tools suggest.

I would recommend Pi most strongly to experienced developers who like constructing their own tooling. It is particularly compelling for AI platform work, experimental coding-agent research, specialised engineering workflows and teams that already have secure execution infrastructure. It is also useful for developers who want to compare models without changing their entire coding environment every time. I would be more cautious for beginners and for organisations expecting enterprise-style policy controls immediately after installation.

The missing sandbox changes how Pi should be deployed, especially when an agent is allowed to operate unattended or can encounter untrusted instructions. Treat Pi as a powerful local developer process rather than a security boundary: for untrusted repositories, unattended operation or environments containing valuable credentials, use an operating-system, container, VM or policy-based sandbox and expose only the resources required for the task.

My overall rating is 8.4/10. That is an editorial verdict rather than the average of the table above, and it could easily become higher for the right user. Pi's greatest weakness and greatest strength are the same thing: it does not finish the agent for you.

Should you switch from Claude Code? Not automatically. Use Claude Code when its built-in permissions, subagents, hooks and workflow fit the way you already work. Evaluate Pi when you want broader ownership of the harness, provider selection, context construction and tool architecture. For advanced developers, I would not even assume the choice has to be exclusive: Claude Code can remain the polished daily driver while Pi becomes the programmable agent laboratory. After enough customisation, that laboratory may turn into the tool you actually prefer.

Licensing and cost

Pi itself is open source and released under the MIT licence. That does not make model inference free: your actual cost depends on the model provider, API usage, supported subscription authentication, local compute and any sandbox infrastructure you choose to run.

References

Pi Coding Agent repository: https://github.com/earendil-works/pi
Pi website and installer: https://pi.dev
npm package: https://www.npmjs.com/package/@earendil-works/pi-coding-agent
Gondolin micro-VM: https://github.com/earendil-works/gondolin
NVIDIA OpenShell: https://docs.nvidia.com/openshell/about/overview
Licence: MIT

GFM vs CommonMark vs Pandoc Markdown: Syntax Compared

Rost — Sat, 08 Aug 2026 10:05:11 +0000

Markdown looks like one language until the same file renders differently on GitHub, Hugo, Obsidian, or Pandoc. And the problem is not that Markdown is unreliable.

It is that "Markdown" describes a family of related syntaxes, parsers, and platform features rather than a single universal document format. CommonMark defines a precise portable core, GitHub Flavored Markdown adds features useful for software collaboration, and Pandoc Markdown expands the language into a serious document-authoring format.

Choosing between them depends on where the document must render. A README file, a Hugo blog post, and an academic paper each have different requirements. This comparison is part of the broader documentation tooling picture and covers the formal dialects, platform-specific extensions, and practical portability rules so you can pick the right syntax for your target environment. For a quick syntax reference, the Markdown cheatsheet covers the essential formatting elements.

Markdown Is Not One Language

The original Markdown syntax was intentionally small and loosely specified. That made it easy to read and implement, but different parsers began interpreting ambiguous input differently.

CommonMark was created to define consistent parsing rules for the fundamental Markdown structures. GitHub Flavored Markdown, usually called GFM, builds on that foundation with several widely used extensions.

Pandoc Markdown takes a different approach. Instead of remaining a small web-oriented syntax, it adds document features such as citations, metadata, footnotes, definition lists, attributes, and mathematical notation.

A simplified relationship looks like this:

flowchart TD
    M[Markdown family] --> C[CommonMark core]
    C --> G[GitHub Flavored Markdown]
    C --> X[Other CommonMark-based renderers]
    M --> P[Pandoc Markdown]
    G --> GH[GitHub platform features]
    X --> H[Hugo with Goldmark]
    X --> GL[GitLab Flavored Markdown]
    P --> PDF[PDF and academic workflows]
    P --> DOCX[DOCX and publishing workflows]

This hierarchy is useful, but it is not exact inheritance in every implementation. Each renderer can enable, disable, or add syntax independently.

The Short Answer

Use CommonMark-compatible syntax when portability matters most.

Use GFM when writing README files, pull requests, issue templates, and technical documentation intended primarily for GitHub-compatible platforms.

Use Pandoc Markdown when the source document must become PDF, DOCX, EPUB, LaTeX, slides, or an academic paper with citations and metadata.

For a Hugo technical blog, use the CommonMark core plus the Goldmark extensions that your site explicitly enables. Do not assume every feature visible on GitHub will work merely because Hugo is described as GFM-compatible.

Opinionated take: if you only remember one rule for a Hugo technical blog, treat CommonMark plus GFM-style tables and task lists as the default, and treat everything else — footnotes, math, callouts, header attributes — as an explicit, tested extension rather than an assumed default. That single habit prevents most of the portability failures described below.

CommonMark: The Portable Core

CommonMark is a formal specification for the basic Markdown language. Its main contribution is not a large collection of features, but consistent parsing.

It defines how parsers should interpret:

Paragraphs
ATX and Setext headings
Block quotes
Ordered and unordered lists
Fenced and indented code blocks
Emphasis and strong emphasis
Links and images
Reference-style links
Inline code
Thematic breaks
Raw HTML blocks
Hard and soft line breaks

A CommonMark document can still behave differently at the presentation layer. CSS, syntax highlighting, heading anchors, HTML sanitization, and link policies are outside the core parsing rules.

CommonMark should therefore be treated as a reliable structural baseline, not a promise that every renderer will produce an identical page.

A Portable CommonMark Example

# Service Deployment

The service exposes a small HTTP API.

## Requirements

- Linux
- Docker
- 8 GB of memory

## Start the service

```bash
docker compose up -d
```

See the [configuration guide](configuration.md) for details.

This type of document works across almost every modern Markdown environment. It uses headings, paragraphs, lists, fenced code, and ordinary links without relying on dialect-specific extensions.

GitHub Flavored Markdown: CommonMark for Software Projects

GitHub Flavored Markdown is a formal dialect based on CommonMark. It preserves the CommonMark parsing model and adds features commonly needed in repository documentation and collaboration.

The formal GFM specification adds:

Pipe tables
Task list items
Strikethrough
Extended autolinks
Restrictions around some raw HTML tags

These extensions are now so common that many users think they are part of standard Markdown. They are not part of the CommonMark core.

GFM Tables

| Backend | Best use |
|---|---|
| Ollama | Local experiments |
| vLLM | Shared inference |
| SGLang | Structured workloads |

A strict CommonMark parser is allowed to treat this as ordinary paragraph text. A GFM-compatible parser recognizes it as a table. For a deeper look at table syntax and alignment options, see Tables in Markdown.

GFM Task Lists

- [x] Install Docker
- [x] Download the model
- [ ] Add monitoring

Task list syntax is useful in issues, pull requests, and project documentation. Outside a supporting renderer, it may appear as an ordinary list containing literal square brackets.

GFM Strikethrough

Use the ~~old endpoint~~ new endpoint.

Strikethrough is widely supported, but it is still an extension rather than portable CommonMark syntax.

GFM Autolinks

GFM recognizes more URL-like and email-like text without requiring angle brackets or explicit link syntax.

Visit https://example.com/docs for details.

In strict CommonMark, explicit autolinks use angle brackets:

<https://example.com/docs>

The explicit form is safer when a document must travel through unknown Markdown processors.

GitHub.com Supports More Than Formal GFM

A frequent source of confusion is the assumption that every Markdown feature visible on GitHub belongs to the GFM specification.

It does not.

GitHub.com adds platform-level processing and features around the GFM parser. Depending on the context, GitHub can support:

Mathematical expressions
Mermaid diagrams
Alerts
Issue and pull request references
User and team mentions
Commit references
Emoji shortcodes
Collapsible HTML sections
Color previews
Repository-relative links
Automatic heading anchors

Some of these features are syntax extensions. Others are post-processing behavior or integrations with GitHub data.

This distinction matters because another renderer may accurately claim GFM compatibility without implementing GitHub's math renderer, Mermaid integration, issue references, or alert styling.

GitHub Mermaid Diagrams

GitHub renders a fenced code block marked mermaid as a diagram:

```mermaid
flowchart LR
    A[Markdown] --> B[Rendered diagram]
```

A generic GFM renderer may display the same block as highlighted source code. The Markdown remains valid, but the enhanced rendering is platform-specific. For a practical introduction to Mermaid syntax, see the Mermaid Diagrams Quickstart.

GitHub Mathematical Expressions

GitHub supports inline and block mathematical expressions using dollar delimiters and additional escaping forms.

The cache size is approximately $2nlhd$ bytes.

$$
C = 2nlhd
$$

Math is not part of formal GFM. Moving this content to another renderer requires a compatible math extension such as KaTeX, MathJax, or Pandoc math support.

GitHub Alerts

GitHub supports alert-style block quotes such as:

> [!WARNING]
> Changing this setting clears the cache.

On GitHub, this can appear as a styled warning. On a plain CommonMark renderer, it usually appears as an ordinary block quote containing [!WARNING].

That fallback is readable, which makes GitHub alerts less dangerous than extensions that disappear completely. They are still not portable presentation elements.

Pandoc Markdown: Markdown as a Document Language

Pandoc Markdown is designed for document conversion rather than one particular website. It uses Markdown as the source syntax for producing HTML, PDF, DOCX, EPUB, LaTeX, presentations, and other formats.

Its default Markdown reader includes a large extension set. Important capabilities include:

YAML metadata blocks
Footnotes
Citations
Multiple table formats
Definition lists
Mathematical notation
Header identifiers and attributes
Code block attributes
Fenced divisions
Bracketed spans
Superscript and subscript
Strikeout
Line blocks
Numbered example lists
Raw LaTeX
Raw HTML
Automatic section numbering
Bibliography processing

Pandoc Markdown is much more expressive than CommonMark or formal GFM. That expressiveness makes it powerful for publishing, but less safe as an interchange format.

Pandoc Footnotes

Markdown has several incompatible dialects.[^dialects]

[^dialects]: CommonMark, GFM, and Pandoc Markdown are three
    important examples.

Footnote syntax is supported by many modern tools, but it is not part of CommonMark or formal GFM.

GitHub currently renders footnotes in several content contexts, but that is a GitHub platform feature rather than a formal GFM guarantee. A renderer claiming only CommonMark or GFM compatibility may not support them.

Pandoc Citations

PagedAttention improves KV cache memory management
[@kwon2023pagedattention].

With a bibliography file and citation style, Pandoc can resolve this into a formatted academic citation and bibliography.

pandoc article.md \
  --citeproc \
  --bibliography references.bib \
  --csl ieee.csl \
  -o article.pdf

The citation syntax remains readable in an unsupported renderer, but it will not become a formatted reference without Pandoc or another compatible citation processor. Pandoc's reader-side flexibility also underpins conversion workflows in the other direction — see converting Word documents to Markdown for a practical example of using Pandoc's extended dialect as an intermediate format.

Pandoc Definition Lists

CommonMark
: A precise specification for core Markdown.

GFM
: A CommonMark-based dialect with software-oriented extensions.

Pandoc Markdown
: An extended authoring format for document conversion.

Definition lists are useful in manuals, glossaries, and technical books. They normally degrade poorly in renderers that do not support them because the colon lines remain visible as plain text.

Pandoc Header Attributes

## Cache Configuration {#cache-config .deployment}

Pandoc interprets the braces as an explicit identifier and class list. Many other Markdown renderers show the attribute text directly in the heading.

This is one of the clearest examples of useful syntax that should not be placed in a document expected to render everywhere.

Pandoc Fenced Divisions

::: warning
Changing this option restarts the server.

Pandoc converts this into a structural division with a class. Templates, CSS, filters, or output writers can decide how that structure should appear.

Most CommonMark and GFM renderers do not recognize the fence. They display the colons and content as ordinary text.

CommonMark vs GFM vs Pandoc Markdown

The following matrix describes the formal dialects, not every feature added by GitHub.com, Hugo, Obsidian, GitLab, or another platform.

Feature	CommonMark	Formal GFM	Pandoc Markdown
Headings	Yes	Yes	Yes
Emphasis	Yes	Yes	Yes
Links and images	Yes	Yes	Yes
Block quotes	Yes	Yes	Yes
Ordered and unordered lists	Yes	Yes	Yes
Fenced code blocks	Yes	Yes	Yes
Raw HTML syntax	Yes	Restricted in some contexts	Yes
Pipe tables	No	Yes	Yes
Task lists	No	Yes	Yes
Strikethrough	No	Yes	Yes
Extended autolinks	No	Yes	Configurable
Footnotes	No	No	Yes
Citations	No	No	Yes
YAML metadata	No	No	Yes
Definition lists	No	No	Yes
Mathematical notation	No	No	Yes
Header attributes	No	No	Yes
Fenced divisions	No	No	Yes
Raw LaTeX	No	No	Yes
Bibliography processing	No	No	Yes

The word "No" does not mean a platform can never support the feature. It means the feature is not guaranteed by that dialect's formal specification.

Which Syntax Works on GitHub?

For README files, issues, pull requests, discussions, and wikis, GFM is the natural baseline.

You can generally use:

CommonMark syntax
Tables
Task lists
Strikethrough
Extended autolinks
Syntax-highlighted code fences
GitHub-specific references
GitHub-supported math
GitHub-supported diagrams
GitHub alerts
Footnotes where supported by the content surface

The portability risk begins when GitHub performs additional rendering beyond formal GFM. Mermaid diagrams, mathematical notation, issue references, and alert presentation may not survive outside GitHub.

For repository files that are also published elsewhere, test the source in the second renderer rather than treating GitHub preview as authoritative.

Which Syntax Works in Hugo?

Hugo uses Goldmark as its default Markdown renderer. Goldmark conforms to CommonMark and provides extensions compatible with important parts of GFM.

In a typical Hugo configuration, the following work well:

CommonMark structure
Fenced code blocks
Pipe tables
Strikethrough
Task lists
Automatic heading IDs
Syntax highlighting
Footnotes when the extension is enabled
Definition lists when enabled
Typographic substitutions when enabled

Hugo also adds features outside Markdown through:

Front matter
Shortcodes
Render hooks
Page resources
Internal reference functions
Template processing
Site configuration

These Hugo features do not travel with the Markdown file. For a practical example of Hugo deployment, see Deploy Hugo to AWS S3.

Hugo Front Matter Is Not Markdown Content

A Hugo page commonly starts with YAML, TOML, or JSON metadata:

---
title: "Markdown Compatibility"
description: "Compare Markdown dialects and renderers."
date: 2026-07-31
tags:
  - Markdown
  - documentation
---

Pandoc can also recognize YAML metadata blocks, but it interprets fields according to its own templates and writers. GitHub normally displays the block as a YAML-like section or treats it as repository metadata only in specific systems.

The same syntax can therefore be recognized in more than one tool without having the same semantics.

Raw HTML in Hugo

Goldmark does not render potentially unsafe raw HTML by default in a standard Hugo configuration.

A block such as:

<div class="notice">
  Restart the service after changing this value.
</div>

may be omitted unless raw HTML rendering is enabled or the content is implemented through a shortcode or render hook.

For a controlled technical blog, enabling raw HTML can be reasonable. It still makes the source less portable and should be a deliberate site-level decision.

Mermaid in Hugo

A fenced mermaid block is still just a code block unless the Hugo theme, render hook, shortcode, or JavaScript pipeline transforms it into a diagram.

GitHub and Hugo may therefore accept identical Mermaid source while using completely different rendering mechanisms.

Which Syntax Works in Pandoc?

Pandoc can read several Markdown dialects explicitly:

pandoc --from=markdown input.md
pandoc --from=commonmark input.md
pandoc --from=gfm input.md
pandoc --from=commonmark_x input.md

This is one of Pandoc's most useful portability features. The operator can tell Pandoc which dialect the source claims to use instead of relying on a vague .md file extension.

Pandoc also lets you enable or disable individual extensions:

pandoc \
  --from=markdown-footnotes-pipe_tables \
  input.md \
  -o output.html

Or start from a narrower format and add one feature:

pandoc \
  --from=commonmark+footnotes \
  input.md \
  -o output.html

You can inspect available extensions with:

pandoc --list-extensions=markdown
pandoc --list-extensions=commonmark
pandoc --list-extensions=gfm

This extension model is powerful, but it means "Pandoc Markdown" is not always one fixed configuration. Build commands and defaults files are part of the document specification.

Which Syntax Works in Obsidian?

Obsidian stores notes as Markdown files, but its authoring model includes several application-specific features.

Common examples include:

Wiki links
Embedded notes
Embedded files
Callouts
Block references
Tags
Properties
Highlighting
Comments
Dataview queries from plugins
Application-specific URI links

A wiki link such as:

[[Markdown Compatibility]]

is meaningful inside an Obsidian vault. GitHub, CommonMark, and a default Pandoc reader normally display it as literal bracketed text.

An embed is even more application-specific:

![[compatibility-table]]

The referenced content is not present in the file itself. Exporting or publishing the note therefore requires an expansion step that resolves the embed.

Obsidian is a good example of why storage in .md files does not guarantee Markdown portability. For a practical look at Obsidian as a knowledge management tool, see Obsidian for Personal Knowledge Management.

Which Syntax Works in GitLab?

GitLab Flavored Markdown uses CommonMark as its core and includes GFM features such as tables and task lists. It then adds GitLab-specific behavior including cross-references, mathematical notation, diagrams, and other collaboration features.

A README written in conservative GFM usually moves between GitHub and GitLab without major damage.

Platform integrations do not travel as reliably. Issue references, user mentions, diagrams, math handling, and special block syntax can behave differently even when the basic Markdown remains readable.

Platform Support Matrix

This matrix describes common default behavior. Themes, plugins, extensions, and configuration can change individual cells.

Feature	GitHub	Hugo Goldmark	Pandoc	Obsidian	GitLab
CommonMark core	Yes	Yes	Yes	Mostly	Yes
Pipe tables	Yes	Yes	Yes	Yes	Yes
Task lists	Yes	Yes	Yes	Yes	Yes
Strikethrough	Yes	Yes	Yes	Yes	Yes
Footnotes	Yes	Configurable	Yes	Yes	Yes
YAML metadata	Context-dependent	Front matter	Yes	Properties	Context-dependent
Math	Yes	Requires setup	Yes	Yes	Yes
Mermaid	Yes	Requires setup	Output-dependent	Yes	Yes
Citations	No native bibliography	Requires tooling	Yes	Plugin-dependent	No native bibliography
Definition lists	No	Configurable	Yes	Limited	Limited
Header attributes	Limited	Renderer-dependent	Yes	Limited	Limited
Wiki links	No	No by default	No by default	Yes	Wiki-dependent
Callouts or alerts	GitHub syntax	Theme or shortcode	Template-dependent	Obsidian syntax	GitLab syntax
Raw HTML	Sanitized or restricted	Disabled by default	Yes	Context-dependent	Sanitized or restricted

"Yes" still does not guarantee identical HTML or visual presentation. It means the environment recognizes the general feature.

Syntax That Is Usually Safe Everywhere

The safest portable subset includes:

ATX headings using #
Ordinary paragraphs
Blank lines between blocks
- for unordered lists
1. for ordered lists
Fenced code blocks using backticks
Inline code using backticks
Emphasis using *text*
Strong emphasis using **text**
Ordinary links
Ordinary images
Block quotes
Thematic breaks
Explicit angle-bracket autolinks

An intentionally conservative document might look like this:

# Deployment Guide

This guide explains how to deploy the service.

## Requirements

- Docker
- Linux
- A supported GPU

## Configuration

Create a file named `compose.yaml`.

```yaml
services:
  application:
    image: example/application:1.0
```

For more information, see the [configuration reference](config.md).

> Back up existing data before upgrading.

This syntax travels well because it does not depend on tables, footnotes, attributes, callouts, or platform processing.

Syntax That Commonly Breaks

Portability problems tend to cluster around a small number of features.

Pipe Tables

Pipe tables are well supported by GFM-oriented tools, but not by strict CommonMark.

A table can degrade into unreadable text when passed through a parser that does not recognize it. For highly portable documents, consider short lists or semantic HTML generated during a build step.

Footnotes

Footnote syntax has become common, but it remains an extension.

Different tools may:

Support only one footnote format
Place footnotes differently
Generate different identifiers
Reject multi-paragraph footnotes
Render the source literally

Use footnotes when the publishing pipeline is known. Avoid depending on them in README files that must render across arbitrary systems.

Heading IDs and Attributes

This Pandoc syntax is not portable:

## Installation {#installation .procedure}

Use an ordinary heading and let the renderer generate its own anchor when portability matters.

Also avoid hard-coding links to auto-generated heading IDs unless every target uses the same slugification rules.

Callouts and Alerts

GitHub, Obsidian, GitLab, MkDocs, Docusaurus, and Hugo themes can all support callout-like blocks, but they often use different syntax.

A portable fallback is an ordinary block quote:

> Warning: Back up the database before upgrading.

It is less visually impressive, but it preserves meaning everywhere.

Wiki Links

Wiki links are concise inside knowledge-management tools:

[[KV Cache]]

They are poor interchange syntax because the target path, file name, heading rules, and resolution behavior belong to the application.

Use standard Markdown links in content intended for publication:

[KV cache](kv-cache.md)

Raw HTML

Raw HTML is the usual escape hatch when Markdown cannot express a layout. It is also a common portability and security failure.

A renderer may:

Remove the HTML
Escape it
Sanitize selected elements
Allow blocks but not inline elements
Refuse Markdown parsing inside HTML
Pass it unchanged only in trusted mode

Use raw HTML only when the publishing target is controlled.

Mathematical Notation

Dollar-delimited math is popular but not universally interpreted.

The source:

The complexity is $O(n^2)$.

may become:

Rendered mathematics
Ordinary text with dollar signs
Incorrect emphasis
Input to a different math parser

Choose one math pipeline and test it in every target environment.

Mermaid and Other Diagram Blocks

A Mermaid code fence is syntactically safe because unsupported renderers normally display it as code.

The semantic result is still different. Readers may see a rendered architecture diagram on GitHub and raw Mermaid source in another environment.

This is graceful degradation, not true compatibility.

The Three Layers of Markdown Compatibility

It helps to separate compatibility into three layers.

Layer 1: Parsing Compatibility

Does the parser recognize the structure?

Examples include headings, tables, footnotes, and fenced divisions.

Layer 2: Transformation Compatibility

Does the platform apply additional processing?

Examples include:

Rendering Mermaid
Resolving citations
Expanding wiki links
Linking issue numbers
Processing shortcodes
Generating a table of contents

Layer 3: Presentation Compatibility

Does the result look and behave appropriately?

Examples include:

Table styling
Syntax highlighting
Alert colors
Heading anchors
Responsive images
Footnote placement
Math fonts

Two platforms can parse identical syntax while producing substantially different presentation.

A Better Portability Model

Instead of asking whether a file is "valid Markdown," ask four narrower questions:

Which dialect is the source written in?
Which parser reads it?
Which extensions are enabled?
Which platform transformations run afterward?

For example:

Dialect: CommonMark plus GFM tables
Parser: Goldmark
Extensions: tables, strikethrough, task lists, footnotes
Platform: Hugo
Additional processing: render hooks and Mermaid JavaScript

That description is much more useful than saying "the site uses Markdown."

Choosing a Dialect by Use Case

README Files

Use GFM.

README files benefit from:

Tables
Task lists
Fenced code
Autolinks
Strikethrough
GitHub references

Avoid excessive dependence on GitHub-only features when the repository is mirrored to GitLab, rendered on a package registry, or included in generated documentation.

Hugo Technical Articles

Use CommonMark-compatible Markdown with a documented Goldmark extension set.

Tables, code fences, footnotes, and Mermaid can be reasonable because you control the build pipeline. Prefer Hugo shortcodes or render hooks over embedding large amounts of raw HTML.

Keep Hugo-specific syntax isolated and easy to find.

Academic Documents

Use Pandoc Markdown.

Citations, bibliography processing, footnotes, metadata, mathematical notation, cross-references, and conversion to PDF or DOCX justify the reduced portability.

Store the Pandoc command, defaults file, filters, bibliography, and templates beside the source. The source file alone does not fully describe the build.

Books and Long-Form Documentation

Pandoc Markdown is usually the strongest of the three options when multiple output formats matter.

Definition lists, citations, attributes, metadata, and structured transformations become more important as document complexity grows.

For web-only documentation hosted in a Git repository, GFM or a CommonMark-based documentation generator may remain simpler.

Notes and Personal Knowledge Bases

Use the native syntax of the selected notes application when application features provide real value.

Obsidian wiki links, embeds, and callouts are useful inside a vault. Treat export as a compilation process rather than assuming the raw files are already portable publications.

Shared Documentation Across Unknown Systems

Use a conservative CommonMark subset.

Avoid:

Wiki links
Platform alerts
Header attributes
Citations
Raw HTML
Custom containers
Application embeds
Shortcodes

Portability usually requires giving up convenience features.

Practical Rules for Portable Markdown

Start with CommonMark Structure

Use CommonMark for the document skeleton:

Headings
Paragraphs
Lists
Links
Images
Block quotes
Code blocks

This ensures that the main meaning survives even when optional extensions fail.

Add GFM Features Deliberately

Tables and task lists are reasonable when all important targets support them.

Do not assume "most tools support GFM" without testing the exact target. Some claim GFM compatibility while enabling only selected extensions.

Isolate Platform Extensions

Keep platform-specific syntax in clearly identifiable blocks.

For example, centralize Hugo shortcodes, Pandoc citations, or Obsidian embeds rather than scattering them through every paragraph.

Isolation makes later conversion easier.

Prefer Graceful Degradation

A Mermaid block degrades into readable source code. A GitHub alert degrades into a block quote.

A wiki embed may degrade into an unexplained file name, while a Pandoc fenced division may expose punctuation around the content.

Choose extensions whose fallback remains understandable.

Do Not Depend on Auto-Generated Heading IDs

Heading anchor algorithms differ between GitHub, Hugo, Pandoc, and documentation generators.

For cross-document links, use renderer-supported explicit IDs only when the target pipeline is controlled. Otherwise, link to the document rather than a generated fragment.

Keep Build Configuration with the Content

Pandoc extensions, Hugo settings, plugins, filters, and JavaScript integrations determine how Markdown behaves.

Commit relevant configuration files with the source:

content/
  article.md
pandoc.yaml
references.bib
config/
  _default/
    markup.yaml
layouts/
  _default/
    _markup/

A .md extension alone does not capture the publishing environment. For a structured approach to documenting these decisions, see Decision Records for AI-Driven Development.

Test Markdown Against Every Important Target

Visual preview in one editor is not enough. The editor may support a richer dialect than the production renderer.

For Pandoc, test explicit input formats:

pandoc --from=commonmark article.md -o commonmark.html
pandoc --from=gfm article.md -o gfm.html
pandoc --from=markdown article.md -o pandoc.html

Warnings and visible source punctuation reveal which features are dialect-specific.

For Hugo, build the production site:

hugo --gc --minify

Then inspect the generated HTML rather than relying only on an editor preview.

For repositories, view the committed file on the actual hosting platform. Local Markdown extensions in VS Code may not match GitHub or GitLab.

Troubleshooting Common Rendering Mismatches

When a file that worked on one platform breaks on another, the failure usually falls into one of a handful of repeatable patterns. The table below lists the symptom as you would actually see it, the most likely cause, and a concrete command or check to confirm and fix it.

Symptom	Likely cause	Confirm and fix
A pipe table renders as one long paragraph with visible `\	` characters	Renderer is strict CommonMark without a tables extension
`[^note]` stays inline as literal text instead of becoming a superscript footnote marker	The footnote Goldmark extension is not enabled	In Hugo, check for `footnote` under `markup.goldmark.extensions` in `hugo.yaml`, rebuild with `hugo --gc --minify`, and look for `<sup>` in the generated HTML
A ``mermaid` fence shows as plain grey source code instead of a diagram	The platform performs no post-processing on the fenced block	GitHub renders it natively; Hugo needs a render hook, shortcode, or JS pipeline — check the built HTML for `<pre><code class="language-mermaid">` versus an `<svg>`
`## Heading {#id}` shows the literal curly braces in the rendered heading text	Header attribute syntax is Pandoc-specific, not CommonMark or GFM	Remove the attribute syntax for portable output, or pre-convert with `pandoc --from=markdown --to=gfm file.md -o out.md`
`[[Note Name]]` displays as literal double square brackets	Wiki link syntax is application-specific to tools like Obsidian	Replace with a standard Markdown link, `[Note Name](note-name.md)`, before exporting outside the vault
`[@kwon2023pagedattention]` stays as plain bracketed text instead of a formatted citation	No bibliography or citeproc pass was applied	Re-run with `pandoc --citeproc --bibliography=refs.bib input.md -o output.pdf` and confirm the CSL style is specified
`> [!WARNING]` renders as an ordinary quoted paragraph instead of a styled alert	Alert styling is a GitHub.com platform feature, not part of formal GFM	Expected outside GitHub; keep the wording readable as a plain block quote rather than depending on the color styling

This is the fastest first pass before assuming a Markdown "bug" — most of these mismatches are a missing extension or a platform-only feature, not broken syntax. For code-fence-specific issues such as missing syntax highlighting or unsupported language identifiers, see the dedicated guide on Markdown code blocks.

Lint the Portable Subset

A Markdown linter cannot guarantee renderer compatibility, but it can remove avoidable ambiguity.

Useful rules include:

Use consistent heading styles
Add blank lines around lists and code blocks
Use fenced rather than indented code
Specify code fence languages
Avoid skipped heading levels
Use consistent list markers
Avoid ambiguous emphasis around punctuation
Keep line endings consistent
Validate links and images

For multi-target publishing, add a build test for each important renderer rather than relying only on syntax linting.

Converting Between Dialects with Pandoc

Pandoc can normalize documents from one dialect to another:

`bash pandoc \ --from=markdown \ --to=gfm \ article.md \ -o article-gfm.md `

Or convert GFM into Pandoc Markdown:

`bash pandoc \ --from=gfm \ --to=markdown \ README.md \ -o document.md `

This is useful, but conversion is not guaranteed to preserve every feature.

Potential losses include:

Platform-specific references
Callout styling
Complex tables
Embedded application objects
Custom attributes
Raw HTML behavior
Plugin syntax
Diagram rendering
Exact whitespace and formatting

Pandoc preserves document structure better than original source formatting. Treat conversion as a build step, not a reversible text formatter.

Recommended Strategy for Hugo Sites

For a Hugo technical blog, the most practical policy is:

Use CommonMark for core prose and structure.
Enable a small documented set of Goldmark extensions.
Use GFM-style tables and task lists where they improve readability.
Implement Mermaid through one consistent render hook or shortcode.
Handle math through one documented KaTeX or MathJax pipeline.
Use Hugo front matter only at the start of content files.
Prefer render hooks and shortcodes over raw HTML.
Keep source links as standard Markdown links where possible.
Test migrated or externally sourced documents through Hugo.
Document any syntax that will not render correctly on GitHub.

This approach accepts that Hugo content is not universally portable while keeping the portability boundary visible.

The worst approach is accidental dialect mixing: GitHub alerts, Obsidian embeds, Pandoc attributes, and Hugo shortcodes placed in the same document without a defined build pipeline.

Decision Table

Use case	Recommended syntax	Reason
Portable plain-text document	CommonMark	Smallest reliable baseline
GitHub README	GFM	Tables, tasks, and repository workflows
GitHub issue template	GFM plus GitHub features	Platform is the intended target
Hugo blog post	CommonMark plus configured Goldmark extensions	Controlled publishing pipeline
Academic paper	Pandoc Markdown	Citations, math, metadata, PDF output
Multi-format book	Pandoc Markdown	Structured conversion to many outputs
Obsidian vault	Obsidian Markdown	Backlinks, embeds, and knowledge workflows
GitHub and GitLab mirror	Conservative GFM	Strong shared feature set
Unknown renderer	CommonMark subset	Lowest compatibility risk

Conclusion

CommonMark, GitHub Flavored Markdown, and Pandoc Markdown are not competing versions of the same product. They solve different problems.

CommonMark provides a dependable parsing foundation. GFM adds practical features for software collaboration, while Pandoc Markdown turns Markdown into a rich source language for publishing and conversion.

The safest rule is simple: write the smallest dialect that satisfies the real destination. Use CommonMark when content must travel, GFM when GitHub-style collaboration is the target, and Pandoc Markdown when document structure and output formats matter more than universal rendering.

Markdown portability is not achieved by avoiding every extension. It is achieved by knowing which extensions are part of the source contract and testing them in every renderer that matters.

References

Data Gravity: The Real Cost of API-First AI

Rost — Fri, 07 Aug 2026 09:54:30 +0000

Every API call feels like a simple transaction - until enough of them accumulate that your fine-tuning data, evaluation harnesses, and tool schemas are all shaped around one vendor, and switching stops being a routing change.

That's data gravity: the same force that made pulling data out of AWS S3 expensive long before AI existed, now operating one layer up the LLM hosting stack. It doesn't require a bad contract or a malicious vendor. It's compounding integration debt - every fine-tuned checkpoint, cached embedding, and evaluation harness tuned to one provider's output format makes the next one cheaper to add and the whole pile more expensive to move.

The mechanism has four stages, and none of them announce themselves. Most teams don't decide to become dependent - they drift from Exploration into Integration, then Optimization, until Dependency feels less like a choice and more like the ground truth of their architecture. Recognizing which stage you're in, and what it costs to reverse, is the point of this article.

The Mechanism: Four Stages of Lock-In

graph LR
    A[Exploration<br/>swap a base URL] --> B[Integration<br/>workflows assume<br/>the API's shape]
    B --> C[Optimization<br/>fine-tunes, caches,<br/>vector stores]
    C --> D[Dependency<br/>product quality =<br/>vendor's model]
    style A fill:#e8f4fd
    style B fill:#cfe8fb
    style C fill:#a8d4f5
    style D fill:#6fb3ea

Exploration. You call an API, prototype, iterate. Switching cost is low - changing a base URL and a key covers most of it.

Integration. You build workflows around the API's shape. Error handling assumes its rate-limit headers. Retry logic matches its backoff curves. Your evaluation harness is tuned to its output format. Switching now means refactoring, not routing.

Optimization. You fine-tune. You cache. You build vector stores and custom pipelines that depend on that provider's embedding space, tokenization, or tool-calling schema. Your data is embedded in their ecosystem. Switching means rebuilding, not refactoring.

Dependency. Your product's performance depends on that provider's model quality. Downgrading to a self-hosted alternative means accepting lower capability. The trade-off stops being architectural and becomes product-level.

Each stage compounds the previous one. The transition from Exploration to Dependency rarely feels like a decision - it feels like progress, right up until the moment a vendor changes the terms.

Why It Matters Right Now

Three forces are making data gravity urgent instead of theoretical.

Open-weight models are closing the capability gap. Moonshot AI's Kimi K3, a 2.8-trillion-parameter sparse mixture-of-experts model released in July 2026, scored 57 on the Artificial Analysis Intelligence Index - third overall, comparable to Claude Opus 4.8 and GPT-5.5, and still behind Claude Fable 5 and GPT-5.6 Sol, but close enough that the gap is now a deliberate trade-off rather than a forced compromise. Qwen and DeepSeek ship under permissive licenses with native support across vLLM and SGLang. For coding and infrastructure tasks specifically, open-weight models routinely land within 5-15% of frontier API quality - close enough that the lock-in cost, not the capability gap, becomes the deciding factor.

Geopolitics is fragmenting data flows. In July 2026, security researchers found that Claude Code had shipped hidden detection code since version 2.1.91 (April 2, 2026) that checked a user's system timezone against Asia/Shanghai and Asia/Urumqi and scanned proxy hostnames against a list of Chinese corporate and AI-lab domains - including Alibaba, Baidu, ByteDance, and Moonshot AI - encoding the match invisibly into the tool's own system prompt. Anthropic called it an anti-distillation experiment; Alibaba responded by banning Claude Code for its employees effective July 10, 2026, and ordering Claude models deleted from company infrastructure. Whatever the intent, the episode is a preview of a world where cross-border AI data flows carry protocol-level risk, not just contractual risk. If your data and your tool's behavior live in someone else's runtime, you're subject to decisions you can't audit - which is the same conclusion LLM Self-Hosting and AI Sovereignty reaches from the policy and jurisdiction side rather than the switching-cost side covered here.

Memory economics are tightening. SK Hynix CEO Kwak Noh-jung told Reuters in July 2026 that 2027 will be the memory industry's worst-ever supply shortage, with customer demand expected to outstrip production capacity "even beyond 2030." SambaNova closed the first tranche of a $1B Series F at an $11B valuation the same month, explicitly to scale inference hardware manufacturing. The narrative that API costs fall indefinitely was already shaky; a multi-year hardware shortage makes owning your inference stack a strategic hedge rather than a hobbyist preference.

The Real Cost Isn't Tokens

The pricing comparison is the wrong frame. It's not "$0.01 per 1K input tokens vs. $0.002 self-hosted" - it's architectural dependency, and the clearest recent proof is OpenClaw's collapse.

OpenClaw grew to roughly 247,000 GitHub stars on the strength of running Claude through flat-rate Pro and Max subscriptions rather than metered API billing. On April 4, 2026, Anthropic revoked the ability to use those subscription OAuth tokens in third-party tools. Users who wanted to keep running OpenClaw with Claude had to switch to pay-as-you-go billing at 10 to 50 times the effective cost of their old plan. That is Dependency, stage four, made visible almost overnight: a huge community had optimized its entire workflow around one provider's specific pricing mechanism, and when that mechanism disappeared, the workflow's economics didn't degrade gracefully - they broke. The OpenClaw vs. Hermes usage data shows a meaningful share of that traffic migrating toward self-hosted and open-weight alternatives in the months after.

The same pattern shows up quietly inside individual companies. A team that builds a code-review agent against one provider's API accumulates fine-tuning data in that provider's format, an evaluation harness tuned to that provider's output shape, and tool-calling integrations built around that provider's schema. None of that is measured in tokens. It's measured in engineering weeks the day you try to leave - the same architectural-dependency problem that the LLM Architecture cluster covers at the routing, cost, and guardrail layer above hosting.

How to Score Your Lock-In

Count how many of these your team has accumulated for a given provider:

Dependency	Do you have this?
Fine-tuning datasets stored in a provider-specific format
Cached embeddings tied to a provider's embedding space
Evaluation harnesses tuned to a provider's output shape
Custom tool schemas built around a provider's tool-calling API
Team knowledge specific to a provider's failure modes and workarounds
Product features that assume a specific model's capability ceiling
Billing or usage patterns tied to a provider-specific plan (subscription vs. metered)

0-2 checked: Exploration - switching cost is still close to zero. 3-5: Integration - expect a real refactor. 6+: Optimization or Dependency - you are not choosing your AI provider anymore; you are renting your architecture from them. The count itself is the warning signal, and it costs nothing to calculate.

What Self-Hosting Actually Costs - and Doesn't

The economics are real but secondary to the lock-in question. Cost Optimization for LLM Systems works through the hardware break-even math in detail - at roughly an hour or more of daily local usage, a consumer GPU like an RTX 4090 typically pays for itself against equivalent API spend within 4-8 months. That analysis is the right place for the $/token comparison; the point worth repeating here is that the break-even calculation only matters once you've decided portability is worth optimizing for. Teams deep in the Dependency stage often find the migration cost dwarfs any hardware savings, which is exactly the trap this article is about.

The Antidote: Portability as a Strategy

The goal isn't to avoid APIs. It's to keep your data layer portable long enough to make deliberate choices instead of drifting into a stage you didn't choose.

Start local, go remote deliberately. Prototype with self-hosted models - llama.cpp, GGUF quantization, or a full comparison of local hosting tools to pick a stack. When a task genuinely needs frontier capability, use the API for that task specifically - but keep the data layer decoupled from which model answered it.

Prefer open-weight to closed-API when the quality gap is close. When a model ships as open weights - Kimi K3, Qwen, Gemma, DeepSeek - you can run it, fine-tune it, quantize it, and own the relationship end to end. The capability gap is a known, shrinking, task-dependent trade-off. The lock-in gap is a slow-moving trap that doesn't announce its size until you try to leave.

Build abstraction where it actually matters. Not "wrap everything behind an interface" - that's a rule that delays the problem without solving it. Build the abstraction around data formats, evaluation logic, and tool schemas specifically: fine-tuning datasets in framework-agnostic formats (JSONL, parquet), evaluation harnesses that score model output rather than a specific API's response shape, and tool-calling logic that translates to and from provider-specific schemas rather than being written against one.

Route deliberately instead of committing to one provider. Model routing strategies - capability-based, cost-aware, latency-aware - let you send routine traffic to a local model and edge cases to a frontier API, which keeps you inside the Integration stage indefinitely instead of drifting into Optimization around a single vendor.

Quantify your lock-in on a schedule. Re-run the scoring table above quarterly per provider. When the count climbs, that's data gravity doing its work whether or not anyone made an explicit decision to let it.

What This Looks Like in Practice

A practical stack that resists data gravity by design:

Inference: llama.cpp for local, single-machine serving; vLLM or SGLang for production-grade self-hosted throughput. All three expose OpenAI-compatible APIs, so application code doesn't need to know which one is behind it.
Fine-tuning: datasets stored in standard formats - JSONL, parquet - never a provider's proprietary fine-tuning-job format.
Evaluation: framework-agnostic harnesses that score outputs, not API response envelopes, so the same eval suite runs whether the model is local or remote.
Tool calling: a provider-agnostic JSON schema translated to and from each vendor's tool-calling format, rather than application logic written directly against one vendor's shape.
Vector stores: local-first options such as Qdrant, Milvus, or Chroma, with embeddings computed through a portable library rather than tied to one provider's embedding endpoint - see chunking strategies in RAG for how this fits the retrieval layer.

This isn't a self-hosting manifesto. Plenty of workloads belong on a frontier API, permanently. It's a recognition that the engineers who can measure and manage data gravity - rather than discover it the day a vendor changes its pricing - end up with more options, not fewer.

The Bottom Line

Data gravity is why open-weight capability matters more than a single benchmark score. A model that runs locally at 85-95% of a frontier model's quality is often the better architectural choice, because you keep the data relationship. The frontier race between GPT-5.6 Sol, Fable 5, Kimi K3, and Qwen is genuinely interesting, but the infrastructure layer underneath it - who holds the fine-tuning data, whose schema the tools speak, whose pricing model the workflow assumes - is what actually determines which teams have options in three years and which ones are renting their architecture from someone else.

Score your lock-in before a vendor's pricing page forces the question for you.

Sources

Claude Code Subagents: Setup, Config, and When to Use Them

Rost — Wed, 05 Aug 2026 09:07:21 +0000

Most Claude Code sessions get slow and cluttered for the same reason: every exploratory grep, every log dump, and every "let me check one more file" stays in the main conversation forever.

Subagents exist to fix exactly that problem. They are one of the agent primitives built into Claude Code for handling noisy, parallelizable work — a way to push the mess into an isolated window and bring back only the summary that matters.

A subagent is not a smarter Claude, and it is not the same thing as a Skill. It is a separate reasoning agent with its own context window, its own tool allowlist, and no memory of your current conversation unless you explicitly fork it. Understanding that distinction is the difference between a subagent setup that quietly saves you context budget and one that just adds latency for no benefit.

Subagents vs Skills vs MCP

Claude Code gives you three extension points that solve different problems, and they get conflated constantly because all three can technically "help with a task."

Layer	What it is	When to reach for it
Skill	Instructions loaded into the main agent's context on demand	Reusable procedures, checklists, playbooks — see Claude Skills for developers
Subagent	A separate agent with its own context window, dispatched for delegated work	Noisy exploration, parallelizable research, anything you want kept out of the main session
MCP server	An external tool/data connector exposed over a protocol	Reaching systems outside the local session — APIs, databases, remote services

A useful rule of thumb: a hook enforces a hard constraint deterministically, a Skill gives the main agent a capability inline, and a subagent is for work you want to delegate and keep out of the main context entirely. If a Skill's job is to orchestrate a tool that doesn't exist yet, that is usually a sign you need an MCP server, not a subagent. Claude Code isn't alone in this shape — OpenCode's ecosystem has a comparable idea in its specialised agents, which split planning, research, and review across dedicated roles in a similar way.

What a subagent actually is

Three properties define a Claude Code subagent, and all three matter for how you use it:

Isolated context. A subagent starts with a fresh window. It does not see your conversation history unless you explicitly fork it, which keeps its output from being polluted by whatever you discussed three turns ago.
A restricted tool allowlist. Subagents can only use a subset of what the parent session already has — they cannot grant themselves new capabilities, and a well-designed subagent should get only the tools its job requires (read-only tools for a research agent, for example).
No cross-subagent visibility. Subagents cannot see each other's work in progress. If task B genuinely needs task A's output, that is a sequential dependency, not something you can parallelize across two subagents.

The trigger for reaching for one is not "this task is hard." It is "this task is noisy" — the kind of work that generates a lot of intermediate output (dozens of file reads, a long log, an exploratory grep across the whole repo) where none of that intermediate material needs to survive into your next conversation turn.

When to use a subagent (and when not to)

Good fits: codebase exploration before a big change, automated test runs where you only care about pass/fail and failure summaries, security or style reviews, and any multi-step research task whose raw output would otherwise flood your main session.

Bad fits: two-second lookups ("what does this function return"), anything requiring tight back-and-forth refinement, and dependent tasks you're tempted to "parallelize" even though the second one needs the first one's answer. Using a subagent for a trivial lookup just adds the overhead of spinning up a fresh context window for no real isolation benefit.

Measuring the payoff: context and cost math

The pitch for subagents is abstract until you put numbers on a real task. Take a common one: grep a ~500-file service for every place a deprecated config key is still read, then report the exact file:line matches.

Approach	Main-session context consumed	What survives into your next turn
Direct exploration, no subagent	~35-45K tokens — every grep hit, every file you opened to double-check, every dead end	All of it, including the wrong turns
Delegated to an Explore subagent	~1.5-3K tokens — one summarized report	Only the findings that mattered

That's roughly a 15-20x reduction in what your main session has to carry for that step, which is the actual mechanism behind "subagents keep sessions faster" — it is not magic, it is context that never gets loaded in the first place.

The cost side compounds the same way. Using the pricing from the Claude Code pricing breakdown, running that same exploration pass on Opus (\$5/MTok input, \$25/MTok output) costs roughly \$0.20-0.25 for the ~40K input tokens alone. Routing it to Haiku (\$1/MTok input, \$5/MTok output) drops that to \$0.04-0.05 — and the main session's Opus budget is never touched by the exploration tokens at all, since it only ever sees the ~2K-token summary.

Defining a custom subagent

Custom subagents live as Markdown files with YAML frontmatter, either project-scoped in .claude/agents/ (committed to the repo, shared by the whole team) or user-scoped in ~/.claude/agents/ (personal tools you bring to every project).

---
name: code-reviewer
description: >
  Reviews staged changes for bugs, security issues, and style violations
  before commit. Use when the user asks to review, audit, or check
  changes prior to committing or opening a PR.
tools: Read, Grep, Glob
model: sonnet
skills:
  - security-checklist
---
You are a careful code reviewer. Read the staged diff, flag concrete
issues with file:line references, and end with a short pass/fail summary.
Do not modify any files.

The description field is the most important line in the file. It is what the parent session's routing logic reads to decide whether this subagent fits the current task. Write it like a job posting — name the trigger condition explicitly, not a vague "helps with code." Vague descriptions get skipped or misapplied by automatic dispatch.

The tools field is your isolation boundary. Give a research subagent Read, Grep, and Glob and nothing else; giving it every tool available defeats the entire point of running it in a restricted sandbox. The optional skills field preloads the full content of named Skills into the subagent's startup context — useful when a subagent needs domain knowledge without spending a turn discovering and loading it mid-task.

Model routing: cheap models for grunt work

Subagents are also where cost control gets real. Route file discovery, log scanning, and other cheap-to-verify work to Haiku, and reserve Sonnet or Opus for the reasoning-heavy steps — architecture decisions, ambiguous debugging, anything where getting it wrong is expensive. Haiku is roughly 15x cheaper per token than Opus, and on the kind of noisy exploration subagents are built for, that gap adds up fast across a real working session.

The Explore, Plan, Execute pattern

For complex, multi-step work, the pattern that holds up in practice is Explore, Plan, Execute — using cheap subagents for the parts that generate noise, and keeping the human review gate in the one place it actually matters.

sequenceDiagram
    participant You
    participant Main as Main session
    participant Explore as Explore subagent (Haiku)
    participant Execute as Execute agent (Sonnet/Opus)

    You->>Main: Describe the task
    Main->>Explore: Delegate codebase exploration
    Explore-->>Main: Return summarized findings
    Main->>Main: Enter Plan mode, propose approach
    Main->>You: Show plan for review
    You->>Main: Approve or adjust
    Main->>Execute: Hand off approved plan
    Execute-->>Main: Apply changes, run tests
    Main-->>You: Report results

The key detail people get backwards is where the review gate belongs. Exploration is cheap, so let a subagent read freely without asking permission first. Planning is analytical, so let the agent design the approach on its own. But before any agent modifies files, you want to see the plan and approve it — that is what Claude Code's plan mode (permissionMode: plan) is for, and it is the same principle discussed in the broader vibe coding best practices around reviewing every diff before it lands.

Common mistakes

A handful of mistakes show up repeatedly once teams start writing custom subagents:

Vague descriptions. "Helps with code" will never route correctly. Name the exact trigger condition.
Over-broad tool access. Giving a read-only research subagent write and bash access removes the isolation guarantee that made it worth creating in the first place.
Parallelizing dependent tasks. If task B needs task A's finished output, run them sequentially — subagents cannot coordinate mid-task the way a shared orchestrator can. For workflows that genuinely need agents talking to each other mid-task, that's a different shape of problem; see multi-agent orchestration patterns if you're building a production system rather than a single-repo workflow.
Using a subagent for trivial work. "Format this JSON" or "run this one command" doesn't need a fresh context window; just do it directly.

Worked example: a code-review subagent end to end

Say you want every non-trivial commit reviewed before it lands. Drop the code-reviewer definition shown earlier into .claude/agents/code-reviewer.md, commit it so the whole team shares the same reviewer, and invoke it with a natural request like "review my staged changes before I commit." Claude Code matches your request against the subagent's description, spins it up with only Read, Grep, and Glob access, and it comes back with file:line-referenced findings and a pass/fail summary — none of the file-by-file noise from getting there ever touches your main session.

What that looks like in the main transcript, annotated:

You:  review my staged changes before I commit

Main: [dispatches code-reviewer subagent — 6 files read, 1 grep pass,
       zero of it shown here]

Main: code-reviewer findings:
      - auth/session.go:142 — token refresh path doesn't handle expired
        refresh token; falls through to nil dereference
      - auth/session.go:203 — style: error wrapped without %w
      PASS/FAIL: FAIL (1 blocking issue)

Six file reads and a grep pass happened, and your main session paid for exactly four lines of it. That gap — everything the subagent did versus the three-line summary you actually see — is the entire value proposition in one transcript.

If your team also uses Spec-Driven Development scaffolds, a review subagent slots naturally into the validation step; see GitHub Spec Kit vs Kiro vs Claude Code SDD Workflows for how that review gate compares across portable and IDE-integrated SDD setups.

Is it worth setting up custom subagents?

Not on day one. The built-in general-purpose subagent already covers most exploration and research delegation without you writing a single YAML file, and a single Explore-Plan-Execute pass is enough for most day-to-day work. Write a custom .claude/agents/*.md file only once you've delegated the same task by hand three times — a code reviewer, a test-runner triager, a docs-lookup agent for one specific internal library. Teams that write five subagents in their first week usually end up with five stale description fields nobody updates when the actual trigger condition drifts, which quietly breaks automatic routing months later. Start with zero custom subagents, add one at a time, and only when repetition — not theoretical usefulness — demands it.

Known limitations

A few rough edges are worth knowing before you build around subagents:

No recursive delegation. A subagent cannot spawn its own subagents. If a task genuinely needs a second layer of delegation, that is a sign you want a different orchestration shape — see multi-agent orchestration patterns for what that looks like outside a single Claude Code session.
No memory across invocations. Every dispatch starts from zero, even if you called the same subagent five minutes ago on a related task. There is no built-in mechanism for a subagent to remember its last run.
Isolation is a tool allowlist, not a sandbox. A subagent with Bash access can still touch the filesystem and network like any other tool call. Restricting tools reduces blast radius; it does not create a hard security boundary.

Troubleshooting

Subagent never triggers. The description is almost always the problem. Rewrite it around the specific trigger condition instead of a general capability statement, and double-check the file lives in .claude/agents/ (project) or ~/.claude/agents/ (personal) with the right extension.

Subagent burns too much context anyway. Check the tools allowlist — an overly broad toolset invites overly broad exploration. Also check whether the task should have been split into two subagents instead of one doing everything.

A listed skill doesn't load inside the subagent. Claude Code skips a missing or disabled skill named in the skills field rather than failing the run, and logs a line to that effect in the debug output (/debug from the main session, then reproduce the dispatch) — something like skill "security-checklist" not found, skipping. Run /doctor afterward to confirm the rest of your setup is healthy.

Results feel inconsistent between runs. This is often a model-routing issue, not a subagent-design issue — reasoning-heavy work assigned to a cheap model will vary more. Move it to Sonnet or Opus and keep Haiku for the deterministic, low-ambiguity steps.

Subagents are one piece of a much larger toolbox; if you're comparing Claude Code against the rest of the AI developer tools ecosystem before committing to this workflow, that overview is a good next stop.

Useful links

Ollama to vLLM: When to Migrate Your Local LLM Server

Rost — Sun, 02 Aug 2026 13:59:56 +0000

Ollama is one of the easiest ways to run a local language model, but convenience can conceal the moment when a local experiment becomes a shared inference service that needs better scheduling and observability.

That is where vLLM becomes relevant. Migrating from Ollama to vLLM is not an automatic upgrade, however. It is a trade: you exchange some of Ollama's simplicity for greater control over batching, memory management, concurrency, distributed inference, and production operations.

This guide covers the practical signals that indicate migration is warranted, the risks of moving too early, and a staged approach that keeps both servers running side by side during validation. The goal is to help you decide based on measurements rather than feature lists. For the wider landscape of local, self-hosted, and cloud options beyond just these two runtimes, see LLM Hosting in 2026: Local, Self-Hosted & Cloud Infrastructure Compared.

Ollama and vLLM Solve Different Problems

Ollama is primarily optimized for convenient model consumption. It gives developers a concise command-line interface, a local API, a model library, Modelfiles, and straightforward support for common desktop and workstation configurations.

vLLM is an inference engine and serving platform. Its central concerns are high-throughput request scheduling, efficient KV cache management, continuous batching, model parallelism, and compatibility with applications built for OpenAI-style APIs.

The distinction matters because the two servers can look similar from the outside. Both can expose a chat API, stream tokens, run quantized models, and serve local applications. Their operating models become visibly different only when the server is placed under sustained or concurrent load.

A useful summary:

Requirement	Ollama	vLLM
Fast local setup	Excellent	More involved
Curated model downloads	Excellent	Usually Hugging Face based
GGUF workflow	First-class	Supported, not main strength
Single-user chat	Excellent	Often unnecessary
Concurrent API traffic	Limited but configurable	Core use case
Continuous batching	Not the primary model	Core feature
Prefix cache reuse	Limited operational control	Built-in optimization
Multi-GPU model serving	Limited compared with vLLM	Tensor and pipeline parallelism
Production metrics	Basic response timing data	Prometheus metrics endpoint
Deployment tuning	Minimal	Extensive

The question is not which server is universally better. It is whether your workload still matches the operating model that makes Ollama attractive. If you want the fuller picture across more than these two runtimes, our comparison of Ollama, vLLM, LocalAI, Jan, LM Studio and other local LLM tools covers the broader field.

Signs You Have Outgrown Ollama

A slow response does not, by itself, justify a migration. Generation speed is often constrained by model size, quantization, memory bandwidth, prompt length, or GPU capability rather than the serving engine, and the stronger migration signals only appear once workload shape itself starts to matter.

Multiple Users Cause Unstable Latency

A local LLM server can feel fast during an isolated test and then degrade sharply when several clients connect. Requests begin waiting behind long generations, time to first token becomes inconsistent, and a single large prompt can affect everyone sharing the model.

Ollama can process parallel requests, and OLLAMA_NUM_PARALLEL controls how many requests a loaded model may handle concurrently — see how Ollama handles parallel requests for the queuing and memory mechanics behind that setting. That parallelism is not free: memory requirements grow with both the configured parallel request count and context length.

This is often the first practical warning. A configuration that works for one 8K conversation may become impossible when four clients each reserve a much larger context.

vLLM is designed to combine work from active requests through continuous batching. Instead of treating each request as an isolated inference job, it continuously updates the batch as sequences arrive, generate tokens, and finish — a scheduling model that generally becomes more valuable as concurrency increases.

GPU Utilization Is Low While Requests Are Queued

A queue does not necessarily mean that the GPU is fully used. In a simple serving arrangement, work may be serialized even though additional requests could have contributed useful computation to the current decode step.

vLLM's scheduler is designed to keep more useful work in flight. PagedAttention manages KV cache memory in blocks, while continuous batching allows active sequences to enter and leave the execution batch dynamically.

The result is not guaranteed to be lower latency for every individual request. Under load, however, it can produce substantially better aggregate throughput and more predictable resource utilization.

Long Prompts Dominate Time to First Token

Long-context coding assistants, RAG pipelines, and agent sessions can repeatedly send large system prompts or shared document prefixes. Processing those input tokens is the prefill stage, and it can dominate time to first token.

vLLM supports chunked prefill and automatic prefix caching. Prefix caching allows later requests to reuse KV cache blocks when their initial token sequence matches an already processed prefix.

This is particularly useful when requests share:

A long system prompt
The same tool definitions
A stable repository summary
Repeated few-shot examples
A common RAG document prefix
A shared conversation history

Prefix caching does not make output generation faster. It reduces repeated prompt computation, so its benefit depends on whether requests actually contain identical reusable prefixes.

You Need More Than One GPU

A model that does not fit on one GPU is a strong reason to consider vLLM. It supports tensor parallelism across GPUs and pipeline parallelism across multiple nodes or devices.

This does not make multi-GPU inference effortless. GPU interconnect bandwidth, PCIe topology, model architecture, container shared memory, and communication overhead still affect performance.

Nevertheless, vLLM provides a deliberate path for distributed inference. Ollama is usually a better match for a single desktop or workstation where the chosen model already fits comfortably.

You Need Production-Level Observability

Ollama API responses expose useful timing fields such as model load duration, prompt evaluation duration, generated token count, and generation duration. These values are enough for local benchmarking and application-level logging.

vLLM exposes Prometheus-compatible metrics through its /metrics endpoint. That makes it easier to track request volume, queueing, time to first token, inter-token latency, cache usage, preemptions, throughput, and request outcomes over time.

Once users depend on the service, observability stops being optional. Without queue, cache, and latency metrics, it is difficult to distinguish an undersized GPU from an oversized context limit, poor scheduling, cold model loading, or simply too many simultaneous requests.

Where vLLM Actually Wins

vLLM's most important advantage is not that it can produce one response faster than Ollama on every machine. The meaningful advantage is that it gives the operator more mechanisms for using expensive accelerator memory and compute efficiently across many requests.

Continuous Batching

Traditional static batching works best when requests have similar input and output lengths. Interactive LLM traffic rarely behaves that way: one user asks for a short classification, another submits a 20K-token prompt, and a third generates several thousand tokens of code.

Continuous batching changes the active batch as requests progress. Completed sequences leave, new sequences enter, and the engine attempts to avoid wasting batch capacity on requests that have already finished.

This improves throughput when traffic is concurrent and uneven. It provides little benefit when a single user sends one request at a time.

Paged KV Cache Management

During generation, the server stores attention keys and values for previously processed tokens. This KV cache can consume a large amount of GPU memory, especially with long contexts and multiple active sequences.

vLLM manages this cache in blocks instead of requiring each sequence to reserve one large contiguous allocation. The approach reduces memory fragmentation and allows available cache capacity to be used more flexibly.

The practical value is higher concurrency within the same memory budget. It does not remove the underlying cost of long context, but it reduces avoidable waste around that cost.

Prefix Caching

Many production requests share a substantial beginning. Tool-enabled agents may send identical function schemas, support bots may use the same policy documents, and coding assistants may repeatedly include the same repository instructions.

Automatic prefix caching can reuse the computed cache for matching prefixes. It is especially useful when a stable, large prefix is followed by a relatively small request-specific suffix.

It is less useful when templates, timestamps, document ordering, or dynamically generated metadata change near the beginning of every prompt. Small differences in tokenization can prevent the prefix from matching.

Parallel and Distributed Inference

vLLM supports several forms of parallelism, including tensor, pipeline, data, expert, and context parallelism. Not every deployment needs these modes, but their availability matters when a service grows beyond one GPU.

For a workstation with two suitable GPUs, tensor parallelism may allow a larger model to run across both devices. For a replicated service, data parallelism can create multiple engine replicas for additional throughput.

These features introduce operational complexity. They should be adopted because measurements demonstrate a capacity problem, not because distributed inference appears more sophisticated.

Broader Production Controls

vLLM exposes controls for GPU memory utilization, maximum model length, maximum active sequences, quantization, cache data types, speculative decoding, tool calling, structured output, model aliases, authentication keys, and distributed execution.

That flexibility makes the server easier to tune for a particular workload, but it also creates more opportunities for an invalid or inefficient configuration. Migrating to vLLM means taking responsibility for those decisions.

Where Ollama Still Wins

A migration guide should not treat Ollama as an inferior preliminary tool. For many local deployments, it remains the better server.

Personal Workstations

For one developer using a chat interface, code assistant, or occasional local API, the operational advantages of vLLM may never compensate for its additional setup.

Ollama installs quickly, downloads models through a simple registry, and hides many model-specific details. It is well suited to experimentation and private desktop use.

GGUF Model Collections

Ollama has a natural workflow around GGUF models and Modelfiles. Existing users may have curated quantizations, adapters, templates, system prompts, and parameters that work reliably with their hardware.

vLLM supports GGUF, but its strongest path is generally through supported Hugging Face model repositories and quantization formats such as AWQ, GPTQ, BitsAndBytes, FP8, or vendor-specific formats. Moving an existing GGUF deployment to vLLM without evaluating a more native checkpoint format can preserve the inconvenience of migration while missing some of the performance advantages.

Mixed CPU and GPU Offloading

Desktop inference sometimes relies on partial GPU offloading because the entire model does not fit in VRAM. This can be practical for occasional use, particularly when latency is not critical.

vLLM is generally most compelling when the model and required KV cache capacity can be served effectively by the available accelerator configuration. A workload that depends heavily on system RAM and CPU offloading may be better suited to Ollama or llama.cpp.

Rapid Model Switching

Ollama makes it easy to pull, run, stop, and switch among many local models. That is useful for evaluation, writing, coding, embeddings, vision, and ad hoc experimentation.

A vLLM deployment is more commonly built around a deliberately selected model that remains loaded as a service. Multi-model deployment is possible, but it requires more explicit resource planning.

Minimal Administration

Ollama is intentionally opinionated. That can be a limitation under load, but it is an advantage when nobody wants to maintain an inference platform, and if the local server has one user, acceptable latency, and no meaningful queue, migration is likely to create work rather than remove it.

Do Not Migrate Based on Tokens per Second Alone

Single-request token generation speed is an incomplete benchmark. Two servers may produce similar decode throughput for one sequence while behaving very differently with eight concurrent clients.

A useful evaluation should measure at least:

Time to first token
Inter-token latency
End-to-end request latency
Prompt processing throughput
Output token throughput
Requests completed per minute
Queue wait time
GPU memory consumption
GPU utilization
Failure and timeout rate

Run the same model family, precision, context length, prompt set, output limit, and concurrency level on both servers. Otherwise, the test is more likely to compare model packaging and configuration than serving engines.

The most useful comparison is a small load test that represents your real traffic. For a shared coding assistant, that might include long system prompts, repeated prefixes, streaming responses, and two to eight simultaneous sessions.

Plan the Model Migration First

Ollama model names do not automatically map to equivalent vLLM model identifiers. An Ollama package may contain a particular GGUF quantization, prompt template, stop-token configuration, and default parameters.

Before changing the server, identify:

The original model family and version
Whether it is a base or instruction-tuned model
The current quantization and effective precision
The prompt or chat template
The configured context length
Stop tokens and generation defaults
Tool-calling or structured-output requirements
Any LoRA adapters or custom system prompts

Then choose a vLLM-supported checkpoint that matches the intended behavior. Do not assume that an AWQ or FP8 checkpoint will behave identically to the GGUF build previously used in Ollama — the model migration is often more significant than the API migration.

Check VRAM Before Starting vLLM

A model fitting into GPU memory does not mean that it can serve the required workload. VRAM must cover more than model weights.

The practical memory budget includes:

model weights
+ KV cache
+ CUDA graphs and runtime allocations
+ temporary workspace
+ multimodal processor caches, if used
+ safety margin

Long contexts and concurrent sequences primarily expand the KV cache requirement. Increasing the maximum context length therefore reduces the number of simultaneous requests that can fit, even if most requests never use the full limit.

Start with a realistic --max-model-len rather than the largest value advertised by the model, and avoid setting GPU memory utilization so aggressively that minor workload variation causes out-of-memory failures. A stable service with slightly less theoretical capacity is more useful than one that fails at its first traffic spike.

A Minimal vLLM Docker Compose Deployment

The following example starts an OpenAI-compatible vLLM server on port 8000:

services:
  vllm:
    image: vllm/vllm-openai:latest
    container_name: vllm
    restart: unless-stopped
    ports:
      - "8000:8000"
    ipc: host
    gpus: all
    volumes:
      - ${HOME}/.cache/huggingface:/root/.cache/huggingface
    environment:
      HF_TOKEN: ${HF_TOKEN:-}
    command:
      - --model
      - Qwen/Qwen3-8B
      - --served-model-name
      - local-model
      - --max-model-len
      - "16384"
      - --gpu-memory-utilization
      - "0.90"
      - --api-key
      - ${VLLM_API_KEY:-change-me}

Create an environment file:

cat > .env <<'EOF'
HF_TOKEN=
VLLM_API_KEY=replace-with-a-long-random-value
EOF

Start the server:

docker compose up -d

Check the logs:

docker compose logs -f vllm

Test the models endpoint:

curl http://localhost:8000/v1/models \
  -H "Authorization: Bearer replace-with-a-long-random-value"

Send a chat request:

curl http://localhost:8000/v1/chat/completions \
  -H "Authorization: Bearer replace-with-a-long-random-value" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "local-model",
    "messages": [
      {
        "role": "user",
        "content": "Explain continuous batching in two paragraphs."
      }
    ],
    "temperature": 0.2,
    "max_tokens": 300,
    "stream": false
  }'

For a maintained deployment, pin the image to a tested vLLM release instead of leaving it on latest. Review release notes before upgrading because command-line options, model implementations, metrics, and engine behavior can evolve. This Compose file is intentionally minimal; for the fuller setup guide — OpenAI API compatibility, PagedAttention tuning, and a deeper vLLM-vs-Ollama comparison — see the vLLM Quickstart.

OpenAI API Compatibility Is Not Complete Interchangeability

Both Ollama and vLLM provide OpenAI-compatible endpoints, which can make the application migration relatively small. In many clients, changing the base URL, API key, and model name is enough to establish a connection.

For example:

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8000/v1",
    api_key="replace-with-a-long-random-value",
)

response = client.chat.completions.create(
    model="local-model",
    messages=[
        {
            "role": "user",
            "content": "What should I monitor on an LLM server?",
        }
    ],
    temperature=0.2,
)

print(response.choices[0].message.content)

Compatibility should still be tested at the feature level. Examine:

Streaming event behavior
Supported request parameters
Chat template selection
Tool-call parsing
Reasoning output handling
JSON or schema-constrained output
Embeddings endpoints
Multimodal inputs
Token usage reporting
Error response formats
Model name discovery
Context-length enforcement

A client that only sends ordinary chat completions will usually be easier to migrate than an agent framework that depends on a particular tool-call parser or nonstandard extension.

Chat Templates Are a Common Migration Failure

Instruction-tuned models expect conversations to be serialized using a specific chat template. The template inserts role markers, separators, control tokens, and generation prompts in the format used during training.

Ollama packages much of this behavior inside its model definition. With vLLM, the template is normally obtained from the model tokenizer configuration, although an operator can provide one explicitly.

A server may start successfully even when the selected template is wrong. The symptoms appear in model behavior:

The model repeats role labels
Responses contain special tokens
System instructions are ignored
Tool calls are malformed
The model continues the user message
Output quality is much worse than expected

Before blaming the inference engine, compare the fully rendered prompt used by each deployment.

Use a Staged Migration

Replacing a working local server in one step creates unnecessary risk. Ollama and vLLM can run side by side on different ports while you validate the new deployment.

Stage 1: Reproduce One Model

Choose the model responsible for most API traffic and match its instruction tuning, context requirement, generation parameters, and chat behavior as closely as possible. Do not begin by moving every experimental model.

Stage 2: Validate API Behavior

Run existing integration tests against the vLLM endpoint, including streaming, cancellation, timeouts, tool calls, malformed requests, context overflow, and concurrent access. Record behavioral differences rather than hiding them behind client retries.

Stage 3: Establish a Baseline

Measure one-request performance first. This confirms that the model is loaded correctly and provides a reference for later tests.

Record prompt tokens per second, output tokens per second, time to first token, total latency, and GPU memory usage.

Stage 4: Add Realistic Concurrency

Test the number of simultaneous requests expected in normal operation and during a plausible peak, using representative prompt and output lengths rather than identical synthetic requests. Watch queueing, cache use, preemptions, time to first token, and tail latency.

Stage 5: Move One Client

Route a noncritical application or a small percentage of traffic to vLLM. Keep Ollama available as a fallback until the new server has operated reliably under real use.

Stage 6: Tune From Measurements

Adjust model length, memory utilization, maximum active sequences, prefix caching, parallelism, and quantization only after identifying a measured constraint. Changing several parameters at once makes performance regressions difficult to explain.

A Practical Migration Checklist

Before switching clients, verify the following:

[ ] The target model is supported by vLLM
[ ] The selected checkpoint and quantization fit in VRAM
[ ] Enough VRAM remains for the required KV cache
[ ] The maximum context length reflects real usage
[ ] The correct chat template is available
[ ] Stop tokens and generation defaults are tested
[ ] Streaming works with existing clients
[ ] Tool calls and structured output are validated
[ ] The public model alias remains stable
[ ] Authentication is enabled
[ ] The server is not exposed directly to the internet
[ ] Prometheus metrics are collected
[ ] GPU metrics are collected separately
[ ] Load tests include realistic concurrency
[ ] Timeouts and cancellations are handled
[ ] A rollback path to Ollama exists

This list is deliberately operational. Installing vLLM is usually easier than proving that it behaves correctly for an existing application.

Security and Network Exposure

Neither a local Ollama endpoint nor a vLLM endpoint should be casually exposed to the public internet. An unauthenticated inference server can consume expensive GPU capacity, reveal model behavior, and become a route for denial-of-service attacks through very long prompts or outputs.

vLLM can require an API key for its OpenAI-compatible endpoints, but an API key is not a complete security boundary. For shared or remote access, place the service behind a reverse proxy or API gateway that provides TLS, network restrictions, request-size limits, rate limits, access logging, and appropriate authentication — the same pattern covered in Ollama behind a reverse proxy with Caddy or Nginx applies just as well in front of vLLM.

Also consider model-specific risks. Multimodal URL loading, custom model code, remote files, and unrestricted tool execution can expand the attack surface beyond ordinary text generation.

When Not to Migrate

Stay with Ollama when:

One or two users access the server
Requests are mostly sequential
The model already delivers acceptable latency
Easy GGUF management is important
CPU or partial GPU offloading is required
Models are changed frequently
Nobody wants to operate additional infrastructure
There is no measured concurrency or throughput problem

A move to vLLM should solve a concrete limitation. "Production" is not a magic threshold that invalidates Ollama, especially for an internal service with modest traffic.

Conversely, do not preserve Ollama merely because it was easier to install. If users regularly wait in a queue, repeated prefixes consume significant prefill time, or a larger model must be distributed across GPUs, the simpler server may have become the more expensive choice operationally.

Keep Ollama for Development and Add vLLM for Shared Serving

The most practical architecture is often not a complete replacement. Developers can keep Ollama running in Docker Compose on their workstations for model exploration, GGUF testing, and private interactive use while a shared vLLM instance serves a stable model to applications and teams. That split also matters for AI sovereignty — keeping both runtimes self-hosted means prompts, weights, and inference logs stay under your control regardless of which server handles a given request.

This separates two different workflows:

Ollama:
experimentation -> model switching -> personal tools -> local chat

vLLM:
selected model -> shared endpoint -> concurrent traffic -> monitoring

The arrangement also lowers migration risk. Models can be tested locally before a suitable checkpoint is promoted to the shared vLLM deployment.

Migration Decision Flow

The following diagram summarizes the key decision points:

flowchart TD
    A[Ollama serving LLM] --> B{Multiple users<br>with unstable latency?}
    B -->|No| C[Stay with Ollama]
    B -->|Yes| D{Long shared<br>prefixes?}
    D -->|Yes| E[Strong vLLM signal]
    D -->|No| F{Need multi-GPU<br>or observability?}
    F -->|Yes| E
    F -->|No| G{Measured concurrency<br>problem?}
    G -->|No| C
    G -->|Yes| E
    E --> H[Plan staged migration]
    H --> I[Validate side by side]
    I --> J[Switch clients gradually]

Conclusion

Ollama is difficult to beat as a local model runner. It removes enough packaging and configuration work that developers can concentrate on the model and application rather than the inference stack.

vLLM becomes the stronger choice when the server itself is the problem to be engineered. Concurrent traffic, queueing, repeated long prefixes, multi-GPU models, capacity planning, and production observability are the migration signals that matter.

Do not migrate because vLLM has a longer feature list. Migrate when measurements show that Ollama's simpler operating model no longer matches the workload. Until that point, simplicity is not a technical weakness; it is an optimization.

Keeping Specs, Tests, And Code In Sync In AI Development

Rost — Sat, 01 Aug 2026 10:43:20 +0000

AI coding agents ship features fast, but specs, tests, and code quietly drift apart. This guide covers a traceability model, spec-to-test and spec-to-code mapping, and the CI checks that catch drift before a merge.

A spec that nobody re-checks against the running system is worse than no spec at all, because it creates false confidence. Reviewers trust the document instead of the diff, and an AI agent asked to "follow the existing pattern" will happily follow whatever the code actually does, even when that contradicts the requirement it was supposed to satisfy.

The fix is not more documentation. It is a small, enforceable link between four things that already exist in most repositories: the requirement, the design decision behind it, the tests that prove it, and the commits or pull requests that changed it.

Once that link exists as data rather than as a shared understanding, you can query it. You can ask which requirements have no test coverage, which tests no longer map to any requirement, and which files changed in a pull request without a matching requirement ID. That query is the actual deliverable of this article, and the rest of the post walks through how to build it with tools you likely already run.

The Drift Problem: Why Specs, Tests, And Code Fall Out Of Sync

Drift shows up in four recognisable shapes, and AI-assisted teams tend to hit all four faster than teams that write every line by hand.

Spec changes, code does not. A requirement gets clarified in a follow-up conversation or a comment thread, but nobody regenerates or edits the implementation to match.
Code changes, spec does not. An agent or a developer fixes a bug or refactors a module, and the spec keeps describing the old behavior as if it were still current.
Tests cover implementation, not intent. Unit tests assert what the code currently does, which is circular: they pass by construction even when the code satisfies the wrong requirement.
Pull requests do not reference requirements. Reviewers approve a diff on the strength of "looks reasonable" because there is no explicit claim to check it against.

Recent process research on AI development frameworks identifies specification drift as a recurring risk precisely because agents regenerate code quickly and repeatedly, and each regeneration is a fresh opportunity for spec and implementation to diverge a little further. The Spec-Driven Development vs Vibe Coding debate is really an argument about this same failure mode: a spec that nobody enforces degenerates into the same drift you get without one, just with extra ceremony.

Modern spec-kit-style workflows increasingly frame this as specification rot: the spec keeps looking authoritative while quietly losing its connection to what the system actually does. The core definition of spec-driven development treats the spec as the source of truth, but a source of truth only stays true if something keeps checking it against reality.

A Traceability Model For AI-Assisted Development

A workable traceability model needs six identifiers that connect a business requirement all the way down to the lines of code and the pull request that implemented it. Most teams already have three or four of these; the missing ones are usually the design decision ID and the explicit link back from tests and commits.

Identifier	Lives in	Example
Requirement ID	`requirements.md` or spec tool	`REQ-014`
Design decision ID	ADR / decision record	`ADR-0032`
Task ID	task breakdown or issue tracker	`TASK-014-3`
Test ID	test file or test name	`test_req_014_password_reset`
Commit / PR link	Git history	`PR #482`
Changed files	Git diff	`auth/reset.go`, `auth/reset_test.go`

The relationships between these identifiers form a graph rather than a straight line, because one requirement can spawn several tasks, and one pull request can touch several requirements at once.

graph TD
    REQ["Requirement<br/>REQ-014"] --> ADR["Design Decision<br/>ADR-0032"]
    ADR --> TASK["Task<br/>TASK-014-3"]
    TASK --> CODE["Code Change<br/>auth/reset.go"]
    TASK --> TEST["Test<br/>test_req_014_password_reset"]
    CODE --> PR["Pull Request<br/>#482"]
    TEST --> PR
    PR --> COMMIT["Commit history"]

Storing this graph as structured data, not prose, is what lets you query it later. GitHub's Spec Kit ecosystem has moved in exactly this direction: extensions like spec-kit-trace scan REQ-XXX tokens embedded in spec files and test files and generate a deterministic matrix from that literal text match, deliberately avoiding fuzzy name-based guessing that produces silent false positives.

Spec-To-Test Mapping: Turning Acceptance Criteria Into Test Cases

Every acceptance criterion in a spec is, by construction, a behavioral assertion: given this state, when the actor does this, then the system should respond that way. That is already the shape of a test case, which is why the strongest SDD workflows generate tests from the same acceptance criteria that generate the code, instead of asking the code-generating agent to also invent its own tests after the fact.

A widely used format for writing these criteria is EARS (Easy Approach to Requirements Syntax), which forces each requirement into an unambiguous, testable pattern such as "When <trigger>, the system shall <response>." That structure maps cleanly onto four categories of test that every requirement should carry:

Positive tests — the happy path the requirement explicitly describes.
Negative tests — inputs or states the requirement says must be rejected.
Boundary tests — the edges of ranges, limits, and thresholds mentioned in the acceptance criteria.
Migration tests — behavior for data or state that predates the requirement, so an old record does not silently bypass a new rule.

Requirement type	Test category to add	Common miss
"System shall reject X"	Negative	Only the accept path is tested
"Limit is N items"	Boundary	N-1, N, and N+1 are not all covered
"New field replaces old field"	Migration	Old records with no new field crash silently
"Within 60 seconds"	Boundary + timing	Test asserts logic, not the actual time budget

Unit tests written this way still matter as the fast, cheap layer of the pyramid; the practical patterns for structuring them are covered in the Go unit testing guide and the Python unit testing guide. What traceability adds on top is a literal, stable requirement token embedded in the test name or a test comment, so a later query can prove — not assume — that REQ-014 has coverage.

Spec-To-Code Mapping: From Design Plans To A Trace Table

Spec-to-test mapping proves behavior; spec-to-code mapping proves scope. It answers a different question: which files were actually supposed to change for this requirement, and did the diff stay inside that boundary or spill into unrelated modules?

A design plan that lists affected files up front — even a rough list — gives you something to diff the real pull request against later. Comments in code should only reference a requirement ID when doing so adds information a reviewer cannot get from the spec itself; a comment repeating the requirement text verbatim is noise, but // enforces REQ-014 boundary: max 5 reset attempts per hour earns its place because the number is otherwise invisible in the diff.

A generated trace table turns this into something reviewable in seconds rather than something a reviewer has to reconstruct by reading both documents side by side:

Requirement	Design decision	Files changed	Tests	Status
REQ-014	ADR-0032	`auth/reset.go`, `auth/reset_test.go`	`test_req_014_*` (4)	Covered
REQ-015	ADR-0032	`auth/reset.go`	none	Gap
REQ-016	—	`auth/notify.go`	`test_notify_basic`	Orphan spec link

That single table surfaces two of the most common failure patterns at a glance: REQ-015 changed code with zero matching tests, and the test attached to REQ-016 does not actually reference a requirement ID, which means either the spec is missing or the test was misfiled.

The Pull Request Workflow: Reviewing Spec, Code, And Test Diffs Together

A pull request built around traceability reviews three diffs side by side instead of one: what changed in the spec, what changed in the code, and what changed in the tests. The review question stops being "does this look right?" and becomes the far more specific "which requirement does this change satisfy, and does the evidence prove it?"

sequenceDiagram
    participant Dev as Developer or Agent
    participant PR as Pull Request
    participant CI as CI Pipeline
    participant Rev as Reviewer

    Dev->>PR: Open PR with spec diff + code diff + test diff
    PR->>CI: Trigger traceability checks
    CI->>CI: Verify REQ-ID present in PR description
    CI->>CI: Run spec-to-test coverage check
    CI->>CI: Run spec-to-code file-scope check
    CI-->>PR: Post trace report as PR comment
    Rev->>PR: Review against "which requirement does this satisfy?"
    Rev->>PR: Approve or request changes

A short, concrete reviewer checklist works better here than a long one, because reviewers skip long checklists under deadline pressure:

Does the PR description name the requirement ID(s) it satisfies?
Does every changed file appear in the design plan's affected-files list, or is the extra scope explained?
Does at least one new or existing test reference each requirement ID touched by this PR?
If the spec changed, did the code and tests change in the same PR, or is there a tracked follow-up?

Automating Traceability In CI

Manual review catches drift only as often as reviewers remember to look for it, which is why the checks above belong in CI rather than in a wiki page nobody re-reads. The same GitHub Actions cheatsheet patterns you already use for build and test jobs apply directly here — traceability checks are just another job in the same pipeline.

Practical automation ideas, roughly in order of effort:

CI checks for spec files — fail the build if a spec file was edited without a corresponding code or test change in the same PR, or vice versa.
Require requirement IDs in PR titles or descriptions — a lightweight regex check (REQ-\d+) blocks merges that don't name what they implement.
Agent-generated trace summaries — have an agent produce a short summary of which requirements a PR touches, for a human to confirm rather than write from scratch.
Test coverage by acceptance criterion, not just by line — line coverage tells you code ran; requirement coverage tells you a claim was checked.
Stale spec warnings — flag specs that have not been touched in N commits touching their linked files, since long-silent specs are the ones most likely to have quietly rotted.

Extensions built on top of GitHub's Spec Kit already implement several of these mechanically: one scans literal REQ-XXX tokens across spec and test files to build a matrix and flag orphaned tests, and a stricter V-Model-oriented pack goes further, generating a paired test specification for every development specification and producing multiple traceability matrices for teams working under regulatory frameworks such as IEC 62304 or ISO 26262. You do not need that level of ceremony for most projects, but the underlying idea — a deterministic, script-generated matrix rather than a hand-maintained spreadsheet — scales down just as well as it scales up.

Using AI Agents For Traceability, Not As An Oracle

AI agents are well suited to the mechanical parts of traceability and poorly suited to being the final judge of whether a requirement was actually satisfied. Three tasks fit an agent's strengths directly:

Compare spec and diff — ask the agent to list every requirement mentioned in the spec files touched by a PR, and every one it did not find corresponding code for.
Find uncovered requirements — ask the agent to scan the test suite for requirement tokens and report which requirements in the spec have none.
Detect code not described by spec — ask the agent to flag changed files or functions that touch requirement-bearing modules but do not correspond to any requirement ID in the diff.

The failure mode to guard against is trusting the agent's summary as ground truth instead of as a reviewer's starting point. An agent can misread a comment, miss a requirement token split across two files, or confidently declare coverage for a test that only exercises the code path superficially. Treat every agent-generated trace report the way you would treat a junior reviewer's pass: useful, fast, and still subject to a second look before it gates a merge. This is the same caution that applies to decision records for AI-driven development — the record only stays trustworthy if something other than the agent that wrote it eventually checks it.

A Minimal Traceability Template You Can Copy

You do not need a heavyweight framework to start. A five-file template, checked into the repository next to the code it describes, covers the essentials:

docs/
  requirements.md     # REQ-IDs with EARS-style acceptance criteria
  design.md           # ADR-IDs, affected files, architecture decisions
  tasks.md            # TASK-IDs mapped to one or more REQ-IDs
  tests.md            # which test files/functions reference which REQ-IDs
  traceability.md     # generated table: REQ -> ADR -> TASK -> files -> tests -> PR

requirements.md, design.md, and tasks.md are written or edited by humans and agents together, the same way the spec-driven development workflow already describes. tests.md and traceability.md should be generated, not hand-maintained, even if the generator is a short script that just greps for REQ-\d+ across the test directory and the spec files — hand-maintained trace tables are themselves a form of drift risk, because nobody updates a spreadsheet under deadline pressure.

Conclusion

Spec-driven development is not finished the moment code comes out of an agent; it is only useful once code, tests, and specs keep each other honest over time, through PRs, refactors, and requirement changes that arrive months apart. A traceability model built from six plain identifiers, enforced by a handful of CI checks, and reviewed with a short PR checklist gets you most of the benefit without the overhead of a full compliance framework. Start with the minimal template, wire the cheapest CI check first — requirement IDs in PR descriptions — and add the trace table and stale-spec warnings once that habit sticks.

Traceability is one piece of a larger testing and documentation discipline covered across the App Architecture in Production cluster, and it sits alongside the tooling questions explored in the AI developer tools cluster for teams choosing which agent workflows to standardize on.

Dead Letter Queues: Handling Poison Messages in Distributed Systems

Rost — Fri, 31 Jul 2026 13:26:50 +0000

A dead-letter queue is the safety net that catches messages your consumers cannot process, so one broken payload does not block or silently drop everything behind it in the queue.

Every message-driven system eventually receives a message it cannot handle: a malformed payload, a schema that changed underneath the consumer, or a downstream call that fails no matter how many times you retry it. Without a dead-letter queue, that message either blocks the head of the queue forever or gets silently discarded, and both outcomes are worse than knowing about the failure.

A DLQ turns an invisible failure into a visible, inspectable one. It gives you a place to quarantine the message, alert on it, and decide — deliberately, not by accident — whether to fix and replay it or discard it for good.

The mechanics differ across brokers, but the underlying pattern is the same everywhere: a delivery-attempt counter, a threshold, and a destination for messages that cross it. This guide covers what a DLQ actually does, how to tell a poison message from a transient failure, when to retry versus discard, and how to replay safely once you have fixed the root cause. For the broader integration-patterns context this pattern sits inside, see the App Architecture.

What Is a Dead Letter Queue

A dead-letter queue is a separate, ordinary queue that a broker or consumer routes a message to after that message fails processing too many times. It is not a special construct — RabbitMQ's dead-letter queue is a regular queue bound to a regular exchange, and an SQS DLQ is a regular standard or FIFO queue. What makes a queue a "DLQ" is purely that something else points failed messages at it.

flowchart LR
    P[Producer] --> Q[Main Queue]
    Q --> C[Consumer]
    C -- ack: success --> Done[Message deleted]
    C -- fail / nack / timeout --> Q
    Q -- retry budget exhausted --> DLQ[Dead Letter Queue]
    DLQ --> I[Inspect / alert]
    I -- fix root cause --> R[Replay to main queue]
    I -- unrecoverable --> D[Archive / discard]

Each broker implements the redirect differently:

Amazon SQS uses a redrive policy with a maxReceiveCount. Once a message has been received that many times without being deleted, SQS moves it to the configured deadLetterTargetArn. AWS explicitly recommends keeping the DLQ's message retention period longer than the source queue's, because the original enqueue timestamp — not the move time — still governs expiry.
RabbitMQ dead-letters a message when it is rejected with requeue=false, its per-message TTL expires, the queue hits a length limit, or a quorum queue exceeds its delivery-limit. You configure this with the x-dead-letter-exchange (and optionally x-dead-letter-routing-key) queue arguments, and RabbitMQ attaches x-death headers recording the reason, the origin queue, and how many times it happened.
Apache Kafka has no broker-native DLQ. Kafka only tracks offsets; it has no concept of a "failed" message. The dead-letter topic pattern is something you build in the consumer, in a Kafka Streams topology, or in a Kafka Connect connector — commonly paired with a retry-topic tier before the terminal DLT, as Spring Kafka's @RetryableTopic and DeadLetterPublishingRecoverer do.
Azure Service Bus dead-letters automatically once a message's delivery count exceeds MaxDeliveryCount (default 10), and also for a handful of system reasons such as TTLExpiredException, HeaderSizeExceeded, and MaxTransferHopCountExceeded, each recorded in the message's DeadLetterReason property.

For a broader view of how brokers and streaming platforms fit together operationally rather than as a reliability pattern, Apache Kafka Quickstart and RabbitMQ on AWS EKS vs SQS cover the infrastructure side of running these brokers.

Poison Messages

A poison message is one that will never succeed no matter how many times a consumer retries it — a malformed JSON payload, a schema field that a producer renamed, a business rule violation, or a bug that throws on a specific input every single time. That is different from a transient failure, where the message is fine but the environment briefly is not: a downstream timeout, a database connection blip, a rate limit response.

Treating both failure types the same way is the most common DLQ mistake. If you dead-letter on the first failure, you punish transient errors that would have succeeded on retry. If you retry poison messages dozens of times before giving up, you waste compute, delay unrelated messages behind them (on ordered queues and partitions), and flood your logs with the same stack trace.

A few detection signals help separate the two:

Exception type. Deserialization errors, validation errors, and ClassCastException-style failures are almost always permanent. Spring Kafka's DefaultErrorHandler explicitly treats certain exceptions as fatal and skips retries for them rather than exhausting the retry budget first.
Repeat count with no variance. RabbitMQ's x-death header array lets you see exactly how many times a message has been dead-lettered and why; a message with a growing count and an identical x-first-death-reason on every cycle is poison, not unlucky.
Consistent failure across replicas. If every consumer instance fails on the same message while succeeding on everything around it, the message itself is the problem, not the infrastructure.

For distinguishing retryable from non-retryable failures at the code level — the same classification a DLQ policy depends on — see Go Error Handling Architecture: Boundaries and Patterns.

Retry vs Discard

The core policy decision behind every DLQ is the retry threshold: how many delivery attempts a message gets before it is quarantined. Get this too low and you dead-letter messages that would have succeeded after a brief downstream hiccup. Get it too high and a poison message sits in the main queue for a long time, consuming worker capacity and — on ordered systems — blocking everything queued behind it.

Current guidance across the major brokers converges on similar numbers:

Broker	Mechanism	Typical threshold
Amazon SQS	`maxReceiveCount` in redrive policy	3–5 for mixed transient/permanent workloads
RabbitMQ (quorum queues)	`delivery-limit` policy argument	3–5, tuned per queue
Azure Service Bus	`MaxDeliveryCount`	Default 10, often reduced for latency-sensitive queues
Kafka (via retry topics)	Retry-count header + retry-topic tier	3–4 retry-topic hops before the terminal DLT

A practical middle ground many teams land on is: start conservative (2–3 attempts), watch the actual failure mix in production, and raise the threshold only for queues where you can show most failures resolve within a few retries. Pair the retry count with exponential backoff and jitter between attempts so a downstream outage does not turn into a retry storm — the same discipline covered in backoff and circuit-breaker design. A circuit breaker at the integration boundary complements this: it stops sending requests to an unhealthy dependency instead of letting every message in the queue individually discover the outage and dead-letter one by one.

Once a message is in the DLQ, "discard" should still be a deliberate action, not neglect. Set a retention period on the DLQ itself — long enough to investigate (AWS recommends the DLQ retention exceed the source queue's; a week is a common floor for RabbitMQ DLQs) — and alert on DLQ depth and age so failures get triaged instead of silently expiring. A message that ages out of the DLQ unexamined is a message you decided to lose without deciding to lose it.

Idempotency matters just as much here as it does anywhere else duplicates can occur: a message that gets redriven from a DLQ back to the main queue is, functionally, a duplicate delivery. If your consumer is not safe to run twice on the same message, redriving from a DLQ can create the exact duplicate-side-effect bug you were trying to avoid. See Idempotency in Distributed Systems That Actually Works for the consumer-side patterns that make redrive safe.

Replay Strategies

Getting a message out of the DLQ correctly is its own discipline, separate from getting it in.

Fix the root cause first. Deploying the consumer fix before replaying is the difference between a clean recovery and re-poisoning the queue with the same failure a second time.
Redrive deliberately, not automatically. SQS supports a redrive-to-source feature that moves messages back to their original queue (or another destination) on demand; RabbitMQ and Kafka require you to build the equivalent consumer or tooling yourself. Either way, treat replay as an operator-triggered action with a record of what was replayed and when.
Preserve ordering where it matters. For Kafka, the dead-letter topic should have at least as many partitions as the source topic and should retain the original message key, so that replayed messages land back on the correct partition and preserve per-key ordering.
Cap replay attempts. A message that fails again after a fix-and-replay cycle is not transient — route it to a permanent archive (a database table, an object-storage bucket) instead of looping it through the DLQ indefinitely. RabbitMQ's own docs warn that a dead-lettered message can be routed between queues only a limited number of times (16) before further TTL-based dead-lettering is disabled.
Never let a DLQ dead-letter into itself. If your DLQ has its own x-dead-letter-exchange (RabbitMQ) or its own redrive policy (SQS) pointed back at the same chain, a replay failure can create an infinite loop. Keep the DLQ's own dead-letter configuration empty, or point it at a strictly terminal archive.
Alert on volume, not just presence. A single message in a DLQ is a data point; a sudden spike is an incident. Wire DLQ depth and message age into the same alerting pipeline you use for everything else — see Modern Alerting Systems Design for Observability Teams for routing and noise-reduction practices that apply directly to DLQ alerts.

If your workflow involves multi-step, long-running processes rather than single messages, the same dead-letter thinking applies at the workflow layer — a saga's compensation logic needs the same "quarantine, inspect, decide" discipline when a step fails permanently instead of transiently. And when the events themselves originate from a database write, the transactional outbox pattern already builds dead-letter handling into the relay worker, so the pattern shows up one layer earlier than the broker.

Where DLQs Fit in the Bigger Picture

A dead-letter queue does not make failures go away — it makes them survivable and reviewable instead of silent. It works best alongside retries with backoff for the transient case, idempotent consumers so redrive is safe, and a circuit breaker so a struggling dependency does not flood the main queue (and, eventually, the DLQ) with the same failure thousands of times over. Treat the DLQ threshold, retention, and alerting as first-class configuration decisions, not defaults you leave untouched, and dead letters become a diagnostic tool instead of a place where data quietly disappears.

DEV Community: Rost

Configure Git User Name and Email Address

How

Useful links

What Comes After LLMs? Mamba, Diffusion & World Models

The pattern: AI's three-year breakthrough cycle

Why transformers are running out of road

Quadratic attention breaks at long context

The training data is running out

Energy and cost are the hard ceiling

Pattern matching is not understanding

State space models: linear scaling with a compressed memory

Diffusion language models: many tokens per forward pass

World models and JEPA: predicting meaning, not tokens

What production systems are doing: hybrid architectures

Where this is heading: the post-transformer era

References

Agent Skills vs MCP Servers: Decision Framework

Agent Skills vs MCP Servers in One Table

The Core Distinction: Procedure vs Capability

A More Precise Rule

When a SKILL.md Is Enough

The Data Is Local or Supplied by the User

The Workflow Depends on Judgment

Portability Matters More Than Central Control

An Existing CLI Already Provides the Capability

When You Need an MCP Server

The State Is Live and Authoritative

Credentials or User Identity Are Involved

Writes Need Transactional Guarantees

Multiple Agents Need the Same Capability

The Context-Cost Question

An Illustrative Token Budget

The Thin-Server Pattern: MCP Below, Skill Above

What the MCP Server Owns

What the Skill Owns

How It Runs in Hermes Agent

How It Runs in OpenClaw

Why the Split Survives Change

A Five-Step Decision Framework

1. Identify the Source of Truth

2. Locate the Trust Boundary

3. Separate Capability From Policy

4. Estimate Context and Operating Cost

5. Test the Boundary, Then the Behavior

Security Rules That Do Not Fit in a Slogan

Common Anti-Patterns

Hiding a Remote API Client in a Skill

Encoding the Workflow in Tool Descriptions

Building an execute_anything Tool

Publishing a Kitchen-Sink MCP Server

Treating Tool Hints as Security Policy

Using MCP for Static Knowledge

Final Decision: Skill, MCP Server, or Both?

References

OpenCode CLI in Practice: Workflows, Automation, and Pitfalls

What OpenCode Actually Is: An Agent Environment, Not a Chat Box

Setup: Install, Connect, and Why Provider Independence Matters

Initialize the Repository with AGENTS.md Before Asking for Code

Work from the Command Line with opencode run

The Best OpenCode CLI Use Cases

1. Repository exploration

2. Small, well-bounded fixes

3. Test generation after implementation

4. Mechanical refactoring

5. Code review

Use Plan and Build Modes as Different Mental Modes

Subagents Are Useful, but Delegation Is Not Free

Use Custom Agents as Permission Boundaries

Configure OpenCode Permissions on Day One

Do not confuse approval prompts with sandboxing

Turn Repetitive Prompts into Custom Commands

Use Skills for Reusable Workflows

MCP: Useful Until the Tool Catalog Becomes the Problem

Local Models in OpenCode: A Real Option, Not a Magic Fix

A Practical Daily Workflow with OpenCode

Step 1: Start from a clean Git state

Step 2: Ask for investigation first

Step 3: Narrow the implementation

Step 4: Inspect the diff yourself

Building an `execute_anything` Tool

Work from the Command Line with `opencode run`