DEV Community: Rost

Trending Open Source AI Projects on GitHub, October 2026

Rost — Wed, 21 Oct 2026 09:34:50 +0000

GitHub's monthly trending page works for a language, because a language filter already defines the set. Open source AI asks a different question: the work spans five languages, and the useful cut is the job a repository does.

This snapshot was read from the public GitHub API on 16 October 2026, and inside each group the repositories are ordered by star count. Stars measure attention, and a skills folder can gather that attention faster than a serving engine, so the group comes before the sort. The license column is part of the ranking as well, because several of the most starred projects are fair-code, brand-restricted, or split across two licenses.

The Rust and Go roundups on this site were built from that same page, with a language filter doing the grouping. Here the eight job groups below replace the filter, and each section states the trade that separates the close options. The closing sections turn the tables into a shortlist you can run, and into a repeatable method for the next snapshot.

How to rank open source AI projects

Search topics, then assign a job

GitHub topics are the workable substitute for a language filter. Sort a topic such as ai-agents, llm, rag, local-llm, or agent-memory by stars, and keep repositories that were pushed in the last few months. Free-text search for "AI" is too wide, and the unfiltered trending page still mixes agents with unrelated tools.

Three public inputs are enough to rebuild the list. Use topic search for the pool, a scenario catalog such as Top Open Source AI for the job labels, and the repository LICENSE file for the last column. That catalog publishes a composite of its own: stars at 70 percent, contributors at 10 percent, and update recency at 20 percent. This article keeps the simpler rule, so the number in the table is the number on GitHub.

Useful starting queries:

Keep software, and set lists aside

A repository earns a row when someone can install it, or when it is a maintained harness that changes how an agent runs. Awesome-lists, prompt directories, and dumps of vendor system prompts collect stars and still leave nothing to deploy. They belong in a reading list, and they stay out of the tables below.

Add a trending column only from your own snapshots

The language roundups could show stars gained in a month because GitHub already computes that delta inside one language. Across these groups, a monthly delta needs two snapshots of your own. Save full_name and stargazers_count on the first of the month, then subtract. Until that series exists, total stars are the honest ranking key. "Trending" in the title means actively maintained projects with the attention to match, as of this date.

flowchart TD
  topics[Topic search on GitHub]
  catalog[Scenario catalog]
  licenseFile[License file]
  drop[Set lists and prompt dumps aside]
  group[Assign each repo to one job]
  rank[Sort by stars inside the job]
  topics --> drop
  catalog --> group
  drop --> group
  group --> rank
  licenseFile --> rank

Where the projects sit

These groups are layers you can stack. A coding assistant can call a gateway, a gateway can call Ollama or vLLM, and a memory service can sit beside either of them. Two products from the same layer are usually a second opinion, and they are rarely a better architecture.

flowchart TB
  agent[Personal agent]
  coder[Coding assistant]
  skills[Skills and harness]
  memory[Memory]
  docs[RAG and documents]
  flow[Workflow builder]
  gate[Model gateway]
  run[Local runtime]
  coder --> skills
  agent --> memory
  coder --> memory
  agent --> docs
  flow --> docs
  agent --> gate
  coder --> gate
  flow --> gate
  docs --> run
  gate --> run

Personal AI assistants

This group is for agents that take actions on a machine or across tools, beyond a single coding session. OpenClaw leads by a wide margin. Hermes is the other general agent at this scale, and AutoGPT is the older name that is still accumulating stars while its license has split. A deeper leaderboard for this pair alone, with downloads, token share, and CVE history, is the OpenClaw vs Hermes Agent comparison.

Project	Stars	License	Repository
OpenClaw	392,126	MIT	openclaw/openclaw
Hermes Agent	254,122	MIT	NousResearch/hermes-agent
Firecrawl	191,908	AGPL-3.0	firecrawl/firecrawl
AutoGPT	187,598	MIT and Polyform Shield	Significant-Gravitas/AutoGPT

Firecrawl is listed here because agents need pages, and it is the web-data project readers actually star. It is a tool, and the assistants call it.

OpenClaw

Metric	Value
Repository	openclaw/openclaw
Stars	392,126
License	MIT
Language	TypeScript

OpenClaw is a personal agent meant to run on your own operating system and carry out tasks, which is the plain reading of a project that calls itself an AI that does things on any platform. The star count is the largest in this article, and the MIT license is the permissive end of the set. Treat the install as a local automation runtime with broad access to the machine, and scope what it is allowed to touch before you leave it running.

Hermes Agent

Metric	Value
Repository	NousResearch/hermes-agent
Stars	254,122
License	MIT
Language	Python

Hermes Agent, from Nous Research, is a general agent designed to accumulate skills and context as it is used. It sits in the same job as OpenClaw, with a Python codebase and the same MIT terms. Choose it when you want that growth loop and you are already comfortable operating a Python agent. Running both Hermes and OpenClaw means two permission models to audit.

AutoGPT

Metric	Value
Repository	Significant-Gravitas/AutoGPT
Stars	187,598
License	MIT outside autogpt_platform; Polyform Shield inside it
Language	Python

AutoGPT is the project that made autonomous agents a mainstream GitHub category, and it is still one of the most starred names in the set. The license is no longer a single grant. Everything outside the autogpt_platform folder is MIT, and everything inside that folder is under the Polyform Shield License, which permits use but reserves the code's commercial value to the producer. Read that split before you build a product on the platform directory.

Firecrawl

Metric	Value
Repository	firecrawl/firecrawl
Stars	191,908
License	AGPL-3.0
Language	TypeScript

Firecrawl turns sites into data that agents can call. The star count puts it among the assistants even though the job is acquisition, which says a lot about what people are wiring up in 2026. AGPL-3.0 matters if you modify it and offer the result as a service. For a personal agent that only calls a local or vendor endpoint, the license is a smaller constraint.

AI coding assistants

Coding agents are the densest part of the 2026 list. OpenCode leads the fully open terminal and editor agents, with Codex and Gemini CLI close behind as the vendor CLIs that happen to be open source. OpenHands, Cline, and Goose are the other programs you can actually run.

Aider (49,567 stars, Apache-2.0), Continue (36,228, Apache-2.0), and Crush (28,669, FSL-1.1-MIT) are real coding tools in the next band. Crush's license is a Functional Source License that converts to MIT on the schedule written in the file. One coding agent is plenty. A second one is for a comparison week, and after that it is configuration you have to keep in sync.

Project	Stars	License	Repository
OpenCode	213,922	MIT	anomalyco/opencode
Codex	129,395	Apache-2.0	openai/codex
Gemini CLI	107,389	Apache-2.0	google-gemini/gemini-cli
OpenHands	91,004	MIT	OpenHands/OpenHands
Cline	70,370	Apache-2.0	cline/cline
Goose	55,324	Apache-2.0	aaif-goose/goose

OpenCode

Metric	Value
Repository	anomalyco/opencode
Stars	213,922
License	MIT
Language	TypeScript

OpenCode is an open source coding agent with a terminal workflow and a large plugin surface. It has become the default name people cite when they want a coding agent they can inspect, and the MIT license matches that use. The repository is anomalyco/opencode. Pick it when the agent itself should be something you can patch.

Codex

Metric	Value
Repository	openai/codex
Stars	129,395
License	Apache-2.0
Language	Rust

Codex is OpenAI's lightweight coding agent for the terminal, written in Rust and licensed Apache-2.0. It is the open client for a hosted model family, which is a different bargain from a local-first agent. Use it when your team already standardizes on OpenAI models and you want a CLI you can automate. The client being open does not move the model weights onto your disk.

Gemini CLI

Metric	Value
Repository	google-gemini/gemini-cli
Stars	107,389
License	Apache-2.0
Language	TypeScript

Gemini CLI brings Gemini into the terminal as an agent, under Apache-2.0. It plays the same role as Codex for a different provider, and the TypeScript codebase is easier to embed in existing Node tooling. Choose it when Gemini is the model you already pay for. Running it beside Codex is reasonable only if each repo has one default agent.

OpenHands

Metric	Value
Repository	OpenHands/OpenHands
Stars	91,004
License	MIT
Language	TypeScript

OpenHands is a software-development agent with a full task loop, now published from the OpenHands organization. The MIT license and the higher-level product make it a platform choice, closer to a hosted development environment you can self-host than to a single binary. Use it when you want issues, sandboxes, and a UI around the agent. For a one-person terminal workflow, OpenCode, Codex, or Goose is the smaller install.

Cline

Metric	Value
Repository	cline/cline
Stars	70,370
License	Apache-2.0
Language	TypeScript

Cline is an autonomous coding agent that ships as an SDK, an IDE extension, or a CLI. Apache-2.0 and the editor integration make it the practical pick for people who live in VS Code and still want a library they can call. It overlaps OpenCode more than it overlaps a chat UI. Standardize on one of them per repository so reviews stay readable.

Goose

Metric	Value
Repository	aaif-goose/goose
Stars	55,324
License	Apache-2.0
Language	Rust

Goose is an extensible local agent that installs, edits, runs, and tests code with whatever model you point at it. The repository now lives at aaif-goose/goose, and the Rust implementation plus Apache-2.0 make it a strong workstation agent. Choose Goose when the model provider should be swappable and the agent should run on the machine that holds the code. It is a quieter star count than OpenCode, and it is a more direct fit for a local model.

Agent skills and harnesses

Topic search will put this group above several real products. Superpowers and ECC are instructions, hooks, and workflows for agents you already run. Spec Kit is GitHub's own process toolkit for spec-driven development. They change agent behavior, and they are still software you compose with a coding assistant from the previous section.

Project	Stars	License	Repository
Superpowers	299,886	MIT	obra/superpowers
ECC	280,382	MIT	affaan-m/ECC
Spec Kit	141,439	MIT	github/spec-kit

Superpowers

Metric	Value
Repository	obra/superpowers
Stars	299,886
License	MIT
Language	Shell

Superpowers is an agentic skills framework and a software-development methodology, distributed mostly as shell and instruction files under MIT. The star count is what you get when a method spreads through coding-agent users in a few months. Install it on top of an agent you already trust, and expect to edit the skills to match the repository. The framework will follow whatever permissions that agent already has.

ECC

Metric	Value
Repository	affaan-m/ECC
Stars	280,382
License	MIT
Language	JavaScript

ECC is a harness for Claude Code, Codex, OpenCode, Cursor, and similar agents, covering skills, memory, and checks around how the agent works. It is the right shape if you want the same loop on more than one coding agent. It is also another moving layer between you and the model. Adopt it when the team has already picked the agents and wants one shared way to brief them.

Spec Kit

Metric	Value
Repository	github/spec-kit
Stars	141,439
License	MIT
Language	Python

Spec Kit is GitHub's toolkit for spec-driven development and the process around it. The maintainer is the platform most of these repositories already live on, and the MIT license is straightforward. Use it when you want a written spec before the agent edits code. It will stay quiet if the team treats the spec as optional paperwork.

AI memory

Memory is the layer that lets an agent resume. The starred projects split into two styles. claude-mem, Beads, and much of Hindsight attach memory to coding sessions. Mem0, Cognee, and Letta are services you call from application agents. Capture policy, dependencies, and self-hosting tradeoffs for these backends are compared in Agent Memory Providers.

Memvid (16,600 stars, Apache-2.0) and Zep (4,961 stars, Apache-2.0) are smaller options in the same job, Memvid as a single-file store and Zep as an integration-heavy memory API. One memory system per agent is the setup you can still explain later.

Project	Stars	License	Repository
claude-mem	101,052	Apache-2.0	thedotmack/claude-mem
Mem0	67,243	Apache-2.0	mem0ai/mem0
Hindsight	50,055	MIT	vectorize-io/hindsight
Cognee	32,364	Apache-2.0	topoteretes/cognee
Beads	27,917	MIT	gastownhall/beads
Letta	25,156	Apache-2.0	letta-ai/letta

claude-mem

Metric	Value
Repository	thedotmack/claude-mem
Stars	101,052
License	Apache-2.0
Language	TypeScript

claude-mem records what a coding agent does, compresses that history, and injects the relevant part into later sessions. It talks to Claude Code, OpenClaw, Codex, Gemini CLI, Hermes, Copilot, and OpenCode, which is why it leads this group. Apache-2.0 fits a sidecar you may want to modify. Use it when the thing you keep forgetting is the coding session itself.

Mem0

Metric	Value
Repository	mem0ai/mem0
Stars	67,243
License	Apache-2.0
Language	Python

Mem0 is a memory layer for agents and applications, with user, session, and agent state behind an API. It is the default name when the product is a support bot or a personal assistant, and you want that memory outside any one coding tool. Apache-2.0 and a hosted option sit side by side, so self-hosting is a choice you can make later. Prefer it over a coding-session hook when many agents should share the same facts about a user.

Hindsight

Metric	Value
Repository	vectorize-io/hindsight
Stars	50,055
License	MIT
Language	Python

Hindsight is Vectorize's agent memory project, aimed at agents that learn from what they have already done, and it integrates with both coding agents and application frameworks. The MIT license is the most permissive in this group. The project publishes strong benchmark claims, and those claims should be re-run on your own transcripts before you bet a product on them. Pick Hindsight when you want a dedicated memory service and you are willing to compare it with Mem0 on your data.

Cognee

Metric	Value
Repository	topoteretes/cognee
Stars	32,364
License	Apache-2.0
Language	Python

Cognee builds a memory and knowledge graph that agents can query, with an emphasis on running that layer on smaller models. It overlaps Mem0 and it also overlaps the RAG tools below, because a graph of your documents behaves like both memory and retrieval. Apache-2.0 keeps the embedding path replaceable. Choose it when the facts you care about are entities and relationships, and a flat vector index has already felt too lossy.

Beads

Metric	Value
Repository	gastownhall/beads
Stars	27,917
License	MIT
Language	Go

Beads stores coding-agent tasks, plans, and dependencies as files that travel with the git branch. The repository is now gastownhall/beads, and it is the same project the January Go roundup listed under its earlier path. MIT and a git-backed store make it the memory tool that your existing review process can see. Use it when context should fork with the branch. Use claude-mem or Mem0 when the memory has to outlive a single repository.

Letta

Metric	Value
Repository	letta-ai/letta
Stars	25,156
License	Apache-2.0
Language	Python

Letta is the stateful-agent platform that grew out of MemGPT. Memory here is part of the agent runtime, so the agent manages its own context as a feature of the platform. That is a deeper commitment than dropping Mem0 under an agent you already run. Apache-2.0 applies to the open platform. Choose Letta when you want the runtime and the memory designed together, and choose Mem0 when you want memory under agents you do not intend to replace.

RAG and knowledge bases

RAG projects turn documents into a context layer. RAGFlow is the end-to-end engine people are starring. LlamaIndex is the document framework, GraphRAG is Microsoft's graph approach, and WeKnora is Tencent's knowledge platform, which also appeared in the January Go roundup and has kept growing.

Project	Stars	License	Repository
RAGFlow	92,289	Apache-2.0	infiniflow/ragflow
LlamaIndex	52,513	MIT	run-llama/llama_index
GraphRAG	36,349	MIT	microsoft/graphrag
WeKnora	34,210	MIT	Tencent/WeKnora

RAGFlow

Metric	Value
Repository	infiniflow/ragflow
Stars	92,289
License	Apache-2.0
Language	Go

RAGFlow is a self-hosted RAG engine that parses messy documents and exposes retrieval to agents, with citations as part of the product. Apache-2.0 and a UI you can hand to a team are why it leads this group by a clear gap. Use it when the job is "put our files behind an answer box" and you want the parser, the index, and the API from one project. Reach for LlamaIndex when you are embedding retrieval inside an application you already control.

LlamaIndex

Metric	Value
Repository	run-llama/llama_index
Stars	52,513
License	MIT
Language	Python

LlamaIndex is a document-processing framework for agents and RAG pipelines. It is a library with many connectors, and the MIT license fits that role. You assemble the index, the store, and the agent yourself, which is the cost of the flexibility. Choose it for product code. Choose RAGFlow when you want the same job as a service you deploy.

GraphRAG

Metric	Value
Repository	microsoft/graphrag
Stars	36,349
License	MIT
Language	Python

GraphRAG builds a graph over a corpus and answers questions by walking that graph, as Microsoft Research's open implementation of the idea. The technique itself is explained in Advanced RAG: LongRAG, Self-RAG and GraphRAG. MIT licensing makes it easy to try beside a vector pipeline. Graph construction costs more up front than a chunk-and-embed index, and it pays that cost back on questions that span many documents. Use it when the corpus has structure you are currently forcing into chunks.

WeKnora

Metric	Value
Repository	Tencent/WeKnora
Stars	34,210
License	MIT
Language	Go

WeKnora is Tencent's knowledge platform for turning documents into a RAG index, an agent, and a wiki that the project describes as self-maintaining. The project license is MIT, with third-party components under their own terms. GitHub reports that license as Other, because the MIT grant sits under a Tencent preamble. It is the closest cousin to RAGFlow in this table, with a Go core. Prefer it when you want the wiki and the agent in the same box, and compare the parsers on a sample of your own PDFs before you migrate a library.

Workflow and agent builders

Builders are where a team draws the flow, or where a developer imports the framework that draws it for them. n8n is the largest by stars and the narrowest by license. Dify and Langflow are the AI-native studios. LangChain is the library underneath a great many of the others.

CrewAI (59,745 stars, MIT) and AutoGen (61,429 stars, CC-BY-4.0) belong in the same conversation as code-first multi-agent frameworks. AutoGen's Creative Commons attribution license is unusual for a code repository, and it is worth reading before you vendor it. LangChain remains the integration surface those frameworks still meet.

Project	Stars	License	Repository
n8n	207,535	Sustainable Use License	n8n-io/n8n
Dify	158,591	Modified Apache-2.0	langgenius/dify
Langflow	155,717	MIT	langflow-ai/langflow
LangChain	147,886	MIT	langchain-ai/langchain

n8n

Metric	Value
Repository	n8n-io/n8n
Stars	207,535
License	Sustainable Use License
Language	TypeScript

n8n is a workflow platform with hundreds of integrations and native AI steps, and it is the project people mean when they want Zapier-style automation they can host. The Sustainable Use License allows internal business use, personal use, and non-commercial use. Distribution is limited to free, non-commercial copies, and files marked .ee need a separate enterprise license. It is source-available fair-code, and it is still the right tool for internal automation if those limits match how you run it.

Dify

Metric	Value
Repository	langgenius/dify
Stars	158,591
License	Modified Apache-2.0
Language	TypeScript

Dify is a workspace for agent workflows and RAG pipelines, with model providers and tools in one deployable stack. The license starts from Apache-2.0 and adds conditions: operating a multi-tenant service, meaning multiple workspaces, needs a commercial license, and the console logo and copyright notice stay on the frontend. Single-tenant internal use is the case the extra clauses leave open. Choose Dify when a team needs one UI from prototype to production, and choose Langflow when the license has to stay plain MIT.

Langflow

Metric	Value
Repository	langflow-ai/langflow
Stars	155,717
License	MIT
Language	Python

Langflow is a visual builder for agents and RAG flows, with an MIT license and a star count essentially tied with Dify. The canvas is the product, and the generated flows are something a developer can still own. Use it when you want a diagram of the pipeline and a license a procurement checklist already understands. The trade is a younger application surface than Dify's workspace features.

LangChain

Metric	Value
Repository	langchain-ai/langchain
Stars	147,886
License	MIT
Language	Python

LangChain is the agent-engineering library that most integration docs still assume. MIT and the breadth of connectors keep it in every shortlist, and the abstraction cost is real once the chain is doing something simple. Use it when you are writing application code and you want the adapters already written. Use Dify or Langflow when the deliverable is a workspace, and use a direct provider SDK when the flow is a handful of calls.

Local inference and chat interfaces

This is the layer under everything else. Ollama is the workstation runner, Transformers is the model-definition library, Open WebUI is the chat front end, llama.cpp is the portable C++ engine, and vLLM is the server engine. Hugging Face Transformers is included because so many of the other projects import it, even though you do not "run" it the way you run Ollama. API maturity, hardware support, and production readiness across the runtimes are compared in Ollama vs vLLM vs LM Studio.

Project	Stars	License	Repository
Ollama	183,334	MIT	ollama/ollama
Transformers	167,734	Apache-2.0	huggingface/transformers
Open WebUI	155,825	Open WebUI License	open-webui/open-webui
llama.cpp	131,311	MIT	ggml-org/llama.cpp
vLLM	93,870	Apache-2.0	vllm-project/vllm

Ollama

Metric	Value
Repository	ollama/ollama
Stars	183,334
License	MIT
Language	Go

Ollama runs current open models locally behind a simple CLI and an OpenAI-compatible API. It was already a leader in the January Go roundup, and at 183,334 stars it is still the default workstation runtime. MIT and a single daemon are the reasons. Use it on a developer machine and on small servers. Move the same weights to vLLM when concurrency, batching, and GPU utilization become the problem.

Transformers

Metric	Value
Repository	huggingface/transformers
Stars	167,734
License	Apache-2.0
Language	Python

Transformers is Hugging Face's library for defining, loading, and running models across text, vision, audio, and multimodal checkpoints. Apache-2.0 and the model hub around it make this the library the rest of the stack imports. You feel it when you are training, converting, or debugging a model. You can ignore it for a long time if Ollama or vLLM already serves the checkpoint you need.

Open WebUI

Metric	Value
Repository	open-webui/open-webui
Stars	155,825
License	Open WebUI License
Language	Python

Open WebUI is the chat interface most people put in front of Ollama and OpenAI-compatible APIs. The Open WebUI License allows redistribution and then restricts branding. Above 50 end users in any rolling 30 days, you may not alter, remove, or replace the Open WebUI name and marks without written permission or an enterprise license. For a personal or small-team install that keeps the name, it is the polished front end. For a product you will rebrand, budget the enterprise conversation first. Docker deployment and the setup options are walked through in the Open WebUI guide.

llama.cpp

Metric	Value
Repository	ggml-org/llama.cpp
Stars	131,311
License	MIT
Language	C++

llama.cpp is the C and C++ inference engine that made local models practical on ordinary hardware. MIT licensing and the GGUF ecosystem are why Ollama and a crowd of smaller apps sit on top of it. Use the project directly when you want the engine, the quantization tools, or a binary with few dependencies. Use Ollama when you want that engine with model management already wrapped.

vLLM

Metric	Value
Repository	vllm-project/vllm
Stars	93,870
License	Apache-2.0
Language	Python

vLLM is a high-throughput serving engine for GPUs, with continuous batching and an OpenAI-compatible API. Apache-2.0 and the deployment focus make it the server counterpart to Ollama. Star count is lower because the audience is people who run GPUs, and that audience is the one that feels the difference. Choose vLLM when more than a few concurrent sessions share one machine.

LLM gateways

A gateway is the OpenAI-compatible front door in front of many providers. LiteLLM is the common choice, New API is the self-hosted distributor, and Bifrost is the smaller high-throughput gateway. Put a gateway in the stack when you have more than one provider, a budget per key, or a need to fail over. A single local Ollama daemon does not need one.

Project	Stars	License	Repository
LiteLLM	61,338	MIT for the core	BerriAI/litellm
New API	50,037	AGPL-3.0	QuantumNous/new-api
Bifrost	8,837	Apache-2.0	maximhq/bifrost

LiteLLM

Metric	Value
Repository	BerriAI/litellm
Stars	61,338
License	MIT outside enterprise/
Language	Python

LiteLLM exposes a large set of provider APIs through one OpenAI-shaped interface, with spend tracking, retries, and guardrails. Code outside the enterprise/ directory is MIT, and that directory has its own license. The Python SDK is what application code imports, and the proxy is what the platform team runs. Use the core when you want one client for many vendors. Check the enterprise directory before you assume the whole repository is MIT.

New API

Metric	Value
Repository	QuantumNous/new-api
Stars	50,037
License	AGPL-3.0
Language	Go

New API is a model hub that aggregates providers and re-serves them as OpenAI, Claude, or Gemini compatible endpoints. It is the gateway people deploy when they also want user accounts, quotas, and a distribution UI. AGPL-3.0 is the term that matters if you modify the hub and offer it as a service. For an internal gateway with upstream keys, it is a concrete alternative to LiteLLM's proxy, with a Go deployment and a stricter license.

Bifrost

Metric	Value
Repository	maximhq/bifrost
Stars	8,837
License	Apache-2.0
Language	Go

Bifrost is a Go gateway for models and tools, with failover, budgets, and an MCP gateway in the same process. The star count is far below LiteLLM, and the January Go roundup already had it on the trending list, so the project is younger than it is obscure. Apache-2.0 avoids the enterprise-directory split. Consider it when the gateway is on the request path of a busy service and you want a small Go binary. Keep LiteLLM when you need the long tail of provider adapters today.

A stack worth running

The tables are a map, and a workstation still needs a short list. For a local setup in October 2026, Ollama plus one coding agent covers the daily loop. Add Open WebUI if the branding clause fits the user count, Mem0 or claude-mem depending on whether memory belongs to an application or to a coding session, and RAGFlow when the documents are the product. Add LiteLLM or New API only after a second model provider shows up.

Layer	Default pick	Choose the alternative when
Runtime	Ollama	vLLM, once several users share a GPU server
Chat UI	Open WebUI	You need to remove the brand above 50 users
Coding agent	OpenCode, Codex, Gemini CLI, or Goose	You self-host the whole task loop: OpenHands
Memory	claude-mem or Mem0	The facts are a graph: Cognee. The facts follow a git branch: Beads
Documents	RAGFlow	You are embedding retrieval in your own service: LlamaIndex
Builder	Langflow	You accept Dify's multi-tenant clause, or n8n's fair-code terms for internal automation
Gateway	LiteLLM core	You want AGPL and a user-facing hub: New API

Skills packs such as Superpowers and ECC come after the agent is chosen. They are easy to star and easy to accumulate, and three harnesses on one repository will fight over the same instructions.

Keeping the ranking current

Repeat the topic queries at the top of this article, assign each new repository to one job, and re-sort that job by stars. Then open the license file. A project that jumped the table on a skills directory, a prompt list, or a weekend of attention can wait for the next snapshot before it replaces a tool you run.

Store the snapshot. A single CSV of full_name, stargazers_count, license, and pushed_at, taken monthly, turns the next edition of this page into a real trending column, the same kind of delta the language articles already show. Until then, the groups are the part that ages well, and the star counts are the part that goes stale on purpose.

Sources

Star counts, languages, and license identities were read from the GitHub repository API and the repository LICENSE files on 16 October 2026. Job labels also draw on the public scenario list at Top Open Source AI, which documents its own composite score separately from the sort used here.

Hindsight for Hermes Agent: Self-Hosted Memory with Local LLMs

Rost — Sun, 18 Oct 2026 02:50:36 +0000

A self-hosted AI agent needs more than a long context window. It needs a way to retain useful facts, retrieve earlier decisions, and revise what it thinks it knows as projects evolve.

Hermes Agent ships with built-in memory files, but an external provider can provide much richer retrieval across sessions. Hindsight is one of the more interesting options in that space. This guide installs it for Hermes with no cloud inference, first as an all-in-one Docker container and then against an existing PostgreSQL server with separate local model endpoints.

The guide covers deployment, choosing and wiring local embedding, reranking, and generative models, controlling what auto-retain writes into the bank, connecting Hermes, and the operational layer: backup, migration, and day-to-day bank hygiene. The examples use a deliberate port scheme — 8080 for the main llama.cpp server, 8081 for embeddings, 8082 for reranking, 11434 for Ollama — so change them to the ports you already use and keep the small servers off the main model's port.

What Hindsight Does

Hindsight is a persistent agent memory engine with three principal operations: retain, recall, and reflect. Retain extracts and stores information; recall combines semantic, keyword, graph, and temporal retrieval; reflect synthesizes an answer from accumulated memories. Background consolidation can turn overlapping observations into a more useful knowledge layer.

It is not just an embeddings database. A vector store can locate similar passages, but a memory system also has to preserve chronology, project boundaries, contradictory evidence, and changes in decisions. Those failure modes are explored in self-reinforcing memory loops.

For the wider market, see agent memory providers compared. Hermes' built-in MEMORY.md, USER.md, and memory modes are covered in the Hermes Agent memory system guide, so this article assumes you know how the built-in layer works.

Why Move Beyond Mnemosyne

Mnemosyne for Hermes offers appealing local-first ideas, including SQLite storage and hybrid retrieval. In practice, installation constraints, version compatibility, write-approval behaviour, and rapid changes around its 4.0 prereleases can make it difficult to operate predictably. Those are operational concerns, not evidence that Mnemosyne's memory model is fundamentally flawed.

Hindsight is not dependency-free either: its Docker image and local models can consume substantial disk space. Running its API as an independent service, with inference elsewhere, keeps upgrades and troubleshooting isolated from the agent itself.

Choose a Deployment

Component	A: All-in-one container	B: Existing PostgreSQL server
Hindsight API and UI	Docker container	Docker container
PostgreSQL	Embedded `pg0` in container	Existing PostgreSQL machine
Generative LLM	External llama.cpp	External llama.cpp
Embeddings	Default local or external	External Ollama or llama.cpp
Reranking	Default local or external	External rerank server
Best for	Quick evaluation	Long-running, isolated installation

Here, "all-in-one" means the Hindsight API, UI, and database are in one container; the inference LLM is still hosted elsewhere. This is intentional: Docker's official Hindsight image does not bundle a ready-to-use local generative llama.cpp server.

flowchart LR
    H[Hermes Agent] -->|Memory API| A[Hindsight API]
    A --> P[(PostgreSQL)]
    A --> E[Embedding endpoint]
    A --> R[Reranking endpoint]
    A --> L[Generative LLM endpoint]

Use LAN-reachable addresses, not localhost, for services running on a different host. Restrict these ports to trusted clients; the example dummy keys are compatibility placeholders, not authentication.

Option A: All-in-One Hindsight Container

This is the simplest way to evaluate Hindsight without filling the Hermes VM with Python, Torch, and local model dependencies. The container runs Hindsight and embedded PostgreSQL, while your main llama.cpp LLM handles memory extraction and reflection.

Two image tags matter here. ghcr.io/vectorize-io/hindsight:latest is the large image — roughly 9 GB on AMD64 — because it bundles a BGE embedder, a MiniLM reranker, and embedded PostgreSQL. latest-slim is roughly 500 MB and expects those two models to live elsewhere, which is what you want once you point Hindsight at the embedding and reranking setup in Serve and Wire Local Models. Once the health check passes, pin a specific tag: a moving tag is a poor match for a database you intend to keep.

Create a directory and a Compose file on the Hindsight host:

mkdir -p ~/services/hindsight
cd ~/services/hindsight
cat > compose.yaml <<'YAML'
services:
  hindsight:
    image: ghcr.io/vectorize-io/hindsight:latest
    container_name: hindsight
    restart: unless-stopped
    shm_size: 1gb
    extra_hosts:
      - "host.docker.internal:host-gateway"
    ports:
      - "8888:8888"  # Memory API
      - "9999:9999"  # Control-plane UI
    environment:
      HINDSIGHT_API_WORKER_ID: hindsight-standalone
      HINDSIGHT_API_LLM_PROVIDER: openai
      HINDSIGHT_API_LLM_BASE_URL: http://WORKSTATION_IP:8080/v1
      HINDSIGHT_API_LLM_MODEL: LocalMegaLLM
      HINDSIGHT_API_LLM_API_KEY: dummy
      HINDSIGHT_API_LLM_MAX_CONCURRENT: "2"
    volumes:
      - hindsight-pg0:/home/hindsight/.pg0
volumes:
  hindsight-pg0:
YAML

Replace WORKSTATION_IP with the host running llama.cpp. On Linux, host.docker.internal resolves only if the container declares the host-gateway mapping shown above; without it, a base URL of http://host.docker.internal:8080/v1 points nowhere. The 1 GB shared-memory reservation is particularly useful for embedded PostgreSQL indexing; Docker's small default /dev/shm can cause failures. A named volume preserves the database when the container is recreated, and a bind mount alternative has to be owned by uid 1000, the user inside the image.

HINDSIGHT_API_WORKER_ID should stay stable across recreations — Docker's default is the container hostname, and a task that was mid-flight when a container died stays stuck under the old worker id.

docker compose up -d
docker compose logs -f --tail=100
curl -f http://localhost:8888/health

If your installed Hindsight version exposes health at a different path, inspect http://localhost:8888/docs and the container logs. The control plane is normally available at http://HINDSIGHT_HOST:9999. Once the basic deployment works, you can also point its embedding and reranking providers at the configuration in Serve and Wire Local Models rather than loading the default models inside the container.

Do not expose ports 8888 or 9999 directly to the internet. Use a private network or an authenticated reverse proxy. Embedded PostgreSQL is convenient for testing, but an externally managed database is easier to back up and maintain.

Option B: Hindsight Against an Existing PostgreSQL Server

For a long-running installation, I prefer keeping the Hindsight API, database, and inference endpoints separate. The Hindsight VM can remain small because it runs orchestration and HTTP services rather than heavy models.

flowchart LR
    H[Hermes VM] -->|port 8888| A[Hindsight VM]
    A -->|port 5432| P[(PostgreSQL host)]
    A -->|port 8080| L[Generative LLM]
    A -->|port 8081 or 11434| E[Embedding endpoint]
    A -->|port 8082| R[Reranking endpoint]

Prepare PostgreSQL and pgvector

Hindsight needs PostgreSQL 14 or newer and a vector extension in the hindsight database. The extension is per database: installing the package on the server does nothing until CREATE EXTENSION runs there. On a Debian or Ubuntu PostgreSQL server, install the pgvector package matching the actual PostgreSQL major version; for example, on PostgreSQL 16:

sudo apt update
sudo apt install postgresql-16-pgvector

The package name and availability depend on your configured apt repositories. If it is not available, follow the pgvector installation instructions, or install a compatible package from the PostgreSQL PGDG repository. Do not blindly change the PostgreSQL major version on a shared server.

Create a dedicated role and database as a PostgreSQL administrator:

CREATE ROLE hindsight LOGIN PASSWORD 'REPLACE_WITH_LONG_PASSWORD';
CREATE DATABASE hindsight OWNER hindsight;

Connect to the new database and enable the extension:

sudo -u postgres psql -d hindsight -c \
  'CREATE EXTENSION IF NOT EXISTS vector;'
sudo -u postgres psql -d hindsight -c \
  "SELECT extname, extversion FROM pg_extension WHERE extname = 'vector';"

The extension normally needs elevated privileges to install; the application role does not need to be a superuser. Hindsight uses pgvector by default and wants 0.8.0 or newer for iterative ANN scans; older servers reject that setting. HINDSIGHT_API_VECTOR_EXTENSION can alternatively select pgvectorscale (a DiskANN option Hindsight documents for very large banks) or vchord, both of which need extra packaging on this exact PostgreSQL build. Switching algorithms after memories exist stops startup and asks for a migration, so choose before the first retain. For a personal Hermes bank, pgvector is the one to install and then stop.

Allow only the Hindsight host through pg_hba.conf, for example:

host  hindsight  hindsight  192.168.1.50/32  scram-sha-256

Set PostgreSQL listen_addresses appropriately and use TLS or a trusted private network for cross-host connections. Back up this database independently of the Hindsight Docker container. If the server is shared with other applications, keep Hindsight in its own database, or set HINDSIGHT_API_DATABASE_SCHEMA to a schema you own; the default schema is public.

Run Hindsight Against PostgreSQL

Create compose.yaml on the Hindsight VM. This version deliberately does not start PostgreSQL locally:

services:
  hindsight:
    image: ghcr.io/vectorize-io/hindsight:latest-slim
    restart: unless-stopped
    ports:
      - "8888:8888"
      - "9999:9999"
    env_file: .env
    environment:
      HINDSIGHT_API_LLM_PROVIDER: openai
      HINDSIGHT_API_LLM_BASE_URL: http://WORKSTATION_IP:8080/v1
      HINDSIGHT_API_LLM_MODEL: LocalMegaLLM
      HINDSIGHT_API_LLM_API_KEY: dummy
      HINDSIGHT_API_LLM_MAX_CONCURRENT: "2"
      HINDSIGHT_API_DB_POOL_MIN_SIZE: "2"
      HINDSIGHT_API_DB_POOL_MAX_SIZE: "10"
      HINDSIGHT_API_WORKER_ID: hindsight-standalone

Store the external database URL in a file called .env:

HINDSIGHT_API_DATABASE_URL=postgresql://hindsight:YOUR_PASSWORD@POSTGRES_IP:5432/hindsight

chmod 600 .env
docker compose up -d
docker compose logs -f --tail=100

If your password contains @, :, /, or other reserved URL characters, URL-encode it before putting it in the connection string. Test that the Hindsight VM can reach PostgreSQL port 5432, and that pg_hba.conf allows the Hindsight host — a server bound only to 127.0.0.1 refuses the connection even when the URL looks right. Allow the first startup to run database migrations; migrations run on startup unless you set HINDSIGHT_API_RUN_MIGRATIONS_ON_STARTUP=false. With the API and database up, continue with Choose Models for Hindsight and Serve and Wire Local Models before connecting Hermes.

Choose Models for Hindsight

Deploy puts the API and database on the network. The next decision is which models fill the three inference roles: embeddings for recall candidates, a reranker to re-order them, and a generative LLM for retain, reflect, and consolidation. A practical default for a CPU Hindsight host whose GPU is already busy with conversation is in-process Qwen/Qwen3-Embedding-0.6B, in-process BAAI/bge-reranker-v2-m3, and a shared llama.cpp chat model for generative work.

Embedding models

Hindsight reads the embedding dimension on startup and sizes the bank to match, so the model you pick at first retain is the model the bank keeps until you rebuild it. Dimensionality and instruction behaviour matter more than leaderboard rank: a stronger 8B embedder is not worth a migration if the bank is already a 384- or 1024-dimensional one. Rows below are sorted from lightest to heaviest; RAM figures are approximate loaded footprint on CPU (FP16 / typical local load), not GGUF file size.

Model	Params	Dim	Context	Loaded	Runs on	Notes
`BAAI/bge-small-en-v1.5`	33M	384	512	~100–130 MB	CPU	Hindsight default; ships in `hindsight:latest`. Fastest local path. Score distribution still calibrates many `recall_min_scores` defaults after you leave it.
`BAAI/bge-base-en-v1.5`	109M	768	512	~400 MB	CPU	Mid-tier English BGE; locks a 768-dim bank.
`sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2`	~118M	384	512	~200 MB	CPU	Hindsight-documented multilingual MiniLM; 50+ languages at the same 384-dim bank width as BGE-small.
`intfloat/multilingual-e5-small`	~118M	384	512	~200 MB	CPU	Default for the `onnx` provider; light multilingual CPU path when you have an ONNX export.
`nomic-ai/nomic-embed-text-v1.5`	~137M	768	8k	~500 MB	CPU	Common Ollama embedder; longer context than classic BGE, still English-first.
`google/embeddinggemma-2`	270M	768	8k	~540 MB	CPU	EmbeddingGemma 2 text-only (disable vision/audio towers). 100+ languages + code; MRL truncates to 512/256/128; instruction prefixes for search vs document. Full multimodal load is 740M.
`BAAI/bge-m3`	568M	1024	8k	~1.1 GB	CPU or GPU	Strong multilingual dual-encoder; same width as Qwen3-0.6B, so a same-dim swap still needs a full re-embed.
`Qwen/Qwen3-Embedding-0.6B`	0.6B	1024	32k	~1.2 GB	CPU	This guide's default. Instruction-aware (search instruction on the query side); comfortable on CPU. Local provider auto-applies the instruction; OpenAI-compatible servers need `HINDSIGHT_API_EMBEDDINGS_QUERY_PREFIX`.
`Qwen/Qwen3-Embedding-4B`	4B	2560	32k	~8 GB FP16 / ~2.5 GB quant	GPU	Qwen3 quality tier; locks a 2560-dim bank. Not viable when the conversational GPU is already saturated.

How the model is run is the other half of the choice. Hindsight's embedding providers, from simplest to most moving parts:

Provider	How it runs	Best when	Notes
`local`	In-process SentenceTransformers; no sidecar	Single Hindsight host, CPU embedder	Default path. Set `HINDSIGHT_API_EMBEDDINGS_LOCAL_MODEL`. No llama.cpp, Ollama, or TEI.
`onnx`	In-process ONNX Runtime	Lighter CPU than Torch; you already have an ONNX export	Needs `hindsight-api-slim[local-onnx]` and `HINDSIGHT_API_EMBEDDINGS_ONNX_*`.
`openai` / `tei`	External OpenAI-compatible or HF TEI endpoint	Embeddings off the API process; shared GPU/CPU host	What Serve and Wire Local Models configures for Ollama / llama.cpp / TEI. Only worth the sidecar if you want isolation or a different machine.

Qwen3-Embedding-0.6B does not need an external embeddings service. On a CPU Hindsight host it loads inside the API process with the local provider — Hugging Face downloads the weights on first start, and the provider applies the model's search instruction itself:

# Compose environment (or equivalent .env keys)
HINDSIGHT_API_EMBEDDINGS_PROVIDER: local
HINDSIGHT_API_EMBEDDINGS_LOCAL_MODEL: Qwen/Qwen3-Embedding-0.6B

Use an external OpenAI-compatible endpoint only when you want the embedder on another machine or off the API process. Ollama is one such host:

# Embeddings: Qwen3-Embedding-0.6B on Ollama
HINDSIGHT_API_EMBEDDINGS_PROVIDER: openai
HINDSIGHT_API_EMBEDDINGS_OPENAI_BASE_URL: http://192.168.1.123:11434/v1
HINDSIGHT_API_EMBEDDINGS_OPENAI_MODEL: qwen3-embedding:0.6b-q8_0
HINDSIGHT_API_EMBEDDINGS_OPENAI_API_KEY: dummy

Replace the host, port, and tag with whatever ollama list shows on your inference machine.

HINDSIGHT_API_EMBEDDINGS_QUERY_PREFIX depends on the provider:

Path	`HINDSIGHT_API_EMBEDDINGS_QUERY_PREFIX`	`HINDSIGHT_API_EMBEDDINGS_PASSAGE_PREFIX`
`local` (`Qwen/Qwen3-Embedding-0.6B`)	Leave unset / empty — the in-process provider applies Qwen's search instruction itself	Leave unset / empty
External Ollama / llama.cpp / OpenAI-compatible	Official Qwen3 retrieval instruction (real newline before `Query:`)	Leave empty — stored memories stay raw text
Server already wraps inputs in Qwen's template	Leave both empty — otherwise the instruction is applied twice	Leave empty

For the external path, set:

# Real newline before Query:. Bash $'...' is the reliable way to pass it.
HINDSIGHT_API_EMBEDDINGS_QUERY_PREFIX=$'Instruct: Given a web search query, retrieve relevant passages that answer the query\nQuery:'
HINDSIGHT_API_EMBEDDINGS_PASSAGE_PREFIX=

In Compose YAML the same string needs an explicit \n (or a literal multiline value). Set the prefixes before the first retain: changing them later means old vectors and new queries no longer sit in the same space.

Two properties decide more than the leaderboard position. Instruction sensitivity: Qwen3 embeddings apply a search instruction on the query side; without the matching prefix on an OpenAI-compatible server, the stored side and the query side stop sitting in the same space. Dimension lock: because the width is read once at startup, moving from 0.6B to 4B later is a bank migration, not a config change — and even a same-dimension swap (BGE-M3 ↔ Qwen3-0.6B, or EmbeddingGemma 2 ↔ nomic at 768) changes the vector space, so bank vectors are not recomputed automatically. Plan a fresh bank or a manual re-embed/reindex; see Export, import, and migrating the memory bank. After leaving BGE-small, probe recall_min_scores against the new score distribution — thresholds calibrated on BGE-small can be too tight for Qwen3-0.6B and starve the reranker of candidates. For benchmarks and GGUF/Ollama background on the Qwen3 family, see Qwen3 Embedding and Reranker models on Ollama.

Reranking models

Embeddings recall a candidate set; the reranker scores each query/passage pair and re-orders it before Hindsight injects it. With the local provider, Hindsight loads a SentenceTransformers cross-encoder (or a generative reranker where supported). With an external Cohere-compatible endpoint, it posts the full Cohere-style body to HINDSIGHT_API_RERANKER_COHERE_BASE_URL. The reranker is optional, but once the embedder is fixed it is usually the largest recall-quality lever — and, on CPU, often most of the recall wall-clock. Rows are sorted from lightest to heaviest.

Model	Params	Type	Context	Runs on	Notes
`cross-encoder/ms-marco-MiniLM-L-6-v2`	23M	Cross-encoder	128	CPU	Hindsight default; ships in `hindsight:latest`. Sub-second for ~20 candidates. Keep as named fallback if a heavier BGE model is too slow.
`cross-encoder/ms-marco-MiniLM-L-12-v2`	~33M	Cross-encoder	512	CPU	Same family, higher accuracy; still English-leaning MS MARCO.
`cross-encoder/mmarco-mMiniLMv2-L12-H384-v1`	~33M	Cross-encoder	512	CPU	Hindsight-documented multilingual MiniLM reranker.
`BAAI/bge-reranker-base`	278M	Cross-encoder	512	CPU	Moderate upgrade over MiniLM; en/zh. Discussed often; skipped when v2-m3 already fits the latency budget.
`BAAI/bge-reranker-v2-m3`	568M	Cross-encoder	8k	CPU or GPU	Practical multilingual default on CPU. On a ~500-memory bank, ~2.6–2.8 s per recall at a 20-candidate floor (~140 ms/candidate; ~93–99% of recall time). Candidate caps 20/40/80; lowering the floor from 20 → 10 roughly halves rerank cost.
`Qwen/Qwen3-Reranker-0.6B`	0.6B	Generative (yes-token)	32k	GPU preferred	Instruction-aware, long context. Not a drop-in cross-encoder: needs recent SentenceTransformers generative-reranker support, and llama.cpp `/rerank` scoring has known wrong-score bugs (hindsight-benchmarks#11). Validate before adopting.
`BAAI/bge-reranker-v2-gemma`	2.5B	Cross-encoder	8k	GPU preferred	Strong multilingual quality; excessive for a few-hundred-memory personal bank when the GPU is already full.
`Qwen/Qwen3-Reranker-4B`	4B	Generative	32k	GPU	Higher-quality Qwen candidate; not viable without dedicated VRAM.
`Qwen/Qwen3-Reranker-8B`	8B	Generative	32k	GPU	Family top tier; same generative-scoring caveats, larger still.

Cloud API rerankers (Cohere, ZeroEntropy, Alibaba qwen3-rerank, TypeSafe, and similar) work in Hindsight but send memory text off-box. For a local-only cross-domain layout they are out of scope.

Two operational constraints dominate the pick. VRAM: if the conversational model already saturates the GPU (for example a Qwen3.8-27B Q5 on a 16 GB card with no headroom), every "GPU preferred" row is off the table and the live choice collapses to MiniLM or BGE-v2-m3 on CPU. Candidate count: HINDSIGHT_API_RERANKER_MAX_CANDIDATES (30 in the external-endpoint example below) bounds how many passages the model re-scores; each candidate is a full forward pass, so cut the floor before you jump a model tier. For Qwen3 via llama.cpp, the GGUF must keep the ranking head — an old convert_hf_to_gguf.py that drops it scores every passage near zero — and even a correct conversion still needs a known-good/known-bad passage test until the generative scoring path is confirmed.

Generative LLM tiers

Embedding and reranking models are not substitutes for the generative LLM used to extract and reason over memories. Hindsight supports a default LLM and operation-specific overrides, which can target the same llama.cpp server or independent endpoints.

Hindsight operation	Purpose	Practical model tier	Qwen examples
Retain	Extract facts, entities, decisions, and chronology	4B minimum for tests; 9B preferred	Qwen3.5-4B / Qwen3.5-9B
Recall	Semantic, keyword, graph and temporal retrieval	No generative LLM required	Qwen3-Embedding-0.6B + reranker
Reflect	Answer questions by synthesizing retrieved memories	9B to 27B+	Qwen3.5-9B / Qwen3.8-27B
Consolidation	Reconcile and compress related facts	9B preferred	Qwen3.5-9B / Qwen3.8-27B
Mental model refresh	Update longer-lived synthesized understanding	9B to 35B	Qwen3.5-9B / Qwen3.6-35B-A3B

These are operational recommendations, not official Hindsight minimum parameter counts. A small model may extract a port number correctly but miss that an earlier architectural decision has been superseded. Structured-output reliability and faithful handling of negation matter more than nominal parameter count alone. Hindsight's own temperature defaults are worth leaving alone — 0.1 for retain, 0.0 for consolidation, 0.9 for reflect — and for retain and consolidation, the non-thinking alias is worth serving if your llama.cpp build has one, because the small Qwen3.5 checkpoints spend many tokens in thinking mode.

For a deployment with an existing Qwen3.8-27B llama.cpp endpoint, start by sharing it across generative Hindsight tasks with low concurrency:

HINDSIGHT_API_LLM_PROVIDER=openai
HINDSIGHT_API_LLM_BASE_URL=http://WORKSTATION_IP:8080/v1
HINDSIGHT_API_LLM_MODEL=LocalMegaLLM
HINDSIGHT_API_LLM_API_KEY=dummy
HINDSIGHT_API_LLM_MAX_CONCURRENT=2
HINDSIGHT_API_RETAIN_LLM_MAX_CONCURRENT=1
HINDSIGHT_API_CONSOLIDATION_LLM_MAX_CONCURRENT=1

Later, route routine extraction to an independent 9B model and reserve the 27B for difficult reflection:

HINDSIGHT_API_RETAIN_LLM_PROVIDER=openai
HINDSIGHT_API_RETAIN_LLM_BASE_URL=http://MEMORY_LLM_HOST:8083/v1
HINDSIGHT_API_RETAIN_LLM_MODEL=MemoryLLM
HINDSIGHT_API_RETAIN_LLM_API_KEY=dummy

HINDSIGHT_API_REFLECT_LLM_PROVIDER=openai
HINDSIGHT_API_REFLECT_LLM_BASE_URL=http://WORKSTATION_IP:8080/v1
HINDSIGHT_API_REFLECT_LLM_MODEL=LocalMegaLLM
HINDSIGHT_API_REFLECT_LLM_API_KEY=dummy

The smaller 4B model is economical but needs testing; 9B is a sensible quality/performance compromise. If retain and consolidation share a small model, lower HINDSIGHT_API_CONSOLIDATION_LLM_BATCH_SIZE to 2 — the default batch of 8 is a long prompt for a small context window. Qwen3.6-35B-A3B is attractive for a separate well-provisioned inference server (it is a mixture-of-experts model with about 3 billion active parameters, so budget it as a small active model), while a 27B dense model is usually unnecessary for every routine retain event. Limiting background concurrency protects the Hermes conversational model from memory-maintenance bursts.

Serve and Wire Local Models

If the in-process local embedding and reranker settings in Choose Models are enough, you can skip the external Ollama and llama.cpp subsections and jump to wiring — or leave Compose on the local defaults shown there. External endpoints matter when the embedder or reranker should live on another host.

External embeddings: Ollama or llama.cpp

External embeddings are for when you want Qwen3-Embedding-0.6B (or another model) on a separate host. Two practical options are Ollama, which handles model acquisition and serving with little configuration, and llama.cpp, which gives precise control over GGUF quantization, CPU threads, device placement, and batching. Model background and benchmarks for both Qwen3 models are in Qwen3 Embedding and Reranker models on Ollama.

Qwen3-Embedding-0.6B returns 1024-dimensional vectors, and Hindsight reads that width on startup and sizes the database to match. A bank created with the bundled 384-dimensional BGE model cannot take this model — start from an empty database or volume, and decide the embedding model before the first retain. An embeddings provider switch later requires re-embedding stored records even if both models use the same dimensionality.

Option 1: Ollama for Qwen3-Embedding-0.6B

Install Ollama on the inference host using its official Linux installation documentation, then start the embedding model:

ollama pull qwen3-embedding:0.6b
ollama serve

For remote access, configure Ollama's OLLAMA_HOST=0.0.0.0:11434 in its service environment, restart the service, and firewall it to the Hindsight VM — the procedure is covered in Ollama remote access. A daemon bound only to 127.0.0.1 is invisible to the Hindsight container. First check the model name with ollama list, and test the standard embedding endpoint:

curl http://EMBED_HOST:11434/api/embed \
  -H 'Content-Type: application/json' \
  -d '{"model":"qwen3-embedding:0.6b","input":"A test memory"}'

Hindsight's OpenAI embeddings provider instead calls Ollama's OpenAI-compatible /v1/embeddings endpoint. Verify that endpoint works with your Ollama version before configuring Hindsight. Ollama is straightforward for embeddings; do not assume its API offers a compatible reranking endpoint. For reranking, prefer the in-process local BGE path below, or a separate llama.cpp / TEI endpoint after you have validated scores.

Option 2: llama.cpp for Qwen3-Embedding-0.6B

Download a verified Qwen3-Embedding-0.6B GGUF conversion. Use the GGUF filename you actually downloaded:

#!/bin/sh
# start-embedding.sh - CPU-only embedding server
set -eu
cd "$HOME/prj"
exec ./llama.cpp/llama-server \
  --host 0.0.0.0 --port 8081 \
  --model "$HOME/ggufs/Qwen3-Embedding-0.6B-Q8_0.gguf" \
  --alias qwen3-embedding-0.6b \
  --embedding --pooling last \
  --ctx-size 4096 --parallel 1 \
  --threads 4 --threads-batch 4 \
  --batch-size 512 --ubatch-size 256 \
  --n-gpu-layers 0

Qwen3 embeddings use last-token pooling, unlike the CLS pooling used by some BGE models. A CPU-only llama.cpp build is preferable if you want absolutely no CUDA initialization on a workstation whose GPU is occupied by a large conversational model. Do not enable --reranking on this embedding server: that combination is a known way to get a vector of zeros.

curl -s http://EMBED_HOST:8081/v1/embeddings \
  -H 'Content-Type: application/json' \
  -d '{"model":"qwen3-embedding-0.6b","input":"A test memory"}' \
  | jq '.data[0].embedding | length'

A full-dimension Qwen3-Embedding-0.6B output should contain 1024 floats. Check that the GGUF conversion and your Hindsight query/document instruction handling are compatible; instruction-sensitive embedding models can lose accuracy when prompted incorrectly.

Reranker: local BGE or Qwen3 via llama.cpp

Reranking is different from embeddings: the model scores each query/document pair and returns a relevance score. On a CPU-only Hindsight host — or whenever the conversational GPU has no VRAM headroom — the practical path is Hindsight's local provider with BAAI/bge-reranker-v2-m3 (or the lighter MiniLM default). No sidecar is required:

HINDSIGHT_API_RERANKER_PROVIDER=local
HINDSIGHT_API_RERANKER_LOCAL_MODEL=BAAI/bge-reranker-v2-m3
HINDSIGHT_API_RERANKER_MAX_CANDIDATES=20

Qwen3-Reranker-0.6B is a generative yes-token scorer, not a classic cross-encoder. It can be served through llama.cpp with a GGUF conversion designed for ranking, but treat that path as experimental until you confirm scores on a known-good/known-bad pair: an old convert_hf_to_gguf.py that drops the ranking head scores every passage near zero, and llama.cpp's native /rerank path has known wrong-score reports for this family.

#!/bin/sh
# start-reranker.sh - GPU or CPU as selected by --n-gpu-layers
set -eu
cd "$HOME/prj"
exec ./llama.cpp/llama-server \
  --host 0.0.0.0 --port 8082 \
  --model "$HOME/ggufs/Qwen3-Reranker-0.6B-Q8_0.gguf" \
  --alias qwen3-reranker-0.6b \
  --embedding --reranking --pooling rank \
  --ctx-size 4096 --parallel 1 \
  --batch-size 512 --ubatch-size 256 \
  --n-gpu-layers 99

Set --n-gpu-layers 0 to run it on CPU. The GPU version needs additional VRAM for compute buffers and context, not just model weights; avoid assuming that the file size is the total GPU allocation.

curl -s http://RERANK_HOST:8082/v1/rerank \
  -H 'Content-Type: application/json' \
  -d '{"model":"qwen3-reranker-0.6b", "query":"PostgreSQL storage", "documents":["Hindsight stores memories in PostgreSQL.","A recipe for bread."], "top_n":2}' \
  | jq

The PostgreSQL document should rank above the unrelated document; all-zero relevance scores mean the GGUF is the wrong conversion, and recall will look random even when the facts are stored. Check the exact results[].index and results[].relevance_score shape: Hindsight expects a Cohere-compatible rerank response, and llama.cpp compatibility can vary across builds.

Wire endpoints into Hindsight

Add the following environment settings to the Hindsight Compose service, or store them in its .env file. Pick one embeddings path — in-process local, Ollama, or llama.cpp — never two at once.

# Option A: in-process Qwen3 on the Hindsight host (CPU; no sidecar)
HINDSIGHT_API_EMBEDDINGS_PROVIDER=local
HINDSIGHT_API_EMBEDDINGS_LOCAL_MODEL=Qwen/Qwen3-Embedding-0.6B

# Option B: llama.cpp embedding service
# HINDSIGHT_API_EMBEDDINGS_PROVIDER=openai
# HINDSIGHT_API_EMBEDDINGS_OPENAI_BASE_URL=http://EMBED_HOST:8081/v1
# HINDSIGHT_API_EMBEDDINGS_OPENAI_MODEL=qwen3-embedding-0.6b
# HINDSIGHT_API_EMBEDDINGS_OPENAI_API_KEY=dummy
# HINDSIGHT_API_EMBEDDINGS_OPENAI_BATCH_SIZE=8
# HINDSIGHT_API_EMBEDDINGS_QUERY_PREFIX=$'Instruct: Given a web search query, retrieve relevant passages that answer the query\nQuery:'
# HINDSIGHT_API_EMBEDDINGS_PASSAGE_PREFIX=

# Option C: Ollama on another host
# HINDSIGHT_API_EMBEDDINGS_PROVIDER=openai
# HINDSIGHT_API_EMBEDDINGS_OPENAI_BASE_URL=http://192.168.1.40:11434/v1
# HINDSIGHT_API_EMBEDDINGS_OPENAI_MODEL=qwen3-embedding:0.6b-q8_0
# HINDSIGHT_API_EMBEDDINGS_OPENAI_API_KEY=dummy
# HINDSIGHT_API_EMBEDDINGS_QUERY_PREFIX=$'Instruct: Given a web search query, retrieve relevant passages that answer the query\nQuery:'
# HINDSIGHT_API_EMBEDDINGS_PASSAGE_PREFIX=

# Option: local BGE reranker (no sidecar; preferred on CPU)
HINDSIGHT_API_RERANKER_PROVIDER=local
HINDSIGHT_API_RERANKER_LOCAL_MODEL=BAAI/bge-reranker-v2-m3
HINDSIGHT_API_RERANKER_MAX_CANDIDATES=20

# Option: llama.cpp Cohere-compatible rerank endpoint (validate Qwen3 scores first)
# HINDSIGHT_API_RERANKER_PROVIDER=cohere
# HINDSIGHT_API_RERANKER_COHERE_BASE_URL=http://RERANK_HOST:8082/v1/rerank
# HINDSIGHT_API_RERANKER_COHERE_MODEL=qwen3-reranker-0.6b
# HINDSIGHT_API_RERANKER_COHERE_API_KEY=dummy
# HINDSIGHT_API_RERANKER_MAX_CANDIDATES=30

The variable names are provider-specific, and a near-miss is silent: HINDSIGHT_API_EMBEDDINGS_BASE_URL is the wrong key, and Hindsight quietly aims at OpenAI's own embedding endpoint instead of your local server. Use HINDSIGHT_API_EMBEDDINGS_OPENAI_BASE_URL. With the Cohere-compatible reranker path, Hindsight posts the full Cohere-style body to HINDSIGHT_API_RERANKER_COHERE_BASE_URL; if the container log shows a 404 for /v1/rerank/rerank, your build appends /rerank on its own and the value should stop at /v1.

For Options B and C, use the HINDSIGHT_API_EMBEDDINGS_QUERY_PREFIX value from Embedding models (Option A leaves it unset). Test query-to-passage retrieval on your actual memories after the first retain.

Control What Gets Remembered

With the stack running, the next failure mode is usually the write path: Hermes defaults to auto_retain: true, so every conversation turn can be extracted into the bank. Incorrect guesses, discarded plans, and exploratory statements become searchable facts, and later recall treats them as evidence — the loop described in self-reinforcing memory loops. Capture policy for Hindsight among other providers is summarized in agent memory providers compared.

Capture modes that exist today

Mode	Hermes / Hindsight settings	What you get
Continuous	`auto_retain: true`, `retain_every_n_turns: 1`	Every turn is retained and extracted
Batched auto	`auto_retain: true`, raise `retain_every_n_turns`	Fewer, larger retain payloads — still automatic, not a review queue
Explicit-only	`auto_retain: false` + tool `hindsight_retain`	Storage only when the agent calls the tool (for example after you say "remember this")
Session synthesis	End-of-session `hindsight_retain` of a summary, or bank reflect / consolidation	A curated write if you or the agent discipline the call — not a built-in flush-for-approval

Setting auto_retain to false disables automatic storage. It does not add a human approval dialog before each write. The agent can still call hindsight_retain whenever its tools allow; treat explicit retain as deliberate capture, not a pending queue.

The approval-queue gap

A common wish is a pending queue of candidate memories that you review and approve before they enter the bank. Hindsight under Hermes does not provide that contract today. Mnemosyne for Hermes implements provider-specific staged writes in some versions; that path is not a shared Hermes standard and does not apply to the Hindsight plugin. Until an upstream approval flow exists, the workable pattern is auto_retain: false, deliberate hindsight_retain, and inspection or deletion in the Hindsight control-plane UI before the next session relies on a bad fact. There is no missing write_approval key for Hindsight to hunt for in config.json.

Bank steering (missions, labels, directives, mental models)

People often call the next layer "finetuning." In Hindsight it is mostly bank configuration, not Hermes config.json alone: missions and extraction settings steer what retain keeps; labels structure entities; directives and mental models shape later synthesis.

Control	Where it lives	How it helps
Retain mission (`retain_mission` / Hermes `bank_retain_mission`)	Bank config	Plain-language "pay attention to X, ignore Y" injected into extraction
Extraction mode (`concise`, `verbose`, `custom`, …)	Bank config	`concise` (default) keeps long-term-worthy facts; `custom` replaces built-in rules via `retain_custom_instructions`
Entity labels	Bank config	Controlled `key:value` vocabulary at retain time → graph links and optional tags; improves retrieval structure, does not by itself reject wrong statements
Directives ("hard rules")	Bank directives API	Explicit instructions enforced during reflect (tag-scoped); constrain synthesis, not a write-approval gate
Mental models	Bank mental-model API	Standing answers to recurring questions, refreshed via reflect from matching memories; derived, so bad source facts still poison them
Reflect / observations mission	Bank config	Steers synthesis and consolidation, not the raw retain firehose

Missions, concise extraction, and labels reduce noise and improve retrieval. They do not replace turning off auto-retain when accuracy matters more than completeness. Prefer a short retain mission that names durable project decisions and ignores chat logistics; add entity labels only when you have a real taxonomy; add mental models for two or three recurring questions after the bank already holds clean facts.

Recommended hygiene profile

A concrete starting point for a personal Hermes bank:

{
  "mode": "local_external",
  "api_url": "http://HINDSIGHT_VM_IP:8888",
  "bank_id": "hermes",
  "memory_mode": "hybrid",
  "auto_recall": true,
  "recall_sync": false,
  "auto_retain": false,
  "retain_async": true,
  "recall_prefetch_method": "recall"
}

On the bank, set a retain mission and leave extraction in concise mode (via the control plane or Banks API). After an explicit retain, open the UI, confirm the observation, and delete mistakes before consolidation or mental-model refresh runs. Leave recall_prefetch_method at recall — reflect spends a chat-model call before every turn and injects synthesis rather than underlying facts.

Connect Hermes Agent to Hindsight

On the Hermes VM, install the catalog plugin, then run the setup wizard. Select Hindsight and the local_external mode; the plugin does not need to start embedded PostgreSQL or run the Hindsight model stack locally, and local_embedded would start a second Hindsight daemon beside the container you just ran.

hermes plugins install hindsight
hermes memory setup hindsight

There will be Interesting set of options of agent role using hindsight

hermes memory status

The external Hindsight connection lives in ~/.hermes/hindsight/config.json. An illustrative configuration that keeps continuous capture (Hermes defaults) is:

{
  "mode": "local_external",
  "api_url": "http://HINDSIGHT_VM_IP:8888",
  "bank_id": "hermes",
  "memory_mode": "hybrid",
  "auto_recall": true,
  "recall_sync": false,
  "auto_retain": true,
  "retain_async": true,
  "retain_every_n_turns": 1
}

For explicit-only retain and bank steering, use the hygiene profile in Control What Gets Remembered instead of leaving auto_retain on. recall_sync: false prioritizes interactive responsiveness but may inject recalled context on the following turn. Use synchronous recall when the current request must incorporate the latest memories before the model starts answering. retain_async: true avoids blocking ordinary responses while memories are processed.

The plugin's recall default is observations only; raw world and experience facts stay in the bank and out of the prompt until consolidation has written an observation, or until you widen recall_types to observation,world,experience. Widening it for the first week and watching the control plane, then narrowing, is a reasonable order.

Built-in Hermes memory stays active next to the provider; keeping MEMORY.md and USER.md on while you learn what Hindsight adds is fine, and the memory-system article is the place to decide whether the flat files should stay.

Restart the gateway if it is already running:

hermes gateway restart
hermes memory status

The status should identify Hindsight as the active provider. For a wider command reference, including Hermes' TUI and plugin commands, see the Hermes Agent CLI cheatsheet.

Validate Memory, Not Just Installation

Ask Hermes to remember a disposable fact about a test project, start another session, and ask for that fact again. Retention is asynchronous by default and extraction waits on the chat model, so give the main llama.cpp server a moment before judging. Then test a harder case: tell Hermes that an earlier decision has changed and ask which decision is current and why. Check Hindsight's UI and stored observations rather than assuming memory status proves ingestion and recall are correct.

If the answer comes back empty, inspect the control plane before changing models. A retain can succeed and still be invisible to recall while recall_types is observations only and consolidation has not run — widening recall_types is the first check. A connection refused in the embedding server log means the container cannot reach the embedding port, and all-zero reranker scores mean the GGUF is the wrong conversion.

Write-path choices (auto_retain, missions, approval gap) are covered in Control What Gets Remembered.

Troubleshooting and Operational Notes

Symptom	Likely cause	First check
Hermes says provider unavailable	Wrong `local_external` URL or network rule	`curl http://HINDSIGHT_VM_IP:8888/health`
Docker starts, PostgreSQL errors	Missing extension, wrong credentials, or insufficient shared memory	Container logs and `pg_extension`
Embeddings return 401 or 404	Incorrect OpenAI-compatible URL, wrong env key name, or model alias mismatch	Direct `/v1/embeddings` test and variable names
Reranking fails	Endpoint or response format mismatch, including doubled `/rerank` path	Direct `/v1/rerank` test and container log
Recall returns nothing usable	`recall_types` limited to observations, or zero reranker scores	Control plane, then `recall_types`, then reranker conversion
Existing bank reports dimension mismatch	Embedding model changed	Rebuild vectors/reimport with a consistent model
Hermes responses slow down	Synchronous recall or shared LLM contention	Turn off sync recall; cap retain concurrency
Irrelevant memories repeat	Weak extraction, noisy retrieval, or insufficient consolidation	Inspect observations and relevance scores

A practical small deployment is 2-4 vCPUs and 4-6 GB RAM for the Hindsight API VM when PostgreSQL and all inference are external. Disk allocation still depends on the chosen Docker image and retained image layers; allocate roughly 30-40 GB initially with room to expand. Size the PostgreSQL host based on actual memory volume, storage performance, and the other databases it serves.

Back up and restore the Hindsight bank

The memory bank is the only state that matters, and it lives in PostgreSQL. In Option B that is a database you administer; in Option A it is the embedded pg0 named volume inside the container. Back it up independently of the Hindsight container — recreating the container without first capturing the volume or the database discards the bank, and hermes backup does not reach a bank running in a separate service.

Option B — external PostgreSQL. Dump the hindsight database in custom format, then restore it into a scratch database to verify the archive before you trust it:

pg_dump -h POSTGRES_IP -U hindsight -Fc hindsight > hindsight-bank.dump

# restore into a scratch database to prove the dump is usable
createdb -h POSTGRES_IP -U postgres hindsight-verify
pg_restore -h POSTGRES_IP -U postgres -d hindsight-verify hindsight-bank.dump

Confirm the pgvector extension and a sample recall exist in the restored copy before you delete the original dump target. Keep the dump on a different host than the Hindsight VM, and run this as a PostgreSQL admin (a role that can create databases) rather than the application role.

Option A — all-in-one container. The embedded database is a named volume, so stop the container and tar the volume through a throwaway image:

docker stop hindsight
docker run --rm -v hindsight-pg0:/data -v "$PWD":/backup alpine \
  tar czf /backup/hindsight-bank.tgz -C /data .
docker start hindsight

The mount point inside the tar container (/data) is arbitrary; it simply aliases the named volume so tar can read it. To restore, stop the container, clear the volume, and untar the archive back into it, or point the container at the external PostgreSQL restore from Option B.

Take a snapshot at three moments rather than on a calendar schedule: before changing the embedding model, before a major Hindsight or PostgreSQL upgrade, and before a reflect or consolidation pass that rewrites the bank. A backup you have restored into a scratch bank is worth more than one you have only written.

Export, import, and migrating the memory bank

Backup and restore protect the bank in place. Export/import moves it between Hindsight instances, and that is what a model change actually forces you to do.

Changing the embedding model is not a configuration edit. The vectors in the bank were computed with the old model, and a different model — for example the bundled 384-dimensional BGE embedder versus 1024-dimensional Qwen3-Embedding-0.6B — writes into a different space and usually a different width. You cannot re-point the existing instance at the new embedder and keep the old vectors; the dimensions must be recalculated by re-embedding every memory with the new embedding model.

Because the embedder and its configuration are set per Hindsight instance, the clean path is to stand up a fresh instance with the new embedding configuration and migrate the bank into it, rather than rebuilding vectors inside the one that is already serving recall. The Hindsight API can move the bank too, but for a one-off change export/import is the easier and more inspectable route:

# export the bank from the old instance
docker exec hindsight \
  hindsight-admin export-bank \
    --bank hermes2 \
    --output /tmp/hermes2.zip

docker cp hindsight:/tmp/hermes2.zip ./hermes2.zip

ls -lh hermes2.zip
unzip -t hermes2.zip

# import the bank into the new instance
docker cp ./hermes2.zip hindsight-q06:/tmp/hermes2.zip

docker exec hindsight-q06 \
  hindsight-admin import-bank \
    --archive /tmp/hermes2.zip

unzip -t verifies the archive before you ship it to the new instance. After the import, re-point Hermes at the new instance's api_url in ~/.hermes/hindsight/config.json, restart the gateway, and run the memory validation from earlier: a retained fact should recall through the new embedder. Leave the old bank and its vectors in the old instance until the new one behaves, so you still have a working fallback.

Monitor, prune, and keep the bank stable

With the bank running, the day-to-day job is watching what the extraction model has decided to store and keeping derived state in check. Use the control plane UI (hindsight-embed -p hermes ui start) to inspect observations, relevance scores, and consolidation output rather than assuming memory status proves ingestion and recall are correct. Duplicated or stale memories are usually a weak extraction or an under-run consolidation, so fix the extraction tier or consolidation cadence before widening retrieval.

Two maintenance traps are worth testing before you rely on the bank. First, deleting an observation does not guarantee every derived representation disappears with it — a reflect fact or graph edge built from it can survive the delete, so verify deletion end to end, the same provenance concern covered in self-reinforcing memory loops. Second, keep HINDSIGHT_API_WORKER_ID stable across recreations and pin a specific image tag: a task that was mid-flight when the container died stays stuck under the old worker id, and a moving tag is a poor match for a database you intend to keep.

References

Upgrading to Go 1.27: What Breaks, What to Adopt First

Rost — Fri, 16 Oct 2026 09:33:44 +0000

Go 1.27 shipped in August 2026, and for most modules the upgrade is a single go.mod line followed by a go fix pass — but a handful of changes bite silently. Here is what actually breaks and what to adopt first.

The headline changes are structural: encoding/json is now backed by the new json/v2 implementation under the hood, the uuid package joined the standard library, and methods can finally declare their own type parameters. None of these force a rewrite, and the Go team kept the usual compatibility promise — almost everything compiles and runs unchanged.

Not everything is quiet, though. Several GODEBUG settings now fail the build at their old values, compress/flate produces different bytes, go test enforces a new stdversion check, and compiler-generated symbol names changed. If you need a syntax refresher on the parts that did not move, the Go cheatsheet stays valid; this guide covers the rest, with commands and fixes for each item.

The upgrade path in one diagram

The Go team recommends a build-first, fix-second order. go fix rewrites code to new APIs and idioms, but only a clean build tells you whether any of the runtime-level changes hit you.

flowchart TD
    A[Install Go 1.27 toolchain] --> B["go build ./..."]
    B --> C{Build passes?}
    C -- "No: removed GODEBUG value" --> D[Set final default or delete the line]
    D --> B
    C -- Yes --> E["go test ./..."]
    E --> F{"stdversion + vet clean?"}
    F -- "No: symbol too new" --> G[Raise go line or swap the symbol]
    G --> E
    F -- Yes --> H["go mod tidy"]
    H --> I["go fix ./..."]
    I --> J[Review JSON golden tests and diffs]

Start with the toolchain itself. If you manage toolchains through the go command, either pin the version in go.mod or install explicitly:

go install golang.org/dl/go1.27@latest
go1.27 download
go1.27 version

Alternatively, set GOTOOLCHAIN=go1.27.0 in the environment, or add toolchain go1.27.0 to go.mod and let the go command fetch it. The module-focused companion on Go workspace structure explains how the go and toolchain lines interact across multiple modules; the short version for a single module is below.

Step 1: bump the `go` line and clear removed GODEBUG settings

go mod edit -go=1.27
go mod tidy
go build ./...

Raising the go directive turns on the new language features (generic methods, promoted-field keys in struct literals, wider function type inference) and makes the go command strict about GODEBUG settings whose support has been removed. Go 1.27 removed these permanently:

asynctimerchan — timer and ticker channels are now always unbuffered
gotypesalias
tlsunsafeekm, tlsrsakex, tls3des, tls10server
x509keypairleaf

The new rule: the go command still accepts these settings in go.mod (godebug entries) or in source (//go:debug comments) if they are set to the final default value that existed before removal. An old value — the one that disabled a behaviour change — fails the build:

//go:debug asynctimerchan=0  // build error: old value of a removed setting
//go:debug asynctimerchan=1  // accepted, but delete the line instead

If your build fails here, the code depended on pre-Go-1.23 buffered timer channels. The fix is in the code, not the config: check Timer.Stop and Timer.Reset usage against the context, cancellation, and timeout patterns that rely on the synchronous channels. The same search applies to the TLS-related settings — anything pinning tls3des=1 or tls10server=1 was keeping old cipher suites alive, and Go 1.27 refuses to compile it.

Step 2: run `go test` — the stdversion trap

go build and go test no longer agree. Go 1.27's go test runs the stdversion vet check by default, which reports standard-library symbols that are too new for the go version in force in the referring file.

// go.mod still says "go 1.26"
func Base(p string) string {
    _, file, ok := strings.CutLast(p, "/") // added in Go 1.27
    if !ok {
        return p
    }
    return file
}

With a 1.27 toolchain, go build succeeds — the compiler has strings.CutLast — while go test fails the vet step because go.mod still declares 1.26. With a real 1.26 toolchain the file would not build at all. The lesson is procedural: never validate an upgrade with go build alone; run the full test suite, then bump the go line. The vet suite in go test also picked up a new printf finding: fmt.Errorf("...: %w", p) where p is a *E and E implements error now warns, because wrapping a pointer produces type assertions and errors.Is checks that fail. That failure mode shows up in error-handling code exactly as described in the Go error handling architecture patterns — check the operand type, not just the verb.

encoding/json: the v2 engine underneath

The biggest under-the-hood change: encoding/json/v2 (experimental since Go 1.25 as GOEXPERIMENT=jsonv2) graduated, and the classic encoding/json package is now backed by the v2 implementation. For v1 callers, behaviour is preserved — nil slices still encode as null, map keys still sort — with one visible difference: the exact text of some error messages changed. If tests or monitoring match on error strings, expect diffs.

Three things can still go wrong:

Error text. Any strings.Contains(err.Error(), ...) assertions keyed to the old v1 wording need updating.
Performance-sensitive paths may shift. Unmarshalling is faster, but the rewrite changes which operations allocate; benchmarks flagged in CI will show it.
Compatibility escape hatch. GOEXPERIMENT=nojsonv2 restores the original v1 implementation at build time. It is temporary — plan to remove it, not to ship with it.

A second-order effect for golden tests: compress/flate's encoder changed in the same release, so archive/zip, compress/gzip, compress/zlib, and image/png outputs may differ byte-for-byte from Go 1.26 even though they decode identically. Regenerate compressed fixtures rather than pinning old bytes.

Migrating to the v2 API deliberately

The v2 API is a separate import (encoding/json/v2) with Marshal, Unmarshal, and an options system. The supported migration path is to make v2 calls behave like v1 first, then tighten selectively:

import (
    "encoding/json/v2"
    "encoding/json/jsontext"
)

// Step 1: identical behaviour to encoding/json — a safe, mechanical change
out, err := jsonv2.Marshal(v, jsonv2.DefaultOptionsV1())

// Step 2: opt in to one v2 default at a time
out, err := jsonv2.Marshal(v,
    jsonv2.DefaultOptionsV1(),
    jsonv2.FormatNilSliceAsNull(false)) // nil slices now encode as []

Once you drop DefaultOptionsV1, v2 defaults apply, and they are stricter and more interoperable than v1:

Behaviour	v1 (`encoding/json`)	v2 default
Invalid UTF-8 in strings	Replaced with U+FFFD	Error
Duplicate names in an object	Allowed	Error
Nil slice / nil map	`null`	`[]` / `{}`
Struct field matching	Case-insensitive fallback	Case-sensitive
Unknown object members	Ignored	Ignored (`RejectUnknownMembers` to error)
Map key ordering	Always sorted	Unsorted by default
`time.Duration`	Built-in string encoding	No default encoding

The unsorted map keys are the one that surprises golden-file tests: v2 skips the sort because sorting costs time. When you need stable output — fixtures, snapshot tests, hashed representations — pass json.Deterministic(true), which restores sorted output for maps and structs. The ORM comparison article's models all carry json tags; those tags keep working in v2, but the format tag option was removed during the experiment and inline was renamed to embed, so grep your model definitions for both before migrating. For low-level needs — streaming, token-by-token processing, custom syntax handling — the companion encoding/json/jsontext package exposes an Encoder/Decoder state machine over JSON text.

stdlib `uuid`: dropping `github.com/google/uuid`

Go 1.27 adds uuid to the standard library, implementing RFC 9562. The API surface covers the common cases:

id := uuid.New()    // currently equivalent to NewV4
v7 := uuid.NewV7()  // time-sortable, 48-bit timestamp + random
u, err := uuid.Parse("550e8400-e29b-41d4-a716-446655440000")

github.com/google/uuid is the second most-depended-on Go module, so this migration matters at scale. The two APIs are close — types are [16]byte values comparable with == and usable as map keys, String(), MarshalText/UnmarshalText round-trip — and the upstream project is preparing go:fix inline shims so go fix can rewrite uuid.NewV4() calls to the standard package and delete the dependency.

The interesting adoption decision is NewV7. Version-7 UUIDs embed a millisecond timestamp in the most significant bits, so values sort in creation order (except when the system clock moves backwards). For database primary keys that means better B-tree locality than random UUIDv4, and created_at-like ordering for free in the identifier itself — the same property that drives time-sortable key discussion in multi-tenant data models.

Language changes: generic methods and friends

Generic methods are the release's most requested language feature: a method declaration may now declare its own type parameters.

// Go 1.27: math/rand/v2 gained a generic method alongside the function
func (r *Rand) N[Int intType](n Int) Int

Two restrictions keep the type system decidable: interface methods may not declare type parameters, and interface methods cannot be implemented by generic methods. Alongside them, struct literal keys may now be any valid field selector (not just top-level field names), and function type inference generalizes to all contexts where a generic function is assigned to a matching function-type variable. None of these break existing code; they remove workarounds — package-level generic helper functions and explicit instantiations.

go fix grew modernizers for adjacent cleanups: atomictypes, embedlit, slicesbackward, and unsafefuncs. One caution from the hands-on upgrade labs: run go fix -fix=atomictypes deliberately rather than in bulk on libraries, since it can change the types of package-level variables that other packages reference. The waitgroup modernizer was renamed to waitgroupgo; note that sync.WaitGroup.Go (from 1.25) already covers most of what it detects — see the bounded concurrency patterns for how it slots into worker-pool shutdown.

Things that quietly break

These are the changes that compile fine and still cost you a release day:

Function-literal symbol names. The compiler now generates simpler, inlining-independent names for closures, and may merge identical function literals. Tests that assert on symbol names in tracebacks or profiles will fail; code that compares function code pointers for equality is exposed in more cases.
gofmt alignment. A fix to column-alignment logic (it previously varied across CPU architectures) reflows a small number of already-formatted files. Expect whitespace-only diffs in a gofmt ./... commit after the upgrade.
go mod tidy rewrites go.mod. For modules on go 1.27, tidy now merges duplicate require blocks into at most two blocks (direct, indirect), preserving comments. Diff your go.mod and resolve any merge-conflict residue before the release commit, not after.
Traceback labels. Tracebacks for modules on go 1.27 include runtime/pprof goroutine labels in the header line; disable with GODEBUG=tracebacklabels=0 if labels carry sensitive data.
go tool trace binds to localhost. -http=:6060 now listens on localhost only; pass -http=0.0.0.0:6060 explicitly when you need remote access (and mind that a profiling port on a shared host is a risk — see the production profiling workflow for the hardened setup).

Experiments worth watching

Two experimental packages shipped in 1.27, both enabled with GOEXPERIMENT=simd at build time:

simd — a portable, vector-size-agnostic API (Int8s, Float32s, ...) that maps to hardware instructions where available. It supports AVX/AVX2/AVX-512 on amd64, NEON on arm64, and WebAssembly SIMD. The standard library already uses this machinery internally: Go's Swiss Table map implementation benefits from it.
simd/archsimd — the architecture-specific layer for code that needs to know the vector width.

On the crypto side, crypto/mldsa implements the post-quantum ML-DSA signature scheme from FIPS 204 in three parameter sets (MLDSA44, MLDSA65, MLDSA87), integrated into crypto/x509 and offered as TLS 1.3 signature schemes. Hybrid post-quantum key exchange can now be forced via Config.CurvePreferences even when the tlsmlkem/tlssecpmlkem GODEBUG fallbacks are set. Neither is a migration item today, but both shape the TLS configuration surface your services will grow into.

New APIs worth adopting immediately

A few additions solve everyday friction and cost nothing to adopt:

// strings.CutLast / bytes.CutLast — the LastIndex dance, replaced
before, after, found := strings.CutLast("a/b/c.tar.gz", ".")

// httptest.NewTestServer — in-memory fake network, auto-cleanup
srv := httptest.NewTestServer(t, mux) // no real port, no defer srv.Close()

httptest.NewTestServer creates an httptest.Server backed by an in-memory network instead of a TCP listener and registers its own t.Cleanup. Its real payoff is pairing with testing/synctest: HTTP round-trips run in synthetic time, making timeout and retry tests fast and deterministic — the combination is demonstrated in the synctest concurrency testing article.

Upgrade checklist

The order matters more than the items; each step assumes the previous one is green.

Step	Small module	Service / monorepo
1. Toolchain	`go install golang.org/dl/go1.27@latest`	Pin `toolchain go1.27.0`; align CI images
2. `go.mod`	`go mod edit -go=1.27 && go mod tidy`	Same, plus resolve `require`-block merge diffs
3. Removed GODEBUG	Grep `//go:debug` and `godebug:`	Also audit deployed configs and flags
4. `go build ./...`	Fix compile errors	Fix per-module; bisect failures
5. `go test ./...`	Fix `stdversion`/`printf` findings	Run before the `go` bump to catch symbol drift
6. `go fix ./...`	Review the diff	One modernizer at a time
7. JSON/golden tests	Regenerate fixtures	Check `flate`-based fixtures (zip/gzip/png) too
8. `GOEXPERIMENT=nojsonv2`	Skip	Temporary only; file issues for what forced it

Closing notes

Go 1.27's breaks cluster around edges most programs never touch — removed GODEBUG opt-outs, symbol-name assertions, compressed-byte fixtures — while its wins land where every program lives: faster JSON, standard UUIDs, generic methods. A clean upgrade is mostly discipline: go build, then go test, then go fix, in that order, with the go line raised before the test run rather than after it. The nojsonv2 escape hatch and the asynctimerchan acceptance-at-final-default rule buy time, but both are scheduled to disappear, so treat this upgrade as the deadline for the deferred fixes it inherits from 1.23–1.26.

Bounding Concurrency in Go: errgroup, Worker Pools, and Backpressure

Rost — Sat, 10 Oct 2026 07:53:44 +0000

Unbounded fan-out is the default failure mode of Go concurrency: every go statement is one more task in flight, with no ceiling and no owner. This article is about giving both.

Cancellation and bounding are separate controls. Cancellation decides when work stops; bounding decides how much of it exists at once. A service can cancel perfectly and still fall over because ten thousand goroutines each decided, independently, that now was a good time to call the database.

The pages around this one cover the other half of the problem. Go context.Context Done Right treats context as the control plane for stopping work; here the subject is the capacity plane — errgroup.SetLimit, semaphore channels, worker pools, and bounded queues — plus the one leak shape that survives errgroup.Wait even when every cancel path looks wired up.

Five primitives, one decision table

Primitive	Partial results	Fail-fast	Backpressure	Dynamic work	Best for
`sync.WaitGroup` + `errors.Join`	yes	no	no	no	wait-all batches where every result counts
`errgroup.WithContext`	no	yes	no	no	fan-out where the first error ends the request
`errgroup.SetLimit(n)`	no	yes	yes (blocks `Go`)	no	bounded fail-fast fan-out
Semaphore channel	yes	manual	yes (blocks acquire)	yes	custom admission control
Worker pool	yes	manual	yes (queue fills)	yes	steady streams, long-lived workers

Two questions pick the row. First: when one task fails, do you still want the others' results? If yes, fail-fast group semantics discard work you needed. Second: does work arrive continuously, or is it a fixed batch you can count before launching? Groups handle batches; pools and queues handle streams.

errors.Join is what makes the plain WaitGroup row viable — it collects every worker's error instead of keeping only the first, which pairs with the boundary-translation rules in Go Error Handling Architecture.

The leak that survives errgroup.Wait

The best-known errgroup failure mode has nothing to do with bounding. It is a cancellation gap, and it hides behind code that looks correct:

g, gctx := errgroup.WithContext(ctx)
results := make(chan int) // unbuffered

go func() { // producer — nobody owns this goroutine
    for i := 0; i < 100; i++ {
        results <- i
    }
    close(results)
}()

g.Go(func() error { // consumer — group member
    for r := range results {
        if r == 5 {
            return fmt.Errorf("save failed")
        }
    }
    return nil
})

err := g.Wait() // returns at the consumer's error

When the consumer returns, gctx cancels and Wait returns. The producer is parked on results <- i — and context cancellation does not unblock a channel send. Nobody owns that goroutine, so it stays parked for the life of the process. Run this shape in a loop and the leak is exactly linear:

Variant	Live goroutines after 500 iterations
Send without a `ctx.Done` arm	501 (+500 leaked, one per iteration)
Send wrapped in `select` with `<-gctx.Done()`	502 (+1 baseline)

// the fix: every channel operation inside a cancellable task
// gets a Done arm
select {
case results <- i:
case <-gctx.Done():
    return
}

The same gap exists on the receive side. The rule is mechanical: every channel operation inside code that can be cancelled gets a <-ctx.Done() arm, and every goroutine has an owner that waits on it or cancels it. goleak in CI catches the cases review misses — it slots into the same automated quality gate as the tools in Go Linters: Essential Tools for Code Quality.

Two related shapes are worth naming. If the producer is a group member, the same code produces not a leak but a deadlock — Wait blocks forever waiting for the parked send, which at least fails loudly. And a producer that sends with select but receives on a channel whose consumer has quit leaks the same way; Done arms belong on both sides.

errgroup.SetLimit: bounded fail-fast fan-out

SetLimit turns an errgroup into an admission-controlled group. Each Go call beyond the limit blocks until a slot frees:

g, ctx := errgroup.WithContext(ctx)
g.SetLimit(8)
for _, item := range items {
    g.Go(func() error {
        return process(ctx, item)
    })
}
err := g.Wait()

Three semantics matter in production. Go blocks, so the loop above applies backpressure to the producer — that is usually what you want, but it means the caller's goroutine can stall, so a request handler feeding a large batch needs its own timeout budget around the whole loop. SetLimit(0) blocks every Go call forever. And changing the limit while goroutines are active panics — the limit is fixed for the life of the group.

SetLimit inherits group semantics: the first error cancels the context and Wait returns the first error, discarding results of work still in flight. It is the right primitive when failing the whole operation on first failure is correct — fetching one page of data, calling a set of independent health checks, fanning a request out to shards.

Semaphore channels: backpressure as a feature

A buffered channel of capacity n is a semaphore with no dependencies:

sem := make(chan struct{}, 8)
var wg sync.WaitGroup
for _, item := range items {
    wg.Add(1)
    sem <- struct{}{} // blocks when 8 tasks are in flight
    go func() {
        defer wg.Done()
        defer func() { <-sem }()
        _ = process(ctx, item)
    }()
}
wg.Wait()

The acquire happens before the go statement, so goroutines are only ever created when a slot exists. The buffer size is the contract: it is simultaneously the concurrency ceiling and the queue of pending admissions.

The primitive earns its place when you need admission control the groups do not express — context-aware acquisition, per-downstream pools, or partial results on failure:

select {
case sem <- struct{}{}: // slot acquired
case <-ctx.Done():      // caller gave up before getting in
    return ctx.Err()
}

Measured on the same workload, the semaphore channel and SetLimit(8) finish within 1.5% of each other — the choice between them is semantics, not performance.

Worker pools: for streams, not batches

A fixed pool of long-lived workers separates admission (the queue) from execution (the workers):

func pool(ctx context.Context, workers int) chan<- func() {
    jobs := make(chan func())
    var wg sync.WaitGroup
    for i := 0; i < workers; i++ {
        wg.Add(1)
        go func() {
            defer wg.Done()
            for {
                select {
                case f, ok := <-jobs:
                    if !ok {
                        return
                    }
                    f()
                case <-ctx.Done():
                    return
                }
            }
        }()
    }
    return jobs
}

Pools fit work that arrives continuously — queue consumers, polling loops, request workers — where a per-batch group would be created and torn down endlessly. Since Go 1.25, sync.WaitGroup.Go removes the Add/Done pairing boilerplate for the common shape. The shutdown contract is the part to get right: closing jobs terminates workers after the queue drains; cancelling ctx abandons queued work. Both are legitimate, but only one can be the documented behavior.

Bounded queues: block or drop

A buffered channel is also a queue with a ceiling. Once the buffer fills, the producer blocks — and that moment is the design decision. Blocking propagates pressure upstream to whoever feeds the queue; dropping sheds load at the cost of lost work. A monitoring pipeline can drop; an order pipeline must block or spill to durable storage.

select {
case queue <- event: // admitted
default:            // queue full: drop policy lives here
    dropped.Add(1)
}

Whatever the policy, make it explicit and measured. A silent default branch is where events go to disappear.

What bounding costs — measured

The same workload, 10,000 tasks of 5 ms simulated I/O each, Go 1.27.1:

Strategy	Peak goroutines	Wall time
Unbounded `go` per task	~10,000 launched	18 ms
`errgroup.SetLimit(8)`	8	6.9 s
Semaphore channel (cap 8)	8	6.6 s

The unbounded run wins on wall time because nothing in the experiment pushes back — ten thousand timer goroutines simply sleep in parallel. That is exactly the trap. The cost of unbounded fan-out is never CPU in your own process; it is ten thousand simultaneous connections, a downstream that starts timing out under the burst, and a memory graph that follows the queue depth of whoever you are calling. On a real dependency, the unbounded run does not finish in 18 ms — it times out. The bounded runs pay 6.6 seconds because 10,000 tasks ÷ 8 slots × 5 ms is 6.25 s of unavoidable serialization, and the cap is the point: it turns an uncontrolled burst into a predictable 6.25-second drain.

One refinement matters when the cap is higher than a handful of slots: keep the cap at or below what the downstream can sustain concurrently, and derive it from that limit (connection pool size, rate limit, worker capacity) rather than from a feeling about "reasonable" parallelism.

Measuring and testing bounded code

Three signals cover most of it. Goroutine count trends (/sched/goroutines:goroutines exported via the Prometheus Go collector) catch leaks as a slope. Scheduler latency (/sched/latencies:seconds) catches saturation — runnable goroutines waiting for CPU time is real pressure regardless of the count. A pprof delta of two goroutine dumps pinpoints where the parked goroutines live.

For the tests, bounded workers are the case Testing Concurrent Go Code with synctest was built for: fake time makes a full 10,000-task drain run in milliseconds, synctest.Wait replaces sleep-and-hope for quiescence, and a bubble fails fast when a worker stays durably blocked — the leak repro above dies in a test bubble instead of in production.

At service scale, the same bounding question reappears between services, where the semaphore becomes a rate limit and the circuit breaker guards the downstream — Circuit Breaker Pattern in Go covers that layer, and Go Microservices for AI/ML Orchestration shows the queue-backed forms the orchestration tier takes.

Choosing

flowchart TD
    A[Work to run concurrently] --> B{Fixed batch or continuous stream?}
    B -- continuous stream --> C[Worker pool or bounded queue]
    B -- fixed batch --> D{Partial results needed<br/>on first failure?}
    D -- yes --> E[WaitGroup + errors.Join<br/>or semaphore channel]
    D -- no --> F[errgroup.WithContext]
    F --> G{Need a concurrency ceiling?}
    G -- yes --> H[errgroup.SetLimit]
    G -- no --> I[group without limit<br/>only when the batch is provably small]
    C --> J{Full queue: block or drop?}
    J -- block --> K[Backpressure upstream]
    J -- drop --> L[Shed load, count the drops]

The table and the flowchart compress to one habit: before writing go, name the ceiling and the owner. The ceiling is SetLimit, a semaphore, or a bounded queue; the owner is a Wait, a ctx.Done arm on every channel operation, or both. Every goroutine-creation site that can answer those two names in one glance is one that will not page anyone at 3 a.m.

Useful links

Clone a VM in Virt-Manager with virt-clone and virt-sysprep

Rost — Fri, 09 Oct 2026 11:36:19 +0000

Virt-Manager (the libvirt GUI) copies a KVM disk and assigns the clone a UUID and MAC. The guest keeps its machine-id and SSH host keys until virt-sysprep clears them.

This walkthrough uses an Ubuntu 26.04 guest and a 14 GB qcow2 image stored at /mnt/extra2/vms. The disk commands are the same for other Linux guests. virt-sysprep does not operate on Windows guests.

The host already needs KVM and libvirt. Install KVM on Ubuntu 24.04 is that setup, and it pulls in virtinst, which provides virt-clone. virt-sysprep, virt-customize, virt-cat, and guestfish come from a separate package:

sudo apt install -y libguestfs-tools

What actually clones a VM

libvirt splits the job across three tools. virt-clone copies the disk images and defines a new domain with the same virtual hardware. It changes the domain name, the UUID, and the host-side MAC. Passwords, the hostname, static addresses, and files inside the guest are unchanged, which is the behaviour virt-clone(1) documents.

virt-sysprep opens the disk with libguestfs and edits the guest filesystem. The virt-sysprep(1) manual is explicit that the edit is in place: there is no output-file flag, so the image you pass to -a is the image that changes. Copy or clone first if you still need the source.

qemu-img info reports whether that file is standalone or the top of a backing chain. qemu-img convert writes a new file with the chain resolved.

flowchart LR
    A[guest shut down] -->|virt-clone| B[new domain and disk]
    B -->|qemu-img info| C{backing file?}
    C -->|yes| D[qemu-img convert]
    C -->|no| E[standalone qcow2]
    D --> E
    E -->|copy, then virt-sysprep| F[template]
    F -->|per-instance customize| G[imported guest]

Flatten before sysprep when a backing line is present. Sysprep the copy, then derive each running guest from that file.

Cloning the disk in Virt-Manager

Shut the guest down. virt-clone rejects a running domain, and a copy of a live qcow2 is not a consistent filesystem.

In Virt-Manager, right-click the guest and choose Clone. Set the new name. On the storage step, give each disk a new path. Leaving the original path in place defines a second domain that opens the same file, so both guests write the same blocks.

The new file lands beside the source. A default pool uses /var/lib/libvirt/images/. A disk that already lives on /mnt/extra2/vms is cloned into that directory. Note the path virt-clone prints, or read it back:

virsh domblklist ubuntu26.04-base14gb-clone

The command-line equivalent, with the name and the disk path chosen explicitly:

virt-clone --original ubuntu26.04-base14gb \
  --name ubuntu26.04-base14gb-clone \
  --file /mnt/extra2/vms/ubuntu26.04-base14gb-clone.qcow2

--auto-clone invents the name and the disk path. That is convenient for a throwaway copy. A template path is easier to keep when you pass --name and --file yourself. Omit --mac, or pass --mac RANDOM, and virt-clone generates an address in the QEMU range 52:54:00:xx:xx:xx.

--reflink asks for a lightweight copy. The Ubuntu 26.04 virt-clone manual limits that flag to raw images on the same btrfs filesystem, so it skips these qcow2 files. A qcow2 overlay is a separate setup, covered with qemu-img create -b below.

Does your clone have a backing file?

Run this on the new disk before you treat it as independent:

qemu-img info --backing-chain /mnt/extra2/vms/ubuntu26.04-base14gb-clone.qcow2

A standalone image lists file format, virtual size, and disk size. An overlay also lists a backing file: line pointing at the image it reads through. virtual size is the guest's disk size, here about 14 GB. disk size is how much space the file currently occupies, which is small for a fresh overlay and grows as the guest writes.

virt-clone is not the usual source of that line. An overlay comes from qemu-img create -b, from a disk-only snapshot, or from a tool that created a qcow2 delta on purpose. The info check is what tells those cases apart from a full copy.

Keep the overlay only when the base will stay where it is and will not be written again. Every write from the clone lands in the small file; reads that miss in the overlay come from the base. Moving or editing the base breaks every guest that names it.

flowchart TB
    subgraph linked
        O[base.qcow2 read-only]
        C1[vm1.qcow2 deltas only]
        C2[vm2.qcow2 deltas only]
        C1 -->|backing file| O
        C2 -->|backing file| O
    end
    subgraph flat
        F1[vm1-flat.qcow2 standalone]
        F2[vm2-flat.qcow2 standalone]
    end

To make a standalone file, convert the overlay. -p prints progress. The destination is a new file; the overlay and the base are left as they are:

qemu-img convert -p -O qcow2 \
  /mnt/extra2/vms/ubuntu26.04-base14gb-clone.qcow2 \
  /mnt/extra2/vms/ubuntu26.04-base14gb-flat.qcow2

Run qemu-img info on the flat file and confirm there is no backing file: line. A template is that standalone file. Overlays are how you stamp guests from it later, not the template itself.

A btrfs reflink is a third case. cp --reflink=auto can share physical extents between two files, and qemu-img info still shows no backing file, because each path is a complete image. That copy is independent at the qcow2 layer even though the filesystem has not duplicated every block yet.

virt-sysprep: resetting the identity

Copy the standalone image, then sysprep the copy. --reflink=auto shares extents when the filesystem supports it and otherwise makes a normal copy, including on ext4. -n is a dry run that discards its writes:

cp --reflink=auto \
  /mnt/extra2/vms/ubuntu26.04-base14gb-flat.qcow2 \
  /mnt/extra2/vms/ubuntu26.04-base14gb-clean.qcow2

virt-sysprep -n -a /mnt/extra2/vms/ubuntu26.04-base14gb-clean.qcow2
virt-sysprep -a /mnt/extra2/vms/ubuntu26.04-base14gb-clean.qcow2 \
  --hostname ubuntu-template

On a default Ubuntu pool the image is often mode 0600 and owned by libvirt-qemu. The manual recommends giving your user write access rather than running as root. When the qemu user owns the file, sudo virt-sysprep -a ... is the invocation that can open it. The guest must be shut down, and nothing else should have the image open.

--hostname is a customize option, and customize is part of the default operation set, so it runs in the same invocation. There is no separate default operation that clears /etc/hostname on its own. --operations replaces that default set when you pass it; --operations defaults,-ssh-userdir keeps the defaults and turns one operation off.

These are the operations that decide whether the clone collides with the original. The list is the one in the current virt-sysprep manual, which is worth reading because a short --operations list silently drops the rest:

Operation	Default	Effect on the guest
machine-id	yes	Clears the machine ID. systemd generates a new one on the next boot when `/etc/machine-id` is empty.
ssh-hostkeys	yes	Removes `/etc/ssh/ssh_host_*`. The next boot creates new host keys.
ssh-userdir	yes	Removes `.ssh` under `/root` and `/home/*`, including `authorized_keys` and private keys stored in the guest.
dhcp-client-state	yes	Removes DHCP client leases.
udev-persistent-net	yes	Removes udev rules that map an old MAC to a fixed name such as `eth0`.
net-hwaddr	yes	Removes `HWADDR` from Fedora and RHEL `ifcfg-*` files. It does not edit Ubuntu netplan.
lvm-uuids, lvm-system-devices	yes	Assigns new LVM UUIDs and removes `/etc/lvm/devices/system.devices`, which otherwise pins the guest to the source disk's WWIDs.
logfiles, bash-history, tmp-files	yes	Removes logs, shell history, and temp files.

ssh-userdir is the one that surprises people cloning a machine they still log into. The template should not carry the source's private keys, so leave the operation on and inject a key per instance later. For a one-off copy that must keep the existing authorized_keys, pass --operations defaults,-ssh-userdir.

user-account is not in the default set, and passwords stay unless you pass --password or --root-password. fs-uuids is also off by default. The manual says it does not update every reference, including /etc/fstab, and that enabling it is likely to make the guest unbootable.

Ubuntu Server images usually have cloud-init, and virt-sysprep has no cloud-init operation. An old instance id under /var/lib/cloud makes the clone skip first-boot setup. Clean it on the template, and tell cloud-init to leave the hostname that --hostname wrote:

virt-customize -a /mnt/extra2/vms/ubuntu26.04-base14gb-clean.qcow2 \
  --run-command 'if command -v cloud-init >/dev/null 2>&1; then cloud-init clean --logs --seed; fi' \
  --write '/etc/cloud/cloud.cfg.d/99-preserve-hostname.cfg:preserve_hostname: true'

The same manual warns against cloning a guest that uses internal full-disk encryption for distribution: every copy shares the volume key. It also notes that deleted files are unlinked, not scrubbed. --scrub overwrites a named file, and virt-sparsify can drop freed space from the image.

How the clone gets its IP address

virt-sysprep has no operation that rewrites an address. A guest that used DHCP asks again. The clone has a new MAC, an empty machine-id, and no old lease file, so the DHCP server on libvirt's default network treats it as a new client.

A static addresses: block in netplan is copied with the disk, and both guests then claim it. Read the files before the first boot. virt-ls and virt-cat are the one-shot tools; guestfish --ro -a <disk> -i is the same inspection as a shell, after which ls /etc/netplan and cat run against the guest:

virt-ls -a /mnt/extra2/vms/ubuntu26.04-base14gb-clean.qcow2 /etc/netplan
virt-cat -a /mnt/extra2/vms/ubuntu26.04-base14gb-clean.qcow2 \
  /etc/netplan/00-installer-config.yaml

Installer images often use 00-installer-config.yaml or 01-netcfg.yaml. Cloud images use 50-cloud-init.yaml, which cloud-init will recreate unless network configuration is disabled. Netplan merges every YAML file in the directory, so a leftover static file still applies after you add a DHCP file.

To make the template a DHCP client, remove those files and upload one replacement. Use the interface name virt-cat showed. On a guest installed under KVM that name is often ens3 or enp1s0, and virt-clone keeps the same virtual PCI slot, so the name stays valid:

cat > /tmp/99-dhcp.yaml <<'EOF'
network:
  version: 2
  ethernets:
    ens3:
      dhcp4: true
EOF

virt-customize -a /mnt/extra2/vms/ubuntu26.04-base14gb-clean.qcow2 \
  --delete '/etc/netplan/*.yaml' \
  --upload /tmp/99-dhcp.yaml:/etc/netplan/99-dhcp.yaml \
  --write '/etc/cloud/cloud.cfg.d/99-disable-network.cfg:network: {config: disabled}'

For a static address, gateway, and DNS, use the same upload with the YAML from How to Change a Static IP Address in Ubuntu Server. Each instance then needs its own file, applied to the instance disk rather than the template.

The name in the YAML has to exist inside the guest. A disk moved from physical hardware often names the NIC enp2s0, while the same system under virtio presents a different name, and netplan then configures an interface that is not there. For a template that should boot on more than one machine type, pin the kernel to eth0 and use that key in netplan. Ubuntu sources /etc/default/grub.d/*.cfg while building grub.cfg:

virt-customize -a /mnt/extra2/vms/ubuntu26.04-base14gb-clean.qcow2 \
  --write '/etc/default/grub.d/99-eth0.cfg:GRUB_CMDLINE_LINUX="$GRUB_CMDLINE_LINUX net.ifnames=0 biosdevname=0"' \
  --run-command 'update-grub'

update-grub runs inside the libguestfs appliance. If grub-probe cannot see the boot disk there, the command fails and the cmdline is unchanged. In that case, set the interface name in netplan to the name the guest already uses, and skip the eth0 switch.

Per-instance customization with virt-customize

Boot instances from copies of the clean file. Do not point a domain at the template itself.

A full copy has no runtime dependency on the template:

cp --reflink=auto \
  /mnt/extra2/vms/ubuntu26.04-base14gb-clean.qcow2 \
  /mnt/extra2/vms/ubuntu26.04-vm1.qcow2

An overlay stores only the blocks vm1 writes. -F qcow2 records the base format so QEMU does not have to probe it:

qemu-img create -f qcow2 \
  -b /mnt/extra2/vms/ubuntu26.04-base14gb-clean.qcow2 \
  -F qcow2 \
  /mnt/extra2/vms/ubuntu26.04-vm1.qcow2

The base path is stored inside the overlay. It has to keep that path, and it has to stay read-only, for every guest built this way.

The virt-sysprep security notes describe a second pass on each new disk. Sysprep writes a random seed into the template; guests cloned from that file would otherwise start with the same seed, which makes first-boot SSH host keys and TCP sequence numbers predictable. --enable customize rewrites the seed. --hostname and --ssh-inject are customize options, so they belong on the same command. Substitute the account that actually exists in the image for ubuntu:

virt-sysprep --enable customize \
  -a /mnt/extra2/vms/ubuntu26.04-vm1.qcow2 \
  --hostname vm1 \
  --ssh-inject "ubuntu:file:${HOME}/.ssh/id_ed25519.pub"

Hand the instance file to libvirt, not the template. If you created it as your user outside a storage pool, libvirt-qemu needs read access or the domain fails at startup with a permission error on the qcow2.

Importing into Virt-Manager

For each instance disk, open Virt-Manager, choose New Virtual Machine, and select Import existing disk image. Point it at ubuntu26.04-vm1.qcow2. The OS entry should be Ubuntu 26.04 when the host's osinfo database has it.

The matching short id is ubuntu26.04. Check the host database before relying on it:

osinfo-query os | grep ubuntu26

If that prints nothing, the installed database is older than the 26.04 entry. --os-variant ubuntu24.04 still selects a virtio-friendly machine type for a guest that is already installed. --import tells virt-install the disk already contains an OS.

sudo virt-install --name ubuntu26.04-vm1 \
  --memory 4096 --vcpus 2 \
  --disk path=/mnt/extra2/vms/ubuntu26.04-vm1.qcow2,format=qcow2 \
  --import \
  --os-variant ubuntu26.04 \
  --network network=default \
  --noautoconsole

--network network=default attaches the NAT network whose dnsmasq instance hands out the DHCP lease. After boot:

virsh domifaddr ubuntu26.04-vm1 --source lease
virsh dumpxml ubuntu26.04-vm1 | grep "<mac"

--source lease reads that dnsmasq table. It stays empty for a static address, for a bridge, or for a NIC that is not on a libvirt virtual network. Those guests need qemu-guest-agent in the image and virsh domifaddr ubuntu26.04-vm1 --source agent, or an ARP lookup with --source arp. virsh net-dhcp-leases default lists every lease on the NAT network when you want the pool view rather than one domain.

Choosing a clone strategy: full copy, overlay, or reflink

Strategy	Space per VM	Time to create	Constraint	Use when
Full copy, or `cp --reflink=auto`	about 14 GB logical; reflink shares blocks until they change	minutes, or near-instant with reflink	reflink needs a filesystem that supports it, such as btrfs	a guest that should outlive the template
qcow2 overlay	megabytes at first	seconds	base path must remain, and the base must stay read-only	many similar guests from one sealed image
`virt-clone --reflink`	raw image, shared extents	near-instant	raw disks on the same btrfs filesystem; qcow2 is skipped	raw images already stored on btrfs

A guest you expect to keep for months is a full copy or a reflink. A short-lived guest can be an overlay, as long as the base file is still there when it runs. Proxmox is the other self-hosted path when you want a datacenter-style template and clone model on its own web UI. Multipass launches Ubuntu cloud images for a single user and applies cloud-init itself, which is a different workflow from sealing a disk you already installed.

Troubleshooting: when the clone will not boot

No address. virt-cat the netplan file and compare the interface key with ip link on the console. A match: macaddress entry still set to the source MAC drops the new NIC. A key that names an interface the guest does not have does the same. Fix the file on the disk, or apply the eth0 cmdline above and rebuild the instance from the template.

Address up, same as the original. The static addresses: block is still in one of the netplan files. virt-sysprep did not remove it. Upload a replacement and delete the other YAML files in /etc/netplan.

SSH says the host key changed, or accepts the clone as the original. Host keys are shared when ssh-hostkeys did not run on the file the domain opens. Confirm with virsh domblklist that the domain uses the sysprepped disk, then check qemu-img info so you did not sysprep an overlay while the guest boots from a different path. If the clone kept the original IP on purpose, clients that stored the old key in known_hosts will warn until that line is removed. That warning is the regeneration the manual describes.

Key login fails on a clone of a machine you could reach before. Default sysprep removed .ssh from the home directories. Inject a key with --ssh-inject, or re-run with --operations defaults,-ssh-userdir if this copy should have kept them.

LVM volumes missing after boot. The guest has /etc/lvm/devices/system.devices from the source, and the virtual disk's WWID changed. The default lvm-system-devices operation removes that file. A hand-picked --operations list has to include it, and lvm-uuids, when the guest uses LVM.

One clone or a template

A single extra machine is virt-clone, a backing-file check, and virt-sysprep on the new disk. A fleet is one sealed standalone file, then a copy or an overlay per guest, with --enable customize on that guest's own disk before it boots.

Overlays make the sealed file part of the runtime path of every guest that names it. Full copies and reflinks do not. Pick that dependency with the lifetime of the guest, and leave fs-uuids off so the bootloader and /etc/fstab still match the filesystem.

Useful links

References

virt-clone(1) on Ubuntu 26.04, including --auto-clone, --file, and --reflink
virt-sysprep(1) operations, in-place edits, and the random-seed note
virt-customize(1) for --hostname, --ssh-inject, --upload, and --run-command
qemu-img info, convert, and create -b / -F
virt-install(1) --import and --os-variant
osinfo-db Ubuntu 26.04 entry, short id ubuntu26.04
machine-id(5) on how an empty /etc/machine-id is replaced at boot

Self-Hosted S3 Alternatives in 2026 (RustFS, Garage, SeaweedFS, Ceph)

Rost — Thu, 08 Oct 2026 08:35:20 +0000

Amazon S3 has become the de facto API for object storage. After the open source MinIO repository was archived in April 2026, seven credible self-hosted alternatives remain.

These options are not interchangeable. Some are full object-storage engines that own their data layout; others are thin gateways that expose files you already have. That split — native store versus filesystem-backed gateway — decides more about backups, recovery, and migrations than any feature list does.

This article compares RustFS, Garage, SeaweedFS, Ceph RGW, VersityGW, S3Proxy, and rclone. Each section covers the storage model, the S3 feature coverage, where the project fits, and the limitation that usually decides the choice. A decision guide for single-server and multi-server deployments closes the article.

For the broader picture — object storage, databases, search, and AI-native data layers — see the Data Infrastructure for AI Systems.

Self-Hosted S3 Storage Comparison

The most important question is whether you need a distributed object store or an S3 interface in front of storage that already exists.

Software	Storage model	Distributed	S3 compatibility	Plain files on disk	Complexity	Best use
RustFS	Native object store	Yes	High, tested subset	No	Low to medium	General MinIO replacement
Garage	Native object store	Yes	Good, narrower API	No	Low to medium	Small distributed clusters
SeaweedFS	Distributed blob and file store	Yes	High	No	Medium	Huge file counts and mixed storage
Ceph RGW	Ceph object gateway	Yes	High	No	High	Large storage infrastructure
VersityGW	S3 gateway over POSIX	Backend dependent	Good	Yes	Low	S3 over normal files
S3Proxy	S3 translation layer	Backend dependent	Moderate	Yes with filesystem-nio2	Medium	Gateway and protocol translation
rclone serve s3	S3 gateway	Backend dependent	Basic, experimental	Yes with local backend	Very low	Simple and lightweight deployments

RustFS is the closest direct replacement for a conventional MinIO deployment. Garage fits small distributed clusters. SeaweedFS and Ceph solve broader storage problems. VersityGW can expose an existing POSIX filesystem through S3 while leaving the object payloads as ordinary files. For a feature-by-feature matrix of Garage, MinIO, and AWS S3, see Garage vs MinIO vs AWS S3.

Two Different Types of S3 Storage

A native object store owns its storage layout:

flowchart TD
    A[Application] -->|S3 API| B[Object Store]
    B --> C[Internal Metadata]
    B --> D[Chunks or Object Layout]
    C --> E[Disks]
    D --> E

RustFS, Garage, Ceph RGW, and SeaweedFS belong to this family. The files visible on the underlying disks are implementation details, and applications should not manipulate them directly.

A filesystem-backed S3 gateway has a different relationship with storage:

flowchart TD
    A[Application] -->|S3 API| B[S3 Gateway]
    B --> C[POSIX Filesystem]
    C --> D[bucket]
    D --> E[path]
    E --> F[ordinary-file.dat]

With software such as VersityGW, the filesystem can remain the authoritative representation. An object such as:

s3://archive/documents/2026/report.pdf

can correspond directly to:

/storage/archive/documents/2026/report.pdf

RustFS: The Closest MinIO Replacement

RustFS is a distributed object storage system written in Rust and released under the Apache-2.0 license. It provides an S3-oriented operating model familiar to MinIO users, which makes it the first project to evaluate when replacing MinIO.

Its current S3 implementation covers the operations most applications expect: buckets, objects, multipart uploads, conditional requests, object metadata, tags, versioning, lifecycle management, object lock, presigned URLs, bucket policies, CORS, notifications, replication configuration, and server-side encryption.

RustFS publishes an S3 compatibility matrix that is a tested subset, not a claim of complete Amazon S3 coverage. The snapshot verified against commit 1e6f5f1e on 9 August 2026 lists 455 implemented standard tests, 5 lifecycle-behavior tests, 17 unimplemented standard tests, and 270 excluded cases that do not block the compatibility gate. The executable lists under scripts/s3-tests are the source of truth when those counts move.

Where RustFS fits

RustFS is a candidate when an application expects a proper S3 object store.

Typical uses include application object storage, backup repositories, media storage, AI and data-pipeline artifacts, internal S3 infrastructure, Kubernetes workloads, and replacement of existing MinIO endpoints.

It works with AWS SDKs and common S3 clients. Applications normally need a custom endpoint, credentials, region settings where required, and path-style addressing.

The important limitation

RustFS owns the storage representation. The files below its data directory are RustFS internals.

That makes RustFS the wrong answer when the requirement is that every S3 object remain an ordinary file you can inspect, copy, rsync, or recover without the object-storage software. That requirement selects a filesystem-backed gateway.

Garage: Distributed S3 Without Ceph-Scale Complexity

Garage is an S3-compatible distributed object store for small and medium self-hosted installations. Deuxfleurs develops it under the AGPL v3 licence and has run it in production since the project's initial releases in 2020.

Garage is built to run across several ordinary servers, including servers in different physical locations. Replication is part of the design.

flowchart LR
    A[Applications] --> B[S3 Endpoint]
    B --> C[Garage Node 1]
    B --> D[Garage Node 2]
    B --> E[Garage Node 3]
    C <--> D
    D <--> E
    E <--> C

Garage is attractive when you have three small servers rather than a rack of dedicated storage hardware. A working setup path, from Docker through cluster layout, replication, and TLS via a reverse proxy, is in the Garage quickstart.

Where Garage fits

Garage makes sense for homelabs, small hosting platforms, geographically distributed servers, replicated backups, and clusters where Ceph would be excessive.

Its scope is narrower than AWS S3. Garage's compatibility table leaves out bucket policies, ACLs, and bucket versioning. Choose Garage over RustFS when lightweight multi-node replication is the primary requirement. Choose RustFS when broader S3 behavior and MinIO-like application compatibility matter more.

SeaweedFS: More Than an Object Store

SeaweedFS started by storing enormous numbers of files efficiently. It is now a distributed storage platform exposing S3, a FUSE filesystem, WebDAV, SFTP, and HDFS. SeaweedFS 4.48 was released on 28 September 2026.

Volume servers pack blobs into append-only volume files. Masters track volumes rather than individual files. A filer provides hierarchical filesystem semantics. The design targets workloads with millions or billions of relatively small objects.

S3 support

SeaweedFS documents versioning, object lock, lifecycle rules, tagging, CORS, checksums, presigned URLs, multipart uploads, bucket policies, IAM, STS, and several server-side encryption modes.

The same cluster can serve S3 table buckets and a built-in Iceberg REST catalog, so Spark, Trino, DuckDB, and similar engines can use Iceberg tables without a separate Hive Metastore or Glue. The project's one-command S3 example listens on port 8333.

Where SeaweedFS fits

Consider SeaweedFS for web crawlers and archives, image and document repositories, billions of small objects, data lakes, mixed S3 and filesystem workloads, and storage that grows by adding volume servers.

There are more components to understand than in a straightforward RustFS installation. SeaweedFS can provide filesystem semantics through its filer and FUSE mount, but objects are stored inside its volume format. It does not satisfy the requirement that every S3 object remain an independent ordinary file on the host filesystem.

Ceph RGW: The Infrastructure-Scale Option

Ceph is broader than S3 storage. A cluster can provide block devices through RBD, a filesystem through CephFS, and object storage through RADOS Gateway (RGW). RGW presents an S3-compatible API backed by RADOS.

flowchart TD
    A[Applications] -->|S3| B[Ceph RGW]
    B --> C[RADOS]
    C --> D[OSD 1]
    C --> E[OSD 2]
    C --> F[OSD 3]

Ceph supports replication, erasure coding, failure domains, multiple gateways, multisite deployments, access policies, and cluster management tooling.

Where Ceph fits

Ceph makes sense when object storage is one part of a larger storage platform: Kubernetes persistent volumes, virtual-machine disks, shared filesystems, large S3 repositories, and clusters that need explicit failure domains.

For a single server with several terabytes, installing Ceph only to obtain an S3 endpoint is difficult to justify. If you already operate Ceph, RGW is the obvious S3 choice. If you do not, RustFS or Garage reaches useful object storage with less machinery.

VersityGW: S3 Over Ordinary Files

VersityGW is the option to evaluate when the filesystem itself matters. It is an Apache-2.0 licensed S3 gateway that can expose POSIX filesystems, ScoutFS, Azure Blob Storage, other S3 services, and custom backends.

Its POSIX backend maps S3 buckets and objects onto directories and files. For example:

s3://website-assets/images/logo.png

maps to a path such as:

/storage/website-assets/images/logo.png

The logo.png payload remains a normal file.

Why plain files are useful

You can inspect storage with ordinary Unix tools:

find /storage -type f
du -sh /storage/*

You can read an object without the S3 server:

cat /storage/archive/2026/source.html

Conventional filesystem backup tools apply directly: rsync, restic, borg, tar, and zfs send. If the gateway software disappears, the payloads are still files in a directory tree.

S3 metadata on a POSIX backend

S3 metadata does not fit in a POSIX file by itself. VersityGW stores it in extended attributes by default. Since March 2026, object metadata is a single user.metadata xattr holding JSON, because one xattr per key hit filesystem key-length limits. A versitygw utils convert-xattr-metadata command rewrites older per-key attributes. Reads still fall back to the old layout when user.metadata is absent.

Filesystems without xattrs can use --sidecar (or VGW_META_SIDECAR) to keep metadata in a separate directory tree. The project marks sidecar and --nometa as less tested than xattrs. --nometa disables metadata storage and is aimed at read-only hosting of a dataset that already exists; bucket policies and ACLs are metadata, so they are unavailable in that mode.

Object versions can live in a separate directory. The project's POSIX example passes --versioning-dir for older versions while the current payload stays at its natural path.

Changing a file behind the gateway can leave S3 metadata, including ETags, inconsistent with the new bytes. A workable operating model is:

Application access:
    Read through S3
    Write through S3

Administrator access:
    Read the filesystem directly
    Inspect the filesystem directly
    Back up the filesystem directly
    Avoid modifying objects behind the gateway

VersityGW 1.8.0, already the version a September 2026 COSI driver release targeted, adds a standalone IAM service (versitygw iam), STS AssumeRoleWithWebIdentity, and IAM policy conditions.

S3Proxy: A General Translation Layer

S3Proxy implements the S3 API and translates requests onto several backends: local files, Azure Blob Storage, Google Cloud Storage, OpenStack Swift, SFTP, and other S3 services. It requires Java 17 or newer.

flowchart TD
    A[S3 client] --> B[S3Proxy]
    B --> C[filesystem-nio2]
    C --> D[Local filesystem]

The recommended on-disk provider is filesystem-nio2. The older filesystem provider is deprecated. Object keys become filesystem paths, and metadata uses user extended attributes, so the payloads stay ordinary files.

The filesystem backend refuses object versioning and server-side encryption. Versioning stays unsupported (supportsVersioning() is false, and those requests remain HTTP 501) because a version layout written to a real disk would become a format the project would have to keep. Server-side encryption is refused for the same class of reason: reporting AES256 over a plaintext file someone can read would be a false claim. The in-memory transient-nio2 backend implements both, and the data dies with the process. The project README also lists bucket policies, object tagging, and ACLs other than private and public-read as unsupported.

S3Proxy is the tool when you need that translation across backends. For a straightforward S3-to-POSIX gateway, VersityGW is the smaller surface.

rclone serve s3: Lightweight but Experimental

rclone serve s3 exposes an rclone backend through an S3-compatible API. The documentation still labels the command experimental.

For a local filesystem:

rclone serve s3 \
  --addr :9000 \
  --auth-key ACCESS_KEY,SECRET_KEY \
  local:/srv/s3

Without --addr, the server binds 127.0.0.1:8080. --auth-key enables Signature Version 4. Multipart uploads are written in part-number order to a temporary object and renamed into place on completion, so a failed upload does not replace an existing object.

Objects written through a local backend become files under that path. The same command can sit in front of other rclone remotes. It is a reasonable choice for internal tooling, a lab, or a temporary migration hop. It is a weak permanent S3 layer for many unrelated production applications.

What About MinIO?

The MinIO GitHub repository was archived on 13 February 2026, briefly unarchived, and archived again on 25 April 2026. It is read-only. The repository notice points operators at AIStor, MinIO's commercial distribution, and states that the community edition is source-only.

An existing MinIO deployment can keep running, particularly when it is isolated and its behavior is already understood. The full timeline, the operational risk, and a phased migration plan are in MinIO CE in 2026.

For the S3 behavior of the system you may be leaving, MinIO as an AWS S3 alternative documents how MinIO maps onto S3. The MinIO command line cheatsheet is the audit companion for a cluster that is still up. For a new installation, an actively maintained alternative is the default.

Plain Files vs Native Object Storage

Use RustFS, Garage, SeaweedFS, or Ceph when the object store should control durability, metadata, distribution, and layout. That buys richer S3 semantics, replication or erasure coding, horizontal scaling, and versioning where the project implements it. The cost is dependence on the engine's on-disk representation.

Use VersityGW, S3Proxy, or rclone when an existing filesystem should remain authoritative. That buys plain recoverable files, conventional backups, direct inspection, and compatibility with ZFS, XFS, ext4, and NFS. Distributed durability then has to come from the filesystem or the storage array, not from the S3 process. On a single server or a NAS, that trade is often the right one.

Choosing the Right Self-Hosted S3 Server

For a single server that should resemble MinIO, start with RustFS.

For three or more modest servers where replication is the main requirement, and the application can live without bucket policies and versioning, choose Garage.

For enormous numbers of small objects, mixed filesystem access, or a data-lake workload that may grow into Iceberg tables, consider SeaweedFS.

When the same cluster must also provide block devices and a distributed filesystem, choose Ceph RGW.

When the requirement is specifically an S3 API over ordinary files — a NAS, an archive, a document repository, a crawler, or a backup target — start with VersityGW.

When the job is translating S3 onto Azure, GCS, Swift, SFTP, or another S3 service, use S3Proxy.

When the deployment is small, temporary, and can tolerate an experimental server, rclone serve s3 may be enough.

Recommended Architecture for a Single Storage Server

If applications should reach the data only through S3, RustFS owns the object layer and the filesystem is an implementation detail:

flowchart TD
    A[Applications] -->|S3| B[RustFS]
    B --> C[Local Storage]
    C --> D[Filesystem or RAID]

If the files themselves must survive the gateway, keep the data in a POSIX tree:

flowchart TD
    A[Applications] -->|S3| B[VersityGW]
    B --> C[POSIX Filesystem]
    C --> D[ext4, XFS, ZFS, NFS]
    D --> E[Plain Files]

In the second design, redundancy is a ZFS mirror, RAID, snapshots, filesystem replication, or a conventional backup tool. The S3 service can be replaced without rewriting the data format.

Final Thoughts

Maintained self-hosted S3 still exists after MinIO's public repository went read-only. The choice that matters is whether S3 owns the bytes or only fronts files that already exist. Every comparison above follows from that answer.

References

Go Web Frameworks in 2026: net/http, chi, Gin, Echo, Fiber Compared

Rost — Mon, 05 Oct 2026 23:53:52 +0000

Go ships a production-capable HTTP server in its standard library, and since Go 1.22 the built-in router handles method matching and path parameters on its own. The framework question has narrowed, not disappeared.

Five stacks cover almost every Go backend decision in 2026: the standard library's net/http, chi, Gin, Echo, and Fiber. Each takes a different position on the same tradeoff — what the library gives you versus what you owe it in dependencies, conventions, and ecosystem compatibility. This article compares them on routing power, handler ergonomics, middleware, and measured performance, then ends with a decision table rather than a single verdict.

If you are starting from zero and want the full build process around your choice — validation, authentication, testing, deployment — the step-by-step version lives in Building REST APIs in Go. Here the focus stays on the stacks themselves.

The post-1.22 ServeMux reset

The reason the framework debate changed shape is one addition to the standard library. Go 1.22 gave http.ServeMux method-aware patterns and path parameters:

mux := http.NewServeMux()
mux.HandleFunc("GET /users/{id}", func(w http.ResponseWriter, r *http.Request) {
    id := r.PathValue("id")
    // ...
})
mux.HandleFunc("POST /users", createUser)

Before this, any API needing GET /users/{id} required a third-party router, which is how Gin, chi, and Echo became defaults. In 2026, the stdlib covers static routes, method matching, single-segment and wildcard parameters, and middleware via plain function wrapping. What it still does not give you: route groups, composable middleware chains, struct binding, or per-route param types.

The contenders

	net/http (stdlib)	chi v5.3.2	Gin v1.12.0	Echo v4.16.0	Fiber v2.52.15
HTTP engine	net/http	net/http	net/http	net/http	fasthttp
Handler signature	`func(w, r)`	`func(w, r)`	`func(c *gin.Context)`	`func(c echo.Context) error`	`func(c *fiber.Ctx) error`
http.Handler compatible	yes	yes	yes	yes	no
Middleware ecosystem	growing, stdlib-shaped	stdlib-shaped	largest, Gin-specific	large, Echo-specific	growing, fasthttp-specific
Binding/validation	manual	manual	built-in tags	built-in	via helpers
HTTP/2	yes	yes	yes	yes	no
Dependencies	0	1	~6	~8	~4

All five use trie- or radix-style routers with path parameters and grouped routes. The structural divide is the last row of the handler-signature story: chi and stdlib speak plain http.Handler; Gin, Echo, and Fiber wrap requests in their own context type. Fiber is the outlier once more because it sits on fasthttp, not net/http at all.

What I benchmarked

Rather than quoting synthetic router benchmarks, I ran the same service on all five stacks. The rig:

Go 1.27.1, Gin v1.12.0, chi v5.3.2, Echo v4.16.0, Fiber v2.52.15
Intel Core Ultra 7 265, Linux, 20 threads, localhost, no TLS
Three routes per stack: GET /health, GET /users/{id}, GET /api/v1/users/{uid}/orders/{oid}
Load: 8 keep-alive workers for 3 seconds against the two-parameter route, after a warmup pass
Every handler marshals a small JSON document per request with encoding/json — so the numbers include serialization, not just routing

The stdlib version of the hot route:

mux.HandleFunc("GET /api/v1/users/{uid}/orders/{oid}", func(w http.ResponseWriter, r *http.Request) {
    w.Header().Set("Content-Type", "application/json")
    w.Write(orderBody(r.PathValue("uid"), r.PathValue("oid")))
})

and the Fiber equivalent:

app.Get("/api/v1/users/:uid/orders/:oid", func(c *fiber.Ctx) error {
    c.Set("Content-Type", "application/json")
    return c.Send(orderBody(c.Params("uid"), c.Params("oid")))
})

Two honesty notes before the numbers. First, this is a localhost micro-workload: no database, no TLS handshake, no middleware chain. Real services bottleneck on I/O — the framework's share of a request that queries PostgreSQL via GORM, Ent, Bun, or sqlc is a rounding error. Second, each stack got its own server process and an identical handler shape, so the comparison is between stacks, not between hand-tuned configurations.

Results

Throughput on the dynamic two-parameter route:

Stack	Requests/sec	Avg latency
Fiber (fasthttp)	205,958	0.034 ms
Echo	169,826	0.042 ms
Gin	169,715	0.042 ms
net/http ServeMux	169,708	0.042 ms
chi	163,017	0.044 ms

The four net/http-based stacks land within 4% of each other. Fiber's fasthttp engine is about 21% ahead on this localhost workload — real, but the kind of gap that a single database roundtrip (0.5–5 ms) makes irrelevant in production.

To see where the per-request cost actually goes, I also benchmarked handler invocation without the network, calling each router directly through http.Handler.ServeHTTP (Fiber through its fasthttp handler):

Stack	Time/op	Bytes/op	Allocs/op
Gin	289 ns	192 B	3
Echo	333 ns	192 B	3
net/http ServeMux	430 ns	240 B	5
Fiber	541 ns*	192 B	3
chi	592 ns	897 B	7

* The Fiber figure carries harness overhead — the test has to copy the request into fasthttp's reusable context each iteration, which the real server avoids by reading from the wire. Treat it as an upper bound; the server-level throughput above is the fair comparison for Fiber.

The allocation profile is the more durable story. Gin and Echo pool their request contexts, so a request through them costs 3 allocations. The stdlib's 5 allocations go to the request-scoped plumbing itself. chi stores route parameters in the request context.Context, which shows up as 897 B and 7 allocations per request — the price of staying a plain http.Handler with rich route info.

Fiber and the fasthttp tradeoff

Fiber is the fastest stack on the server benchmark, and the speed comes with a compatibility bill. fasthttp is a from-scratch HTTP implementation with its own request/response types, so:

net/http middleware does not attach. Any library expecting an http.Handler or *http.Request needs a conversion or a Fiber-native port.
No HTTP/2. fasthttp does not implement it; Go's net/http gets h2 from the standard library for free over TLS.
WebSocket, streaming, and http.Flusher-style tricks all take the fasthttp path, which differs from the stdlib behavior most Go documentation assumes.

What Fiber buys in exchange: the fastest numbers in the table above, an Express-like API, and lower allocations than the framework benches suggest once the wire parsing advantage is included. It fits when responses are small, routing dominates, HTTP/2 is terminated upstream, and the middleware you need exists in the Fiber ecosystem. Those conditions hold for some edge services and hold for almost nothing else.

chi: the boring middle path

chi is a router and a middleware pipeline, nothing more. Handlers are func(w, r), middleware is func(http.Handler) http.Handler, and everything written for the stdlib attaches without adapters:

r := chi.NewRouter()
r.Use(RequestID, RealIP, Logger, Recoverer)
r.Route("/api/v1", func(r chi.Router) {
    r.Get("/users/{uid}/orders/{oid}", getOrder)
})

That is the whole pitch: route groups and a middleware stack on top of the standard interface. The allocation table prices it honestly — richer route context per request than the stdlib alone, in exchange for composable middleware and nested routes. For services that want structure without a framework identity, chi plus net/http is the configuration most Go teams converge on.

Gin vs Echo, in code

Gin and Echo are near-twins on performance — within noise in both benchmarks — so the choice between them is API taste and error-handling philosophy.

Echo centralizes errors in an HTTPErrorHandler. Handlers return error, and one function maps errors to responses:

func getOrder(c echo.Context) error {
    o, err := store.Order(c.Param("uid"), c.Param("oid"))
    if err != nil {
        return echo.NewHTTPError(http.StatusNotFound, "order not found")
    }
    return c.JSON(http.StatusOK, o)
}

Gin handlers return nothing; errors accumulate on the context and you render them where you choose:

func getOrder(c *gin.Context) {
    o, err := store.Order(c.Param("uid"), c.Param("oid"))
    if err != nil {
        c.JSON(http.StatusNotFound, gin.H{"error": "order not found"})
        return
    }
    c.JSON(http.StatusOK, o)
}

Echo's model maps cleanly onto Go error handling architecture with wrapped errors and centralized boundary handling — one place translates errors.Is results into status codes. Gin's model matches the more common pattern of deciding per handler. Beyond errors, Gin's deciding advantages are ecosystem size and built-in struct binding with validation tags; Echo's are a consistent context API and stronger built-in middleware. Both integrate Swagger generation through their own adapters — the per-framework setup is covered in Adding Swagger to Your Go API.

One more axis: context propagation. Gin and Echo wrap context.Context inside their own context types, so passing cancellation into handlers requires an extraction step. The patterns for keeping cancellation, timeouts, and values straight across any of these wrappers are in Go context.Context Done Right.

The timeouts nobody sets

Every stack in this comparison leaves server timeouts at zero defaults unless you set them — and zero means no limit. A client that opens a connection and sends nothing holds a goroutine and a file descriptor indefinitely:

srv := &http.Server{
    Addr:              ":8080",
    Handler:           mux,
    ReadHeaderTimeout: 5 * time.Second,
    ReadTimeout:       10 * time.Second,
    WriteTimeout:      30 * time.Second,
    IdleTimeout:       120 * time.Second,
}
srv.ListenAndServe()

ReadHeaderTimeout is the one to set first — it bounds slowloris-style connection exhaustion without breaking long uploads the way a blanket ReadTimeout can. WriteTimeout must cover the slowest legitimate response, which makes it a policy decision tied to your timeout budgets rather than a copy-paste constant. The same discipline applies per framework: Gin, Echo, and Fiber all accept server-level timeout configuration, and none of them enable it by default.

Choosing

flowchart TD
    A[New Go HTTP service] --> B{Internal tool, plugin,<br/>or zero-dependency requirement?}
    B -- yes --> C[net/http ServeMux]
    B -- no --> D{Want built-in binding,<br/>validation, big middleware catalog?}
    D -- no --> E[chi + net/http]
    D -- yes --> F{Team convention?}
    F -- error-returning handlers --> G[Echo]
    F -- largest ecosystem, Gin idioms --> H[Gin]
    A --> I{Small responses, extreme throughput,<br/>no HTTP/2, upstream TLS termination?}
    I -- all true --> J[Fiber]
    I -- no --> E

Situation	Pick
Internal service, CLI endpoint, plugin, library	net/http alone
Most APIs — structure without framework lock-in	chi on top of net/http
Public API, larger team, validation and binding out of the box	Gin
Public API, centralized error translation, consistent context API	Echo
Proven routing/serialization bottleneck, tiny JSON, no HTTP/2	Fiber

If the honest default for a new service in 2026 has to be named: start with net/http, add chi when middleware or route groups start to ache, and move to Gin or Echo only when binding, validation, or team convention justifies a framework identity. Fiber stays a specialist tool — worth benchmarking against when throughput is proven to be the problem, priced against the compatibility bill it charges.

Migrating between stacks

Migration cost tracks the handler-signature divide. stdlib ↔ chi is mechanical: handlers keep the same signature, routes move from mux.HandleFunc patterns to r.Get, middleware attaches to the chi stack. Gin ↔ Echo is a context-type translation — c.Param, c.JSON, and middleware logic port nearly line-for-line. Anything moving to or from Fiber is a rewrite of the HTTP layer, because *http.Request-based code, net/http middleware, and h2 assumptions do not survive the fasthttp boundary; the least painful route is keeping business logic in framework-free packages and regenerating the transport layer, which is also the structure that makes the next migration cheap.

Useful links

Best Newsletter Platforms in 2026: Pricing, Hosting and SEO

Rost — Mon, 05 Oct 2026 10:38:35 +0000

Newsletter platforms split into thin email senders and full publishing systems with archives and memberships. That split decides hosting, SEO, and migration more than any editor does.

This comparison covers Buttondown, EmailOctopus, beehiiv, Kit, Ghost, Substack, MailerLite, Brevo, and Sender. Prices are in USD, reflect the public pricing in October 2026, and normally rise as the mailing list grows on subscriber-based plans.

Each platform section describes what the product actually does, its pricing tiers, and where the public archive lives. The closing sections cover the three archive ownership models, canonical URLs, sender authentication, and API publishing, because those decisions outlast any single vendor choice.

Platform models: email-first, publishing-first, hybrid

Email-first services assume that another system owns the website and the content. Publishing-first services want the newsletter platform itself to become an important public destination. A few deliberately sit between those two models, and the architectural choice carries consequences for SEO, archives, and migration.

flowchart TD
    A[Newsletter platforms] --> B[Email-first]
    A --> C[Publishing-first]
    A --> D[Hybrid]

    B --> B1[EmailOctopus]
    B --> B2[Brevo]
    B --> B3[Sender]
    B --> B4[MailerLite]

    C --> C1[Substack]
    C --> C2[beehiiv]
    C --> C3[Ghost]

    D --> D1[Buttondown]
    D --> D2[Kit]

Pricing and features at a glance

Platform	Free plan	Paid entry point	Public archive	External canonical support	API	Main model
Buttondown	100 subscribers	From $9/mo	Yes	Yes	Yes	Hybrid
EmailOctopus	2,500 subscribers, 10,000 emails/mo	$10/mo, or $9/mo annual	Individual web copies	No documented per-issue override	Yes	Email-first
beehiiv	2,500 subscribers, unlimited sends	From $43/mo annual	Yes	Limited for external-site workflows	Yes	Publishing-first
Kit	10,000 subscribers, unlimited broadcasts	From $33/mo annual	Yes	Limited	Yes	Hybrid
Ghost(Pro)	No permanent free hosted plan	From $18/mo annual	Yes	Yes	Yes	Publishing-first
Substack	Unlimited free publishing	10% of paid revenue	Yes	Limited	Limited	Publishing-first
MailerLite	250 subscribers, 2,500 emails/mo	From $12/mo	Yes	Limited	Yes	Email-first
Brevo	300 sends/day	From $9/mo	Web campaign copies	No comparable publication archive	Yes	Email-first
Sender	2,500 subscribers, 15,000 emails/mo	From $7/mo annual	Web campaign copies	Limited	Yes	Email-first

The free allowance is a useful starting point, but it should not dominate the decision. A newsletter with 500 subscribers may be operationally better served by a $10 email platform than by a free system that also wants to become the primary CMS.

Buttondown

Buttondown stays focused on email while adding publishing features: Markdown support, custom sending domains, a hosted archive, an API, and external canonical URLs for individual issues. A newsletter can therefore keep a public Buttondown presence while the permanent version lives on the publisher's own site.

flowchart LR
    A[Canonical website issue] --> B[Buttondown campaign]
    B --> C[Subscribers]
    B --> D[Buttondown archive]
    D --> A

Buttondown pricing

Plan or tier	Price	Limits / notable features
Free	$0	First 100 subscribers
Up to 1,000 subscribers	$9/mo	Core newsletter features
Around 5,000 subscribers	$29/mo	Subscriber-based pricing
Around 10,000 subscribers	$79/mo	Subscriber-based pricing
Around 20,000 subscribers	$139/mo	Subscriber-based pricing
Tagging and segmentation	+$9/mo	Optional add-on
Analytics	+$9/mo	Optional add-on
RSS-to-email	+$9/mo	Optional add-on
Paid subscriptions	+$9/mo	Optional add-on
Multiple newsletters	+$29/mo	Optional add-on
Automations	+$29/mo	Optional add-on
Custom-domain archive	+$29/mo	Optional add-on
Whitelabeling	+$79/mo	Optional add-on

The modular pricing keeps a simple newsletter inexpensive, but stacking several add-ons raises the monthly cost quickly. Buttondown fits publishers whose website remains the authoritative copy and who want distribution plus a secondary public archive rather than a replacement for the website.

EmailOctopus

EmailOctopus takes the conventional email-marketing approach: subscriber management, forms, campaigns, templates, automations, analytics, and API access, without trying to become a publishing destination. For a website that already has its own CMS or static-site generator, that is usually exactly what is required.

Each sent campaign receives a public web version, but EmailOctopus does not automatically build a publication-style archive containing every issue. Its normal model is for the publisher to create the archive page separately.

EmailOctopus pricing

Plan	Price	Subscribers	Email allowance	Notable differences
Starter	$0	2,500	10,000/mo	EmailOctopus branding, reports retained 30 days, 1 form and 1 landing page
Pro monthly	$10/mo	Tier-based	Tier-based	Branding removed, permanent reports, unlimited forms and landing pages
Pro annual	From $9/mo	Tier-based	Tier-based	Same Pro features, 10% annual discount
Connect Starter	$0	2,500	Unlimited via own SES account	Amazon SES delivery, EmailOctopus branding
Connect Pro	From $8/mo annual	Tier-based	Unlimited via own SES account	Full design control and permanent reports

EmailOctopus Connect is the option for users already operating AWS infrastructure: EmailOctopus manages contacts and campaigns, while Amazon SES performs the actual delivery.

A sensible website-first archive structure is:

example.com/newsletter/
    2026-08/
    2026-09/
    2026-10/

with EmailOctopus responsible for sending those issues to subscribers. The hosted web copies are useful for "view in browser" links and sharing, but the permanent archive does not need to depend on them.

beehiiv

beehiiv treats newsletters as a publishing business rather than an email list. It combines email, a hosted website, public posts, subscription forms, recommendations, analytics, advertisements, paid subscriptions, digital products, and growth tooling, and the free tier already includes a public website with unlimited email sends.

That positioning makes beehiiv attractive when the newsletter itself is the publication. For an established publication with its own CMS, the same breadth creates overlap with the existing content system.

beehiiv pricing

beehiiv was in the middle of a plan-name transition in October 2026, so customers may see the newer Free/Lite/Pro naming or the previous Launch/Scale/Max terminology depending on account and renewal state.

Plan	Price	Subscriber capacity	Notable features
Launch / Free	$0	Up to 2,500	Unlimited sends, website, custom domain, analytics, recommendations, API access
Scale	From $49/mo monthly or $43/mo annual	Tier-based, up to 100,000	Automations, Ad Network, paid subscriptions, digital products, advanced analytics
Max	From $109/mo monthly or $96/mo annual	Tier-based	Branding removal, RSS-to-send, dynamic content, more publications, priority support
Enterprise	Custom	100,000+ / custom	SSO, dedicated infrastructure options, account management

Email sending is unlimited across the normal newsletter plans, so costs are driven mainly by subscriber count and feature tier rather than campaign volume. beehiiv charges a 0% platform fee on paid subscriptions on the relevant paid plans, although normal payment-processing fees still apply.

Kit

Kit, formerly ConvertKit, sits between creator marketing software and newsletter publishing. Its free tier is unusually generous: up to 10,000 subscribers with unlimited email broadcasts, landing pages, forms, tagging, segmentation, and API access, designed to keep audience growth inexpensive. Paid plans add more sophisticated automation and analytics.

Kit also provides public newsletter pages, which lets broadcasts function as web content rather than existing only inside inboxes. It makes the most sense when email subscribers are a core business asset and features such as sequences, segmentation, products, and creator-oriented automation matter.

Kit pricing

The prices below use the 1,000-subscriber tier and annual billing for the paid plans.

Plan	Price	Subscribers	Notable features
Free	$0	Up to 10,000	Unlimited broadcasts, forms and landing pages, tagging, segmentation, API
Creator	$33/mo annual	1,000 at quoted tier	Visual automations, sequences, integrations, RSS campaigns, branding removal
Creator monthly equivalent	About $39/mo	1,000	Same feature tier without annual discount
Pro	$66/mo annual	1,000 at quoted tier	Advanced analytics, subscriber scoring, referral system, unlimited users
Pro monthly equivalent	About $79/mo	1,000	Same feature tier without annual discount

Kit's paid prices increase with subscriber count.

Ghost

Ghost is a complete open-source publishing platform with newsletters built in: a website, CMS, memberships, newsletters, subscriptions, themes, APIs, and strong SEO controls. It can be self-hosted or purchased as the managed Ghost(Pro) service.

Unlike most hosted newsletter platforms, self-hosted Ghost is open source, and hosting, database operation, and bulk email delivery then become separate infrastructure costs. Ghost is strong on SEO architecture: posts can specify external canonical URLs, and themes can access that canonical URL directly, which allows a Ghost archive to function as an index to content whose authoritative copy lives elsewhere.

newsletter.example.net/
    Issue A -> example.com/newsletter/issue-a/
    Issue B -> example.com/newsletter/issue-b/

For organizations that need both a CMS and newsletters, Ghost is one of the most complete systems available. Deploying it only to send email, however, is unnecessarily heavy.

Ghost(Pro) pricing

The prices below are annual-billing equivalents, and Ghost(Pro) pricing also changes with audience size.

Plan	Price	Included members at entry tier	Notable features
Starter	$18/mo	1,000	Website, custom domain, newsletter, unlimited sends
Publisher	$29/mo	1,000	Custom themes, paid subscriptions, advanced analytics, integrations
Business	$199/mo	Higher limits	15 staff users, priority support, larger usage limits
Custom	Quote	Custom / unlimited	Dedicated IP, SLA, advanced configuration

Substack

Substack removes most infrastructure decisions. A publication gets email delivery, public posts, archives, subscriber management, comments, recommendations, community features, and paid subscriptions without a monthly platform charge, so its pricing model is fundamentally different from subscriber-tiered services.

Substack's public archive is central to the product: the platform expects its own web pages to be part of the publication rather than temporary copies of messages sent from another CMS. That is convenient for newsletter-first publishing, but it gives an external website less control over the public archive and the SEO architecture.

Substack pricing

Plan / usage	Cost	Notes
Free publication	$0	Unlimited free subscribers and publishing
Free newsletters	$0	No monthly platform fee
Paid subscriptions	10% of subscription revenue	Charged by Substack
Payment processing	Additional	Stripe processing charges also apply

This makes Substack extremely inexpensive for a large free newsletter. The economics change once significant paid subscription revenue is involved: a newsletter producing $10,000 per month in subscription revenue would pay approximately $1,000 per month to Substack before payment-processing fees.

MailerLite

MailerLite combines traditional email marketing with websites, landing pages, signup forms, and automation. Its free tier became substantially smaller in June 2026, dropping to 250 active subscribers and 2,500 email sends per month; in return, the free plan includes access to a broader set of design and publishing tools than many basic free newsletter products.

MailerLite provides web versions of campaigns and a newsletter archive component that displays previous campaigns on a MailerLite-hosted page. That makes its public archive functionality stronger than EmailOctopus, although MailerLite is still primarily an email-marketing platform rather than an independent publishing ecosystem.

MailerLite pricing

Plan	Price	Subscribers / sending	Notable features
Free	$0	250 subscribers, 2,500 emails/mo	3 automations, 3 forms, 1 landing page, 1 small website
Comfort	From $12/mo	Subscriber-tiered; monthly allowance is 10x tier size	More templates, dynamic email, auto-resend, preference center
Power	Subscriber-tiered	Unlimited sending under fair-use policy	Higher feature limits and more advanced functionality
Enterprise	Custom	200,000+ subscribers	Custom account arrangement

Brevo

Brevo is broader than a newsletter platform. Its product includes marketing email, transactional email, SMS, WhatsApp, automation, web push, sales tools, and customer-data functionality, and a newsletter is only one part of that platform. Its pricing is based primarily on sending volume rather than the number of stored contacts.

Brevo pricing

Plan	Price	Sending / contacts	Notable features
Free	$0	300 emails/day, up to 100,000 stored contacts	Campaigns, transactional email, basic automation and reporting
Starter	From $9/mo	From 5,000 emails/mo	No daily sending limit; multiple volume tiers
Standard	From $18/mo	From 5,000 emails/mo	Unlimited-contact automation, A/B testing, advanced reports
Professional	From $499/mo	From 150,000 emails/mo	Multichannel features, 10 users, advanced analytics
Enterprise	Custom	Custom	Enterprise controls and support

The free plan's 300-email daily limit is unusual. For a list with 250 subscribers it is perfectly usable; for a newsletter with 1,500 subscribers it becomes inconvenient because one full-list campaign cannot be delivered under the free daily allowance. Brevo becomes considerably more attractive when the same organization also needs application-generated email or multichannel marketing.

Sender

Sender competes more directly with EmailOctopus and MailerLite. It offers newsletters, forms, landing pages, automation, segmentation, and transactional email, with one of the more generous free allowances in this group.

Sender pricing

Plan	Price	Subscribers	Email allowance	Notable features
Free Forever	$0	2,500	15,000/mo	Automation, forms, landing pages, transactional email; Sender branding
Standard annual	From $7/mo	From 1,000	Roughly 12x subscriber tier/mo	Branding removal, SMS, more templates and experiments
Standard monthly	From about $9/mo	From 1,000	Tier-based	Same plan without annual discount
Professional	From $14/mo annual	From 1,000	Roughly 24x subscriber tier/mo	Advanced automation and reporting, priority support
Enterprise	Custom	Custom	Custom	Enterprise sending infrastructure

At the free tier, 2,500 subscribers and 15,000 monthly sends allow approximately six full-list newsletters per month. Sender therefore compares well with EmailOctopus on raw free capacity, with the tradeoff that it has fewer integrations and less of a publication-oriented ecosystem than platforms such as Kit or beehiiv.

Public archives: three ownership models

Archive functionality decides what happens to a multi-year back catalogue when the platform changes. Three models cover most cases.

Publication archive

Platforms such as Substack, beehiiv, and Ghost treat each newsletter issue as part of a public publication on the provider's domain:

provider.example/publication/issue-1
provider.example/publication/issue-2
provider.example/publication/issue-3

The provider effectively becomes both newsletter infrastructure and CMS.

Hosted campaign copies

Email-marketing services usually create a browser-readable version of an email after it has been sent:

email-provider.example/campaign/abc123

These URLs are useful for "view in browser" links and sharing. They should not necessarily be treated as the permanent content archive.

Website-owned archive

A publisher with an existing website can keep the permanent copies itself:

example.com/newsletter/issue-1/
example.com/newsletter/issue-2/
example.com/newsletter/issue-3/

The email provider then becomes replaceable infrastructure: changing newsletter vendors does not require changing historical URLs. For established websites this is the cleaner model, and it is the newsletter application of the POSSE principle, which favors the copy you control over the copy a platform controls. For a static site, the archive is regular site content, as in the Hugo deployment to AWS S3 guide.

Canonical URLs and newsletter SEO

If the complete issue appears both on an independent website and on a provider-hosted page, search engines can see two very similar documents. A canonical URL provides an explicit preferred version:

<link rel="canonical"
      href="https://example.com/newsletter/issue-42/">

Buttondown and Ghost are particularly strong for this architecture because both provide mechanisms for associating content with an external canonical URL. Other newsletter services tend to assume that either their own hosted page is the publication or that their web version is simply an email mirror.

This does not mean that every duplicate issue creates an SEO problem: search engines routinely consolidate duplicate or near-duplicate pages, and explicit canonical control simply makes the intended architecture clearer. Once the archive is in place, the self-hosted SEO tools comparison covers verification and monitoring, the WordPress SEO plugins comparison covers the metadata layer for WordPress-based sites, and IndexNow can tell search engines that a new issue has been published.

Custom sending domains and sender authentication

A serious newsletter should normally use an address under a domain controlled by the publisher:

newsletter@example.com

rather than:

examplepublication@gmail.com

A custom domain can be authenticated with SPF, DKIM, and DMARC and can remain stable when the newsletter platform changes, so infrastructure can move from one provider to another while readers continue to see the same sender identity. This section covers the sending side; for mailbox hosting on the same domain, see the hosted email comparison instead.

A practical setup sequence for a new sending domain:

Choose the sending domain, usually the main domain or a subdomain, and note it in the newsletter platform.
Create the sending address in the platform and record the DKIM selector it publishes.
Add the DNS records the platform provides: an SPF TXT record listing the sending service, the DKIM TXT record under the selector, and a DMARC TXT record at _dmarc.
Verify that each record resolves before sending:

   # SPF: which servers may send for the domain
   dig TXT example.com +short

   # DKIM: the key published under the platform's selector
   dig TXT selector1._domainkey.example.com +short

   # DMARC: the policy record for the domain
   dig TXT _dmarc.example.com +short

If a record is missing or mistyped, the platform's dashboard usually flags the domain as unverified, and sends either bounce or land in spam.

API access and automated publishing

For an occasional newsletter, manually creating a campaign is not a problem. For a technical publication, API access becomes useful because newsletter generation can be integrated into the normal content pipeline:

flowchart LR
    A[Markdown source] --> B[Static site generator]
    B --> C[Website]
    A --> D[Newsletter build step]
    D --> E[Newsletter API]
    E --> F[Subscribers]

The same source material can generate the permanent page, the newsletter excerpts, the campaign metadata, and the links without repeatedly copying content between systems. The send step is a plain HTTP call; endpoint paths and payload fields differ per platform, but the pattern is the same:

curl -sS -X POST "https://api.example.com/v1/campaigns" \
  -H "Authorization: Bearer $NEWSLETTER_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d @issue.json

This is one reason API-oriented platforms such as Buttondown, EmailOctopus, Kit, and Ghost remain attractive even when their visual editors are not the main selling point.

Choosing the architecture, not the editor

The platform choice reduces to one decision: which system owns the content, and which system merely delivers it. A newsletter without an existing website saves a considerable amount of setup by letting a publishing-first platform own the posts, the archive, and the subscriptions. An established website usually gets a simpler long-term architecture by keeping the permanent archive on its own domain and treating the provider as delivery infrastructure that can be replaced without touching historical URLs.

Whatever the split, three details are cheap to settle before the first send and expensive to unwind later: a custom sending domain with SPF, DKIM, and DMARC in place, an explicit decision about where each issue's canonical copy lives, and an export path that keeps the subscriber list in a standard format.

References

Self-Hosted SEO Tools and Platforms: Open Source Guide

Rost — Sun, 04 Oct 2026 11:03:32 +0000

Self-hosted SEO now covers crawlers, Search Console dashboards, rank trackers, MCP servers, and local AI. This guide compares the main open-source platforms and which features work without paid SEO APIs.

The term covers very different systems. Some projects work from your own website and first-party search data alone, while others self-host the interface but still require DataForSEO, a commercial SERP provider, or another paid API for their most valuable features.

A dashboard wrapped around a paid data service is architecturally different from a crawler, a Search Console analyzer, or a local AI system that keeps working when every commercial API key is removed. That difference, not the location of the source code, decides which platforms fit a no-paid-API stack, and it is the axis of the comparison below. These systems run alongside deploy pipelines and indexing signals - the publishing infrastructure mapped in the Web Infrastructure hub of this site.

What Is a Self-Hosted SEO Platform?

A self-hosted SEO platform runs at least the application layer on infrastructure you control. Depending on the project, the application crawls websites, analyzes Search Console data, tracks rankings, inspects Core Web Vitals, generates reports, or exposes SEO functions to AI agents.

The question that separates the projects is where the data comes from. A typical commercial SEO platform combines several expensive datasets:

keyword database
+
SERP collection
+
backlink crawler
+
site crawler
+
Search Console
+
analytics
+
content analysis

Open-source projects reproduce some of these layers well. Site crawling, technical SEO, Search Console analysis, local page analysis, Core Web Vitals, and LLM-assisted recommendations are all practical to self-host.

Other layers are much harder. A global backlink index requires continuously crawling a substantial part of the public web, and reliable worldwide keyword-volume and SERP datasets require either large-scale data collection or access to commercial sources. Open-source projects that promise Ahrefs-like data usually obtain at least part of it from another API.

Four Types of Self-Hosted SEO Tool

The current ecosystem separates into four categories.

Technical crawlers

These inspect the actual website and detect broken links, duplicate titles, missing metadata, canonical errors, heading problems, redirect chains, and accessibility issues.

Examples:

They normally provide substantial value without any paid SEO API.

Search performance and monitoring platforms

These combine crawl data with first-party search information, particularly Google Search Console.

Examples:

For an established website, this category is often more useful than a generic keyword database because it works with real impressions, clicks, CTR, rankings, and the pages Google already associates with your domain.

Rank and SERP tools

These collect search engine results and track positions.

Examples:

The difficulty here is operational rather than conceptual. Search engines rate-limit automated access, change markup, present CAPTCHAs, and personalize results. A self-hosted SERP system can work, but it requires more maintenance than calling a commercial SERP API.

Full SEO platforms

These attempt to bring keyword research, backlinks, competitor analysis, site audits, rank tracking, and AI workflows into one product. The most visible example is OpenSEO. These systems provide an attractive interface, and their external data often still comes from commercial APIs, so inspect the data path before deployment.

Self-Hosted SEO Tools Compared

GitHub star and fork counts are approximate snapshots from September 2026. They are adoption signals, not quality scores.

System	Type	GitHub stars	Forks	Web UI	CLI	MCP	Local LLM	Paid SEO API needed for core value	License
OpenSEO	Full SEO platform	20.5K	2.6K	Yes	Partial	Yes	Not primary	Yes, DataForSEO	MIT
SerpBear	Rank tracker	2.1K	294	Yes	No	No	No	No, if using own proxies	MIT
LibreCrawl	Technical crawler	980	206	Yes	Limited	No	No	No	MIT
SiteOne Crawler	Technical crawler / QA	917	84	Reports / desktop	Yes	No native	Yes	No	MIT
CrawlSEO	Monitoring platform	601	92	Yes	No	Yes	Agent-side	No	MIT
SEO Skill	SEO CLI / agent backend	532	42	No	Yes	Yes	Agent-side	No	Apache-2.0
Scouter	AI-native crawler	71	7	Yes	Some	Yes	Yes	No	MIT
OpenGSC	GSC platform	27	19	Yes	Limited	Yes	Yes	No for core	MIT
OpenSERP	SERP API / CLI	Varies	Varies	API docs	Yes	Add-on	No	No	MIT

The most important column is whether useful functionality survives when commercial SEO API keys are removed. OpenSEO's 20,000 stars reflect broad appeal and an all-in-one proposition, not self-containment, and a young project such as Scouter can offer a more interesting local-LLM architecture with far fewer stars. Before choosing, it is better to ask:

Does the project solve my problem?
Does its core still work without a paid API?
Can I export my data?
Can I automate it?
Is the project maintained?
Can I replace individual components later?

OpenSEO

OpenSEO is the largest project in this comparison, with roughly 20.5 thousand GitHub stars and 2.6 thousand forks at the time of writing. The project describes itself as an open-source alternative to Semrush and Ahrefs and provides keyword research, rank tracking, competitor intelligence, backlinks, site audits, AI visibility, MCP, and agent skills.

Its strongest feature is product integration: a modern application with project context, SEO workflows, and an MCP interface for clients such as Claude Code, Codex, OpenClaw, and Hermes, instead of assembling several command-line utilities.

The data dependency is the other half of the picture. The project documentation explicitly requires a DataForSEO API key for SEO data, and DataForSEO is a separate pay-as-you-go commercial service. Self-hosting OpenSEO does not self-host the underlying dataset. The architecture is:

flowchart LR
    A[Website] --> B[OpenSEO]
    C[AI Agent] -->|MCP| B
    B --> D[DataForSEO]
    B --> E[Google Search Console]
    B --> F[OpenSEO database]

If you already use DataForSEO, OpenSEO is an open-source control plane around it, and self-hosting still gives you control over projects, credentials, application state, and agent integration. It is not a replacement for paid external SEO data.

Best use

Choose OpenSEO when you want a modern Semrush-style interface and are comfortable paying DataForSEO directly instead of paying for a traditional SEO SaaS subscription. Do not choose it if the requirement is that the system retains most of its value with no paid SEO API at all.

SiteOne Crawler

SiteOne Crawler is one of the most self-contained tools in the list. It has around 917 GitHub stars and 84 forks and is designed as a cross-platform website crawler for SEO, security, accessibility, performance, and quality assurance.

The SEO checks cover the technical layer: metadata, links, redirects, page structure, status codes, sitemaps, and related issues. SiteOne also inspects security headers, TLS, accessibility, caching, and performance, which makes it useful for developer-managed websites where SEO is part of a wider deployment-quality process.

Browser rendering for modern sites

SiteOne can optionally render pages through Chromium using the Chrome DevTools Protocol. That matters for React, Vue, Angular, and other sites where important links or content only appear after JavaScript execution. For static systems such as Hugo, browser rendering is usually unnecessary, but it is useful when testing embedded applications, client-side widgets, or pages with significant JavaScript behavior.

Local LLM support

SiteOne's optional AI layer supports OpenAI, Anthropic, Gemini, and arbitrary OpenAI-compatible endpoints. The OpenAI-compatible path can point at Ollama, vLLM, LocalAI, self-hosted gateways, or other compatible servers, so a local model can inspect selected pages without sending content to a cloud provider.

A fully local architecture:

flowchart LR
    A[Website] --> B[SiteOne]
    B --> C[Deterministic crawl results]
    C --> D[Local OpenAI-compatible API]
    D --> E[Ollama / llama.cpp / vLLM]
    E --> F[SEO suggestions]

SiteOne determines facts such as missing descriptions or broken links, while the model proposes better wording or summarizes the findings.

CI/CD quality gate

A practical workflow:

flowchart LR
    A[Git push] --> B[Build website]
    B --> C[Local preview]
    C --> D[SiteOne crawl]
    D --> E{Critical issues?}
    E -->|Yes| F[Fail pipeline]
    E -->|No| G[Deploy]

The crawl runs against a local preview before deploy, and the pipeline fails on critical issues. That gives engineering teams an automated check instead of a dashboard that is reviewed occasionally.

Best use

Choose SiteOne for technical websites, static-site generators, CI/CD pipelines, or any environment where SEO checks should behave like automated software-quality checks. It is a strong option when the priority is genuine local value rather than external SEO datasets.

LibreCrawl

LibreCrawl is a web-based multi-user SEO crawler with around 980 GitHub stars and 206 forks. The project positions it as an open-source alternative to desktop crawlers such as Screaming Frog.

LibreCrawl extracts page titles, descriptions, headings, links, response information, and other technical SEO data. It supports JavaScript rendering, configurable crawl depth, URL filtering, proxies, robots.txt behavior, and exports to CSV, XLSX, JSON, and XML. Unlike a desktop-only crawler, it runs as a web application and supports concurrent users with separate sessions.

External dependencies

The core crawler works locally. PageSpeed Insights integration can use Google's API and benefits from an API key for higher limits, but the crawler itself does not require a commercial SEO data service.

Because LibreCrawl operates against the rendered website rather than the CMS, it works with:

Hugo
WordPress
Ghost
Drupal
Next.js
Astro
React
custom websites

Limitations

LibreCrawl does not offer the agent-oriented architecture of Scouter or SEO Skill: there is no central MCP workflow, and local LLM integration is not its main focus. Its strength is straightforward crawling and data export.

Best use

Choose LibreCrawl when you want a browser-based, open-source crawler with a familiar human workflow and good export support.

Scouter

Scouter is a younger project, but its architecture is the most ambitious in this list for AI-native SEO workflows. It has roughly 71 GitHub stars and 7 forks, so its deployment history is tiny compared with OpenSEO or SerpBear. The documented feature set includes a self-hosted web UI, multi-user support, JavaScript rendering, internal PageRank, custom XPath and regex extractors, SQL access to crawl data, AI page categorization, AI bulk generation, and a native MCP server.

Bring your own LLM

Scouter supports hosted and local models; the documented local options are Ollama and vLLM. The resulting architecture:

flowchart TD
    A[Website] --> B[Scouter crawler]
    B --> C[Crawl database]
    C --> D[Web UI]
    C --> E[MCP]
    C --> F[AI assistant]
    F --> G[Local Ollama / vLLM]
    E --> H[AI agent]

This is closer to an "SEO database for agents" than to a traditional crawler.

MCP integration

Without MCP, an AI assistant usually needs reports pasted into its context or custom scripts written around exported files. With MCP, the agent asks the SEO system directly for structured crawl evidence, triggers operations, and retrieves relevant findings as needed. Workflows such as this become realistic:

Find pages with weak internal linking.

For each page:
- show inlink count
- identify semantically related pages
- propose links
- do not modify content

The crawler supplies the facts, the model provides the interpretation.

Limitations

Scouter is new. A 2026 project with around 60 commits and a small user base is promising infrastructure rather than a proven production standard, and its more complex stack means more operational components than a single-binary crawler such as SiteOne.

Best use

Choose Scouter when MCP, multi-user web access, SQL over crawl data, and local AI integration are important enough to justify running a younger platform.

SEO Skill (iannuttall/seo)

SEO Skill takes a different approach: instead of a large browser dashboard, it provides a local seo CLI, a TypeScript library, a packaged agent skill, and an MCP server. The project has around 532 GitHub stars and 42 forks.

The workflow combines evidence from your own crawl, Google Search Console, and optional analytics into reports that humans or AI agents can inspect and act upon. Useful commands:

seo report
seo quick-wins
seo second-page
seo technical-watch
seo refresh-priorities
seo report --json
seo mcp serve

The project exposes more than 70 SEO audit tools through its CLI and MCP layer.

First-party data first

SEO Skill treats paid research providers as optional enrichment rather than mandatory infrastructure. It can use local crawl data, Google Search Console, and analytics data, and it can optionally connect to DataForSEO, Semrush, or Ahrefs. If none of the commercial providers are configured, the first-party workflow still works.

For an established website, a query with

8,000 impressions
average position 8.2
CTR 1.1%

describes your actual opportunity, while a third-party estimate claiming that a related keyword receives 12,000 searches per month describes a market you cannot verify from your own data.

Agent architecture

flowchart LR
    A[Website] --> B[SEO Skill crawl]
    C[Google Search Console] --> B
    D[Analytics] --> B
    B --> E[MCP server]
    E --> F[Hermes / Claude / Codex]
    F --> G[Local LLM]

The LLM receives structured evidence from the SEO tool instead of scraping dashboards or guessing metrics.

Limitations

SEO Skill is not a polished multi-user SEO dashboard. If you want a visual application that marketing staff can browse all day, another system may fit better. External competitor, backlink, and keyword research remains limited unless optional third-party providers are added.

Best use

Choose SEO Skill when you want an agent-first, scriptable SEO backend built around first-party evidence rather than a large GUI. For developers, static-site publishers, and automated Git workflows, it is a strong fit.

CrawlSEO

CrawlSEO is a self-hosted SEO monitoring dashboard with around 601 GitHub stars and 92 forks. It combines Google Search Console, a site crawler, Core Web Vitals, SEO opportunities, and MCP access, which puts it between a crawler and a full SEO platform.

The core data sources are:

Google Search Console
local site crawling
Core Web Vitals / PageSpeed data
stored historical observations

CrawlSEO can provide ongoing monitoring without a paid SEO provider.

Optional external research

CrawlSEO can integrate with DataForSEO for keyword research and backlink information, but those features are optional, and Google Autocomplete provides a free fallback for keyword suggestions. The core stack runs without an external SEO dataset:

GSC
+
crawler
+
Core Web Vitals
+
MCP

MCP tools

CrawlSEO exposes ten MCP tools covering sites, site overview, keywords, pages, traffic, crawling, crawl issues, Core Web Vitals, and opportunities. An agent can ask questions such as:

Which pages lost impressions this month and also have technical crawl issues?

That correlation between search performance and technical state is more actionable than a generic SEO score.

Best use

Choose CrawlSEO if you want a persistent self-hosted dashboard built around Search Console and ongoing monitoring rather than one-off audits. It is particularly suitable for established sites that already receive meaningful Google search impressions.

OpenGSC

OpenGSC is another Search Console-centered platform. It is a young project, with roughly 27 stars and 19 forks, but it has accumulated a substantial feature set. The core proposition: Search Console contains valuable first-party SEO data, but Google's own interface is limited for longitudinal analysis, cross-site monitoring, content-decay detection, and agent workflows.

OpenGSC adds:

multi-site GSC dashboards
rank tracking from first-party data
striking-distance queries
content-decay detection
cannibalization analysis
site auditing
indexing tools
alerts and digests
MCP
optional AI SEO tools

Built-in site audit

The crawler operates with zero external SEO APIs and checks up to hundreds of pages for broken internal links, title issues, missing descriptions, H1 problems, noindex pages, canonical mismatches, thin content, missing image alt text, and slow responses. The platform retains useful local functionality even when its optional research integrations are disabled.

AI and external providers

OpenGSC supports multiple AI providers and custom OpenAI-compatible endpoints, which makes local inference possible depending on the configured server. Some extended research features can use DataForSEO or other external providers, but they are not required for the Search Console dashboard and local audit layer.

Caution around optional indexing features

OpenGSC includes optional indexing-oriented infrastructure that goes beyond normal Search Console analysis. Evaluate those features independently; they are not required for ordinary SEO monitoring. For conservative deployments, the sensible configuration is:

GSC analytics
+
site audit
+
MCP
+
optional local AI

rather than enabling every available module.

Best use

Choose OpenGSC if Search Console is the center of your SEO workflow and you want a self-hosted interface around your own search-performance data.

SerpBear

SerpBear is one of the most mature open-source rank trackers, with around 2.1 thousand GitHub stars and 294 forks. Its purpose is narrower than OpenSEO or CrawlSEO: track where a domain appears in Google for a configured set of keywords.

It provides unlimited domains, unlimited tracked keywords, ranking history, email notifications, a SERP API, Google Search Console integration, keyword research through Google Ads integration, and PWA/mobile access.

The SERP collection problem

The application, database, scheduler, and UI are self-hosted, but Google results still have to be collected somehow. SerpBear can use your own proxies, scraping services, or commercial SERP APIs. If you operate your own proxies, no commercial SEO API is inherently required, but the operational burden moves to you:

commercial SERP API:
  money cost
  low operational burden

self-hosted scraping:
  low API cost
  higher operational burden

Best use

Choose SerpBear when rank tracking is the main requirement and you are comfortable managing the SERP acquisition layer yourself. It is a specialized tool that works well alongside a crawler or Search Console platform.

OpenSERP

OpenSERP is not a complete SEO dashboard. It is infrastructure: a self-hosted SERP API and CLI.

It supports search against Google, Bing, DuckDuckGo, Yandex, Baidu, and Ecosia, and returns normalized results through JSON and other formats. OpenSERP can also expose SERP features such as AI summaries, answer boxes, People Also Ask, and related searches, and it can optionally extract content from result pages.

Why it matters

Commercial SEO tools often hide SERP acquisition behind a polished application. OpenSERP provides the lower-level component, which makes it useful for building rank tracking, competitor discovery, SERP-intent analysis, query clustering, agent search tools, and local research pipelines.

Example architecture:

flowchart LR
    A[Keyword list] --> B[OpenSERP]
    B --> C[Search engines]
    B --> D[Local result store]
    D --> E[SEO analysis]
    E --> F[Local LLM]

The project also provides SDKs and an MCP integration, making it suitable as infrastructure behind other systems. For the wider self-hosted search landscape, see Beyond Google: Alternative Search Engines Guide.

Limitations

Running the API yourself does not make search engines easier to scrape. CAPTCHAs, blocking, rate limits, regional variation, and result-layout changes remain real problems, and proxies may eventually become necessary at meaningful scale. OpenSERP is best treated as a controlled local SERP source, not a free substitute for a commercial global search dataset.

Best use

Choose OpenSERP when you want to own the SERP collection pipeline or need search-engine results as an input to custom SEO automation.

Which Tools Work Without Paid SEO APIs?

This comparison is more useful than "open source versus proprietary."

System	Technical audit	GSC analysis	SERP/rank tracking	Local AI	Useful with zero paid SEO APIs?
OpenSEO	Yes	Yes	Yes	Agent-oriented	Partly, but core research depends on DataForSEO
SiteOne	Excellent	No	No	Yes	Yes
LibreCrawl	Excellent	No	No	No	Yes
Scouter	Excellent	No core GSC focus	No	Yes	Yes
SEO Skill	Yes	Excellent	First-party/optional providers	Via agent	Yes
CrawlSEO	Yes	Excellent	GSC based	Via agent	Yes
OpenGSC	Yes	Excellent	GSC based	Yes	Yes
SerpBear	No	Yes	Excellent	No	Yes, with own scraping/proxies
OpenSERP	No	No	SERP infrastructure	No	Yes, operationally limited

The functions easiest to self-host are the ones based on your own site and your own search data.

Hugo, WordPress, Ghost, or Something Else?

Most of these systems do not care which CMS or static-site generator produced the website. A crawler sees HTTP and HTML, so Hugo, WordPress, Ghost, Drupal, Astro, Next.js, Jekyll, and custom applications can all be analyzed by the same crawler.

The difference appears when you want to modify the site. A WordPress SEO plugin can directly update post metadata because it runs inside WordPress. SiteOne or Scouter cannot automatically know how your Hugo front matter is structured unless another integration layer tells them. The separation:

flowchart LR
    A[CMS / static-site generator] --> B[Published website]
    B --> C[Self-hosted SEO system]
    C --> D[Findings]
    D --> E[Human or AI agent]
    E --> F[Source changes]
    F --> A

For modern development workflows, this separation is often an advantage rather than a problem.

Self-Hosted SEO for WordPress

For WordPress, a self-hosted SEO platform normally sits beside a conventional SEO plugin:

flowchart TD
    A[WordPress]
    A --> B[The SEO Framework / SEOPress]
    B --> C[Metadata / schema / sitemap]
    A --> D[Published site]
    D --> E[SiteOne / CrawlSEO / Scouter]
    E --> F[SEO findings]

The plugin controls:

canonical URLs
robots metadata
schema
XML sitemaps
titles and descriptions
redirects

The external platform observes:

technical health
crawl structure
search performance
Core Web Vitals
rank changes
content opportunities

These are complementary responsibilities; plugin-by-plugin comparisons of Yoast, Rank Math, SEOPress, The SEO Framework, and Slim SEO belong to the WordPress side of the stack, not to the monitoring layer described here.

Self-Hosted SEO for Hugo

Static-site generators such as Hugo benefit particularly from external SEO tooling because they usually lack a large CMS plugin ecosystem. A good Hugo workflow treats SEO like software quality:

flowchart LR
    A[Markdown] --> B[Hugo build]
    B --> C[Preview site]
    C --> D[SiteOne]
    D --> E{Technical checks pass?}
    E -->|No| F[Fix source]
    E -->|Yes| G[Deploy]

If you deploy Hugo to S3, the crawl step slots into the same pipeline that builds and publishes the site, and IndexNow notifications complement the crawler by telling engines when new URLs exist.

Production monitoring then uses Search Console:

flowchart LR
    A[Published Hugo site] --> B[Google]
    B --> C[Search Console]
    A --> D[Local crawler]
    C --> E[SEO Skill / CrawlSEO]
    D --> E
    E --> F[Prioritized opportunities]

The workflow evaluates the website that users and search engines actually receive, not the editor's view of it.

Local LLMs in Self-Hosted SEO

Local LLMs add another layer, but they should not become the source of truth. A model is good at:

rewriting titles
proposing descriptions
clustering queries
summarizing crawl findings
identifying semantic gaps
suggesting internal links
comparing content structure
drafting update plans

It is poor as the authoritative source for:

HTTP status codes
rankings
clicks
impressions
indexing state
canonical correctness
backlink counts
Core Web Vitals

The architecture:

flowchart LR
    A[Crawler / GSC / SERP source] --> B[Structured evidence]
    B --> C[Local LLM]
    C --> D[Recommendation]
    D --> E[Review]

rather than asking the model "Audit my site and tell me how it ranks."

SiteOne and Scouter provide native routes to local models. SEO Skill and CrawlSEO are useful when the LLM lives in an external agent such as Hermes, Claude Code, or another MCP client. Running the model on your own infrastructure keeps your content there, the same argument as the broader LLM self-hosting and AI sovereignty case applied to SEO. For the model server itself, the Ollama CLI cheatsheet covers serving and model management.

MCP in Self-Hosted SEO

Model Context Protocol changes how SEO software interacts with AI systems.

Without MCP:

SEO tool
-> export CSV
-> paste into AI
-> ask question

With MCP:

AI agent
-> request crawl evidence
-> request GSC evidence
-> request affected URLs
-> analyze
-> propose change

The agent retrieves only the evidence it needs. Projects in this comparison that already support MCP: OpenSEO, SEO Skill, CrawlSEO, Scouter, OpenGSC, and OpenSERP through its MCP package. If you have built the server side of this integration, the MCP server implementation notes in Go show what an MCP server looks like from the inside.

A properly designed SEO agent could:

1. Find pages losing impressions.
2. Check whether the pages have crawl issues.
3. Read their dominant Search Console queries.
4. Find weak internal linking.
5. Propose a change.
6. Modify the source on a Git branch.
7. Run technical validation.
8. Open a pull request.

The SEO system remains the evidence provider, and the agent becomes the workflow engine.

What Can Actually Be Replaced Locally?

A realistic self-hosted stack can replace a substantial share of commercial SEO functionality.

Capability	Practical self-hosted option
Technical crawling	SiteOne, LibreCrawl, Scouter
Metadata checks	SiteOne, LibreCrawl, Scouter
Internal link analysis	Scouter, SiteOne, crawl exports
Search performance	GSC + SEO Skill / CrawlSEO / OpenGSC
Content decay	GSC-based platforms
Core Web Vitals	CrawlSEO, Lighthouse, PageSpeed
Rank tracking	SerpBear, OpenSERP
SERP inspection	OpenSERP
AI analysis	Local Ollama / vLLM / llama.cpp
Agent automation	MCP-enabled SEO systems
CI quality gates	SiteOne
Historical monitoring	CrawlSEO / OpenGSC

The two major gaps remain global keyword data and backlinks.

What Is Difficult to Self-Host?

Global keyword volume

Search volume looks simple when displayed as one number:

"local llm" -> 12,100 searches/month

but obtaining that number reliably requires access to a very large dataset. Google Search Console only shows queries for which your own site already appeared; it cannot describe the demand landscape for a topic you have never covered. Google Ads provides some keyword information, but building a Semrush-like global keyword database locally is not realistic for most individuals.

Global backlink index

Backlink analysis is more infrastructure-heavy. Ahrefs, Semrush, Majestic, and similar companies continuously crawl huge portions of the web, normalize URLs, identify links, deduplicate data, and maintain historical indexes. An individual can crawl selected competitor sites or monitor links discovered through Search Console and analytics, but that is not equivalent to a web-scale backlink database. Open-source SEO products integrate DataForSEO, Ahrefs, or another provider for these two categories because the underlying data is expensive to collect.

A Practical Fully Self-Hosted Stack

If the goal is useful SEO without a paid SEO API, combine specialized components rather than looking for one giant application:

flowchart TD
    SITE[Website]
    GSC[Google Search Console]
    CRAWL[SiteOne]
    SEO[SEO Skill]
    SERP[OpenSERP]
    AGENT[AI Agent]
    LLM[Local LLM]
    GIT[Git / CMS]

    SITE --> CRAWL
    SITE --> GSC
    GSC --> SEO
    CRAWL --> SEO
    SERP --> SEO
    SEO --> AGENT
    AGENT --> LLM
    AGENT --> GIT

The responsibilities:

SiteOne:
  technical crawl
  CI regression checks

SEO Skill:
  Search Console
  first-party opportunity analysis
  MCP

OpenSERP:
  optional SERP observations

Local LLM:
  interpretation and language tasks

Agent:
  workflow orchestration

Git/CMS:
  controlled publication changes

This stack cannot reproduce every Ahrefs feature, but it covers most of the work involved in improving an existing website.

Choosing a Tool for Your Requirement

Primary requirement	Tool
Deterministic technical checks, CI/CD use, optional local AI	SiteOne
Human-friendly web crawler with exports and JavaScript rendering	LibreCrawl
AI-native crawler: MCP, SQL access, multi-user UI, local Ollama/vLLM	Scouter
CLI/MCP backend combining crawl and Search Console evidence	SEO Skill
Persistent dashboard: GSC, crawling, Core Web Vitals, agent access	CrawlSEO
Search Console-centered dashboard: decay, striking-distance queries, agent access	OpenGSC
Keyword rank tracking, self-managed proxies or SERP provider	SerpBear
Raw self-hosted SERP infrastructure for scripts and agents	OpenSERP
Integrated Semrush-style product with DataForSEO underneath	OpenSEO

Recommended Stacks by Website Type

Small WordPress blog

A conventional WordPress SEO plugin plus occasional technical crawling is enough:

The SEO Framework / Yoast / SEOPress
+
SiteOne or LibreCrawl

There is little reason to deploy five services for a small site.

Established WordPress publication

WordPress SEO plugin
+
CrawlSEO or SEO Skill
+
Search Console
+
technical crawler

Add a local LLM only if you have enough content that automated analysis saves meaningful time.

Hugo or static technical site

SiteOne
+
SEO Skill
+
Search Console
+
Git CI

This makes SEO part of the engineering workflow.

AI-heavy self-hosted environment

Scouter or SiteOne
+
SEO Skill
+
MCP agent
+
local Ollama / llama.cpp / vLLM
+
Git review workflow

This is where self-hosting provides the most architectural freedom.

Conclusion

The open-source SEO ecosystem is large enough that "self-hosted SEO" is a real category rather than a collection of abandoned scripts. No single open-source application reproduces every useful feature of Ahrefs or Semrush without external data: the global keyword databases and web-scale backlink indexes remain expensive because the underlying data is expensive to collect.

The practical opportunity is in the layers you can own: technical crawling, first-party Search Console analysis, local SERP observation at moderate scale, local LLM interpretation of that evidence, MCP as the interface to agents, and CI/CD as a regression gate. For most technically managed websites, that combination is more valuable than recreating a commercial suite, and it answers the question that matters: which capabilities do you need, which data can you collect yourself, and where does an external dataset genuinely add value.

References

WordPress SEO Plugins Compared: Yoast to Local AI

Rost — Sat, 03 Oct 2026 12:37:47 +0000

WordPress SEO plugins add the metadata, sitemaps, schema, and AI layer above WordPress core, and in 2026 that layer includes plugins that run inference on your own hardware.

The market is no longer a choice between Yoast and a few competitors. It now contains commercial SEO suites, lightweight open-source plugins, AI assistants that call hosted APIs, and newer plugins that talk to local models through Ollama, llama.cpp, or any OpenAI-compatible endpoint.

The answer to "which WordPress SEO plugin should I use" depends on what you need: a complete marketing suite, a quiet technical SEO layer, or a local AI assistant operating alongside a conventional plugin. This article compares the three models, walks through the major commercial, open-source, and local-AI plugins, and places the plugin layer inside the wider Web Infrastructure hub of this site.

What a WordPress SEO Plugin Does

A WordPress SEO plugin sits between WordPress content and the HTML, feeds, sitemaps, and machine-readable metadata consumed by search engines and social platforms.

Its most important jobs are deterministic:

generate SEO titles and meta descriptions;
define canonical URLs;
control index, noindex, follow, and nofollow;
create XML sitemaps;
add Schema.org structured data;
add Open Graph and social metadata;
manage redirects and 404s;
generate breadcrumbs;
expose SEO fields in the editor;
integrate with Google Search Console or analytics;
provide content and internal-link analysis.

A conventional flow looks like this:

flowchart LR
    A[WordPress content] --> B[SEO plugin]
    B --> C[Title and description]
    B --> D[Canonical and robots]
    B --> E[Schema JSON-LD]
    B --> F[XML sitemap]
    B --> G[Open Graph]
    C --> H[Published HTML]
    D --> H
    E --> H
    G --> H
    F --> I[Search engines]
    H --> I

A canonical URL should not depend on whether an LLM happens to make a good decision on a particular day. Sitemap output stays in the same deterministic layer; IndexNow handles the separate question of telling engines when URLs change.

AI is useful for a different class of tasks:

proposing alternative titles;
rewriting descriptions;
identifying missing sections;
suggesting internal links;
generating image alt text;
clustering topics;
summarizing SEO findings;
assisting editors with content improvements.

That gives a second workflow:

flowchart LR
    A[Post or page] --> B[SEO analysis]
    B --> C[AI provider]
    C --> D[Suggested change]
    D --> E[Human review]
    E --> F[Approved update]

Deterministic output stays with the SEO plugin; language generation goes to the AI layer, and the two can live in the same plugin or in two separate ones.

The Three WordPress SEO Plugin Models

Most WordPress SEO products now fall into one of three models.

Commercial SEO Suites

Yoast SEO, Rank Math, All in One SEO, and SEOPress combine a free WordPress plugin with paid features, commercial services, or premium editions. Their main advantage is maturity: large user bases, established migration paths, compatibility with popular themes and page builders, and extensive documentation.

Lightweight Self-Hosted SEO Plugins

The SEO Framework and Slim SEO take a quieter approach. Their core SEO functionality runs inside WordPress and focuses on automatically generating correct metadata rather than turning SEO into a large marketing dashboard. They fit site owners who already understand technical SEO and do not need continuous scoring or editorial coaching.

AI SEO Assistants

A newer category adds generative AI to WordPress. Some plugins call OpenAI, Anthropic, Gemini, Groq, OpenRouter, or other hosted providers. Others talk to Ollama, llama.cpp, vLLM, LM Studio, or another self-hosted model endpoint.

The distinction matters for privacy and cost: a plugin can be installed on your own WordPress server while still sending every article to an external AI service.

Commercial WordPress SEO Plugins Compared

The four dominant commercial or freemium products have substantially larger deployment histories than the newer AI-focused plugins.

Active-installation figures below are approximate WordPress.org values as of September 2026.

Plugin	Active installs	Main strength	Complexity	Best suited for
Yoast SEO	10M+	Mature defaults and editorial guidance	Medium	General WordPress sites
Rank Math	4M+	Broad feature set and detailed controls	Medium-High	Power users and feature-rich sites
All in One SEO	2M+	Complete SEO and marketing suite	Medium-High	Business and WooCommerce sites
SEOPress	300K+	Technical controls and privacy-oriented design	Medium	Developers and agencies

Yoast SEO

Yoast SEO is developed by Yoast and remains the most widely installed dedicated WordPress SEO plugin. WordPress.org reports more than 10 million active installations, which gives it an unusually large production footprint.

Yoast covers the expected SEO fundamentals: titles, descriptions, canonical URLs, XML sitemaps, schema, breadcrumbs, robots controls, social metadata, readability analysis, and content optimization. Its strongest feature is not one specific algorithm but maturity: themes, hosts, migration tools, and other plugins generally understand Yoast metadata.

The disadvantage is accumulated complexity. Yoast now includes a large amount of editorial guidance, commercial functionality, integrations, and AI-related features, which can feel excessive when all you need is a reliable metadata and sitemap layer.

Installation is simple and its setup flow suits non-specialists. Choose Yoast when compatibility, documentation, and proven behavior matter more than minimalism.

Rank Math

Rank Math has grown into the largest direct alternative to Yoast, with more than 4 million active WordPress installations. It takes a more feature-heavy approach and exposes many controls that other products either hide or split into separate modules.

Rank Math includes titles, descriptions, XML sitemaps, schema, redirection management, 404 monitoring, Search Console-related features, content analysis, and AI-oriented tooling. Its LLMS Txt module, toggled from the Rank Math dashboard, generates and maintains an llms.txt file at the site root that points AI crawlers at the pages that matter most; other suites added similar generation during 2025, so check the exact edition and version before relying on it.

In 2026 Rank Math added MCP functionality. The MCP server exposes tools for running SEO audits, analyzing posts, reviewing metadata scores and Schema markup, checking links and redirects, retrieving Search Console keyword data with impressions, position, and CTR, fetching robots.txt and llms.txt, and updating supported settings. The vendor documents client support for MCP-capable assistants such as Claude Desktop and GitHub Copilot, so an external agent can read the plugin's SEO state without a browser session.

Its strength is breadth, and that breadth creates complexity. A small publication may not need dozens of modules, scores, dashboards, and integrations, and enabling too much functionality turns the SEO plugin into another administration platform that needs ongoing maintenance.

Installation is still straightforward, but configuring Rank Math well requires more decisions than installing a lightweight plugin. Choose Rank Math when you want most SEO functions concentrated in one product and do not mind a larger configuration surface.

All in One SEO

All in One SEO, commonly called AIOSEO, is one of the longest-running WordPress SEO products. WordPress.org reports more than 2 million active installations.

AIOSEO combines titles, descriptions, canonical controls, XML sitemaps, schema, redirects, local SEO, WooCommerce SEO, Search Console integration, internal-link assistance, author SEO, rank-related features, and AI content tools. The product increasingly resembles a complete SEO and marketing platform rather than a narrow technical plugin.

That makes it useful for commercial sites and less compelling for a technically managed publication that prefers small components. Many of the more advanced features belong to the paid product, so the useful feature boundary depends on the edition in use.

The setup wizard is approachable, but the full product has a substantial feature surface. Choose AIOSEO when you want a broad business-oriented SEO suite, particularly for WooCommerce or sites where marketing staff manage SEO directly in WordPress.

SEOPress

SEOPress is a mature alternative developed by the SEOPress team, with more than 300,000 active installations. It appeals to developers and agencies because it combines strong technical controls with a comparatively restrained interface.

SEOPress supports titles, descriptions, canonical URLs, XML and HTML sitemaps, Open Graph, analytics integration, structured data, redirects, content analysis, llms.txt, and AI-search-oriented functionality. It also provides hooks, REST support, WP-CLI integration, and white-label options. Its PRO edition adds AI metadata and image alt-text generation through hosted providers (GPT, Claude, Mistral, DeepSeek, and Gemini), which keeps the local-inference question to the plugins in the later section.

Compared with Yoast and AIOSEO, SEOPress feels less focused on coaching the editor through every paragraph. The tradeoff is that many advanced functions are part of the commercial edition, so a site owner should compare the exact edition rather than assume every advertised feature belongs to the free plugin.

Configuration complexity is moderate. Choose SEOPress when you want a mature general-purpose SEO suite with stronger developer controls and less intrusive branding.

Lightweight SEO Plugins: The SEO Framework and Slim SEO

Not every site needs a large SEO suite. For many technical publishers, the ideal SEO plugin generates correct metadata, stays predictable, and stays out of the way. The SEO Framework and Slim SEO are the two strongest examples of that philosophy.

Plugin	Active installs	GitHub stars	GitHub forks	Main strength	Complexity
The SEO Framework	200K+	480+	60+	Quiet, deterministic SEO	Low
Slim SEO	70K+	90+	20+	Automatic configuration	Low

GitHub numbers should be interpreted carefully for WordPress plugins. WordPress.org remains the main installation and update channel, so active installations are usually a stronger adoption signal than stars alone.

The SEO Framework

The SEO Framework is developed by Sybre Waaijer and CyberWire. Its core plugin is released under GPLv3 and has more than 200,000 active WordPress installations.

It automatically handles titles, descriptions, canonical metadata, Open Graph, structured data, robots directives, and XML sitemaps. Its main advantage is restraint: it tries to produce sensible SEO output without filling the editor with keyword scores, traffic-light indicators, or a long list of upsells.

The SEO Framework is not an AI SEO platform. That means local LLM functionality has to come from another plugin or integration.

Installation is simple and default behavior is sensible for many sites. Choose The SEO Framework when you want a reliable technical SEO layer that can later be combined with a separate AI assistant.

Slim SEO

Slim SEO is developed by eLightUp and has more than 70,000 active WordPress installations. Its source is publicly available on GitHub and the project emphasizes automatic configuration.

Slim SEO generates titles, descriptions, Open Graph metadata, XML sitemaps, schema, breadcrumbs, and related metadata with relatively little configuration. Recent releases added AI-generated meta titles and descriptions through providers including OpenAI, Anthropic, Gemini, and OpenRouter.

Its weakness for privacy-focused self-hosting is that the built-in AI path is oriented toward hosted providers rather than a generic local OpenAI-compatible server. The core SEO engine remains local, but AI usage is not automatically fully self-hosted.

Installation is one of the simplest in this comparison. Choose Slim SEO when you prefer automation over configuration and do not need a large SEO administration suite.

WordPress Plugins with Local LLM Support

This is where the market becomes more experimental. The important question is no longer simply whether the WordPress plugin is open source. You need to ask where the inference runs and whether a vendor backend sits between WordPress and the model.

A useful classification:

Plugin	Role	Local/self-hosted LLM	Vendor backend required	Maturity
AI Engine	General AI framework	Yes, OpenAI-compatible	No	Established
Coretex SEO	AI-assisted SEO	Yes, Ollama	No	New
SEO Auditor Tools	SEO audit + AI fixes	Yes, Ollama	No	New
PIV AI SEO Assistant	Metadata and image alt generation	Yes, OpenAI-compatible	No	New
Synthocode Content Copilot	Editor AI assistant	Yes, Ollama	No	New

The maturity column matters: several local-AI SEO plugins are promising architecturally but have tiny WordPress.org user bases compared with the established SEO plugins.

AI Engine

AI Engine is developed by Meow Apps and has around 90,000 active installations. It is not primarily an SEO plugin; it is an AI framework for WordPress.

Its main strength is provider flexibility. AI Engine supports hosted providers but also accepts custom OpenAI-compatible endpoints such as Ollama, LM Studio, vLLM, llama.cpp, and LocalAI. It also provides developer APIs, hooks, and MCP functionality.

That makes AI Engine interesting when you want to keep deterministic SEO in The SEO Framework, SEOPress, or another conventional plugin while adding local AI workflows separately. It can become the connection between WordPress and a private inference server.

The downside is that it does not automatically replace a dedicated SEO plugin. You design or configure the actual SEO workflow yourself, and some advanced AI Engine functionality belongs to commercial modules.

Choose AI Engine when you want a programmable local AI layer rather than a narrow "generate meta description" button.

A typical architecture looks like this:

flowchart LR
    A[WordPress] --> B[The SEO Framework or SEOPress]
    A --> C[AI Engine]
    C --> D[OpenAI-compatible endpoint]
    D --> E[llama.cpp / Ollama / vLLM]
    E --> F[Local LLM]

Coretex SEO

Coretex SEO is a newer AI-assisted SEO plugin built around recommendations rather than automatic modification.

It can analyze titles, descriptions, headings, internal links, and image SEO, then place proposed changes into an approval queue with a before/after preview. Nothing is written to the site until an administrator accepts the suggestion. It supports commercial AI providers as well as a local Ollama instance, and the plugin documentation states that no Coretex account or Coretex-hosted backend is required.

The approval workflow is one of its strongest design choices: AI-generated SEO changes remain suggestions, and the model has no standing authority to modify pages.

The main concern is maturity. Its active-installation count is still very small, so it does not have the deployment history of Yoast, Rank Math, The SEO Framework, or AI Engine.

Choose Coretex when local Ollama integration and review-first AI suggestions matter more than a long production track record.

SEO Auditor Tools

SEO Auditor Tools combines a WordPress SEO audit with a large collection of additional site-management utilities.

Its audit layer covers SEO, accessibility, performance, and best practices. It also includes sitemap and robots tooling, redirects, image tools, llms.txt, and an AI helper that can connect to a self-hosted Ollama server.

The AI workflow is conservative: it generates structured proposals, validates them, shows a preview, and requires administrator approval before writing changes. It also records changes and supports undo.

The downside is scope. The plugin bundles SEO, performance, security, analytics, image tools, redirects, and other functions in one package, which is a lot of responsibility for a relatively new plugin.

Choose it if you want an all-in-one local audit and remediation tool and are comfortable evaluating a younger project carefully.

PIV AI SEO Assistant

AI SEO Meta and Image Alt Text Generator is a focused helper rather than a complete SEO engine.

Its most important feature for self-hosting is support for arbitrary OpenAI-compatible endpoints. That allows it to work with Ollama, LM Studio, private gateways, and other compatible servers without forcing content through a vendor-owned AI backend.

The plugin focuses on titles, descriptions, keywords, social metadata, and image alt text. It can write into metadata managed by established plugins such as Yoast, Rank Math, AIOSEO, SEOPress, and The SEO Framework.

It does not replace those plugins. Canonical URLs, schema, sitemap generation, and other deterministic SEO functions still belong to the underlying SEO engine.

Choose PIV when you already have an SEO plugin you trust and want to add local AI-assisted metadata generation on top.

Synthocode Content Copilot

Synthocode Content Copilot is an editor-side writing assistant for Gutenberg.

It can generate titles, descriptions, article ideas, summaries, tags, translations, and FAQ schema. It supports hosted providers as well as local Ollama, allowing content generation to remain inside your own infrastructure.

Its advantage is simplicity: instead of becoming a full technical SEO platform, it assists the person writing or updating a post.

That is also its limitation. It does not replace a proper SEO plugin for canonicals, sitemaps, redirects, schema management, or site-wide technical controls.

Choose it when you mainly want a private AI assistant inside Gutenberg.

Where Does Inference Run? Four "Self-Hosted" Architectures

The term "self-hosted" is used too loosely for WordPress SEO plugins. There are at least four distinct architectures.

1. Plugin installed locally, vendor service required

WordPress
    |
    v
Plugin
    |
    v
Vendor cloud

The plugin code runs on your server, but important functionality depends on the vendor.

2. Plugin installed locally, direct cloud API

WordPress
    |
    v
Plugin
    |
    v
OpenAI / Anthropic / Gemini / other API

There is no plugin-vendor backend, but content still leaves your infrastructure.

3. Plugin installed locally, local model

WordPress
    |
    v
Plugin
    |
    v
Ollama / llama.cpp / vLLM

This is genuinely local inference.

4. Deterministic SEO plugin plus separate local AI

flowchart LR
    A[WordPress] --> B[SEO plugin]
    B --> C[Metadata / schema / sitemap]
    A --> D[AI integration]
    D --> E[Local LLM]
    E --> F[Suggestions]
    F --> G[Human review]

For technically managed sites, the fourth model is usually the most attractive. It keeps critical SEO infrastructure independent of model availability and separates deterministic decisions from language-generation tasks. The same data-residency argument applies at the workflow level, not just the model level, as the data gravity and API lock-in analysis shows. For choosing between Ollama, llama.cpp, vLLM, LM Studio, and similar servers, the local LLM hosting comparison covers API support, hardware compatibility, and production readiness.

Pointing a WordPress Plugin at a Local Endpoint

Whichever plugin in the previous section you choose, the configuration reduces to one thing: an OpenAI-compatible base URL. Working through the checks below with curl first tells you whether a later problem is in the plugin or in the model server.

Start the model server and make sure a model is pulled. For Ollama:

   ollama serve
   ollama pull llama3.2

The Ollama CLI cheatsheet covers the full command set.

Verify that the OpenAI-compatible endpoint answers:

   curl http://127.0.0.1:11434/v1/models

The response lists pulled models. An empty list or a connection error means the server is not ready, not that the plugin is broken.

Send one completion in the same shape the plugin will send:

   curl http://127.0.0.1:11434/v1/chat/completions \
     -H 'Content-Type: application/json' \
     -d '{"model":"llama3.2","messages":[{"role":"user","content":"Write a 155-character meta description for a post about Hugo sitemaps."}]}'

Enter the base URL in the plugin's AI settings. The defaults for common runtimes:

| Runtime | Local base URL |
|---|---|
| Ollama | http://127.0.0.1:11434/v1 |
| LM Studio | http://127.0.0.1:1234/v1 |
| vLLM | http://127.0.0.1:8000/v1 |
| llama.cpp (llama-server) | http://127.0.0.1:8080/v1 |

Most of these servers need no authentication locally; when the plugin demands an API key, any non-empty string (for example ollama) satisfies the field.

Run the plugin's connection test, generate a single description, then inspect the rendered page's <head> to confirm the meta tag actually changed. If the test fails while the curl calls succeed, check that the base URL includes the /v1 suffix and that the model name matches the output of ollama list exactly.

Two security checks belong in the same pass:

Ollama binds to 127.0.0.1 by default. If WordPress runs on the same host, leave it that way. Setting OLLAMA_HOST=0.0.0.0:11434 exposes an unauthenticated inference endpoint to the network, where anyone can use your GPU and the models you loaded. If the plugin must reach the server from another machine, the Tailscale or WireGuard path in remote Ollama access replaces opening a public port.
On the WordPress side, a local endpoint requires lifting WordPress's default block on HTTP requests to private addresses. The connector plugins that do this restrict the exception to the endpoint you configure, so other plugin HTTP calls cannot be pointed at internal addresses. Before enabling a local provider, check which hosts and ports your connector allowlists.

Which Plugin Combination Makes Sense?

For many sites, the best answer is not one plugin.

Conventional blog or small business site

Use:

Yoast
or
Rank Math

Both have large ecosystems and require little custom engineering. Yoast is the more conservative choice. Rank Math suits users who want more features and fine-grained controls in one place.

Developer-managed WordPress site

Use:

The SEO Framework
or
SEOPress

The SEO Framework is particularly attractive if you want the SEO layer to be small and predictable. SEOPress makes more sense if you want richer built-in controls without moving to the largest commercial suites.

Privacy-focused site using a local LLM

A clean architecture:

The SEO Framework
+
AI Engine or PIV
+
llama.cpp / Ollama / vLLM

This separates responsibilities:

The SEO Framework:
  canonical URLs
  robots
  schema
  sitemap
  social metadata

AI layer:
  title suggestions
  descriptions
  content review
  alt text
  editorial assistance

Local LLM:
  inference

That architecture is easier to reason about than letting one AI-heavy plugin control every SEO function.

Gutenberg-focused writer

Use a conventional SEO plugin plus Synthocode if the main requirement is editor assistance.

Experimental local-AI SEO workflow

Coretex SEO or SEO Auditor Tools are interesting, particularly because both use approval-oriented workflows. Their small deployment bases mean they should be evaluated more carefully before being made responsible for important production metadata.

What WordPress SEO Plugins Do Not Solve

Even the largest WordPress SEO plugins operate mostly inside one website. They can optimize metadata, site structure, schema, content, internal links, and redirects.

They cannot independently create the large external datasets provided by products such as Ahrefs or Semrush. A WordPress plugin cannot cheaply reproduce a web-scale backlink index or a global keyword-volume database. When a plugin offers those features, the data usually comes from the vendor or another external API.

The broader distinction:

WordPress SEO plugin:
  controls and optimizes the website

SEO platform:
  observes the website in the wider search ecosystem

A serious self-hosted SEO platform belongs outside WordPress. Technical crawlers, Search Console analysis, local rank tracking, MCP tools, and local LLM analysis can all be self-hosted separately; the self-hosted SEO tools and platforms comparison covers that layer in detail.

AI Should Not Become the Source of Truth

Generative AI is useful for SEO, but it should not determine facts that ordinary software can measure.

Bad architecture:

LLM decides:
  whether page is indexed
  whether canonical is correct
  current ranking
  number of broken links
  Core Web Vitals

Better architecture:

crawler / Search Console / WordPress:
  collect facts

LLM:
  interpret facts
  suggest improvements

For example:

FACT:
  page ranks around position 7

FACT:
  page has high impressions

FACT:
  CTR is unusually low

FACT:
  title is generic

AI TASK:
  propose three titles that better match the dominant queries

Proposing titles from collected facts is a safer use of an LLM than asking it to perform an SEO audit from memory.

WordPress SEO in 2026: What Has Changed

The most visible change is not that SEO plugins acquired more AI buttons. It is that WordPress can connect to AI systems through standard interfaces.

WordPress 7.0 introduced the AI Client, which exposes AI providers through a connector system visible under Settings → Connectors. Two connectors target self-hosted inference directly: AI Provider for Ollama registers Ollama as a provider and handles the localhost allowlisting, port registration, and keyless local authentication, while rtCamp's Universal OpenAI Connector registers an openai_compatible provider that accepts any compatible endpoint, including LM Studio and private gateways. Custom connectors must allowlist localhost hosts and safe ports themselves; without that, requests from the AI Client to a local server fail silently.

AI Engine supports arbitrary OpenAI-compatible endpoints including Ollama, LM Studio, vLLM, llama.cpp, and LocalAI. There is also a separate AI Provider for OpenAI Compatible Servers plugin for the WordPress AI Client, designed to connect WordPress to self-hosted inference servers. Either path keeps a local model behind the WordPress installation:

flowchart LR
    A[WordPress] --> B[AI integration]
    B --> C[OpenAI-compatible API]
    C --> D[llama.cpp]
    D --> E[Local model]

Products such as Rank Math and AI Engine are adding MCP support. That points toward AI agents interacting with WordPress and SEO tools through controlled interfaces rather than through embedded plugin screens. The MCP server implementation notes in Go show what that server side looks like.

Conclusion

There is no single best WordPress SEO plugin because the products now solve different problems. The decision is an architectural one: keep canonical URLs, sitemaps, schema, and indexing rules in deterministic code you can audit, verify the local endpoint with plain curl before trusting a plugin's AI settings, and use a model only where language reasoning adds value, with generated changes passing human review before publication.

If the goal grows beyond one plugin and reaches parts of Semrush, Ahrefs, or another external SEO platform, the architecture has to move outside WordPress entirely, into the self-hosted crawler, Search Console, and local LLM layer described in the companion platform comparison.

References

gstack: AI Software Engineering Stack

Rost — Mon, 28 Sep 2026 05:33:00 +0000

AI coding agents can already write functions, modify repositories, run tests, and open pull requests. The harder problem is getting an agent to follow a repeatable engineering process before, during, and after the code is written.

gstack, a project from Garry Tan originally built around Claude Code, takes a different route: instead of replacing your coding agent with another platform, it wraps the agent you already use with specialized skills, browser tooling, reviews, safety controls, and release processes. The project describes the result as a virtual engineering team -- twenty-three specialists and eight power tools, all slash commands, all Markdown, MIT licensed.

The comparison targets depend on which part of gstack you need: skill collections such as Superpowers, specification systems such as OpenSpec and GitHub Spec Kit, methodologies such as BMAD, orchestration platforms such as Ruflo, or your own maintained set of agent skills. Several of these combine with gstack rather than replace it, and the wider ecosystem they all belong to is mapped in the AI Developer Tools hub of this site.

What Is gstack?

gstack is an open-source collection of AI engineering workflows. In gstack's framing, software development consists of several different kinds of reasoning, and asking one generic coding prompt to perform all of them is a poor abstraction -- so the project exposes specialized skills instead:

product exploration
product and CEO review
architecture review
developer experience review
design review
implementation review
browser-based QA
investigation and debugging
security analysis
documentation
benchmarking
release preparation
deployment
retrospectives

The project presents these roles as a virtual engineering team: a CEO who rethinks the product, an eng manager who locks architecture, a designer who catches "AI slop", a reviewer who finds production bugs, a QA lead who opens a real browser, a security officer who runs OWASP and STRIDE audits, and a release engineer who ships the PR. The toolchain around the skill definitions is TypeScript and Bun: a setup script, generated skill documentation, session hooks, state under ~/.gstack/, a bundled browser, and a set of standalone CLIs.

gstack is not a foundation model and not a replacement for Claude Code; it is a process layer running on top of an agent harness:

flowchart TD
    A[LLM] --> B[Claude Code or another supported harness]
    B --> C[gstack skills and workflow rules]
    subgraph G[gstack process stages]
        D1[Planning]
        D2[Architecture review]
        D3[Design review]
        D4[Code review]
        D5[Browser QA]
        D6[Security]
        D7[Release and deployment]
        D8[Learning and memory]
    end
    C --> G
    G --> E[Git repository, browser, and development tools]

Two structural properties decide where gstack fits. First, the project describes it as a process rather than a collection of tools: the skills run in the order a sprint runs -- think, plan, build, review, test, ship, reflect -- and each skill hands its artifacts to the next, so /office-hours writes a design doc that /plan-ceo-review reads, and /plan-eng-review writes a test plan that /qa picks up. Second, gstack is not Claude Code-only: ./setup auto-detects the agents installed on the machine, and ./setup --host <name> targets Codex CLI, OpenCode, Cursor, Factory Droid, Kiro, Slate, OpenClaw, and Hermes, while a 2KB instruction-only digest in the repo covers rules-reading agents that need no install at all.

Why gstack Exists

A blank Claude Code session is extremely flexible, and that flexibility is also one of its weaknesses. Consider a feature request such as:

Add organization-level API tokens to the application.

A capable agent might immediately inspect the repository and start modifying authentication code, whereas a senior engineer would first ask who owns the tokens, whether users can belong to multiple organizations, how tokens are revoked, whether permissions are inherited, what happens to existing authentication, whether tokens should expire, how secrets are displayed, and which audit events are required. The agent may discover some of these questions eventually, but there is no guarantee it will discover them before implementation. gstack moves that discipline into reusable workflows: instead of idea -> coding agent -> code, the change passes product review, technical planning, architecture review, implementation, code review, browser QA, and release as named stages, each with its own command (the full sequence is in A Practical gstack Workflow below).

This does not make the AI correct; it changes the probability distribution of its mistakes. The agent is pushed to challenge assumptions earlier, inspect evidence, review its own work from several perspectives, and verify the application instead of stopping when the code compiles.

How gstack Turns Markdown Skills into Process

Most of gstack's capabilities originate in Markdown skill definitions. An agent skill can describe:

when it should run
what context it should inspect
what questions it should ask
which tools it may use
which commands it should execute
which evidence it must gather
which checks must pass
how the result should be structured

Such a skill acts somewhere between documentation, a reusable prompt, a standard operating procedure, and executable workflow configuration. The underlying mechanics are covered in Claude Skills and SKILL.md for Developers; gstack adds infrastructure on top of that primitive: generated skill definitions, startup and completion hooks, state management under ~/.gstack/, browser automation, safety mechanisms, repository inspection, opt-in telemetry, cross-session memory managed by /learn, and optional persistent knowledge through the separate GBrain project, which /setup-gbrain can stand up as a local PGLite database, a Supabase project, or a remote MCP endpoint.

The Most Important gstack Skills

The exact collection changes quickly, but several workflows illustrate how the system is intended to be used.

office-hours

/office-hours belongs near the beginning of a project or feature. It runs six forcing questions about the problem before any code is written, and in the README's worked example it reframes a request for a "daily briefing app" into a personal chief-of-staff AI, then writes the design doc that every downstream skill reads. For vague input like "we need better project search", the output is a requirement, not code.

plan-ceo-review

/plan-ceo-review examines the product-level assumptions behind a plan. It works in four scope modes -- Expansion, Selective Expansion, Hold Scope, Reduction -- and can challenge scope, identify missing opportunities, reduce unnecessary work, or suggest framing the problem differently. It runs before the requirements are fixed, a stage most coding-agent tools do not have.

plan-eng-review

/plan-eng-review shifts the perspective toward engineering: architecture, data flow, diagrams, edge cases, a test matrix, failure modes, and security concerns. It stays separate from the product review because merging both into one large prompt makes the model mix product decisions with implementation decisions.

plan-design-review and design-review

gstack treats visual and interaction design as a separate discipline. /plan-design-review rates each design dimension from 0 to 10, describes what a 10 looks like, and edits the plan to close the gap, with "AI slop" detection as a named check. The later /design-review runs the same audit against the actual implementation and fixes what it finds with atomic commits and before/after screenshots. For web applications, both pair with gstack's browser automation.

review

/review performs engineering review against repository changes from a staff-engineer perspective: it auto-fixes the obvious findings, flags the rest for approval, and keeps an advisory simplification lens for over-built code. Code written successfully is not necessarily code that should be merged.

investigate

/investigate enforces a systematic debugging rule the project calls the Iron Law: no fixes without investigation. It traces data flow, tests hypotheses, and stops after three failed fix attempts instead of continuing to thrash. It also auto-activates /freeze, which locks edits to the module under investigation.

qa and qa-only

/qa has the agent operate a browser, interact with the application, find bugs, fix them with atomic commits, re-verify, and generate a regression test for every fix. /qa-only runs the same methodology report-only. Many coding agents stop verification at "tests passed"; for a web application the browser is where integration errors, layout problems, incorrect flows, authentication failures, and JavaScript exceptions usually become visible.

ship, land-and-deploy, and canary

The release chain is three skills rather than one. /ship syncs main, runs tests, audits coverage, pushes, and opens the pull request, bootstrapping a test framework if the project has none. /land-and-deploy merges, waits for CI and the deployment, and verifies production health. /canary then runs a post-deploy monitoring loop that watches for console errors, performance regressions, and page failures.

autoplan, spec, learn, and retro

/autoplan runs the CEO, design, DX, and engineering review pipeline automatically -- engineering always last, so the shipping gate reviews the final amended plan -- and surfaces only taste decisions for approval. /spec turns vague intent into a precise, executable spec in five phases (why, scope, technical with mandatory code-reading, draft, file) with an outside-review quality gate before filing. /learn manages what gstack has learned across sessions -- patterns, pitfalls, and preferences -- with review, search, prune, and export. /retro produces a team-aware weekly retro; /retro global runs it across all your projects and AI tools.

Browser Automation in gstack

On supported macOS systems (macOS 15+), gstack drives the Aside browser first -- your real browser, with your real logged-in sessions, in tabs the agent opens for itself and closes when done. When Aside is unavailable, gstack falls back to its own Chromium-based engine, which ./setup builds and which runs a persistent daemon rather than launching a fresh browser for every command:

flowchart LR
    A[Coding Agent] --> B[gstack Browser CLI]
    B --> C[Local Browser Service]
    C --> D[Aside or bundled Chromium]
    D --> E[Application]

Persistent browser state lets cookies, authentication sessions, and tabs survive between operations, which makes browser-based QA practical. /open-gstack-browser exposes the fallback engine headed, with a sidebar agent that routes fast actions (click, navigate, screenshot) to Sonnet and reading or analysis to Opus. When the agent hits a CAPTCHA, an auth wall, or an MFA prompt, $B handoff opens a visible browser at the same page with cookies and tabs intact; you solve it, and $B resume continues where the agent left off. The agent suggests a handoff automatically after three consecutive failures. /pair-agent shares the browser with other agents -- OpenClaw, Hermes, Codex, Cursor, or anything that can curl -- with scoped tokens, tab isolation, rate limiting, and per-tab activity attribution.

The persistent engine also increases the security surface, since an agent with access to authenticated sessions holds a meaningful privilege. gstack ships a layered prompt-injection defense for this: content filters (datamarking, hidden-element stripping, ARIA scrubbing, URL blocklist) on every page read, plus a local ML classifier in a sidecar subprocess that scans page-derived content before the agent sees it, with a verdict combiner that requires classifier agreement before blocking. Page content is treated as untrusted input -- the agent takes syntax from a page, never instructions. Checks before and while using browser-driven QA:

Decide in advance which authenticated environments the agent may operate in, and prefer an isolated profile for QA work when the fallback engine is in use.
Know the emergency kill switch: GSTACK_SECURITY_OFF=1 disables the security layer -- do not leave it set.
The persistent daemon retains cookies and sessions between runs, so stop it when you are done and confirm nothing is left running, for example ps aux | grep -i chrom.
Read the hooks and safety mechanisms in the cloned repository before enabling them -- they are plain files, so review them the way you would review CI configuration.
After updating the clone, re-run ./setup so generated components stay in sync with the skill definitions.

Safety Guardrails and Second Opinions

Three power tools act as session-level safety switches. /careful warns before destructive commands -- rm -rf, DROP TABLE, force-push, git reset --hard -- and activates by saying "be careful"; recursive deletes of the root or home directory and force-pushes to the default branch are hard-denied. /freeze restricts file edits to one directory so the agent cannot "fix" unrelated code while debugging, and /guard activates both at once.

Second-opinion reviews cross harnesses: on Claude Code, /codex sends the work to OpenAI Codex CLI for an independent review, challenge, or consultation; on the other harnesses, /claude-code does the reverse. Each report identifies the provider that actually completed the review.

A Practical gstack Workflow

You do not need every gstack skill for every change. A reasonable feature workflow:

flowchart TD
    A[Feature idea] --> B[office-hours]
    B --> C[plan-ceo-review]
    C --> D[Create implementation plan]
    D --> E[plan-eng-review]
    E --> F[Implement]
    F --> G[review]
    G --> H[qa]
    H --> I[ship]

Which review skills to add depends on who the software is for:

Building for	Plan stage (before code)	Live audit (after shipping)
End users (UI, web app, mobile)	`/plan-design-review`	`/design-review`
Developers (API, CLI, SDK, docs)	`/plan-devex-review`	`/devex-review`
Architecture (data flow, perf)	`/plan-eng-review`	`/review`
All of the above	`/autoplan`	--

For a trivial bug fix, going straight to investigate, implementation, review, and tests is often enough. A tool-neutral version of the same shape -- spec, design, tasks, implement, validate -- is in Spec-Driven Development Workflow From Requirements to Code. The project's README describes running ten to fifteen of these sprints in parallel, each in its own isolated workspace; the sprint structure is what the project says keeps parallel agents from becoming sources of chaos.

Installing gstack

The current installation expects a working Claude Code setup, Git, Bun v1.0+, and, on Windows, Node.js -- Bun has a known bug with Playwright's pipe transport on Windows, so the browse server falls back to Node.js there. If you have not configured Claude Code yet, start with the Claude Code overview first. On macOS, the Aside browser (macOS 15+) is recommended for the browser skills; without it, the bundled Chromium daemon is used.

Verify your prerequisites: git --version and bun --version (the toolchain is Bun-based).
Clone gstack into the Claude skills directory:

   git clone --single-branch --depth 1 \
     https://github.com/garrytan/gstack.git \
     ~/.claude/skills/gstack

Run the setup script from the cloned directory:

   cd ~/.claude/skills/gstack
   ./setup

Setup installs and generates the components required by the supported skills and builds the bundled browser; a Chromium install failure is best-effort, setup records the reason, finishes registering every skill, and prints which skills are affected.

Add a ## gstack section to the project's CLAUDE.md. The project's install instructions include this step, and it is what makes Claude Code route the skills: use /browse from gstack for all web browsing, never use mcp__claude-in-chrome__* tools, and list the available skills.
Verify the install: check that the generated files are present in the cloned directory, start a Claude Code session, and run /office-hours on a scratch project to confirm the skill is recognized.

Team mode

For repositories, gstack offers a team-oriented setup where developers share one workflow instead of individually configured environments:

(cd ~/.claude/skills/gstack && ./setup --team) && \
  ~/.claude/skills/gstack/bin/gstack-team-init required && \
  git add .claude/ CLAUDE.md && \
  git commit -m "require gstack for AI-assisted work"

required blocks AI-assisted work in the repo without gstack; swap it for optional to nudge teammates instead of blocking them. No files are vendored into the repo: every Claude Code session starts with a fast auto-update check (throttled to once per hour, network-failure-safe, silent), which removes version drift across the team. Personal configuration improves one developer; repository-level configuration creates a shared engineering convention.

Other harnesses, upgrades, and uninstall

Other agents: ./setup --host codex, --host opencode, --host cursor, --host factory, --host kiro, --host slate, --host openclaw, and --host hermes install the skills into each agent's own skills directory. The 2KB instruction-only digest at agents-digest/gstack-AGENTS.md covers agents that only read rules files.
Command naming: skills register with short names by default (/qa, /review); ./setup --prefix switches to namespaced names (/gstack-qa), which matters when you run other skill packs alongside gstack.
Upgrades: re-run ./setup after a git pull (required on Windows, where installs are file copies), or use the /gstack-upgrade skill; setting auto_upgrade: true in ~/.gstack/config.yaml keeps the install current automatically.
Telemetry is off by default and asks for opt-in on first run. If you opt in, it sends skill name, duration, success/fail, gstack version, and OS -- never code, file paths, repo names, or prompts. gstack-config set telemetry off disables it at any time.
Uninstall: ~/.claude/skills/gstack/bin/gstack-uninstall removes skills, symlinks, ~/.gstack/ state, project-local state, browse daemons, and hook registrations.

Verify the install and fix common failures

./setup fails -- confirm Bun is on your PATH with bun --version; the generated components are built by the Bun toolchain.
Skills not recognized by Claude Code -- confirm the clone actually lives in ~/.claude/skills/gstack, that the project's CLAUDE.md has a gstack section, and re-run ./setup.
/browse reports NEED_ASIDE or ASIDE_NOT_RUNNING -- the probe is telling you it will use the fallback browser. That is normal on Linux and Windows; on macOS it means Aside is not open or signed in.
The fallback browser fails -- cd ~/.claude/skills/gstack && bun install && bun run build.
Stale install after an update -- run /gstack-upgrade, or set auto_upgrade: true in ~/.gstack/config.yaml.

Trying gstack Without Adopting Everything

Use gstack on a real but non-critical feature rather than migrating your development process. The project's quick start is the same trial, and it ends with "stop there":

/office-hours -- problem definition
/plan-ceo-review -- product reasoning
/review -- engineering verification, after implementation
/qa -- runtime verification, for web projects

If those stages surface findings your normal Claude Code workflow misses, the rest of the system is worth exploring; if they mostly produce additional text without changing engineering decisions, adopting the entire stack probably will not help.

What gstack Does Well

Separated engineering roles. Instead of one giant "be a senior engineer" instruction, product strategy, architecture, UX, QA, security, and release engineering each get their own reasoning mode.

Verification, not just generation. Review, browser QA with regression-test generation, security audits, benchmarking, and the ship-deploy-canary chain are first-class workflows in gstack rather than optional afterthoughts.

Inspectable. Much of the behavioral layer is plain Markdown files developers can read and modify, unlike the internal workflows of a proprietary autonomous agent. The repo also ships audit tooling for the stack itself: gstack-context-bill reports what an installed skill tree costs in tokens, and gstack-egress writes a hash-chained receipt for every off-machine send, telemetry included.

Team infrastructure. Skills can encode engineering conventions -- instead of typing

Remember to check API compatibility, run integration tests,
inspect the browser console, and update the changelog.

in every session, the requirements live in a reusable workflow, and team mode makes that workflow a repository requirement.

Where gstack Can Be Too Much

gstack is intentionally opinionated, and that limits its fit: a mature organization may already have architecture review procedures, release tooling, CI gates, QA automation, security scanning, ADR conventions, specification templates, and code review policies, and adding another complete methodology on top creates overlap instead of clarity.

There is also a context and token cost: each additional review stage adds repository inspection, model reasoning, and potentially more external model calls. The goal is the minimum reliable process needed to ship correct software, not a maximum number of AI reviews; gstack-context-bill can quantify what your installed skill set actually costs per session before you decide how much of it to keep.

gstack works best as a toolbox whose workflows you select and adapt, not as ceremony for every commit.

gstack Alternatives and Combinations

The closest alternatives, and the layer each occupies:

System	Primary focus	Workflow style	Agent portability	Best fit
gstack	Full engineering workflow	Role-oriented skills and tools	10 agents via `./setup --host`	End-to-end AI-assisted engineering
Superpowers	Engineering methodology	Automatic composable skills	High	Disciplined coding and TDD
OpenSpec	Change specifications	Lightweight spec artifacts	High	Brownfield feature development
GitHub Spec Kit	Spec-driven development	Structured multi-stage workflow	High	Formal requirements-to-code process
BMAD Method	AI-driven agile development	Adaptive roles and workflows	High	Larger end-to-end projects
Ruflo	Multi-agent orchestration	Agents, swarms, memory	Platform-oriented	Parallel autonomous agent systems
Custom skills	Your own process	Fully customizable	Potentially very high	Mature teams with established practices

gstack + Superpowers: implementation discipline inside the roles

Both are skill frameworks, so they overlap the most. The division of labor when combining them: gstack supplies the surrounding roles -- product, design, QA, release -- while Superpowers supplies the discipline inside the implementation phase (TDD, planning before implementation, systematic debugging, subagent review). Install both skill sets, then trim the overlapping skills so the agent never sees two conflicting instructions for the same phase; if command names collide, install gstack with ./setup --prefix so its skills register as /gstack-* and coexist with the other pack. Install and workflow details are in the Superpowers quickstart.

gstack + OpenSpec: durable specs, live reviews

OpenSpec keeps human and agent aligned around explicit change specifications -- artifacts for the proposed change, specifications, design decisions, and implementation tasks. The key property is persistence: a chat conversation disappears into context history, but a specification remains in the repository where humans and future agent sessions can review it. gstack adds the product review before the spec exists and the review and QA after it is implemented:

flowchart LR
    A[Feature request] --> B[gstack product review]
    B --> C[OpenSpec change]
    C --> D[Implementation]
    D --> E[gstack review]
    E --> F[gstack QA]

A concrete sequence: run /office-hours and /plan-ceo-review, capture the outcome as an OpenSpec change, implement against it, then run /review and /qa. Note that gstack also ships its own /spec skill, which archives specs under ~/.gstack; if OpenSpec owns the specification, keep gstack's /spec out of the loop so the two do not diverge. The OpenSpec quickstart covers the explore-propose-apply-archive loop in detail.

gstack + GitHub Spec Kit: pick one planning backbone

Spec Kit's core workflow is a sequence of explicit stages -- constitution, specify, plan, tasks, implement, converge -- and it has expanded into bug-fixing, idea-assessment, extensions, presets, and integrations. Since both Spec Kit and gstack center the planning stage, running both full flows duplicates work. If requirements traceability and formal stages matter, let Spec Kit own the specification backbone and use gstack for the layers Spec Kit does not enforce -- product review, design review, browser QA, and shipping. A broader comparison of spec-driven setups, including Kiro and Claude Code, is in GitHub Spec Kit vs Kiro vs Claude Code SDD Workflows.

BMAD and Ruflo: different axes

BMAD is a broader AI-driven development methodology whose adaptive workflows cover product thinking, specifications, architecture, and implementation, scaling the ceremony to the size of the work. It and gstack both play the process-backbone role, so choose one as the backbone rather than running both in full; gstack's individual skills can still be selected alongside a methodology.

Ruflo targets multi-agent orchestration: coordinated workers, shared memory, swarms. gstack applies multiple specialist perspectives to one engineering workflow; an orchestration platform applies multiple executing agents to one engineering objective. The boundary blurs -- gstack can call external tools and additional models, and orchestrators can implement structured engineering roles -- but the decision is independent: if the problem is that the agent skips engineering discipline, a skill framework is the direct fix; if it is running ten agents concurrently across many tasks and repositories, an orchestrator sits above a workflow like gstack rather than replacing it.

Custom skills: the most customizable layer

You can also skip the framework entirely and create a small collection of skills for the procedures your team already follows:

skills/
  architecture-review/
  api-review/
  database-migration-review/
  incident-analysis/
  release-check/
  security-review/

Each skill encodes organization-specific knowledge a generic framework cannot know. A database migration skill can require rollback analysis, table-lock analysis, index impact review, migration duration estimation, deployment ordering, and compatibility with the previous application version; an API review skill can require backwards compatibility, authentication checks, pagination consistency, idempotency analysis, rate-limit behavior, and OpenAPI changes. A practical path: start from the gstack skills you actually use, copy their structure into your own skills/ directory, and rewrite the checks around your conventions.

Four Layers: Skills, Specifications, Methodologies, Orchestrators

Four layers cover most of these tools, and they show how the combinations above fit together:

Skills answer "how should the agent behave?"

gstack, Superpowers, and custom agent skills.

Specification systems answer "what exactly are we building?"

OpenSpec and GitHub Spec Kit; the underlying spec-driven concepts and terminology are defined in What Is Spec-Driven Development?.

Methodologies answer "how should the project move from idea to software?"

BMAD, Superpowers, and parts of gstack.

Orchestrators answer "how should multiple agents execute work?"

Ruflo and other multi-agent runtimes.

The layers compose; a development environment can contain all four:

flowchart TD
    A[Product requirement] --> B[Specification system]
    B --> C[Engineering workflow]
    C --> D[Agent orchestrator]

    D --> E[Implementation agent]
    D --> F[Test agent]
    D --> G[Review agent]
    D --> H[QA agent]

    E --> I[Repository]
    F --> I
    G --> I
    H --> I

gstack already spans several of these boundaries.

Should You Use gstack?

The project's README describes the audience as technical founders and CEOs who still want to ship, first-time Claude Code users who want structured roles instead of a blank prompt, and tech leads and staff engineers who want rigorous review, QA, and release automation on every PR. gstack is worth trying if you use coding agents extensively and the limiting factor is no longer code generation itself. Typical symptoms:

the agent starts implementing before understanding the problem
implementation plans miss architectural implications
generated code passes tests but fails in the browser
reviews are inconsistent between sessions
release steps are repeatedly forgotten
different developers prompt the agent in completely different ways
useful engineering instructions remain buried in CLAUDE.md files
you repeatedly type the same review prompts manually

If your automation already provides strong deterministic gates and the agent only handles small, well-specified tasks, gstack adds little.

gstack and the Direction of AI Software Development

The shift gstack sits in tracks in generations: code completion (2022-2023), coding agents (2024-2025), specifications and agent workflows (2025-2026), and programmable AI engineering organizations. The products will change, but the model remains one component; engineering quality increasingly depends on the surrounding system:

persistent specifications
reusable skills
repository knowledge
browser access
tests
deterministic tools
review loops
security controls
memory
human approval boundaries
orchestration

Conclusion

gstack is the engineering process around a coding agent, packaged as inspectable, version-controlled skills, and its value is in forcing that process onto the agent rather than in any single skill.

Start from the trial sequence and keep only the skills that earn their place. Beyond it, the direction is composable layers -- specification, skills, deterministic verification, orchestration -- each doing what the others cannot.

References

gstack repository -- source, skill definitions, and setup
gstack skill deep dives -- philosophy, examples, and workflow for every skill
Aside browser -- the browser gstack drives first on macOS
Superpowers repository -- source, skills, and plugin manifests
OpenSpec repository -- source, docs, and the CLI package
GitHub Spec Kit documentation -- official Spec Kit workflow reference
BMAD-METHOD repository -- the adaptive AI-driven agile methodology
Ruflo repository -- the multi-agent orchestration platform

Self-Hosted Deep Research Systems: 12 Tools Compared

Rost — Fri, 25 Sep 2026 23:30:04 +0000

Deep Research has become its own category of software, not just a model pointed at a search box. This article compares twelve self-hosted systems and the research architectures behind them.

The line that actually matters is not whether a product ships a button labelled Deep Research, but what happens after the first round of retrieval. A genuine system notices that its plan was incomplete, chases a newly discovered lead, weighs conflicting sources, and only then writes the report.

Below I compare twelve open-source and self-hosted projects that implement this loop in different ways: recursive research trees, planner-plus-subagent designs, evidence-gap loops, perspective-driven question generation, and model-driven agentic search. For each one I cover the architecture, local-LLM support, RAG or private-document access, deployment complexity, and the license you actually inherit if you self-host. Where a system is also a full product (Open WebUI, Vane), I keep the focus on how it researches and link out to the dedicated guide for installation and configuration. Deep Research is one of the more demanding applied workloads in the AI Systems - it stresses retrieval, planning, and multi-step orchestration all at once, rather than any single layer in isolation.

What Is Deep Research?

A conventional AI web-search workflow is mostly linear. Even when several searches are performed, the model usually just creates related queries, retrieves documents, and summarizes what it finds:

Question
  |
Search
  |
Retrieve pages
  |
Summarize
  |
Answer

Deep Research adds a layer: the research process itself becomes adaptive. After the first pass, the system can branch, re-check, and keep going until the evidence is sufficient.

flowchart TD
    A[Research question] --> B[Build research plan]
    B --> C1[Investigate topic A]
    B --> C2[Investigate topic B]
    B --> C3[Investigate topic C]

    C1 --> D1[Discover new question]
    C2 --> D2[Find conflicting evidence]
    C3 --> D3[Identify missing information]

    D1 --> E1[Research new question]
    D2 --> E2[Verify competing claims]
    D3 --> E3[Search for missing evidence]

    E1 --> F[Combine evidence]
    E2 --> F
    E3 --> F

    F --> G[Evaluate remaining gaps]
    G -->|More research needed| B
    G -->|Enough evidence| H[Generate cited report]

The distinction matters. A system that searches five times is not necessarily performing Deep Research; a stronger system starts with one question, discovers an unexpected implementation detail, opens a new research branch around it, compares primary and secondary sources, and revises its original assumptions. For the broader Search vs Deep Search vs Deep Research distinction and how cloud offerings frame the same idea, see Search vs Deep Search vs Deep Research in 2026.

There is no single Deep Research architecture. Current self-hosted implementations generally fall into five groups:

Recursive research trees.
Planner and subagent architectures.
Evidence-gap-driven research loops.
Perspective and question-driven research.
Agentic iterative search.

The first four provide more explicit research structure. The fifth can still perform surprisingly deep investigation when paired with a strong reasoning and tool-calling model, but much of the strategy is delegated to the model itself. The evidence-gap loop in particular is a system-level cousin of the self-reflective retrieval used in Self-RAG-style pipelines — deciding whether to retrieve again, judging relevance, and critiquing the draft before answering. See Advanced RAG: LongRAG, Self-RAG and GraphRAG for that pattern at the retrieval-pipeline level.

Self-Hosted Deep Research Systems Compared

The table below summarizes the major systems. "Recursive depth" does not mean that multiple web searches are possible; it means the system has some mechanism for deriving additional investigation from intermediate findings.

System	Local LLM	Web Research	Private Docs / RAG	Planning	Recursive / Adaptive Depth	UI	Research Style
GPT Researcher	Yes	Yes	Yes	Yes	Excellent	Web UI	Recursive breadth/depth research tree
Unsloth Studio	Excellent	Yes	Yes	Yes	Very good	Excellent	Planned evidence-driven research
Local Deep Research	Excellent	Yes	Yes	Yes	Excellent	Web UI	Multiple strategies plus autonomous agent
STORM / Co-STORM	Yes	Yes	Custom corpus possible	Yes	Very good	Basic / demo UI	Perspective and follow-up-question research
DeerFlow	Yes	Yes	Yes	Excellent	Excellent	Good	Planner plus subagents and long-horizon agents
Onyx	Yes	Yes	Excellent	Yes	Excellent	Excellent	Multi-step enterprise Deep Research
Open Deep Research	Yes	Yes	Via tools / MCP	Excellent	Excellent	LangGraph oriented	Planner plus parallel researchers
Open WebUI	Excellent	Yes	Excellent	Model driven	Good	Excellent	Agentic iterative search and link following
Khoj	Yes	Yes	Excellent	Yes	Moderate	Good	Personal knowledge plus autonomous research
SurfSense	Yes	Yes	Excellent	Yes	Good	Excellent	Web/data research plus knowledge workspace
Vane	Yes	Yes	File search	Limited	Limited	Excellent	Search-first answering engine
Deep Research by lukeswade	Excellent	Yes	Research library	Yes	Excellent	Web UI	Gap-driven iterative investigation

One point stands out: there is no direct relationship between UI sophistication and research depth. Open WebUI and Vane provide polished interfaces, while GPT Researcher and STORM are centered more on the research algorithm. Conversely, Onyx and Unsloth Studio try to provide both a strong user experience and a substantial research workflow. Most of these systems run against the same local inference backends covered in the LLM Hosting guide.

Deep Research Architectures

Before comparing individual products, it is useful to understand the architectural differences.

Style	Representative Systems	Main Idea
Recursive research tree	GPT Researcher	Explicit breadth and depth generate new research branches
Planner plus subagents	DeerFlow, Open Deep Research	Planner decomposes work and independent agents investigate pieces
Evidence-gap driven	Unsloth Studio, Local Deep Research, lukeswade/deep-research	Findings are evaluated and missing evidence triggers another research round
Perspective driven	STORM / Co-STORM	Research is expanded by generating perspectives and follow-up questions
Multi-step research workflow	Onyx	Multiple research tasks gather and synthesize web and private knowledge
Agentic iterative search	Open WebUI	Model decides when to search, read, verify, and search again
Knowledge-first research	Khoj, SurfSense	Research combines private information with external sources
Search-first answering	Vane	Search and retrieval are optimized primarily for cited answers

The categories overlap. Local Deep Research offers several research strategies, and DeerFlow 2.0 is a general-purpose agent platform that can perform research rather than a research-only application. The distinction is nevertheless useful when choosing a system: a recursively branching researcher behaves differently from a chat interface whose model simply has a search_web tool. Systems that retrieve private documents alongside the web lean on the same retrieval patterns described in the RAG cluster.

License Comparison

Licensing is particularly important if the system will become part of an internal platform, commercial service, or redistributed product.

System	License	Licensing Notes
GPT Researcher	MIT	Current `pyproject.toml` declares MIT; some older package metadata still reports Apache-2.0
Unsloth Studio	AGPL-3.0	Studio UI is AGPL-3.0; core Unsloth remains Apache-2.0
Local Deep Research	MIT	Permissive open-source license
STORM / Co-STORM	MIT	Permissive open-source license
DeerFlow	MIT	Applies to current DeerFlow 2.0 repository
Onyx	MIT plus Enterprise License	Core is MIT; `ee` directories use the Onyx Enterprise License; `onyx-foss` is 100 percent MIT
Open Deep Research	MIT	Repository was archived in August 2026
Open WebUI	Open WebUI License	Current versions include branding restrictions; older code has MIT/BSD history
Khoj	AGPL-3.0-or-later	Network copyleft should be considered for modified hosted deployments
SurfSense	Apache-2.0	Current repository declares Apache-2.0
Vane	MIT	Formerly known as Perplexica
Deep Research by lukeswade	MIT	Permissive open-source license

For private self-hosting, none of these licenses prevents normal use. The differences matter when modifying the software, offering it to other users, embedding it into another commercial application, or redistributing derivatives. MIT and Apache-2.0 are generally the simplest options for integration. AGPL-3.0 deserves closer review for network-accessible modified deployments, and Open WebUI's current license adds its own branding conditions.

GPT Researcher

GPT Researcher is developed by Assaf Elovic and contributors as an autonomous research agent focused specifically on comprehensive online investigation. It is one of the clearest reference implementations of what "Deep Research" means when the term describes an algorithm rather than a user-interface feature.

Its strongest feature is explicit breadth and depth. Deep Research mode exposes parameters such as deep_research_breadth, deep_research_depth, and concurrency, allowing one investigation to generate several branches and those branches to generate additional research. This creates a real research tree rather than a fixed collection of search queries.

That approach also has costs. Recursive expansion can produce many retrieval and LLM operations, and the quality of the final result depends heavily on the model's ability to formulate useful research questions, extract evidence, and avoid propagating weak assumptions into deeper levels. GPT Researcher is also more research-engine oriented than applications such as Open WebUI or Unsloth Studio.

Installation is moderate rather than trivial: the project uses Python and ships a web application, while useful deployments also require suitable model and search providers. Current project metadata declares the MIT license. Choose GPT Researcher when explicit research depth, configurable recursion, and a research-first architecture matter more than an all-purpose local AI workstation.

Unsloth Studio

Unsloth Studio is developed by the Unsloth team as part of the broader Unsloth ecosystem. Originally best known for efficient model fine-tuning, Unsloth has expanded Studio into a local AI environment for inference, chat, tools, RAG, model management, and now Deep Research.

The interesting aspect of Studio is how tightly research is integrated with local model operation. Its Deep Research workflow includes a planning stage, plan review, evidence gathering, report generation, document handling, and failure handling when research steps fail to gather evidence. For users already running GGUF or other local models, this makes Studio considerably more convenient than assembling a separate research framework, inference server, and frontend.

Studio does not expose the same simple breadth/depth research-tree abstraction as GPT Researcher. Much of the workflow is organized around a research plan and evidence collection rather than arbitrary recursive expansion, and the feature is newer than some dedicated research projects. The quality of local research also remains sensitive to context length, output limits, tool use, and the reasoning quality of the selected model.

Installation is relatively friendly because Unsloth now provides Studio and desktop-oriented workflows across major platforms, although GPU and model configuration can still become substantial for advanced local deployments. The Studio component is AGPL-3.0, while the core Unsloth package remains Apache-2.0. Choose Unsloth Studio when Deep Research should be part of a broader local-model workstation rather than a standalone research service.

Local Deep Research

Local Deep Research is maintained by LearningCircuit and contributors as a privacy-oriented open-source research assistant. Its explicit goal is systematic research using web sources, academic databases, private documents, and local language models.

Its major advantage is architectural flexibility. Rather than enforcing one research algorithm, Local Deep Research supports pipeline-oriented strategies as well as a LangGraph agent strategy in which the model can decide what to search, which specialist sources to use, and when enough evidence has been collected. Academic sources such as arXiv, PubMed, Semantic Scholar, and other search mechanisms make it particularly attractive for technical and scientific research.

The downside of flexibility is complexity. Different strategies can behave substantially differently, which makes results harder to characterize with one simple "depth" parameter. It also has more moving pieces than a conventional chat UI, and users looking only for fast AI-assisted web answers may find it unnecessarily elaborate.

The project supports local operation and has developed a substantial application around the underlying research engine. Its license is MIT. Choose Local Deep Research when privacy, local inference, multiple research strategies, academic information sources, and control over the research process are more important than a minimal setup.

STORM and Co-STORM

STORM, developed by Stanford OVAL, stands for Synthesis of Topic Outlines through Retrieval and Multi-perspective Question Asking. It was designed around knowledge curation and long-form report generation rather than general AI chat.

STORM's distinctive technique is perspective-driven research. It tries to discover different perspectives on a subject and uses question asking to broaden the information collected before writing an outline and article. Co-STORM extends the concept toward collaborative human-AI knowledge curation. This can uncover dimensions of a topic that a conventional list of keyword searches may overlook.

STORM is less suitable as a general local AI frontend. Its workflow is strongly oriented toward researching and writing structured, Wikipedia-like articles, and the project's own documentation notes that generated output should not automatically be considered publication-ready. It is therefore better understood as a specialized research and knowledge-curation engine than a replacement for Open WebUI.

Installation is Python oriented, and the pipeline can be customized with different models and retrievers. STORM uses the MIT license. Choose it when the objective is broad topic exploration, perspective discovery, structured outlines, and long-form knowledge synthesis.

DeerFlow

DeerFlow is developed by ByteDance and the DeerFlow community. Its name originally stood for Deep Exploration and Efficient Research Flow, but an important distinction now exists between the original 1.x Deep Research framework and DeerFlow 2.0.

DeerFlow 1.x was specifically designed around Deep Research. DeerFlow 2.0 is a ground-up rewrite into a more general SuperAgent harness capable of orchestrating subagents, memory, sandboxes, tools, and skills. For research, this architecture is powerful because a coordinator can delegate different parts of a problem to separate agents and later synthesize their findings - the same planner-plus-subagent decomposition covered in Multi-Agent Orchestration Patterns.

The trade-off is that DeerFlow 2.0 is no longer a narrowly optimized research engine. It is closer to a general long-horizon agent platform in which research is one workload among coding, artifact generation, and other tasks. If the requirement is a small dedicated research service, this additional machinery can be unnecessary.

Deployment is consequently more involved than a simple search UI, although the architecture provides much more room for customization and extension. DeerFlow is MIT licensed. Choose DeerFlow when Deep Research is expected to become one capability inside a broader multi-agent automation environment.

Onyx

Onyx, originally known as Danswer, is developed by DanswerAI and positioned as a self-hostable AI application layer for organizations. It combines chat, agents, web search, RAG, MCP integration, many enterprise data connectors, and a dedicated Deep Research capability.

Onyx stands out because research can span both the public web and a substantial private knowledge environment. Its Deep Research implementation is a real multi-step research flow rather than merely a search-result summarizer, and the project has published results and execution logs for DeepResearch Bench. For organizations that need research across internal documentation, indexed applications, and external sources, this is a particularly strong combination.

The cost of those capabilities is infrastructure complexity. Onyx is a larger platform than GPT Researcher or a lightweight local research project, and many of its strengths only matter when connectors, indexing, authentication, document stores, and organizational data are actually being used.

Onyx can be deployed self-hosted, including in restricted environments. Most of the main repository is MIT licensed, while code under ee directories uses the Onyx Enterprise License; a separate onyx-foss repository is maintained as a fully MIT-licensed variant. Choose Onyx when Deep Research must coexist with serious enterprise RAG and organizational knowledge retrieval.

Open Deep Research

Open Deep Research was developed by LangChain as an open implementation of a configurable Deep Research agent. It combines planning, research, report generation, multiple model providers, search tools, and MCP integrations using LangGraph.

Its architecture is particularly interesting for developers. Research work can be decomposed and parallelized, making it a useful reference for planner-researcher-synthesizer designs. Because the project was built around LangGraph rather than a monolithic UI, it is also easier to study as an implementation pattern for building custom research agents.

There is one major problem for new deployments: LangChain archived the repository on August 21, 2026, and it is now read-only. The code remains useful, but starting a production system around an archived reference project creates an obvious maintenance risk.

The project is Python based and uses an MIT license. Choose it today mainly for architectural study, experimentation, or as a source of implementation ideas rather than as the default foundation for a new long-lived installation.

Open WebUI

Open WebUI is one of the most popular general-purpose self-hosted interfaces for local and remote LLMs. Its recent agentic tool architecture gives models access to web search, URL fetching, knowledge bases, files, memory, code execution, and other tools. For installation, RAG setup, and the full feature set, see the Open WebUI guide.

Open WebUI's research model is interesting because the research loop is largely controlled by the language model. In native agentic mode the model can search, inspect snippets, fetch full pages, identify missing information, follow newly discovered URLs, cross-check sources, and repeat the process before generating an answer. With a capable reasoning and tool-calling model, this can produce genuine investigative behavior without a dedicated fixed research tree.

The limitation is precisely that this structure is model driven. Open WebUI does not provide the same explicit breadth/depth research topology as GPT Researcher, and there is less deterministic control over how many independent branches will be explored. A weak tool-calling model can stop too early, search poorly, or fail to follow important leads.

Installation is among the easiest in this comparison, particularly for users who already run Ollama, llama.cpp, vLLM, or another OpenAI-compatible inference server. Current releases use the Open WebUI License, which retains substantial permissive characteristics but adds branding restrictions; earlier portions of the project have MIT and BSD-3-Clause history. Choose Open WebUI when you want excellent local-LLM integration and a general AI interface in which research is one of many agentic capabilities.

Khoj

Khoj is developed as a self-hostable personal AI and "second brain." It combines local or cloud language models with web retrieval, personal documents, semantic search, custom agents, automations, and an experimental /research mode.

Its strongest use case is research that crosses the boundary between public information and a user's existing knowledge base. A question can be investigated in the context of PDFs, Markdown files, notes, office documents, or connected information rather than treating every task as web research from scratch. This makes Khoj useful for ongoing personal or team knowledge work.

Khoj is not primarily designed around a visible recursive research tree. Its research functionality is better understood as autonomous investigation within a larger personal knowledge system. Users seeking explicit breadth/depth controls or a dedicated research-engine API may prefer GPT Researcher or Local Deep Research.

Self-hosting is supported and the system can work with local models including the Llama, Qwen, Gemma, and Mistral families. Khoj is licensed under AGPL-3.0-or-later. Choose it when Deep Research should be closely integrated with a long-lived personal knowledge base rather than treated as an isolated web research job.

SurfSense

SurfSense is an open-source research workspace that evolved from a NotebookLM-style knowledge system toward an agent-oriented open-web research platform. It combines a searchable knowledge base with web and platform-specific data connectors, reports, automations, MCP access, and local model support.

SurfSense's distinctive advantage is its data surface. It is designed to give agents structured access not only to ordinary web pages and search results but also to sources such as Reddit, YouTube, Google Maps, and other live information services. Research findings can then remain in the same environment as uploaded documents and previously collected knowledge.

It is less purely focused on the research algorithm than GPT Researcher or STORM. A significant part of SurfSense's value comes from retrieval infrastructure, connectors, knowledge management, and downstream artifacts rather than an explicit recursively expanding research graph.

Self-hosting is supported through Docker-oriented installation, and local models can be connected through common local inference interfaces. The current repository is Apache-2.0 licensed. Choose SurfSense when the difficult part of research is obtaining, structuring, retaining, and reusing information from many different data sources.

Vane, Formerly Perplexica

Vane, formerly known as Perplexica, is an open-source AI answering engine designed as a self-hosted alternative to search-first products such as Perplexity. It combines an AI chat interface, search backend, citations, local model support, and semantic search over uploaded files. For the Docker quickstart, SEARXNG_API_URL wiring, and Ollama/llama.cpp setup, see Vane (Perplexica 2.0) Quickstart With Ollama and llama.cpp.

Vane is good at the search-and-answer experience. It classifies questions, conducts web research, retrieves useful information, and generates cited responses through a polished interface. Most deployments back it with SearXNG as the search layer. For users who mainly want a private AI search engine backed by SearXNG and local models, it provides a much more focused experience than a large general agent platform.

Its limitation in this comparison is research depth. Although the system can run research operations, its architecture is still primarily that of an answering engine rather than a recursively branching Deep Research framework. It should therefore not be treated as equivalent to GPT Researcher merely because both can perform multiple searches before responding.

Vane is relatively straightforward to deploy with Docker and supports common model providers and local inference systems. It is MIT licensed. Choose Vane when the primary requirement is high-quality self-hosted AI search with citations rather than long-running autonomous investigation.

Deep Research by lukeswade

The lukeswade/deep-research project is a smaller self-hosted research system, but it implements one of the more interesting workflows in this comparison. It supports both cloud models and local OpenAI-compatible endpoints including llama.cpp, LM Studio, Ollama, vLLM, and MLX - for the server side of that endpoint, see llama.cpp Quickstart with CLI and Server.

Its research loop is explicitly gap driven. A run decomposes the question into targeted searches, reads relevant pages, produces per-source notes containing evidence, and analyzes what remains unknown before deciding what should be searched next. Higher depth settings allow several rounds and progressively larger source budgets, while saturation detection can stop research early when new searches cease producing useful information.

It does not have the ecosystem, organizational connectors, or general AI workstation capabilities of Onyx, Open WebUI, or Unsloth Studio. It is much more narrowly focused on doing one thing: researching a question deeply and storing the resulting research in a searchable local library.

That focus also makes deployment relatively understandable. The system has a web UI and is particularly friendly to local OpenAI-compatible model servers; the documentation recommends capable models and supports using a smaller fast model for high-volume note processing. It is MIT licensed. Choose it when local inference, transparent evidence collection, and information-gap-driven research matter more than a large surrounding platform.

Which Deep Research System Should You Choose?

There is no single winner, because these systems solve somewhat different problems.

For an explicit research algorithm with understandable depth controls, GPT Researcher remains one of the clearest starting points. Its breadth/depth model makes it easy to reason about why research expands and how expensive a run may become.

For a strongly local workflow, Local Deep Research and Unsloth Studio are particularly attractive. Local Deep Research provides more research-strategy flexibility, while Unsloth Studio integrates research with model management, inference, RAG, and the broader local-model workflow.

For long-form knowledge curation, STORM remains unusually interesting because its multi-perspective question-generation technique attacks a problem that many research systems ignore: discovering the questions that the original user did not know to ask.

For multi-agent systems, DeerFlow represents a different direction. Instead of building a specialized research loop, it treats research as a long-running agent task that can be delegated among subagents and combined with tools, memory, code execution, and other capabilities.

For organizations, Onyx has one of the strongest combinations of Deep Research, RAG, web investigation, and enterprise knowledge connectors. Its heavier infrastructure is justified when internal information sources matter as much as the public web.

For an existing local AI installation, Open WebUI may be all that is necessary. A sufficiently capable local model with native tool calling can repeatedly search, read pages, follow links, verify information, and fill gaps without installing a separate research engine.

Finally, lukeswade/deep-research is worth watching precisely because it is smaller. Its gap-driven workflow is conceptually clean, supports llama.cpp directly through an OpenAI-compatible endpoint, and separates expensive planning and synthesis from high-volume per-source processing.

How to Tell Real Deep Research from Repeated Search

The most useful way to evaluate these systems is not to ask whether they have a button labelled "Deep Research." Instead, inspect what happens after the first round of information is collected.

A genuine research system should be able to discover that its original plan was incomplete. It should recognize a contradiction, missing source, unexpected implementation detail, or newly relevant subtopic and change its subsequent investigation accordingly. That is the line separating sophisticated retrieval from research.

For self-hosting, the ecosystem is now broad enough that the choice is no longer simply between a cloud Deep Research service and a home-grown script. There are dedicated recursive researchers, academic knowledge-curation systems, enterprise research platforms, local-model workstations, general agent frameworks, and lightweight gap-driven tools.

The right choice therefore depends less on which project advertises the most features and more on the research architecture you want to operate: explicit recursion, multi-agent delegation, evidence-gap analysis, perspective discovery, or model-driven autonomous search.