DEV Community: WonderLab

Open Source Project #168: LoopX — Long-Horizon Agent Control Plane Running on Top of Codex/Claude Code

WonderLab — Tue, 01 Sep 2026 01:55:48 +0000

Introduction

"Keep the loop moving. Keep the judgment human."

This is article #168 in the "One Open Source Project a Day" series. Today's project is LoopX — a long-horizon agent control plane, 5,288 Stars, Apache-2.0, authored by huangruiteng.

LoopX addresses a problem that a single session can't solve: an agent can finish a task in one conversation, but what about work that spans days or weeks? Objectives shift, evidence goes stale, decision gates appear, agents hand work off to other agents… chat memory and a timer are nowhere near enough to govern any of that.

LoopX isn't an agent framework. It's a control plane running on top of your existing agent harnesses (Codex App / Claude Code / OpenCode / etc.): it owns the persistent state, decides "what happens next, whether to continue, and what needs human judgment," while the agent harness executes one bounded turn at a time.

What You'll Learn

LoopX's core concepts: the roles of Goal / Gate / Todo / Evidence / Quota
The Five Questions framework for the control plane
Integration patterns with nine agent harnesses
Available capability paths (Issue Fix / Explore / Auto Research, etc.)
Quick start: install, connect a project, run your first loop

Prerequisites

Experience with Claude Code, Codex, or similar AI coding agents
Basic familiarity with long-horizon work management (goals, task decomposition, review workflows)
Python fundamentals

Project Background

Overview

LoopX's core mental model is an agent-native Kanban for long-running work.

A regular Kanban manages human tasks. LoopX "cards" carry: identity (who owns this task), authority (who can claim it), evidence (what was accomplished), and continuation (context for the next step). "Moving a card" isn't clicking a button — it's a validated operator like claim, gate, monitor, or writeback. The board is a projection view; LoopX state is the source of truth.

Author / Team

Author: huangruiteng
Primary language: Python 3.11+
License: Apache-2.0 (from v0.4.8; earlier releases under MIT)
Website: huangruiteng.github.io/loopx
User manual: Feishu Wiki

Project Stats

⭐ GitHub Stars: 5,288+
🍴 Forks: 476+
📄 License: Apache-2.0
📅 Created: 2026-05-31

Core Concepts

LoopX compresses control-plane responsibilities into five questions:

Question	What LoopX keeps visible
What is the objective?	The active goal, explicit scope, and current authority
What happens next?	Ordered user and agent todos, ownership, claims, and leases
What needs human judgment?	Concrete user gates — not a vague "waiting for owner"
What evidence changed?	Compact run history, validation, blockers, accepted writeback
May the loop continue?	Quota, capabilities, safe fallback paths, scheduler hints

These five questions map to five product promises: long-horizon state / semantic decisions / human-agent collaboration / recovery / governance.

The Control-Plane Stack

objective / issue / project
   │
   ▼
LoopX state: objective + gates + todos + scope + evidence + quota
   │
   ├─ human judgment needed? ── yes ──▶ ask a concrete question and wait
   │
   ├─ safe fallback available? ────────▶ run one bounded agent slice
   │
   ▼
Codex / Claude Code / Cursor / shell agent executes one turn
   │
   ▼
write evidence + handoff + next todo ──▶ quota decides the next tick

The agent harness executes the work. LoopX governs the state that lets work continue across runs.

Four-Layer Responsibility Separation

Layer	Responsibility
Kernel	Owns durable Goal, Todo, Gate, Evidence, Quota, Recovery, Scheduling state
Capability	Defines a stable, provider-neutral contract for one bounded, verifiable caller outcome
Provider	Calls external systems and returns bounded observations and readback
Extension	Packages and operates an optional Provider with install/upgrade/rollback lifecycle

Execution path: Agent → Capability → Provider. Control path returns: Provider readback → Capability transition → Kernel.

Quick Start

Installation

# Requires Python 3.11+
python3 -m pip install --upgrade loopx
loopx workflow-skills --install
loopx doctor

macOS/Linux: POSIX shell. Windows: PowerShell 7 (no compatibility layer needed).

Connect a Project

cd /path/to/your-project
loopx connect
loopx status

For a first-time setup, use the guided path:

loopx start-goal --guided --project . --goal-text "Your long-running objective"

The Core Loop — Five Steps

Any custom runner calls these five commands:

loopx quota should-run    # should this registered agent act right now?
loopx todo claim          # claim the current work slice
loopx todo update         # record what changed
loopx refresh-state       # prepare context for the next turn
loopx quota spend-slot    # account for a completed, validated slice

Integration with Nine Agent Harnesses

Agent Harness	Recommended entry
Codex App	Ask the agent to run `loopx connect`, then use `$loopx <task>` or `/skills`
Codex App (SSH)	`loopx agent-onboard --agent-type codex-app-ssh`
Codex CLI	Connect inline, then `$loopx <task>` or `/skills`
Claude Code	Install the opt-in adapter, then `/loopx <task>` + `/loop`
KunlunCode	`loopx-kunluncode connect` + native Goal Pro
OpenCode	Install the static command facade; opt into `--with-goal-bridge` for recurring goals
Pi	`loopx slash-commands --install --surface pi`, then `/loopx <task>`
ZCode	`loopx slash-commands --install --surface zcode`
DeepSeek Harness (dsh)	`loopx[deepseek-harness]` + `loopx turn run-once`
Cursor / shell / custom	Manual `loopx connect` + call LoopX from your runner

All integrations share the same control-plane state. Switching harnesses doesn't lose Goal state.

Capability Paths

LoopX wraps generic control-plane primitives into capability paths with concrete owned outcomes:

You need to...	Capability	Command
Turn a public issue into a reviewable, evidence-backed change	Issue Fix	`loopx issue-fix`
Qualify the exact final diff before delivery	Change Quality	`loopx capability show change-quality-qualification`
Explore uncertain research without losing hypotheses and findings	Explore	`loopx capability show explore`
Rebase decisions on current evidence and verified outcomes	Decision Context	`loopx capability show decision-context`
Produce scheduled or triggered reports with receipts	Periodic Report	`loopx capability show periodic-report`

List all installed capabilities:

loopx capability list --format json

Auto Research: Parallel Multi-Agent Work

LoopX ships a built-in KNN demo that shows proposer / executor / evaluator/promoter agents working in parallel:

# Browse available presets
loopx preset list
loopx preset show daily-triage

# See the Auto Research demo path
# demo/auto_research/README.md

Evidence of Long-Running Work

LoopX is explicit: these are not one-turn demos.

OpenViking open-source contribution arc (200+ elapsed hours)

The author manages their own contributions to the OpenViking repository using LoopX, spanning 200+ hours of elapsed project time across many bounded turns. The Issue Fix capability maintains rolling repository context, revision-stamped fix knowledge, and reviewer-facing preferences across that window.

Auto ML Experiment (200+ elapsed hours)

A redacted owner-run showcase: in a 200+ hour elapsed window, hypotheses, matched evidence, invalid lineages, running replicates, and promote/stop gates stay visible in one graph.

Independent user cases from the community:

One user: 13h+ C++ precision improvement task, stayed aligned across multiple stages, triggered public research, improved final precision
One user: 4-day unattended run, ongoing useful work, periodic report surface
One user: 7 merged PRs, Engine refactor, reported 1B+ token scale

Dashboard and Observability

# Launch local dashboard (PWA mode)
loopx dashboard

Or build the experimental Tauri desktop window:

cd apps/desktop/loopx-control-plane
npm install
npm run dev

Session dash for tracking multi-goal fleet progress:

loopx dash

Daily state inspection:

loopx status
loopx history --goal-id your-project-goal
loopx quota should-run --goal-id your-project-goal
loopx diagnose
loopx review-packet      # compact owner-facing view: decisions, evidence, gates

Resources

🌟 GitHub: huangruiteng/loopx
📦 PyPI: loopx
🌐 Website: huangruiteng.github.io/loopx
📖 User manual (Feishu): CaL5wMk9ui17ngkWzeUcMlAYnZg
📚 Developer Book: Chinese · English
💬 Discord: discord.gg/XmGgQyCFZd

Summary

LoopX represents a clear-eyed engineering judgment: the agent problem isn't just "can it do this in one turn" — it's "after multiple days, multiple runs, and multiple harnesses, can you still tell where the work is, what changed, and who does what next?"

Three things worth noting:

"Control plane" vs. "agent framework" is a deliberate positioning. LoopX doesn't execute work — it manages state. This is the same design thinking as Kubernetes being an "orchestration plane" rather than a "container runtime." It means you can swap agents (Codex today, Claude Code tomorrow) without losing Goal state or evidence history.

The Five Questions framework is an actionable design. Many systems claim to "support long-horizon agents," but "long-horizon" stays vague. LoopX breaks it down into five concrete questions: objective / next step / human judgment gates / evidence changes / whether to continue. Each maps to a piece of code, a CLI command, and a verifiable contract.

What it doesn't do is also a design choice. LoopX explicitly states: it's not an autonomous production controller, it doesn't grant credentials, it doesn't approve destructive operations, it won't mark an unverified run as successful. At a time when agent capability boundaries are still unclear, this kind of explicit self-limitation is engineering honesty.

If you have multi-day AI agent work — machine learning experiments, large code refactors, sustained open-source contributions — LoopX offers the most complete open-source control-plane solution currently available.

Explore PrimeSkills — a curated marketplace of AI agents and skills, each validated against real enterprise workflows. No hype, just what actually works.

Visit my personal site for more insights and interesting products.

Code Agent Anatomy (16): AgentTeams — Why Is One Agent Not Enough?

WonderLab — Tue, 01 Sep 2026 01:54:28 +0000

Starting with a Real Dilemma

Imagine you give a code agent this task:

"Help me refactor this 5000-line Python project entirely into async style, fill in the unit tests, then run a CI verification."

What happens when a single agent receives this?

It will modify files one by one, then go write tests, then run CI after the tests are done. The whole process runs serially, with frequent file reads, file writes, and model calls along the way — just iterating over all the files might consume tens of thousands of tokens, filling the context window to the brim.

What's more problematic: refactoring files and writing tests are two things that could actually happen simultaneously. But a single agent can't parallelize — it has only one ReAct loop and can only do one thing at a time.

This is the problem AgentTeams is trying to solve.

Conclusions First

AgentTeams is an experimental multi-agent collaboration system in MyCodeAgent. Its core design philosophy is:

Problem	AgentTeams' Solution
Task too long, single agent context can't hold it	Break the task across multiple agents, each agent only sees its own portion of context
Need to process multiple subtasks in parallel	Multiple agents run simultaneously, independent of each other
Results need to be aggregated	Coordinator agent collects output from each sub-agent and merges into a final answer

It provides six tools: TeamCreate, SendMessage, TeamStatus, TeamDelete, TeamFanout, TeamCollect, and supports three execution modes: in-process (in-process), tmux (dedicated terminal), auto (auto-select).

But this system was ultimately removed from the stable release. Understanding why it was designed and why it was removed is more valuable than simply learning "how to use it."

1. What Makes Multi-Agent Systems Really Hard

Many people, hearing "multi-agent" for the first time, think it sounds simple: isn't it just running a few more model instances?

In reality, once you have multiple agents, you immediately face a series of problems that simply don't exist in single-agent systems:

Problem 1: How do you provide context?

Each agent has its own context window. The primary agent can't just copy the entire conversation history for each sub-agent — that would cause context explosion. So you need to decide: how much context does each sub-agent get? Which parts? In what format?

Problem 2: How do you merge results?

Sub-agent A modified utils.py, sub-agent B also modified utils.py — they're unaware of each other's existence. How do you merge after both are done? Who decides which version is more accurate?

Problem 3: How do you handle failures?

A single agent failure means the current task fails — just retry. But if 10 sub-agents are running in parallel and 3 fail, what then? Re-run all of them? Only re-run the failed ones? Do the failed parts affect the results of other agents?

Problem 4: Who coordinates?

If a primary agent is responsible for coordinating other agents, the primary agent itself becomes a complex state machine — it needs to know which subtasks are complete, which are still running, which failed, and needs to merge all results at the right moment. This coordination logic is itself enormously complex.

AgentTeams' design is an attempt to answer these four questions.

2. The Basic Model of AgentTeams

AgentTeams abstracts multi-agent collaboration into the concept of a Team.

Primary agent (coordinator)
    │
    ├── TeamCreate: create a team
    │
    ├── TeamFanout: distribute tasks to multiple members
    │       │
    │       ├── Member agent 1 ← runs independently, has its own context
    │       ├── Member agent 2 ← runs independently, has its own context
    │       └── Member agent 3 ← runs independently, has its own context
    │
    ├── TeamCollect: wait and collect all members' results
    │
    └── TeamDelete: clean up the team after task completion

Each team has a coordinator (usually the primary agent that created the team) and several members (agents executing subtasks). Members are mutually independent and don't communicate directly — all messages are relayed through the coordinator.

This design solves the "how to merge results" problem: the coordinator is the only role with a global view, and it decides how to merge.

Member agent execution modes:

in-process: runs in the same Python process, lightweight, but can't truly parallelize (limited by the GIL)
tmux: runs in a dedicated tmux pane, truly parallel, but with higher startup overhead
auto: auto-select based on environment

3. A Concrete Use Case

Using the earlier "refactor + test" example, the AgentTeams workflow looks roughly like this:

Primary agent:
1. Call TeamCreate, create a team
2. Scan the project, divide 50 Python files into 5 groups of 10 each
3. Call TeamFanout, distribute 5 "refactor these 10 files" tasks to 5 member agents
4. Wait, while also starting to write the integration test framework code
5. Call TeamCollect, collect the refactoring results from all 5 members
6. Merge results, resolve conflicts
7. Run CI verification
8. Call TeamDelete, clean up the team

5 member agents work simultaneously, each only handling 10 files, so the context window isn't filled by the entire project. The primary agent can do other work while waiting.

This flow is logically sound. The problems arise at the implementation level — which is what article 19 will cover.

4. The Difference Between Sub-Agents and Team Members

Having read this far, you might ask: how is this different from the Task sub-agent discussed in article 08?

This is an excellent question, and the answer is critical:

Task sub-agent (stable feature):

The primary agent delegates a specific task to a sub-agent
The sub-agent completes it and returns the result to the primary agent
The entire process is synchronous: the primary agent waits while the sub-agent finishes, then continues
Suited for "I'm not good at this, let an expert handle it" scenarios

AgentTeams members (experimental feature):

The primary agent distributes multiple instances of the same type of task to multiple members
Members run in parallel, the primary agent doesn't wait for them — it can continue doing other things
TeamCollect eventually collects all results together
Suited for "the same thing needs to be done many times, and can be done in parallel" scenarios

Simply put: Task is "you go do it, I'll wait"; AgentTeams is "you all go do it separately, I'll do something else first, tell me when you're done."

5. Why This Direction Matters

AgentTeams was ultimately removed, but the direction it represents — parallel multi-agent collaboration — is a real and important need in the AI agent field.

Currently almost all mature agent frameworks (LangGraph, AutoGen, CrewAI, etc.) are trying to solve similar problems, but with different approaches:

Some use graphs (DAGs) to describe dependencies between agents
Some use role-playing to let different agents play different roles
Some use shared blackboards to let agents communicate through a central state

AgentTeams chose the team + coordinator model, which is the design most closely resembling human team collaboration.

Having understood the design motivation, the next article looks at how messages are passed between agents — these are the blood vessels of the entire collaboration system.

Design Highlights

1. Context Isolation Is the First Principle

Each member agent has its own context and won't be "distracted" by seeing too much irrelevant information. This solves the context pollution problem when a single agent handles long tasks.

2. Coordinator Has the Only Global View

Members don't communicate directly; all coordination goes through the primary agent. This simplifies state management — you don't need to handle the complex situation of agent A and agent B messaging each other and waiting on each other.

3. Three Execution Modes for Three Scenarios

in-process is lightweight and fast, suited for development and testing; tmux is truly parallel, suited for production use; auto lets the framework decide, lowering the barrier to use.

Summary

Question	AgentTeams' Answer
Why do we need multiple agents?	Single agent context is limited and can't parallelize
What is the core abstraction?	Team = coordinator + multiple members
How do members run?	Three modes: in-process / tmux / auto
Difference from Task sub-agent?	Task is synchronous delegation, AgentTeams is parallel distribution
Final fate?	Experimental, removed from stable release (reason in article 19)

About the Source Code for This Series

All analysis in this series is based on the open-source project MyCodeAgent.

The AgentTeams implementation code has been removed from the stable release and is preserved in Git history (commit f497b172). You can view design traces via: docs/research-archive.md, docs/plans/2026-07-12-lean-runtime/tasks/M2-03-remove-agent-teams.md.

git clone https://github.com/chendongqi/MyCodeAgent
cd MyCodeAgent
cp .env.example .env   # Fill in your LLM API key
uv sync
uv run python main.py

Visit PrimeSkills — a curated AI Agent and skills marketplace where every piece of content is validated against real enterprise workflows. No hype, only things that actually work.

For more practical insights and interesting products, visit my homepage

Code Agent Anatomy (15): Harness Design Part 5 — Observability

WonderLab — Mon, 31 Aug 2026 07:30:59 +0000

The Last Piece of the Puzzle

The previous four articles dissected the agent's control flow, context engineering, tool pipeline, and fault recovery. All of these mechanisms share a common problem: when something goes wrong, how do you know what happened?

An agent running in the background — which tool the model chose, how many times history was compacted, which permission rule blocked a call — these all happen inside the code but are invisible by default. This article dissects MyCodeAgent's observability system, examining how it turns internal behavior into queryable records.

Conclusions First

The observability system consists of three layers:

Layer	Component	Responsibility
Emission layer	`RuntimeRunner._emit()`	Unified event entry point — all observable facts in the loop originate here
Routing layer	`CompositeRuntimeEventSink`	Delivers one event to multiple sinks simultaneously, mutually isolated
Persistence layer	`TraceRuntimeEventSink` + `TranscriptRuntimeEventSink`	Write to JSONL diagnostic logs and append-only fact transcript respectively

One event is emitted, two paths are taken simultaneously — diagnostics and recovery each get what they need.

1. Unified Emission Point: All Events Originate from One Place

Every "thing worth recording" in the loop is emitted through the same method:

# runtime/loop.py  RuntimeRunner
def _emit(self, event_type: str, payload: dict[str, Any], *, step: int) -> None:
    self._emit_runtime_event(
        run_id=self._get_transcript_run_id(),
        step=step,
        event_type=event_type,
        payload=payload,
    )

_emit accepts three elements: event type, payload, step number — constructing a RuntimeEvent object and handing it to the sink router.

Call sites are scattered throughout the loop, but there's only one entry point. This means: adding a new sink (for example, sending events to Prometheus) only requires modifying the CompositeRuntimeEventSink constructor — the loop itself doesn't need to change.

Emission points cover all key moments in the loop:

message          → message written to history (user/assistant/tool roles)
state_transition → state transitions (USER_INPUT / TOOLS_EXECUTED / MODEL_RECOVERY_RETRY ...)
tool_lifecycle   → tool four phases (requested / started / completed / failed)
checkpoint       → context compaction checkpoint
terminal         → loop termination (completed / max_steps / token_budget ...)
prompt_assembly  → fingerprints of each prompt layer per step
tool_schema      → hash of tool schema (detects tool list changes)
model_output     → raw model response (including token usage)

2. Dual Sink: Diagnostics and Persistence Each Serve Their Own Purpose

# runtime/events.py
def create_runtime_event_sink(trace_logger, recorder) -> CompositeRuntimeEventSink:
    return CompositeRuntimeEventSink(
        (TraceRuntimeEventSink(trace_logger), TranscriptRuntimeEventSink(recorder))
    )

CompositeRuntimeEventSink delivers the same event to two sinks:

def emit(self, event: RuntimeEvent) -> None:
    for sink in self.sinks:
        try:
            sink.emit(event)
        except Exception as error:
            # Sink failure only logs a warning, does not affect loop state
            logger.warning("Runtime event sink failed for %s: %s", event.type, error)

Isolation is key: a Transcript sink write failure doesn't affect the Trace sink, nor does it affect the loop continuing to run. Failures in observability infrastructure must not stop the agent.

Trace Sink: Diagnostic Logs

TraceRuntimeEventSink translates events into diagnostic format and forwards them to TraceLogger, writing to memory/traces/trace-{session}.jsonl. This is the log for development and debugging:

Each line is a JSON object with timestamp, session_id, step, event name, and payload
tool_lifecycle events are additionally split into tool_call (when called) and tool_result (when completed), allowing HTML reports to display them separately
After the file is written, an HTML report can be generated for direct human reading

Transcript Sink: Recoverable Facts

TranscriptRuntimeEventSink only handles five types of "recoverable fact" events (message/state_transition/tool_lifecycle/checkpoint/terminal), writing to memory/transcripts/transcript-{session}.jsonl. This is the log for crash recovery (see article 14) — it doesn't record debug details, only the minimal set of facts needed to reconstruct state.

The same event is emitted, two paths each take what they need: Trace takes all details, Transcript takes only the facts needed for state recovery.

3. TraceLogger: JSONL Streaming Write

# extensions/tracing/logger.py
def log_event(self, event: str, payload: dict[str, Any], step: int = 0) -> None:
    event_obj = {
        "ts": _utc_now().isoformat(),
        "session_id": self.session_id,
        "step": step,
        "event": event,
        "payload": self._sanitizer.sanitize(payload),  # sanitize first
    }
    self._current_run_events.append(event_obj)  # keep in memory for this run
    self._write_line(event_obj)                  # write to disk immediately
    self._update_stats(event, payload, step)     # update statistics

def _write_line(self, event_obj: dict[str, Any]) -> None:
    with self._lock:                             # write lock to prevent concurrent corruption
        if self._file_handle:
            self._file_handle.write(json.dumps(event_obj, ensure_ascii=False) + "\n")
            self._file_handle.flush()            # flush immediately on every write

Every log_event flushes immediately — same strategy as Transcript, aimed at shrinking the crash window: if the process crashes between two events, already-flushed events won't be lost.

_current_run_events keeps all events from the current run in memory, consumed when generating the HTML report (at finalize()).

4. Prompt Fingerprint: Detecting Prompt Drift

At the start of each step, trace_model_request_state() computes and emits fingerprints for each layer of the prompt:

# runtime/events.py  trace_model_request_state()
current = {
    "constitution": prompt_assembly.constitution_fingerprint,
    "tool_contracts": prompt_assembly.tool_contracts_fingerprint,
    "project_rules": prompt_assembly.project_rules_fingerprint,
    "runtime_signals": prompt_assembly.runtime_signals_fingerprint,
}
emit("prompt_assembly", {
    ...
    "changed_layers": [key for key, value in current.items()
                       if previous.get(key) not in (None, value)],
})

changed_layers lists the layers that changed in this step compared to the previous step. For example, if Skills were updated, the tool_contracts layer fingerprint changes, and the trace will show "changed_layers": ["tool_contracts"].

This solves a debugging pain point: model behavior suddenly changes at a certain step, but it's unclear why. By comparing fingerprints, you can precisely identify which prompt layer drifted.

The same treatment applies to tool schemas:

fingerprint = hashlib.sha256(
    json.dumps(tools_schema, ...).encode()
).hexdigest()
emit("tool_schema", {"fingerprint": fingerprint, "changed": previous != fingerprint})

5. TraceSanitizer: Sanitization Makes Logs Shareable

Trace logs may contain sensitive data. TraceSanitizer performs a scan before each event is written to disk:

# extensions/tracing/sanitizer.py
SENSITIVE_KEYS = {
    "api_key", "token", "password", "authorization", "session_id", ...
}
PATTERNS = [
    (re.compile(r"sk-[a-zA-Z0-9]{20,}"), "sk-***"),      # OpenAI key
    (re.compile(r"Bearer\s+[a-zA-Z0-9._+/=-]{20,}"), "Bearer ***"),
]

def _sanitize_dict(self, data: Dict[str, Any]) -> Dict[str, Any]:
    for key, value in data.items():
        if key.lower() in self.SENSITIVE_KEYS:
            result[key] = "***"     # key hit: replace value directly
            continue
        if "path" in key.lower() and isinstance(value, str):
            result[key] = self._sanitize_string(value)  # sanitize usernames in paths
            continue
        result[key] = self.sanitize(value)              # recursively handle nested structures

Two types of coverage:

Key blacklist: any key matching api_key, token, etc. gets its value replaced with *** regardless of the value
Value regex: patterns matching sk-..., Bearer ..., etc. cover cases where the key name is innocuous but the value is sensitive

Usernames in paths (/home/alice/, /Users/alice/) are also replaced with ***, preventing developers from accidentally leaking their local username.

TRACE_SANITIZE=false disables this; it's enabled by default.

Design Highlights

1. Event-Driven, Not Log Instrumentation

Observable points in the loop emit semantic events (state_transition, tool_lifecycle) rather than print or logger.info. Events have structured payloads that downstream can process with code; logs can only be read by human eyes.

2. Sink Failure Doesn't Affect the Loop

CompositeRuntimeEventSink calls each sink inside try/except; failures only log a warning. Observability infrastructure is a "side channel" for the agent, not the main channel. Issues with the main channel are what matter — side channel failures shouldn't bring down the main channel.

3. Two JSONL Files, Two Perspectives

The Trace JSONL is the diagnostic perspective: what happened, at which step, when — aimed at developer debugging. The Transcript JSONL is the recovery perspective: which facts need to be retained to reconstruct state — aimed at crash recovery. The same event stream, two purposes, written to disk separately.

4. Fingerprint Rather Than Diff

Prompt change detection uses fingerprints (sha256) rather than storing full text for diffing: saves storage, comparison is O(1), and in the trace you just look at changed_layers to know which layer changed — no need to open two files and compare manually.

Summary

Design Choice	Approach	Engineering Value
Emission point	Unified `_emit()`, not scattered	Adding a sink doesn't require modifying the loop
Routing	CompositeRuntimeEventSink	Multiple sinks mutually isolated; single failure doesn't affect others
Diagnostic log	TraceLogger JSONL + flush	Streaming write, small crash window, can generate HTML report
Recovery log	TranscriptRuntimeEventSink	Writes only minimal fact set, not mixed with debug details
Drift detection	Prompt fingerprint	O(1) comparison, precisely locates which prompt layer changed
Sanitization	TraceSanitizer key blacklist + value regex	Logs are shareable without fear of leaking API keys

This completes all of Part 4 Harness Engineering. A recap of these five articles:

11: Control flow — single loop, immutable state machine, completion gate, exhaustive termination paths
12: Context engineering — History and ModelView separation, read-time projection, dual-trigger compaction
13: Tool pipeline — concurrent grouping, four-checkpoint execution, two-layer byte budget
14: Fault recovery — classified retry, Transcript fact log, UncertainAction explicit modeling
15: Observability — event-driven, dual-sink routing, fingerprint drift detection

These five mechanisms together form the "skeleton" of an agent framework — determining how long it can run, how much it can self-heal after errors, how much it can recover after a crash, and how deep you can investigate when something goes wrong.

About the Source Code for This Series

All analysis in this series is based on the open-source project MyCodeAgent.

The source code has been annotated at key locations following the order in which topics are covered in this series — you can read the articles alongside the code, or clone the repo and run, modify, and extend it to build your own agent.

git clone https://github.com/chendongqi/MyCodeAgent
cd MyCodeAgent
cp .env.example .env   # Fill in your LLM API key
uv sync
uv run python main.py

Visit PrimeSkills — a curated AI Agent and skills marketplace where every piece of content is validated against real enterprise workflows. No hype, only things that actually work.

For more practical insights and interesting products, visit my homepage

Open Source Project #167: Cumora — Team Chat Where AI Agents Are First-Class Teammates

WonderLab — Mon, 31 Aug 2026 07:29:47 +0000

Introduction

"Where agent teams gather."

This is article #167 in the "One Open Source Project a Day" series. Today's project is Cumora — a cross-platform collaboration tool where AI agents and humans share the same chat room, 3,248 Stars, MIT license, authored by yetone.

Cumora addresses a real gap: most AI agent tools are either question-answer tools ("you ask, it answers") or fully autonomous agents running in isolation — but real team work requires humans and agents to share context, know what each other is doing, and coordinate. Cumora's answer: treat agents as genuine team members with the same standing as humans. Same chat history, same Kanban board, same calendar. Agents have their own memory, personas, real email addresses, can claim tasks proactively, and coordinate with other agents without stepping on each other.

What You'll Learn

Two agent runtime modes: the difference between Cumora Cloud and BYOA (Bring Your Own Agent)
Local engine options for BYOA: Claude Code / Codex / OpenCode and more
Three-layer anti-collision coordination for multi-agent rooms
Technical architecture: React + Express + Postgres + Redis + Kubernetes
Local development setup

Prerequisites

Familiarity with LLMs and AI agents at a basic level
Basic Node.js/TypeScript development experience
Some background with Electron or cross-platform desktop apps is helpful

Project Background

Overview

Cumora's core premise: agents are not tools, they're teammates. So it's not a "plug in an AI assistant" product — its agents share the same first-class status as humans: same group chats, same DMs, same Kanban cards, same calendar. Each agent has a name, avatar, memory, and will speak up unprompted, claim tasks, and even has a real personal email address.

Author / Team

Author: yetone
Primary language: TypeScript
License: MIT
Website: cumora.ai
Web App: app.cumora.ai
iOS beta: TestFlight

Project Stats

⭐ GitHub Stars: 3,248+
🍴 Forks: 397+
📄 License: MIT
📅 Created: 2026-08-17

Two Agent Runtime Modes

Cumora Cloud (Managed)

Each agent runs in its own Kubernetes pod, with the brain being a multi-hop tool-calling loop on the OpenAI Responses API. Available tools include bash commands, file operations, browser, email, memory, and skills.

Advantage: zero setup, always online. Drawback: uses Cumora's OpenAI quota; you can't bring your own Claude Code or local agent.

BYOA — Bring Your Own Agent

Run the agent's brain on your own machine (laptop or VPS):

# Install and run the agent daemon
npx cumora agent computer

Supported local engines for BYOA:

Command	Agent Engine
`claude`	Claude Code (Anthropic)
`codex`	OpenAI Codex CLI
`grok`	Grok Build (xAI)
`cursor-agent`	Cursor Agent
`opencode`	OpenCode
`pi`	Mario Zechner's Pi
`gemini`	Gemini CLI

Key design: the I/O surface (the cumora CLI protocol) is fully decoupled from the brain. Commands like cumora reply, cumora dm, cumora memory, cumora workspace, cumora card are thin shims that POST their argv to /runtime/cli. The transport layer (Server-Sent Events + REST) is engine-agnostic. BYOA only swaps the brain and the host — it reuses everything else.

Security: the server never holds the user's provider API keys. Your Claude Code / Codex credentials stay on your machine.

A single daemon can host multiple independent agents, each with their own isolated home directory, memory, skills, and notes.

Computer: A Unified Mental Model

Cumora introduces "Computer" as a first-class concept — whether cloud or local, an agent always runs on some Computer:

Computers
──────────────────────────────
☁  Cumora Cloud      ● online
   engine: managed · 4 agents

💻 MacBook Pro        ● online
   Claude Code · 3 agents
   "Iris is thinking…"

🖥  prod-vps-01        ○ offline
   Codex · 2 agents

Creating an agent means "pick which Computer it lives on." If a Computer goes offline, its agents show as sleeping rather than broken. There's no special "BYOA agent" type — just agents on different Computers.

Multi-Agent Coordination: Three Defense Layers Against Collisions

This is the deepest engineering in Cumora. When multiple agents share a chat room, the problem is simple: they can all wake up simultaneously, read the same messages, and each decide to respond — so the same thing gets done twice.

There are two failure modes:

Race collisions: two agents simultaneously INSERT a message, both posting "3" in a counting game
Brain misjudgment: the agent's view is correct (it sees the latest messages) but the model still makes the wrong decision

These require different fixes: code mechanisms for collisions, prompt engineering for misjudgments. Never use a prompt to fix a race condition, and never add a code mechanism when the model is making a clear decision in front of correct state.

Defense Layer 1: Freshness Gate

Before an agent can submit a reply, the server checks: "Is the last message you saw actually the latest?"

Agent reads messages → decides to reply → submits reply
                                               ↓
                                     Server checks:
                              seen_cursor >= latest_msg_id?
                                    yes → allow
                                    no  → HOLD (push newer messages to agent to re-decide)

A HELD reply isn't discarded — the agent sees the newer messages and re-decides whether it still needs to respond.

Defense Layer 2: Atomic Task Claiming

Kanban card claiming is an atomic operation. Two agents cannot "simultaneously claim" the same card — the server uses a database-level lock to ensure only one agent can successfully claim a task. Others receive "already claimed" and do not retry.

Defense Layer 3: Small-Brain Triage Gate

When an agent is woken up, a lightweight cheap model first decides: "Does this message actually need a response from me?"

New message arrives
    ↓
Small brain (cheap model): is this directed at me?
    yes → wake the big brain (big model) for full processing
    no  → ignore, no big model tokens consumed

This reduces unnecessary token usage and lowers the probability of multiple agents waking simultaneously and making conflicting decisions.

CI includes a dedicated guard: npm run guard:big-brain — verifies that only agent turns may call the big model, catching accidental usage.

Technical Architecture

 Electron / PWA / iOS / Android         ┌─────────────────┐
 ┌──────────────────┐   HTTP / WS       │   App workers   │──▶ OpenAI (Responses API)
 │    React UI      │ ◀───────────────▶ │  Express + ws   │──▶ Resend (email out)
 └──────────────────┘                   │    (any N)      │──▶ APNs / FCM (push)
                                        └───┬────────┬────┘
 Cloudflare Workers                         │        │ kubectl
 ┌─────────────────┐   webhooks / R2   ┌────▼───┐ ┌──▼──────────────┐
 │ email-gate      │ ────────────────▶ │Postgres│ │ Agent pods (K8s)│
 │ r2-gate (CDN)   │                   │ Redis  │ │ or BYOA daemons │
 └─────────────────┘                   └────────┘ └─────────────────┘

Layer	Tech	Responsibility
Frontend	React 18 + Vite + TypeScript + Tailwind	Pure UI; desktop/mobile/web/admin share the same component tree
Backend	Express + ws + Drizzle ORM	Stateless Node service, horizontally scalable
Data	Postgres (source of truth) + Redis (pub/sub fan-out + presence)	Multiple instances stay in sync via Redis bus
Agent runtime	Kubernetes pods (cloud) / BYOA daemon (local)	Two paths, unified `cumora` CLI protocol
Workers	Cloudflare Workers	Inbound/outbound email gateway, signed CDN

Repository layout:

Path	What it is
`src/`	React renderer (desktop/mobile/web/admin)
`server/`	API + WebSocket + agent runtime
`electron/`	Desktop shell (auto-update)
`ios/`, `android/`	Capacitor native shells
`agent-cli/`	npm package `cumora` — the BYOA daemon
`agent-fuse/`	Go FUSE driver mounting cloud agent workspaces
`workers/`	Cloudflare Workers (email gate, CDN)
`benchmarks/`	Multi-agent coordination benchmarks (chain/counting/werewolf/kanban)

Local Development

# Prerequisites: local Postgres and Redis
createdb -h localhost cumora
export OPENAI_API_KEY=sk-...

npm run setup          # install dependencies
npm run dev:all        # Vite renderer on :5180 + API server on :5181

Open http://localhost:5180 (PWA mode) or run npm run electron:dev for the desktop window.

The schema is created idempotently on boot. An empty database seeds a starter team (6 agents, 3 humans, 9 conversations) with zero messages — everything that appears in chat is produced live.

OPENAI_API_KEY is the only hard requirement; everything else has a sane local default:

DATABASE_URL  # default: postgres://$USER@localhost:5432/cumora
REDIS_URL     # default: redis://localhost:6379
PORT          # default: 5181

Tests

npm test                   # unit tests (node:test) for server + workers
npm run test:integration   # integration suite (needs local Postgres/Redis)
npm run typecheck && npm run server:typecheck
npm run guard:big-brain    # CI guard: only agent turns may use the big model

Resources

🌟 GitHub: yetone/cumora
🌐 Website: cumora.ai
🖥️ Web App: app.cumora.ai
📱 iOS beta: TestFlight
📥 Desktop download: cumora-releases
📖 BYOA docs: docs/BYOA.md
📖 Coordination design: docs/COORDINATION.md

Summary

Cumora represents a specific judgment: the next frontier for AI agents isn't stronger solo agents — it's genuine collaboration between agents and humans.

Three things worth noting:

"First-class citizen" is an engineering commitment, not just a product claim. Agents and humans share the same data model (kind='agent' vs kind='user', both in the participants table), the same messaging system, and the same Kanban board. Whatever an agent can do, a human can too — and vice versa. This is not "AI as a plugin."

Multi-agent coordination engineering is far harder than it looks. COORDINATION.md documents real lessons learned the hard way: using prompts to fix race conditions (wrong), using code mechanisms to replace model judgment (wrong), silent model version upgrades silently breaking coordination behavior (a real production incident). This document is a rare and valuable record of multi-agent systems engineering experience.

The BYOA security model is worth studying. The server never holds provider API keys. The I/O interface is decoupled from the brain, which means any agent engine that supports CLI interaction can in principle be plugged in. This is an extensible design rather than hardcoded integrations for a handful of specific agents.

If you're building workflows that require human-agent collaboration, or studying the engineering behind multi-agent coordination, Cumora's codebase and documentation are both worth your time.

Explore PrimeSkills — a curated marketplace of AI agents and skills, each validated against real enterprise workflows. No hype, just what actually works.

Visit my personal site for more insights and interesting products.

Code Agent Anatomy (14): Harness Design Part 4 — Fault Recovery

WonderLab — Sun, 30 Aug 2026 10:38:52 +0000

Errors Are the Norm, Not the Exception

For a long-running agent, errors are inevitable: the model occasionally returns empty content, context window exceeded, network jitter, a process crashes mid-tool-execution...

The question isn't "how to avoid errors" — it's what the agent should do after an error occurs. This article dissects MyCodeAgent's fault recovery system: runtime tiered retries, the persistent fact log, and state recovery after a crash.

Conclusions First

The fault recovery system has two dimensions:

Dimension	Mechanism	Problem Solved
Runtime self-healing	Error classification + tiered retry	Don't give up immediately on model errors; try automatic recovery first
Crash recovery	Transcript fact log + ResumeLoader	Resume from the interruption point after a process crash, no need to start over

The two dimensions work together: runtime self-healing handles predictable errors, while Transcript ensures that even if self-healing fails and the process crashes, no completed work is lost.

1. Error Classification: Understand the Error Type First

# runtime/model_errors.py
class ModelErrorKind(str, Enum):
    EMPTY_RESPONSE    = "empty_response"    # Model returned empty content — no text and no tool_call
    PROMPT_TOO_LONG   = "prompt_too_long"   # Context limit exceeded, model refused to process
    MAX_OUTPUT        = "max_output"        # Model output truncated (finish_reason="length")
    API_ERROR         = "api_error"         # RuntimeError, usually an API layer issue
    UNKNOWN_MODEL_ERROR = "unknown_model_error"

classify_model_error() accepts an exception object or response metadata and outputs a classification result with a recoverable flag. Classification is the prerequisite for retry decisions — different errors have completely different recovery strategies, and blanket retries only waste quota.

def _looks_like_prompt_too_long(message: str) -> bool:
    # Uses keyword matching rather than specific exception types
    # Reason: different providers throw different exception classes,
    # but error messages usually contain these phrases
    patterns = ("prompt too long", "context length", "context window",
                "too many tokens", "maximum context length", "request too large")
    return any(pattern in message for pattern in patterns)

String matching rather than exception types — this is an intentional design choice. Different LLM providers throw different exception classes, but the wording in error messages is highly consistent, making keyword matching more universal.

2. Runtime Self-Healing: Three Error Types, Three Recovery Paths

Errors occur inside the inner while True loop; successful recovery triggers continue to retry the current step without consuming the outer step quota.

Path 1: PROMPT_TOO_LONG → Compact and Retry

Model call throws exception → classify → PROMPT_TOO_LONG
  → reactive_compact() (LLM summarizes old history, produces a checkpoint)
  → build_model_view() (read-time projection, gets the compacted message list)
  → inner continue, retry invoke_raw()
  → compaction fails → MODEL_RECOVERY_FAILED → terminate

The retry limit is 1 (hardcoded in _get_model_recovery_limit). Compaction itself can also fail (LLM call timeout, insufficient turns) — if so, it goes directly to the termination path without looping forever.

Path 2: EMPTY_RESPONSE → Inject Hint and Retry

The model returned empty content (response_text is empty and no tool_calls) — usually because the model "hesitated" — unsure whether to continue with a tool or give a final answer.

if classification.kind is ModelErrorKind.EMPTY_RESPONSE and retry_count < retry_limit:
    hint = "Last response had empty content and no tool_calls. Please reply with a final answer in content, or use a tool call."
    # Append the hint to the end of this round's messages — NOT written to history_manager
    # Only affects this retry's model view, does not pollute the permanent history
    messages = base_messages + [{"role": "user", "content": hint}]
    continue  # inner retry

Notice the hint message is not written to history_manager — it's only temporarily appended to the message list for this send. This keeps history clean; the model won't see this "debug hint" on the next step.

The retry limit defaults to 1 (configurable via empty_response_retry_limit).

Path 3: MAX_OUTPUT → No Recovery Currently

finish_reason="length" means the model output was forcibly truncated. In theory this could be recovered with a "continuation" strategy, but the current recovery limit is 0:

if kind is ModelErrorKind.MAX_OUTPUT:
    return int(getattr(host, "max_output_recovery_limit", 0) or 0)
    # Returns 0 → retry_count >= retry_limit immediately → terminate directly

This effectively means no retry — it goes to MODEL_ERROR termination. This is an MVP placeholder: the structure is reserved, but the strategy hasn't been implemented yet.

3. Transcript: Append-Only Fact Log

Runtime self-healing handles "predictable errors." But what if the process crashes outright? In that case, a persistent fact log is needed — one that can reconstruct "where things were before the crash."

# runtime/transcript.py
class TranscriptStore:
    """One JSONL file per session, one event per line, append-only."""

    def append_event(self, event: TranscriptEvent) -> TranscriptEvent:
        with self._lock:
            self._repair_trailing_record()  # First repair any potentially incomplete last line
            with self.path.open("a", encoding="utf-8") as handle:
                handle.write(line)
                handle.write("\n")
                handle.flush()  # Flush immediately on every write to reduce data loss window

Five event types cover all key facts in the loop:

Event Type	Records	Recovery Use
`MESSAGE`	Message role + content + metadata	Reconstruct HistoryManager
`STATE_TRANSITION`	Transition reason + details	Reconstruct last LoopState
`TOOL_LIFECYCLE`	Four-phase tool status	Identify which tools didn't complete
`CHECKPOINT`	Compaction summary + split point	Reconstruct CompactStore
`TERMINAL`	Termination reason	Determine if completed normally

_repair_trailing_record() handles a special case: the process crashes between write and \n, leaving a partial JSON on the last line. Before each write, it checks and truncates this line, ensuring every line in the file is valid JSON:

def _repair_trailing_record(self) -> None:
    if data.endswith(b"\n"):
        return  # Normal ending, no repair needed
    # No newline at end: try parsing the last line
    try:
        json.loads(tail.decode("utf-8"))
        # Parse succeeded: just missing the newline, add it
        with self.path.open("ab") as handle:
            handle.write(b"\n")
    except (UnicodeDecodeError, json.JSONDecodeError):
        # Parse failed: this line is a partial write, truncate it
        handle.truncate(tail_start)

4. ResumeLoader: Rebuilding State from the Event Stream

After a crash and restart, ResumeLoader "replays" the event stream from the JSONL file to reconstruct ResumeState:

# Iterate over all events, handling each by type
for event in events:
    if MESSAGE:    → history_messages.append(...)
    if CHECKPOINT: → checkpoint = payload (passed to CompactStore later)
    if TERMINAL:   → terminal = payload (determine if it completed normally)
    if TOOL_LIFECYCLE: → tool_events[(run_id, tool_call_id)] accumulates phase states

Tool state handling is the most complex part, because a tool call has four phases and a crash can occur between any of them:

requested → started → completed / failed

ResumeLoader makes four judgments for each tool call's set of states:

State Set	Meaning	Recovery Handling
Contains `completed`	Successfully completed	`completed_tool_results`, don't replay
Contains `failed`	Already failed	`failed_tool_results`, don't replay
Only `requested`, no `started`	Requested but not yet started	`pending_tool_calls`, can be re-triggered
Has `started`, but no `completed`/`failed`	Started but result unknown	`uncertain_actions`

5. UncertainAction: Explicit Modeling of Uncertainty

started but no result is the most tricky situation — the tool executed, but we don't know if it succeeded.

UNSAFE_UNCERTAIN_REPLAY_TOOLS = {"Edit", "Bash", "Task"}

uncertain_actions.append(
    UncertainAction(
        tool_name=tool_name,
        tool_call_id=tool_call_id,
        step=step,
        replay_allowed=tool_name not in UNSAFE_UNCERTAIN_REPLAY_TOOLS,
    )
)

replay_allowed is classified by idempotency:

Read/Grep/Glob: idempotent, replay has no side effects, replay_allowed=True
Edit/Bash/Task: have side effects, unknown if the previous run succeeded — blindly replaying might duplicate file modifications or command executions, replay_allowed=False

Uncertainty is not silently handled — it's explicitly exposed to the user: when recovering via CLI, the uncertain actions list is printed, letting the user decide whether to continue. This is "transparent fault recovery" — the framework doesn't pretend to know what happened; instead, it honestly reports.

Design Highlights

1. Classification Before Retry

All recovery paths start with classification; different errors take different strategies. This avoids the most common fault-tolerance anti-pattern: blindly retrying all errors, wasting retry budget on both transient and unrecoverable errors alike.

2. Inner while Loop Isolates Retries

Retries happen via continue in the inner loop; the outer step counter step stays unchanged. The max_steps=50 quota is entirely spent on productive ReAct iterations, not consumed by error recovery.

3. Transcript Append-Only, Never Modified

Consistent with the HistoryManager design principle: write only, never modify historical events. This guarantees the integrity of the event stream — the state reconstructed after a crash and restart is identical to the state before the crash, with no risk of "introducing new problems during repair."

4. Uncertainty Explicitly Modeled

Uncertain actions are not an implementation detail — they're a design concept. It acknowledges that "there are situations the framework cannot automatically recover from." Rather than silently skipping or pretending to recover, it honestly informs the user and lets humans make the judgment.

Summary

Design Choice	Approach	Engineering Value
Error classification	Keyword matching + recoverable flag	Cross-provider compatible, classification before retry
Retry isolation	Inner while + continue	Retries don't consume step quota
Persistence	Append-only JSONL + flush	Small crash window, file always reconstructable
Last-line repair	`_repair_trailing_record`	Handles edge case of crash between write/flush
Uncertainty	UncertainAction + replay_allowed	Side-effect tools not blindly replayed, transparently informs user

About the Source Code for This Series

All analysis in this series is based on the open-source project MyCodeAgent.

git clone https://github.com/chendongqi/MyCodeAgent
cd MyCodeAgent
cp .env.example .env   # Fill in your LLM API key
uv sync
uv run python main.py

Visit PrimeSkills — a curated AI Agent and skills marketplace where every piece of content is validated against real enterprise workflows. No hype, only things that actually work.

For more practical insights and interesting products, visit my homepage

One Open Source Project a Day (No. 166): Needle 2 — A 14 MB On-Device Tool-Calling Model

WonderLab — Sun, 30 Aug 2026 00:35:02 +0000

Introduction

"The 2-bit model you deploy is the model that was trained."

This is the 166th article in the "One Open Source Project a Day" series. Today's project is Needle 2.

On-device deployment of large language models runs into the same three-way constraint every time: parameter count, memory, and latency. To get AI running on a phone or embedded device, you either sacrifice accuracy, speed, or both.

Cactus Compute found a counterintuitive way around this wall: don't build a general model — build only for tool calling.

Needle 2 is a foundation model designed specifically for tool calling, structured extraction, and device control:

45M parameters, CQ2-bit quantized
Single 14 MB binary — no separate model files to manage
Fixed ~28 MB session RAM — memory does not grow with conversation length
Matches FunctionGemma 270M (six times the size) on tool-calling benchmarks

9.2k Stars, Apache 2.0, already deployed in production on the Pebble Index Ring and other wearables.

What You Will Learn

Needle 2's SAN (Simple Attention Network) architecture and its three core design decisions
Why CQ2-bit quantization has an advantage over post-training quantization
How confidence gating enables intelligent local/cloud routing
How to wire up tools with the @needle.tool decorator
The full LoRA fine-tuning → .cact export pipeline

Prerequisites

Familiarity with the concept of Function Calling / Tool Use
Python basics (comfortable with decorators)
A rough understanding of model quantization (know what INT4/INT8 means)

Project Background

What It Is

Needle 2 is not a scaled-down general-purpose chat model. It is a foundation model designed from scratch specifically for tool-calling tasks.

The question it answers is: if a model only needs to do one thing well — "receive a natural-language query → decide which tool to call → generate correct parameters → return a structured result" — how small can it get?

The answer is 14 MB, 28 MB at runtime.

To put those numbers in context: the average mobile app install is larger than this. 28 MB of RAM is zero pressure on any Android phone in 2026. On a Raspberry Pi 5 it decodes at 500+ tok/s.

Author / Team

Company: Cactus Compute, Inc. (San Francisco)
Core members: Henry Ndubuaku, Karen Mosoyan, Jakub Mroz, and several co-founders
Focus: On-device AI inference startup; the Needle series is its flagship product
Contact: founders@cactuscompute.com

Project Stats

⭐ GitHub Stars: 9,200+
🍴 Forks: 592
📄 License: Apache 2.0
🧪 Paper: arxiv.org/abs/2607.18363
🤗 Weights: HuggingFace Cactus-Compute/needle2
📦 Install: pip install cactus-needle

Core Features

What Problem It Solves

Needle 2 provides three capabilities — all variations of the same fundamental transformation: natural language → structured output.

Natural-language query
    ↓
Needle 2 inference engine (14 MB binary, 28 MB RAM)
    ↓
┌──────────────┬──────────────────┬─────────────────┐
│  Tool calls  │ Structured JSON  │  Device actions  │
└──────────────┴──────────────────┴─────────────────┘

Every output includes a confidence score that you can threshold to decide whether to handle the request locally or escalate to a cloud model.

Usage Scenarios

Offline voice assistant on wearables
- Pebble Index Ring already uses Needle in production. Voice commands are parsed into API calls locally — no network required.
IoT and smart home control
- Run local voice control on a Raspberry Pi or ESP32. "Turn off the bedroom lights" → turn_off(room="bedroom") — zero cloud latency.
Document structured extraction
- Extract unstructured text (invoices, tables, reports) into strongly-typed JSON. Schema constraints guarantee the output format is correct.
On-device mobile AI agent
- Run a local agent on Android/iOS that handles simple tasks, and routes complex ones to the cloud based on confidence — saves API costs and reduces latency.
Spatial computing
- Decodes at 400–1,500 tok/s on Meta Quest 3S or Apple Vision Pro — fast enough for real-time spatial computing scenarios.

Quick Start

pip install cactus-needle

Tool calling:

import needle

@needle.tool
def get_weather(city: str) -> dict:
    """Get the current weather for a city.

    Args:
        city: The city name to query.
    """
    return {"city": city, "temp_c": 27, "sky": "clear"}

@needle.tool
def send_message(to: str, body: str) -> dict:
    """Send a message to a contact.

    Args:
        to: Recipient name or number.
        body: Message content.
    """
    return {"sent": True}

agent = needle.Needle(tools=[get_weather, send_message])
result = agent.run("What's the weather like in Lagos right now?")
print(result["results"])  # [{"city": "Lagos", "temp_c": 27, "sky": "clear"}]

Structured extraction:

from typing import Annotated
import needle

schema = {
    "invoice_number": str,
    "total_amount": Annotated[float, needle.Field(gt=0)],
    "line_items": list[{"description": str, "qty": int, "unit_price": float}],
}

text = "Invoice #1042, Total: $384.00. 3x Widget @ $128.00"
data = needle.extract(text, schema)
# {"invoice_number": "1042", "total_amount": 384.0, "line_items": [...]}

Confidence gating:

result = agent.complete("Set an alarm for 8am tomorrow")
if result["confidence"] < 0.7:
    # Not confident enough — escalate to cloud
    response = cloud_llm.complete(result["query"])
else:
    # Handle locally
    execute_tool_calls(result["tool_calls"])

Core Features

1. @needle.tool decorator

Decorate any Python function; the docstring is automatically parsed into a tool schema
needle.Field constraints (range, regex, enum) are encoded into the JSON Schema and enforced at the grammar level during inference

2. Grammar-constrained decoding

Structured output is not a soft request ("please output JSON"). The inference engine only permits grammatically legal characters at each token position
Grammar constraints also allow skipping 98% of the vocabulary projection computation — a major source of speed

3. Confidence score

Every call returns confidence: the minimum of a posterior calibration head score and the decoding probability
"The failure mode is escalation, not wrong execution" — better to hand off to the cloud than to execute incorrectly

4. Tool retrieval

When more than 5 tools are registered, embedding-based retrieval kicks in automatically; only the top 5 most relevant tools per turn enter the context
"An unselected tool is unreachable, not merely unlikely" — eliminates hallucinated tool calls at the root

5. Engram memory system

World knowledge is stored in a hashed n-gram table; retrieval costs zero arithmetic operations
Knowledge and computation are fully decoupled — this is what makes the Hadamard MLP's parameter savings possible

Benchmark Results

Task	FunctionGemma 270M (f16)	LFM2.5 230M (f16)	Needle 2 (CQ2)
Mobile Actions (961 rows)	64.0%	69.1%	63.7%
DroidCall (200 rows)	17.5%	—	17.0%
Seal-Tools in-domain	16.3%	26.9%	32.6%
Seal-Tools out-of-domain	15.6%	17.0%	28.7%
Model size	270M (f16)	230M (f16)	45M (CQ2)
MFLOPs / token	540	460	70

Needle 2 matches or beats the strongest competitors on three out of four benchmarks at one-sixth the parameter count — the hallmark of a purpose-built model.

Deep Dive

SAN Architecture: Three Counterintuitive Design Decisions

Needle 2 uses a Simple Attention Network (SAN) instead of a standard Transformer, built around three design choices.

Decision 1: Hadamard MLP instead of a standard FFN

The FFN layer in a standard Transformer accounts for roughly two-thirds of total parameters. SAN replaces dense projections with a fixed Walsh-Hadamard transform — a linear transformation with no learnable parameters — consuming almost none of the parameter budget.

Standard FFN (parameter-heavy):  x → W1 → ReLU → W2 → output
Hadamard MLP (almost parameter-free):  x → H (fixed transform) → output

This is not a capability trade-off. The "knowledge storage" job is simply moved entirely to the Engram memory system.

Decision 2: Engram memory = hashed n-gram table

World knowledge ("Paris is the capital of France," "syntax of list.append()") lives in a hashed n-gram lookup table. Retrieval at inference time costs zero multiplications and zero parameter consumption.

This separation keeps the "computation part" of the model (Attention + Hadamard) extremely lean without reducing knowledge density.

Decision 3: Multi-lane hyper-connections

A 27-layer × 512-wide network uses multi-lane residual connections for more flexible routing than standard residuals — effectively gaining the routing capacity of a wider model without increasing width.

CQ2-bit: Train for Quantization From Day One

Most model quantization is a post-training operation: train at float16/bfloat16, then quantize to INT4 or INT8. This inevitably loses precision — more so at lower bit widths.

Needle 2's Cactus Quants (CQ2-bit) is optimized for 2-bit quantization from the start of training:

Standard pipeline: FP16 training → post-training quantization → INT4 deploy (precision loss)
CQ2 pipeline: CQ2-aware training → CQ2 deploy (training target = deployment target)

The official framing: "The 2-bit model you deploy is the model that was trained." No train-to-deploy precision gap. Quantization is a design goal, not a compromise.

Three Sources of Inference Speed

Source 1: Weights unpacked in vector registers, never decompressed to memory

Traditional quantized models decompress weights from low precision to high precision before each computation step. CQ2-bit weights can be operated on directly within SIMD registers, eliminating the decompression step entirely.

Source 2: Grammar constraints skip 98% of vocabulary projection

Standard LLM generation evaluates softmax over the entire vocabulary (tens of thousands of tokens) at every step. Needle 2's outputs are structured JSON; at any given position, only a tiny number of characters are grammatically legal. The engine can skip the vast majority of candidates before softmax.

Source 3: Automatic CPU instruction-set detection

A single binary includes optimized paths for multiple instruction sets: SDOT (ARMv8.4), NEON (generic ARM), AVX2 (x86), RISC-V Vector Extension, WASM SIMD — the runtime picks the best path automatically.

LoRA Fine-Tuning: From Data to .cact File

Fine-tuning Needle 2 is far simpler than fine-tuning a general-purpose LLM, because the task scope is narrow and well-defined.

Step 1: Prepare data

{"query": "Remind me about the meeting tomorrow at 3pm", "tools": [{"name": "set_reminder", ...}], "answers": [{"name": "set_reminder", "arguments": {"time": "tomorrow 3pm", "message": "meeting"}}]}
{"query": "What's the weather today?", "tools": [{"name": "set_reminder", ...}], "answers": []}

Important: include roughly 1/8 irrelevant examples ("answers": []) to prevent the model from calling a tool on every single input.

Step 2: Data augmentation (optional)

needle generate-data --augment data.jsonl --num-samples 1000

Step 3: LoRA training

# CPU training
needle finetune data.jsonl --epochs 10 --out adapter.pkl

# GPU acceleration
pip install "cactus-needle[gpu]"
needle finetune data.jsonl --epochs 10 --out adapter.pkl

# Apple Silicon
pip install "cactus-needle[metal]"
needle finetune data.jsonl --epochs 10 --out adapter.pkl

Step 4: Export to .cact

# The LoRA adapter is merged into weights at export time
needle build checkpoints/needle2.pkl --lora adapter.pkl --out tuned.cact

Step 5: Load and use

agent = needle.Needle(tools=[my_tools], weights="tuned.cact")

Post-fine-tuning benchmarks show accuracy improvements of 21–58 percentage points, with fine-tuned Needle 2 surpassing DeepSeek V4 Flash on domain-specific tasks — the core advantage of a specialized model trained on a fixed tool set.

Deployment Speed by Device

Device	Decode Speed	RAM Usage
Raspberry Pi 5	500+ tok/s	~28 MB
Apple Vision Pro	1,500 tok/s	~28 MB
Meta Quest 3S	400–800 tok/s	~28 MB
Samsung A-series phone (<$200)	300–700 tok/s	~28 MB
ESP32-S3 (microcontroller)	Supported (needs external RAM)	~28 MB

The RAM footprint is fixed — it does not grow regardless of conversation length. This is the direct result of the 256-token sliding-window KV cache and is a hard requirement for embedded deployment.

Project Links & Resources

Official Resources

🌟 GitHub: https://github.com/cactus-compute/needle
📚 Website: https://cactuscompute.com/needle
🤗 Weights: huggingface.co/Cactus-Compute/needle2
📄 Paper: arxiv.org/abs/2607.18363
📦 Install: pip install cactus-needle

Related Resources

Cactus Compute Blog — design rationale for Needle 1
SAN Architecture Paper — full architectural details

Summary

Key Takeaways

Specialized beats general (on fixed tasks): 45M parameters focused on tool calling matches or exceeds models six times larger on three benchmarks
CQ2-bit = train-to-deploy: quantization is not a precision trade-off; it was the design target from day one
SAN triple combo: Hadamard MLP saves parameters, Engram memory stores knowledge, multi-lane residuals expand routing capacity
Confidence gating is calibrated, not guessed: it is a reliable probability signal that can drive local/cloud routing decisions
Fixed 28 MB RAM: the 256-token sliding window guarantees memory does not grow with conversation length — a hard requirement for embedded deployment

Who This Is For

On-device AI / edge computing engineers who need reliable tool-calling capability on resource-constrained hardware
IoT and smart home developers building offline voice command parsing with zero cloud dependency
Mobile app developers who want to embed a local agent without relying on cloud APIs
AI researchers interested in SAN architecture, CQ2-bit quantization, and on-device inference optimization

One-Line Verdict

Needle 2 proves one thing: when you narrow the problem scope enough, 14 MB can beat 270 MB. That is not a trick — it is an architectural philosophy.

Check out PrimeSkills — a curated marketplace of AI agents and skills that have been validated in real-world, enterprise-grade workflows. No fluff, just what actually works.

Find more useful knowledge and interesting products on my Homepage

Enterprise Knowledge Base (04): HyperGraphRAG Benchmark — Multi-hop Reasoning with Hypergraph Structure

WonderLab — Sat, 29 Aug 2026 02:43:42 +0000

What This Article Is About

This is the fourth benchmark in the series. The previous three articles tested four frameworks:

QAnything v2 (vector RAG, article 02)
LightRAG 1.5.6 (graph + vector hybrid, article 02)
GraphRAG 3.1.1 (LLM-driven knowledge graph, article 03)
HippoRAG 2.0 (hippocampus-inspired PPR graph diffusion, article 03)

This one tests the last framework: HyperGraphRAG 1.0.6 (NeurIPS 2025).

What HyperGraphRAG Actually Does

Traditional knowledge graph edges are binary: entity A → entity B. This means a relationship like "A, B, and C jointly participate in an event" must be decomposed into multiple binary edges — and the decomposition loses the joint semantic context.

HyperGraphRAG uses hyperedges to address this: a single hyperedge can connect any number of entities at once.

Traditional graph:  A → B, B → C, A → C  (3 separate edges, context fragmented)
Hypergraph:         {A, B, C} connected by a single hyperedge (joint context preserved)

The pipeline from documents to hypergraph:

Documents → LLM extracts entity sets (co-occurring entities per passage, not pairwise)
          → Build hypergraph (each entity set = one hyperedge)
          → BGE vectorizes entities
          → Query: vector recall for seed entities → hypergraph diffusion
                 → extract relevant passages → LLM answer

The theoretical expectation: for "multi-entity joint participation" relationships, hypergraph extraction should surface more signal than binary graph extraction.

Deployment

Installation

HyperGraphRAG is not on PyPI. There is no pip install hypergraphrag, and pip install -e . fails because there is no setup.py or pyproject.toml. The working approach: add the source directory to PYTHONPATH and install the requirements from the README:

pip install -r requirements.txt   # graspologic, nano-vectordb, etc.

# Run with:
PYTHONPATH=/path/to/HyperGraphRAG python run_eval.py

SiliconFlow API: Two Hidden Constraints

Constraint 1: Content filter blocks environment variable patterns (HTTP 400, code 20015)

The test documents contain many .env example snippets like:

EMBEDDING_MODEL=jina-embeddings-v4
EMBEDDING_BINDING=jina
EMBEDDING_ASYMMETRIC=true

The ALLCAPS=value format triggers SiliconFlow's content safety filter. Fix: regex-strip any lines matching sensitive patterns before sending embedding requests.

Constraint 2: bge-large-en-v1.5 has a ~512-token input limit

Testing revealed that inputs over roughly 1000 characters return 400. Dense markdown tables (every | and % consumes a token) can exceed 512 tokens at just 900 characters.

Final setting: truncate each text to 700 characters (word-boundary truncation), giving a buffer for high-density content.

Network Stability During Long Runs

GLM-4-flash and SiliconFlow both produced intermittent ConnectTimeout and HTTP 500 errors during the 7-hour indexing run. Two necessary changes:

Set LLM client timeout to 180s, with 6 retry attempts (10/20/30/40/50/60s backoff)
Switch from one-shot rag.insert(all_docs) to per-document insert with a progress file: each successfully indexed document is immediately checkpointed; a crash and restart continues from where it left off

Without both changes, finishing a 443-minute index build in a single run is unrealistic.

Benchmark Results

Numbers

Metric	HyperGraphRAG 1.0.6
Index build time	443.2 min
Boundary refusal rate	26.3%
P90 latency	49.3 s
Average latency	29.6 s
Single-hop match	0.081
Multi-hop match	0.171
Boundary match	0.021

Answer matching uses Jaccard keyword overlap, not LLM judge; LLM is GLM-4-flash throughout

Multi-hop: 0.171 — Tied with LightRAG, Not the Breakout

The theoretical advantage of hyperedges didn't materialize here. HyperGraphRAG's 0.171 on multi-hop is within noise of LightRAG's 0.178.

A few plausible explanations:

The test documents are technical reference materials (configuration guides, API docs). Multi-entity joint relationships are rare in this genre — that's precisely where hyperedges help most, but the signal isn't there to exploit
Hyperedge quality depends on LLM extraction quality; GLM-4-flash's performance limits what the graph can represent
BGE truncation to 700 characters may have dropped some semantic signal from longer entity descriptions

Index Build Time: 443 Minutes

GraphRAG took 31 minutes, HippoRAG took 36 minutes. HyperGraphRAG took 443 minutes — roughly 7.4 hours.

The cause: HyperGraphRAG issues multiple LLM calls per chunk to extract entity sets, and llm_model_max_async=1 (GLM rate limit). With 124 chunks, ~4 LLM calls each, plus the accumulated timeout retries, the total inflates significantly.

In a production setup with a higher-concurrency LLM (GPT-4o), the indexing time would likely drop to 30–60 minutes. GLM's low concurrency is the amplifying factor here.

Boundary Refusal: 26.3% — Better Than Expected for Graph RAG

5 of 19 boundary questions (26.3%) were correctly declined. This is higher than GraphRAG's 15.8% and HippoRAG's 5.3%.

Likely reason: HyperGraphRAG's hybrid mode (local + global search) may be more willing to return "not found in the knowledge base" when neither local graph traversal nor global context returns relevant hits.

Full Five-Framework Comparison

Combining all four articles:

Framework	Type	Index time	Refusal rate	P90 latency	Single-hop	Multi-hop
QAnything v2	Vector RAG	~5 min	26.3%	52.2 s	0.111	0.162
LightRAG 1.5.6	Graph+Vector	~8 min	10.5%	19.4 s	0.082	0.178
GraphRAG 3.1.1	Graph RAG	31 min	15.8%	32.4 s	0.107	0.211
HippoRAG 2.0	Graph RAG	36 min	5.3%	84.0 s	0.122	0.163
HyperGraphRAG 1.0.6	Hypergraph RAG	443 min	26.3%	49.3 s	0.081	0.171

Patterns across the five frameworks:

Index cost and multi-hop performance are decoupled: GraphRAG achieved the highest multi-hop score (0.211) with a 31-minute build. HyperGraphRAG spent 443 minutes and got 0.171. More indexing time does not buy better retrieval.
LightRAG has the best cost-effectiveness: 8-minute build, second-best multi-hop (0.178), lowest P90 latency (19.4s). Its only weakness is low refusal rate (10.5%) — the graph tends to produce an answer regardless of relevance.
Refusal capability is a design decision, not an emergent property: QAnything's 26.3% refusal rate comes from explicit confidence-threshold filtering. Graph RAG frameworks don't ship with a refusal mechanism and show near-zero rates as a result.
Single-hop: graph structure adds nothing: HippoRAG's 0.122 is the best among graph RAG approaches, but vector RAG and graph RAG both cluster in the 0.08–0.12 range. For direct factual retrieval, knowledge graph overhead doesn't help.

Which Framework Should You Choose?

Based on the five-framework data:

Default to LightRAG for most use cases: fast indexing, strong multi-hop, lowest latency. Add a confidence post-processing step to improve refusal behavior if needed.

Choose GraphRAG if:

Multi-hop reasoning is your primary use case (policy cross-referencing, causal chain analysis)
You have LLM budget for the indexing cost
You're using a model with reliable structured output (GPT-4o, Claude) to enable community reports; the GLM structured output issues in this test disabled that feature entirely
30-minute build times are acceptable

Choose QAnything if:

You need strong out-of-the-box refusal capability with minimal configuration
You want a self-contained deployment without managing graph infrastructure

Wait on HyperGraphRAG: the hyperedge concept is theoretically compelling, but version 1.0.6 has real production friction:

No PyPI package; source-only installation
7-hour build time in this configuration
Multi-hop performance didn't surpass LightRAG in a technical document domain
Check back when a production-grade release lands

Code Highlights

Full code at llm-in-action/kb-04-hypergraphrag-eval/.

Text truncation + sanitization

import re

_SENSITIVE_LINE_RE = re.compile(
    r'[^\n]*(?:'
    r'api[_\-]?key|secret|token|password|Authorization'
    r'|sk-[A-Za-z0-9]{10,}'
    r'|[A-Z][A-Z0-9_]{3,}=[^\s\n]+'  # ENV_VAR=value
    r')[^\n]*',
    re.IGNORECASE,
)
_MAX_CHARS = 700  # bge-large-en-v1.5 via SiliconFlow ~512 tokens

def _truncate(text: str) -> str:
    text = _SENSITIVE_LINE_RE.sub("[REDACTED]", text)
    if len(text) <= _MAX_CHARS:
        return text
    truncated = text[:_MAX_CHARS]
    last_space = truncated.rfind(" ")
    return truncated[:last_space] if last_space > 0 else truncated

Per-document insert with checkpoint resume

def build_index(rag, doc_files: list[Path]) -> float:
    progress = load_progress()  # load completed list
    for fpath in doc_files:
        if fpath.name in progress["indexed"]:
            continue  # skip already done
        content = fpath.read_text().strip()
        rag.insert([content])              # insert one at a time
        progress["indexed"].append(fpath.name)
        save_progress(progress)            # checkpoint immediately

Next: Series Wrap-Up — benchmark methodology limitations, what this test set measures (and doesn't), and how to design a production-grade RAG evaluation from scratch.

Check out PrimeSkills — a curated marketplace of AI agents and skills validated in real-world, enterprise-grade workflows. Not demos — things that actually work in production.

Find more on my Homepage

One Open Source Project a Day (No. 165): ntfy — Push Notifications with a Single curl

WonderLab — Sat, 29 Aug 2026 02:42:24 +0000

Introduction

"Push notifications without complexity."

This is the 165th article in the "One Open Source Project a Day" series. Today's project is ntfy (pronounced "notify").

You have probably run into this situation before: a long-running script finishes and you want to know the result. A backup job completes successfully and you want a quick tap on your shoulder. A server's disk is nearly full and you need to know immediately.

There are plenty of solutions — Email, Slack, a Telegram bot — but every single one requires creating an account, configuring an API key, and installing an SDK. By the time you are done, the notification plumbing has become more work than the original task.

ntfy strips the whole thing down to its essence:

curl -d "Backup complete!" ntfy.sh/my-channel

That is it. One line, and the message lands on your phone. No account, no API key, no configuration whatsoever. 33.8k Stars, fully self-hostable, and a staple tool for ops engineers, developers, and automation enthusiasts.

What You Will Learn

How ntfy's publish-subscribe model works
Advanced message features: priority, tags, action buttons, delayed delivery, and attachments
How to run a private ntfy server with Docker
Authentication and access control configuration
Integrating ntfy with Alertmanager, GitHub Actions, Home Assistant, and more

Prerequisites

Basic familiarity with HTTP concepts (GET/POST requests)
Comfortable with Linux command-line basics (curl)
Docker fundamentals for the self-hosting section

Project Background

What It Is

ntfy is an HTTP-based publish-subscribe (pub-sub) push notification service. Its design is deliberately minimal:

Publisher: send an HTTP request to a "topic"
Subscriber: subscribe to that topic in the phone app or browser

Topics require no pre-registration. The message body is the notification content. There are no extra layers of abstraction.

The official ntfy.sh public service is free to use (with rate limits). You can also self-host it entirely — both options run the same open-source code and expose identical features.

Author

Author: Philipp C. Heckel (website: heckel.io)
Background: German software engineer and independent open-source author; also created the ZFS backup tool znapzend among others
Origin: Started in 2021 from a personal itch — "I just want a notification when my script finishes"

Project Stats

⭐ GitHub Stars: 33,800+
🍴 Forks: 1,600+
📄 License: Apache 2.0 + GPLv2 (dual license)
🌐 Public service: ntfy.sh
📚 Docs: docs.ntfy.sh
📱 Android: Google Play + F-Droid (both free)
🍎 iOS: App Store

Core Features

What Problem It Solves

ntfy is a minimal message broker that reduces "send a notification" to the simplest possible HTTP request:

Publisher (anything that can make an HTTP request)
    curl / shell script / GitHub Actions / Prometheus Alertmanager ...
         ↓  POST/PUT  ntfy.sh/my-topic
ntfy server (public ntfy.sh or self-hosted instance)
         ↓  push delivery
Subscriber (any client subscribed to that topic)
    mobile app / browser / CLI ...

The topic name doubles as both the "channel address" and the first layer of access control — there is no registration flow, no API key to manage. An unpredictable topic name is itself the first barrier to unauthorized access (formal authentication is available too).

Usage Scenarios

Script and job completion notifications
- Long-running backup jobs, builds, or data pipelines push their result to your phone the instant they finish. No more staring at a terminal.
Server monitoring alerts
- Pair ntfy with Uptime Kuma, Prometheus Alertmanager, or Healthchecks.io to receive alerts the moment a service goes down.
CI/CD pipeline notifications
- GitHub Actions or GitLab CI pushes success or failure to your phone on completion. No polling the web UI.
Smart home events
- Home Assistant automation rules fire ntfy notifications: "Front door unlocked," "Washing machine done."
Personal automation scripts
- Any shell script scenario — scheduled tasks, file downloads finishing, scrapers completing — one curl line handles the notification.

Quick Start

Option 1: Use the public service — zero configuration

# Step 1: Install the app on your phone (iOS / Android)
# Open app → Add subscription → enter a topic name, e.g. "my-alerts-abc123"

# Step 2: Send a notification (from any device)
curl -d "Hello from ntfy!" ntfy.sh/my-alerts-abc123

# With a title
curl \
  -H "Title: Backup complete" \
  -d "All files successfully backed up to S3" \
  ntfy.sh/my-alerts-abc123

Option 2: Self-hosted server (Docker)

# Minimal start
docker run -p 80:80 -it binwiederhier/ntfy serve

# With persistent storage
docker run \
  -v /var/cache/ntfy:/var/cache/ntfy \
  -v /etc/ntfy:/etc/ntfy \
  -p 80:80 \
  -it binwiederhier/ntfy serve

Replace ntfy.sh with your server's address and everything else stays identical.

Core Features

1. Message Priority (levels 1–5)

# Urgent: heavy vibration + full-screen alert
curl -H "Priority: urgent" -d "Disk almost full!" ntfy.sh/alerts

# Low: silent delivery
curl -H "Priority: low" -d "Scheduled backup complete" ntfy.sh/alerts

Level	Name	Android Behavior
5	urgent	Heavy vibration + full-screen popup
4	high	Long vibration + default sound
3	default	Default behavior
2	low	No vibration or sound
1	min	Collapsed under "other notifications"

2. Tags and Emoji

Tag names that match emoji short codes are automatically converted:

curl \
  -H "Tags: warning,computer" \
  -d "CPU usage exceeded 90%" \
  ntfy.sh/alerts
# Phone shows: ⚠️ 💻 CPU usage exceeded 90%

3. Action Buttons (up to 3)

Add tappable actions directly to the notification — no need to open the app:

# Button that triggers an HTTP request
curl \
  -H "Actions: http, Silence alert, https://myserver.com/api/silence, method=POST" \
  -d "CPU sustained high load. Silence alert?" \
  ntfy.sh/alerts

# Action types:
# view      → open a URL
# http      → send an HTTP request
# broadcast → Android broadcast intent
# copy      → copy text to clipboard

4. Delayed Delivery

# Send in 30 minutes
curl -H "In: 30min" -d "Time to drink some water" ntfy.sh/reminders

# Send tomorrow at 9am
curl -H "At: tomorrow, 9am" -d "Weekly standup reminder" ntfy.sh/reminders

# Dead man's switch: if the script doesn't ping again within 1 hour, fire an alert
curl -H "In: 1h" \
  -H "Title: Script heartbeat lost" \
  -d "No heartbeat in over 1 hour — please investigate!" \
  ntfy.sh/watchdog

5. Attachments

# Upload a local file (max 15 MB, expires after 3 hours)
curl -T screenshot.png \
  -H "Filename: screenshot.png" \
  ntfy.sh/alerts

# Attach an external URL (no server storage used)
curl \
  -H "Attach: https://example.com/report.pdf" \
  -H "Filename: monthly-report.pdf" \
  -d "Monthly report is ready" \
  ntfy.sh/reports

6. Message Templates

ntfy ships with built-in templates for Alertmanager and Grafana. Configuring the webhook URL is enough:

https://ntfy.sh/my-alerts?template=alertmanager

Custom Go templates can process arbitrary JSON payloads.

Competitive Comparison

Dimension	ntfy	Telegram Bot	Slack Webhook	Pushover
No account needed	✅	❌ Requires Telegram	❌ Requires Slack	❌ Requires signup
Self-hostable	✅ Full support	❌	❌	❌
Free self-hosted	✅ Completely free	—	—	❌ Paid
Free Android app	✅	✅	✅	❌ Paid app
Action buttons	✅	✅	✅	✅ Limited
Delayed delivery	✅	❌	❌	❌
Publish method	Plain HTTP	Bot API	Webhook	API
Open source	✅	—	—	❌

Deep Dive

Architecture: Minimal Pub-Sub

ntfy's architecture is deliberately simple:

┌──────────────────────────────────────────────┐
│                ntfy server                    │
│                                              │
│  HTTP API ──→ Message routing ──→ Topic      │
│                                    ↓         │
│                               Message cache  │
│                            (SQLite / PgSQL)  │
│                                    ↓         │
│              ┌─────────────────────┤         │
│              ↓                     ↓         │
│          WebSocket              FCM/APNs     │
│        (Web / CLI)           (Mobile push)  │
└──────────────────────────────────────────────┘

Message routing: The topic name is the routing key. Incoming messages are broadcast to all active subscribers.

Message cache: The server caches messages for 12 hours by default. Clients that reconnect can pull any messages they missed. This is a key advantage over pure webhook solutions — the recipient does not need to be online at the moment of delivery.

Push channels:

Web / desktop: Server-Sent Events (SSE) or WebSocket
Android: Firebase Cloud Messaging (FCM), or direct long-polling in the F-Droid build (no FCM dependency)
iOS: APNs via the official ntfy.sh relay

Technology Stack

The backend is written in Go — a well-suited choice:

Produces a single binary with zero external dependencies
Handles many concurrent WebSocket connections efficiently
Cross-platform compilation covers Linux, macOS, and Windows

The database defaults to SQLite; switching to PostgreSQL is a single line in the config file:

# /etc/ntfy/server.yml
database-url: "postgres://user:pass@localhost/ntfy"

SQLite's single-writer constraint means it cannot scale horizontally; use PostgreSQL for high-concurrency production deployments.

Self-Hosting Configuration

A production-ready server configuration:

# /etc/ntfy/server.yml

# Base settings
base-url: "https://ntfy.example.com"
listen-http: ":80"
listen-https: ":443"
key-file: "/etc/letsencrypt/live/ntfy.example.com/privkey.pem"
cert-file: "/etc/letsencrypt/live/ntfy.example.com/fullchain.pem"

# Storage
cache-file: "/var/cache/ntfy/cache.db"
cache-duration: "12h"
attachment-cache-dir: "/var/cache/ntfy/attachments"
attachment-total-size-limit: "5G"
attachment-file-size-limit: "15M"
attachment-expiry-duration: "3h"

# Access control (close anonymous access — login required)
auth-file: "/var/lib/ntfy/user.db"
auth-default-access: "deny-all"

# Declarative user provisioning (auto-created on startup)
auth-users:
  - "alice:$2a$10$hashhere:admin"
auth-access:
  - "alice:*:rw"        # alice can read/write all topics
  - "*:public-*:ro"     # everyone can subscribe to public-* topics

# Web push (browser notifications)
web-push-public-key: "BNbxxx..."
web-push-private-key: "xxx..."
web-push-file: "/var/lib/ntfy/webpush.db"

# Email forwarding
smtp-sender-addr: "mail.example.com:587"
smtp-sender-user: "ntfy@example.com"
smtp-sender-pass: "your-smtp-password"
smtp-sender-from: "ntfy@example.com"

Authentication

Three authentication options are supported:

# Username + password
curl -u alice:password -d "Private message" ntfy.example.com/private-topic

# Bearer token (recommended — more secure than a raw password)
curl -H "Authorization: Bearer tk_AbcDefGhi..." \
  -d "Private message" ntfy.example.com/private-topic

# URL parameter (for environments where setting headers is inconvenient)
curl -d "Private message" \
  "ntfy.example.com/private-topic?auth=dXNlcjpwYXNz"

Tokens support expiry and can be revoked at any time — safer than embedding a plain password in scripts.

Integrations

GitHub Actions:

- name: Send build notification
  run: |
    curl \
      -H "Title: ${{ github.repository }} build ${{ job.status }}" \
      -H "Priority: ${{ job.status == 'success' && 'default' || 'high' }}" \
      -H "Tags: ${{ job.status == 'success' && 'white_check_mark' || 'x' }}" \
      -d "Branch: ${{ github.ref_name }}, commit: ${{ github.sha }}" \
      ${{ secrets.NTFY_URL }}/${{ secrets.NTFY_TOPIC }}

Prometheus Alertmanager:

# alertmanager.yml
receivers:
  - name: "ntfy"
    webhook_configs:
      - url: "https://ntfy.sh/my-alerts?template=alertmanager"
        send_resolved: true

Home Assistant:

# configuration.yaml
notify:
  - platform: rest
    name: ntfy
    resource: https://ntfy.sh/my-home-alerts
    method: POST_JSON
    title_param_name: title
    message_param_name: message

Project Links & Resources

Official Resources

🌟 GitHub: https://github.com/binwiederhier/ntfy
📚 Docs: https://docs.ntfy.sh
🌐 Public service: https://ntfy.sh
📱 Android App (F-Droid): f-droid.org/packages/io.heckel.ntfy
🐛 Issues: github.com/binwiederhier/ntfy/issues

Related Resources

Uptime Kuma — self-hosted monitoring that pairs naturally with ntfy alerts
Healthchecks.io — cron job heartbeat monitoring with ntfy support
ntfy Android source — Android client in its own repository

Summary

Key Takeaways

Minimal pub-sub model: topic = address, HTTP request = publish, no account, no SDK
Rich message features: priority, tags, action buttons, delayed delivery, attachments — covers virtually every notification use case
Complete self-hosting: single Go binary, docker run in one line, SQLite or PostgreSQL your choice
Strong integration ecosystem: Alertmanager, GitHub Actions, Home Assistant, Uptime Kuma all work out of the box
Message cache: 12-hour cache ensures offline subscribers don't miss messages — more reliable than pure webhooks

Who This Is For

Ops and DevOps engineers: server alerts, CI/CD notifications, cron job monitoring
Indie developers and solo founders: notification needs for automation scripts — cheap, zero-dependency, no lock-in
Self-hosting enthusiasts: full ownership of notification infrastructure, no third-party service dependency
Smart home builders: event push from Home Assistant, Node-RED, and similar platforms

One-Line Verdict

ntfy is the best argument for "simple is best" — it reduces push notifications to a single curl command, yet has enough depth to handle serious production requirements when you need it.

Check out PrimeSkills — a curated marketplace of AI agents and skills that have been validated in real-world, enterprise-grade workflows. No fluff, just what actually works.

Find more useful knowledge and interesting products on my Homepage

Code Agent Anatomy (13): Harness Design Part 3 — Tool Execution Pipeline

WonderLab — Fri, 28 Aug 2026 00:38:08 +0000

The Journey of a Tool Call

The model outputs a tool_call — say Edit("foo.py", ...) — and the result ends up written into history. What happens in between?

This post walks that path end-to-end, disassembling two core modules: ToolOrchestrator (the dispatch layer, managing concurrency and result budgets) and ToolExecutor (the execution layer, managing permissions, optimistic locks, and circuit breakers).

The Conclusion First

The tool execution pipeline has two layers with completely separate responsibilities:

Layer	Module	Responsible for
Dispatch	`ToolOrchestrator`	Concurrent grouping, ordering guarantees, result budget truncation
Execution	`ToolExecutor`	Permission check → optimistic lock injection → circuit breaker check → tool.run()

The model produces a batch of tool_calls; Orchestrator handles "how to run this batch"; Executor handles "how to execute one tool safely." There's a clean interface boundary between the two layers.

1. Dispatch Layer: Concurrent Grouping, Ordering Guarantees

# tools/orchestrator.py  ToolOrchestrator
SAFE_TOOL_NAMES   = {"Read", "Grep", "Glob"}   # read-only, safe to parallelize
UNSAFE_TOOL_NAMES = {"Edit", "Bash", "Task", ...}  # side-effecting, must be serial

The model may request multiple tools in a single step — for example, Read three files and then Edit one. The Orchestrator's first job is batching:

# partition_tool_calls()
# input:  [Read, Read, Edit, Grep, Grep, Edit, Read]
# output: [concurrent(Read,Read), serial(Edit), concurrent(Grep,Grep), serial(Edit), concurrent(Read)]

The rule is simple: consecutive safe tools are merged into one concurrent batch; a write tool breaks the batch.

Concurrent batches run via ThreadPoolExecutor, but there's a key detail — result ordering is preserved:

def _run_batch_concurrently(self, batch, ...):
    # submit returns a Future immediately without blocking — all tools start nearly simultaneously
    futures = {
        offset: executor.submit(self._execute_plan, plan, ...)
        for offset, plan in enumerate(batch.calls)
    }
    # .result() blocks until done; results stored by offset, preserving original position
    for offset, future in futures.items():
        observations[offset] = future.result()
    # rebuild list in offset order — thread completion order is non-deterministic, this forces the original
    return [observations[idx] for idx in range(len(batch.calls))]

Thread B may finish before thread A, but in the returned list A always precedes B. The order the model requested is the order written into history.

Serial batches are a plain for-loop: one completes before the next begins. Total time is the sum of all tool times, but there's no race condition.

2. Execution Layer: Four-Gate Pipeline

Each tool's actual execution goes through ToolExecutor.execute(), which is a linear pipeline — any gate failure short-circuits immediately:

argument parsing → [Gate 1] permission check → [Gate 2] optimistic lock injection → [Gate 3] circuit breaker check → tool.run()

Gate 1: Permission Check

# tools/permissions.py  RiskClassifier
# Decision priority:
# 1. Read/Grep/Glob → ALLOW (read-only, no risk)
# 2. Edit → check runtime_mode (read-only sub-agent → DENY)
# 3. Bash → regex blacklist → greylist → whitelist
# 4. unknown tool → DENY (fail-closed)

Bash is the most complex case. Blacklist hits are immediate DENY without asking the user:

_BASH_DENY_PATTERNS = (
    (re.compile(r"sudo"), "sudo crosses the process privilege boundary"),
    (re.compile(r"rm(?:\s|$)"), "destructive delete command"),
    (re.compile(r"bash\s+-c"), "nested shell execution bypasses command classification"),
    (re.compile(r"`|\$\("), "shell command substitution executes nested commands"),
    ...
)

Greylisted commands (mv, pip install, chmod, etc.) go through the ASK policy, which in the current MVP implementation defaults to DENY when ask_policy="deny" — meaning unknown-risk commands are not executed by default, forcing the model to find another approach.

The critical design principle: fail-closed. Being in the tool Registry only means the model can "see" the tool. Execution rights still have to pass through the permission gate.

Gate 2: Optimistic Lock Injection (Edit only)

# tools/executor.py
if name == "Edit":
    parameters = self.registry.inject_optimistic_lock_params(name, parameters)

After a Read tool executes, the framework caches the file's mtime + size. Before an Edit executes, the framework automatically injects the cached expected_mtime_ms into the parameters. If the file was externally modified between the Read and the Edit, the Edit detects the mtime mismatch and returns a CONFLICT error instead of silently overwriting.

This solves a subtle problem: the model reads a file and decides to modify it, but the file may have been changed by the user or another tool in the interim. The optimistic lock makes this class of "write clobber" detectable.

Gate 3: Circuit Breaker Check

# tools/circuit_breaker.py
# Three states: CLOSED (normal) → OPEN (disabled) → HALF_OPEN (probe after cooldown)
if not self.registry.is_available(name):
    return self.registry.create_circuit_open_result(name, parameters)

After 3 consecutive failures (default threshold), the circuit breaker opens and the tool is temporarily disabled for 300 seconds. This prevents a broken tool from retrying repeatedly, consuming tokens and step budget. After the cooldown period, the breaker enters HALF_OPEN — it lets one probe through: success restores CLOSED, failure resets the timer and continues OPEN.

tool.run() and the Exception Safety Net

try:
    result = tool.run(parameters)
    if not isinstance(result, ToolResult):
        raise TypeError(...)
except Exception as exc:
    # all uncaught exceptions are caught here, converted to EXECUTION_ERROR ToolResult
    # guarantees no exception propagates upward; loop always receives a ToolResult
    # a single tool crash cannot interrupt the entire agent
    return ToolResult(status=ERROR, error_code=EXECUTION_ERROR, ...)

Any internal exception from the tool is caught here and converted into a standard ToolResult. This is the tool pipeline's last safety net: a single tool crashing doesn't crash the whole loop.

3. Result Post-Processing: Two-Layer Byte Budget

After a tool executes, its result passes through three post-processing steps:

execution result
  → _normalize_empty_result()   empty output gets a placeholder text
  → _apply_observation_limit()  initial truncation by line/byte count
  → _apply_result_budget()      two-layer byte budget final truncation

Two-layer budget prevents tool output from overflowing the context window:

Layer 1 (single-tool cap, default 50KB):
    single tool output > 50KB → force_truncate → full content spills to disk file
    result includes the file path; model can reference it on demand

Layer 2 (batch total cap, default 200KB):
    total after per-tool truncation still > 200KB → sort by size descending,
    force-truncate one by one until total is under the limit
    greedy strategy: truncate the largest first to minimize truncation count

Results already truncated by Layer 1 are marked replaced=True in metadata; Layer 2 skips them to avoid double-truncating the same result.

4. Lifecycle Events: Full Observability

Every tool call passes through four lifecycle states, all emitted as events to the trace/transcript:

requested → started → completed / failed

requested: model requested the tool (recorded before argument parsing, regardless of outcome)
started: passed permission check, entering actual execution
completed: execution succeeded (including partial status)
failed: execution failed (including permission denial, circuit break, exception)

These four states let the trace fully reconstruct "the story of a tool call": was it requested? Was it denied or did it actually run? How long did it take? What was the result?

Design Highlights

1. Dispatch and execution are separated

Orchestrator doesn't care how a single tool executes — only "how to schedule this batch." Executor doesn't care how many tools are concurrent — only "is this one tool safe." Separation of responsibilities makes both layers independently testable and evolvable.

2. Writes are forced serial; order matches model request

After concurrent execution, results are force-reordered by original offset — this isn't just an ordering concern, it's a semantic one. When the model issues [Edit A, Edit B], it expects A before B. Wrong write order in history will confuse the model on the next step.

3. Multiple safety boundaries

Permission blacklist (rule layer) → optimistic lock (data layer) → circuit breaker (availability layer) → exception safety net (stability layer). Each layer solves one class of problem, with no overlap.

Summary

Design choice	Approach	Engineering value
Concurrency strategy	Read-only concurrent, writes serial	Safe, no race conditions; Read/Grep/Glob concurrency improves throughput
Ordering guarantee	Offset-based reordering	Model semantics not scrambled by concurrent execution
Permission design	Fail-closed + regex blacklist	Dangerous commands never reach tool.run()
Optimistic lock	Read caches mtime, Edit auto-injects	Write clobber is detectable, not silent
Circuit breaker	Three-state + cooldown	Broken tools don't retry and drain step budget
Byte budget	Single-tool + batch total, two layers	Tool output can't overflow the context window

About the Source Code

All analysis in this series is based on the open-source project MyCodeAgent.

The source code includes comments at key locations aligned with the walkthrough in each post — you can read the articles alongside the code, or clone it directly to run, modify, and extend, building your own agent on top of it.

git clone https://github.com/chendongqi/MyCodeAgent
cd MyCodeAgent
cp .env.example .env   # fill in your LLM API key
uv sync
uv run python main.py

Explore PrimeSkills — a curated marketplace of AI agents and skills, each validated against real enterprise workflows. No hype, just what actually works.

Visit my personal site for more insights and interesting products.

One Open Source Project a Day (No. 164): next-forge — A Production-Grade Next.js SaaS Starter

WonderLab — Fri, 28 Aug 2026 00:36:57 +0000

Introduction

"Stop wasting time on boilerplate. Start building."

This is the 164th article in the "One Open Source Project a Day" series.

Two hundred is a number worth pausing on. Over these 200 projects we have covered infrastructure engines, AI frameworks, developer tooling, content platforms — the whole spectrum of open-source creativity. For article 200, I wanted to pick something that represents the journey from idea to product in the most direct way possible.

next-forge is exactly that.

Its positioning is unambiguous: a production-grade Next.js SaaS starter template. Not a toy demo. Not a "you still need to wire up 10 more libraries" half-finished scaffold. A battle-tested, full-stack foundation that is ready to deploy on day one. Created by Vercel engineer Hayden Bleasel, drawing on more than a decade of building web applications, with 7,700+ Stars and an MIT license.

If you have ever built a SaaS or an indie product, you know the pain: just getting authentication, payments, email, a database, error monitoring, and analytics all configured can eat a full week — and none of it has anything to do with your actual business logic. That is exactly the problem next-forge solves.

What You Will Learn

The five design principles of next-forge (Fast / Cheap / Opinionated / Modern / Safe) and the engineering philosophy behind them
How a Monorepo with 7 apps and 20+ packages is organized
The design rationale behind key integrations: Clerk (auth), Stripe (payments), Prisma + Neon (database)
How to initialize a production-ready SaaS project with a single command
The essential difference between next-forge and other SaaS starter templates

Prerequisites

Basic familiarity with Next.js (App Router knowledge is a plus)
Comfortable reading TypeScript
A rough idea of what a Monorepo is ("multiple projects in one repository") — no deep expertise required

Project Background

What It Is

next-forge is a Turborepo Monorepo template designed for developers who want to ship a SaaS or full-stack web application without spending weeks on infrastructure decisions.

It is not just a Next.js app scaffold. It is a curated collection of technology decisions, each integration pre-selected, pre-configured, and verified to work together. You get a working foundation; you supply the business logic.

The target audience is precise:

Indie developers and one-person companies who want to validate ideas fast and not drown in configuration
Technical leads on small teams who need a proven architectural baseline rather than having to debate "which auth library should we use"
Experienced full-stack engineers who want a production-ready template they can actually deploy, not just reference

Author / Team

Primary author: Hayden Bleasel, a design engineer from Australia based in San Francisco
Organization: Vercel (now at OpenAI)
Background: 10+ years building web applications; also maintains Ultracite and several other open-source projects
Origin: Distilled from real SaaS development experience — not a teaching exercise

Project Stats

⭐ GitHub Stars: 7,700+
🍴 Forks: 712+
📄 License: MIT
🌐 Website: next-forge.com
📚 Docs: next-forge.com/docs
🎮 Demo: demo.next-forge.com

Core Features

What Problem It Solves

next-forge addresses the foundation problem in SaaS development: all the "must-haves" you need to wire up before you write a single line of business logic.

A typical SaaS needs user authentication, billing, a database, transactional email, error tracking, product analytics, documentation, and more. next-forge has all of these pre-integrated and ready to go:

Without next-forge:
idea → pick auth library → configure Stripe → choose ORM → integrate email →
set up error tracking → scaffold docs → ... → two weeks later you finally
start on actual business logic

With next-forge:
npx next-forge@latest init → fill in environment variables → write business logic

Usage Scenarios

Rapid SaaS launch
- You want to validate a subscription product's market fit without spending two weeks on infrastructure. next-forge ships with subscription plans, a checkout flow, and user management — you can have a working "sign up → pay → dashboard" path in a day.
Indie side projects
- Time after hours is precious. next-forge offloads all the "not worth doing yourself" configuration work so every available hour goes toward product differentiation.
Team project architectural baseline
- No need to bikeshed "which auth provider should we use" — next-forge's technology choices are battle-tested and can serve as the team's starting point for any new project.
Learning Next.js production patterns
- For developers who want to understand what Next.js App Router looks like in a real production codebase, next-forge is a genuine reference — not a tutorial demo, but code that is actually deployable.

Quick Start

# Initialize a new project with one command
npx next-forge@latest init

# Requirements: Node.js 20+, Bun (recommended), Stripe CLI

After initialization:

# Install dependencies
bun install

# Start all apps in development mode simultaneously
bun run dev

Running apps:

localhost:3000 — Main application (authenticated user interface)
localhost:3001 — Marketing website (public-facing)
localhost:3002 — API service
localhost:3003 — Email template preview server
localhost:3004 — Project documentation
localhost:6006 — Storybook component library

Core Features

1. Full Authentication System (Clerk)

Email login and social login (Google, GitHub, etc.) out of the box
Routes automatically grouped by authentication state: (authenticated)/ and (unauthenticated)/
Organization/team support for B2B SaaS scenarios

2. Subscription Billing (Stripe)

Subscription plans, one-time payments, and usage-based billing supported
Webhook handling pre-configured in apps/api
Customer Portal for self-service subscription management

3. Type-Safe Database (Prisma + Neon)

Prisma ORM provides end-to-end type-safe queries
Defaults to Neon (serverless PostgreSQL), pay only for what you use
Callable directly from React Server Components — no separate API layer needed

// Query directly in a Server Component
import { database } from '@repo/database';

const users = await database.user.findMany({
  where: { active: true }
});

4. Email System (Resend + React Email)

Write email templates as React components — real JSX, not HTML strings
apps/email provides a live preview server
Resend handles delivery with high deliverability rates

5. Observability (Sentry + BetterStack)

Sentry for error tracking and performance monitoring
BetterStack for log aggregation and alerting
Both pre-configured; effective from the moment you deploy

6. Product Analytics (PostHog + Google Analytics)

PostHog for product analytics, feature flags, and A/B testing
Google Analytics for traffic metrics
Both wrapped in a unified @repo/analytics package

7. Security (Arcjet)

Rate limiting
Bot detection
Email validation to block fake signups
Automatically disabled in development so local debugging is unaffected

8. Real-Time Collaboration (Liveblocks)

Live Cursors
Avatar Stacks
Useful for SaaS products with collaborative features

9. Internationalization (Languine)

i18n solution pre-integrated and ready to use
Translation resources managed centrally in @repo/internationalization

10. Push Notifications (Knock)

In-app notifications, email notifications, and SMS notifications unified in one service
Visual notification workflow editor

Competitive Comparison

Dimension	next-forge	create-t3-app	Manual Setup
Integrations included	20+ services pre-wired	Core stack only, rest manual	Everything manual
Multi-app architecture	✅ 7 independent apps	❌ Single app	Design it yourself
Production-validated	✅ Real SaaS background	✅ Widely used	Depends on your experience
Documentation	✅ Full docs site	✅ Active community	—
Payment integration	✅ Stripe pre-configured	❌ Not included	Manual integration
Observability	✅ Sentry + BetterStack	❌ Not included	Manual integration
Collaboration features	✅ Liveblocks	❌ Not included	Manual integration
Time to first run	One command	One command	Days to weeks

Why choose next-forge?

Zero decision cost: Every domain (auth / payments / database) has already been decided for you, with a solution that is verified to work
Not a demo — a production template: Every integrated service is something real SaaS products actually use in production
Monorepo architecture: Marketing site, app, docs, and email templates are separate and independently deployable

Deep Dive

Architecture: Why a Monorepo

next-forge's choice of Turborepo Monorepo over a single-app repository has clear engineering reasoning.

A complete SaaS typically involves several distinct but related "applications":

A marketing website (for potential users, SEO-first)
A main application (for paying users, feature-first)
An API service (for webhooks, cron jobs, etc.)
A documentation site (for users and developers)

Keeping these in one repository while maintaining independent deployability and sharing auth logic, UI components, and type definitions is precisely the use case Monorepos were built for.

next-forge/
├── apps/
│   ├── app/         ← Main application (localhost:3000)
│   │   └── env.ts   ← Aggregates all env vars this app depends on
│   ├── web/         ← Marketing website (localhost:3001)
│   ├── api/         ← API + Webhook handling (localhost:3002)
│   ├── email/       ← Email template preview server (localhost:3003)
│   ├── docs/        ← Mintlify documentation (localhost:3004)
│   ├── storybook/   ← Component library (localhost:6006)
│   └── studio/      ← Prisma visual database editor (localhost:3005)
│
└── packages/
    ├── auth/              ← Clerk authentication wrapper
    ├── database/          ← Prisma + Neon client
    ├── payments/          ← Stripe wrapper
    ├── email/             ← React Email templates
    ├── analytics/         ← PostHog + GA unified entry point
    ├── observability/     ← Sentry + BetterStack
    ├── security/          ← Arcjet protection
    ├── collaboration/     ← Liveblocks real-time features
    ├── notifications/     ← Knock notifications
    ├── internationalization/ ← i18n management
    ├── feature-flags/     ← Vercel Flags + PostHog
    ├── storage/           ← Vercel Blob file storage
    ├── ui/                ← Shared UI components (built on shadcn/ui)
    ├── design-system/     ← Fonts, themes, global styles
    └── typescript-config/ ← Shared TypeScript configuration

Key design rule: apps must not depend on each other. web cannot import modules from app. Shared code lives exclusively in packages/, referenced as @repo/xxx.

// ✅ Correct: shared logic via package
import { auth } from '@repo/auth';
import { database } from '@repo/database';
import { Button } from '@repo/ui';

// ❌ Wrong: cross-app imports
import { UserCard } from '../../app/components/UserCard';

This boundary is enforced in CI via Turborepo's boundaries check:

bun run boundaries  # detects cross-app dependency violations

Environment Variable Management: the `env.ts` Pattern

next-forge uses an elegant solution to the "scattered environment variables" problem: every app has an env.ts file at its root that aggregates and validates all the environment variables the app needs.

// apps/app/env.ts
import { env as authEnv } from '@repo/auth/env';
import { env as databaseEnv } from '@repo/database/env';
import { env as paymentsEnv } from '@repo/payments/env';

export const env = {
  ...authEnv,
  ...databaseEnv,
  ...paymentsEnv,
  NEXT_PUBLIC_APP_URL: process.env.NEXT_PUBLIC_APP_URL!,
};

Each packages/xxx/env.ts uses @t3-oss/env-nextjs for runtime validation — if a required variable is missing, the app fails at startup rather than at runtime. This pattern answers the perennial question: "What environment variables do I need to configure?" — just read apps/app/env.ts and the answer is complete.

Package Design Philosophy: Self-Contained, Replaceable

Every package in next-forge follows one principle: self-contained, exposing a clean interface, with an implementation that can be swapped at any time.

Take @repo/auth as an example:

// packages/auth/index.ts — the public interface
export { auth, currentUser, signIn, signOut } from './clerk';

// Consumers are completely unaware of Clerk's existence.
// If you ever need to replace Clerk, only this package changes.
// App code remains untouched.

This design minimizes the cost of migrating between vendors. If Clerk stops working for you, switching to Auth.js or Supabase Auth is a change to one package, not a project-wide refactor.

End-to-End Type Safety

next-forge maintains type safety from database schema to frontend UI:

Prisma Schema (database structure definition)
    ↓ prisma generate
Prisma Client (type-safe query functions)
    ↓ import { database } from '@repo/database'
Server Component (direct call, no API layer needed)
    ↓ returns data to
Client Component (TypeScript types automatically inferred)

No type breakage anywhere along the chain. Change a database field and TypeScript will immediately tell you at compile time exactly where the rest of the code needs to follow.

Deployment Options

next-forge supports three deployment approaches:

Option	Best For
Vercel (recommended)	Fast launch, zero ops overhead, pay-per-use
Netlify	Vercel-like experience with different preferences
Docker self-hosted	Full infrastructure control or compliance requirements

Vercel deployment is the simplest path — each app connects independently to its subdirectory in the GitHub repository, with turbo.json already defining the correct build dependency graph.

Project Links & Resources

Official Resources

🌟 GitHub: https://github.com/vercel/next-forge
📚 Docs: https://next-forge.com/docs
🌐 Website: https://next-forge.com
🎮 Demo: https://demo.next-forge.com
🐛 Issues: github.com/vercel/next-forge/issues

Related Resources

Turborepo Documentation — understanding the Monorepo build system
Next.js App Router Docs — mastering file-system routing and React Server Components
shadcn/ui — the component library underpinning next-forge's UI layer

Summary

Key Takeaways

Five design principles (Fast / Cheap / Opinionated / Modern / Safe) guide every technology choice in the template
7 independent apps + 20+ shared packages in a Monorepo — clear responsibilities, independently deployable
env.ts pattern provides centralized environment variable management with runtime validation, eliminating "what variables do I need?" as a question
Self-contained package design makes vendor replacement a single-package change, not a project-wide migration
One command to initialize a complete SaaS skeleton with auth, payments, database, email, and monitoring already wired together

Who This Is For

Indie developers validating SaaS ideas: next-forge handles the foundation so you can spend 100% of your time on product
Small teams that don't want to reinvent the wheel: start from a proven architecture and focus on what makes your product different
Developers learning Next.js production patterns: next-forge is one of the rare "actually production-grade" reference architectures, not a tutorial

One-Line Verdict

If you are building a SaaS or an indie product, next-forge is not "an option worth considering" — it is "the starting point you should default to." Every hour you would have spent on configuration is already handled; everything left is genuinely yours to build.

Check out PrimeSkills — a curated marketplace of AI agents and skills that have been validated in real-world, enterprise-grade workflows. No fluff, just what actually works.

Find more useful knowledge and interesting products on my Homepage

Code Agent Anatomy (12): Harness Design Part 2 — Context Engineering

WonderLab — Thu, 27 Aug 2026 01:50:35 +0000

The Loose End from Last Time

The previous post (Part 11) dissected the control flow: a single main loop, an immutable state machine, and completion gates. One line of code was glossed over:

# loop.py _prepare_step_context()
model_view = host.context_engine.build_model_view(...)
messages = model_view.messages

What build_model_view() actually does is far more than "fetch the message history." A long-running agent accumulates an ever-growing conversation history — exceed the token limit and it errors out. The subtler point: even before hitting the limit, the agent must decide each step — what exactly should get sent to the model? System prompt, history, tool outputs… cramming everything in is no better than careful curation.

This post dissects that pipeline: from "sensing we're almost out of headroom" to "deciding what the model gets to see."

The Conclusion First

MyCodeAgent breaks this problem into three cleanly separated concerns:

Concern	Module	One-liner
What History is	`HistoryManager`	Append-only fact log — messages are never deleted
What the model sees	`ModelView` + `ProjectionBuilder`	Read-time projection, collapsing old messages post-compaction
When to compact	`ContextBudgetPolicy`	Dual-source estimation + threshold decision

Three layers, clear responsibilities, no cross-contamination. Compaction doesn't mean deleting history — it means reading history a different way.

1. History Is a Fact Log — Never Modified

# runtime/history.py
class HistoryManager:
    def __init__(self, ...):
        self._messages: List[Message] = []  # append-only, never deleted

    def append_user(self, content: str, ...) -> Message: ...
    def append_assistant(self, content: str, ...) -> Message: ...
    def append_tool_result(self, ...) -> Message: ...

    def get_messages(self) -> List[Message]:
        return list(self._messages)  # returns a copy — caller cannot mutate

HistoryManager is a pure append-only fact list. It has no knowledge of "compaction," and no message ever disappears because of it.

This matters: it means that when an agent crashes, the complete history can be reconstructed from the transcript — nothing is ever lost to a compaction operation (see Part 10).

2. The Compaction Decision: How to Tell "We're Almost Full"

# runtime/context/budget.py
class ContextBudgetPolicy:
    def should_compact(self, *, messages, pending_input, last_usage_tokens) -> CompactDecision:
        # default: 128000 × 0.8 = 102400 tokens
        threshold = int(self.config.context_window * self.config.compression_threshold)

        # Source 1: estimate from message content (character count // 3)
        estimated_from_messages = self.estimate_tokens(messages, pending_input)
        # Source 2: actual usage from last LLM call + new input
        estimated_from_usage = int(last_usage_tokens or 0) + len(pending_input or "") // 3

        # take the more pessimistic estimate
        estimated = max(estimated_from_messages, estimated_from_usage)

        if message_count < 3:
            return CompactDecision(False, "messages_not_enough", ...)
        if estimated < threshold:
            return CompactDecision(False, "below_threshold", ...)
        return CompactDecision(True, "threshold_exceeded", ...)

Two design details worth noting:

1. Why not count tokens precisely?

Exact token counting requires calling a tokenizer, which is model-specific — different models tokenize differently, and calling it has overhead. character count // 3 is a conservative approximation (an average across mixed Chinese/English text). Better to compact early than to react after an error.

2. Why take the max of two sources?

Estimating purely from the message list underestimates — tool_calls JSON, tool_name fields, and other metadata all consume tokens but don't live in content. Meanwhile, last_usage_tokens is the actual number the LLM consumed on the previous step — more accurate, but it only knows the past, not the current new input. Taking max is an engineering habit of pessimistic estimation: compact a little early rather than not at all.

3. How Compaction Works: Non-Destructive Checkpoints

# runtime/context/compact.py
class ContextCompactor:
    def compact(self, messages: list[Message]) -> dict:
        # 1. segment messages into turns at user-message boundaries
        rounds = self.round_segmenter.identify(source_messages)

        # 2. retain the most recent min_retain_rounds turns verbatim (default: 10)
        retain_start_round = len(rounds) - min_rounds
        retain_start_idx = rounds[retain_start_round].start_idx
        messages_to_compact = source_messages[:retain_start_idx]

        # 3. call LLM to summarize the "old" messages
        summary = self.summary_generator(messages_to_compact)

        # 4. store summary + split-point in CompactStore
        checkpoint = self.compact_store.create_checkpoint(
            summary=summary,
            retain_start_idx=retain_start_idx,
            ...
        )
        # note: source_messages is never modified

The keyword here is non-destructive: compaction does not modify any message in HistoryManager. It only writes a checkpoint to CompactStore recording "what the summary says" and "which index marks the start of verbatim history."

RoundSegmenter divides the message list into turns at user-message boundaries. Compaction granularity is always a complete turn — splitting a user and assistant message from the same exchange would break semantic continuity.

4. Read-Time Projection: What the Model Sees ≠ What's Stored

# runtime/context/projection.py
class ProjectionBuilder:
    def project(self, source_messages: list[Message]) -> ProjectionResult:
        checkpoint = self.compact_store.active_checkpoint

        if not checkpoint:
            # no compaction yet: projection = full history
            return ProjectionResult(messages=source, projection_mode="full_history")

        # checkpoint exists: projection = [summary message] + source[retain_start_idx:]
        summary = Message(content=checkpoint.summary, role="summary", ...)
        return ProjectionResult(
            messages=[summary] + source[retain_start_idx:],
            projection_mode="compact_checkpoint",
        )

This is the single most important step in the whole context engineering stack: read-time projection.

If source_messages has 200 entries and the checkpoint's retain_start_idx is 120, the projected result is:

[summary (covering the first 120 messages)] + [source[120:] — the most recent 80 messages verbatim]

The model sees 81 messages (1 summary + 80 verbatim), but HistoryManager still holds all 200 originals.

5. MessageNormalizer: Format Conversion

History stores Message objects (the runtime's internal format); the LLM API expects a list of OpenAI-style dicts. MessageNormalizer handles that conversion:

# runtime/context/normalizer.py
class MessageNormalizer:
    def _normalize_one(self, msg: Message) -> list[dict]:
        if msg.role == "user":
            return [{"role": "user", "content": msg.content}]
        if msg.role == "assistant":
            return [self._assistant_message(msg)]  # restores tool_calls
        if msg.role == "tool":
            return [self._tool_message(msg)]        # links tool_call_id
        if msg.role == "summary":
            # summary becomes a system message, prepended to history
            return [{"role": "system", "content": f"## Archived History Summary\n{msg.content}"}]

A summary-role message is converted to a system-role message before being sent to the model — the model treats it as background context rather than conversational history. That's a subtle but intentional semantic choice.

6. ModelView: What the Model Saw on This Step

All of the above converges in ContextEngine.build_model_view(), which produces a ModelView:

# runtime/context/engine.py (core logic of build_model_view)
def build_model_view(self, *, history_manager, pending_input, ...) -> ModelView:
    # 1. Bounded history projection (projection + normalize)
    source_messages = history_manager.get_messages()
    projection = self.projection_builder.project(source_messages)
    history_messages = self.normalizer.normalize(projection.messages)

    # 2. System layer (agent persona + tool contracts + project rules)
    system_messages = self.context_builder.get_system_messages()

    # 3. Dynamic system: Session Memory (cross-run summaries)
    dynamic_messages = [{"role": "system", "content": rendered}] if self.session_memory else []

    # 4. Assemble: system first, history last
    messages = list(system_messages) + dynamic_messages + list(history_messages)

    return ModelView(
        messages=messages,
        system_message_count=len(system_messages),
        history_message_count=len(history_messages),
        source_message_count=projection.source_message_count,
        projection_mode=projection.projection_mode,
        ...
    )

Final message order sent to the model:

[Constitution][Tool Contracts][Code Law]   ← system layer (Part 04)
[Session Memory?]                           ← cross-run summary (Part 10)
[summary? + most recent N turns verbatim]  ← history projection

ModelView is more than a message list — it carries metadata: projection_mode (was it compacted?), source_message_count vs. history_message_count (before vs. after compaction counts). These feed the trace, letting you observe "how much the model actually saw this step."

7. Two Trigger Paths

Two places in the loop can trigger compaction:

Per-step (proactive)              On model call (reactive)
        ↓                                  ↓
compact_if_needed()            classify exception → PROMPT_TOO_LONG
  → budget estimate > threshold  → reactive_compact()
  → compactor.compact()          → compactor.compact()
  → record checkpoint             → rebuild model view, inner continue retry

Proactive compaction (compact_if_needed) runs before each step, prior to assembling the model view. If the estimate says we're almost out of headroom, compact first, then assemble — the model gets the compacted version on the next step.

Reactive compaction (reactive_compact) is the fallback. If the estimate didn't trigger but the model actually returns PROMPT_TOO_LONG, force-compact immediately, rebuild the model view, and continue to retry the current step. There's a retry limit; if exhausted, the agent takes the MODEL_RECOVERY_FAILED termination path.

Design Highlights

1. History and ModelView are separate

HistoryManager is an immutable fact log; ModelView is a per-step ephemeral view. Because they're separate, compaction never loses history, crashes can be fully recovered, and during debugging you can compare "what was stored" against "what the model actually saw."

2. Read-time projection, not write-time deletion

Compaction results are stored as a checkpoint (summary + split index), folded in dynamically each time history is read. This is safer than "delete messages on compact" — you can roll back at any time (clear the checkpoint and you're back to full history), and a compaction bug can never corrupt history.

3. Dual-path safety net

Proactive estimation + reactive exception catching: both can trigger compaction. Even if the estimation algorithm has error, the agent self-heals on a model error rather than crashing.

Summary

Design choice	Approach	Engineering value
History storage	Append-only fact log	Compaction never loses history; full crash recovery
Compaction method	Read-time projection + checkpoint	Rollback-safe; compaction bugs can't corrupt history
Token estimation	Dual-source max	Pessimistic; fires early rather than late
Trigger paths	Proactive estimation + reactive exception catch	Fallback when estimation fails; self-heals rather than crashes
ModelView	Message list + observable metadata	Can compare "what was stored" vs. "what the model saw"

About the Source Code

All analysis in this series is based on the open-source project MyCodeAgent.

git clone https://github.com/chendongqi/MyCodeAgent
cd MyCodeAgent
cp .env.example .env   # fill in your LLM API key
uv sync
uv run python main.py

Explore PrimeSkills — a curated marketplace of AI agents and skills, each validated against real enterprise workflows. No hype, just what actually works.

Visit my personal site for more insights and interesting products.

Open Source Project #163: OpenWiki — LangChain's Self-Maintaining Codebase Wiki CLI, Built for Agents

WonderLab — Thu, 27 Aug 2026 01:49:26 +0000

Introduction

"The self-maintaining wiki. Built for agents, explored by humans."

This is article #163 in the "One Open Source Project a Day" series. Today's project is OpenWiki — LangChain AI's open-source CLI for automatic codebase documentation generation and maintenance, 15,600 Stars, MIT license.

OpenWiki addresses a problem every engineering team has: written documentation goes stale, but undocumented codebases leave AI agents disoriented. OpenWiki's solution: let an agent write the docs, let an agent maintain them. The output is a Markdown wiki that lives in your repository — agents can read it as memory, and humans can explore it through an interactive visualization.

What You'll Learn

Two modes: the difference between a code wiki (for a repository) and a personal wiki (for personal knowledge)
Grounded Claims: how material facts are tracked back to exact code lines
Coding-agent integrations with Claude Code, Codex, and OpenCode
Support for 13 model providers and 9 knowledge-source connectors
Interactive node-graph visualization and static site export
CI auto-update (GitHub Actions / GitLab CI / Bitbucket Pipelines)

Prerequisites

Basic command-line experience
Familiarity with LLM APIs (API keys, model names)
Git workflow basics are helpful

Project Background

Overview

OpenWiki is built on Deep Agents, LangChain's deep agent framework. It's not a "generate once and forget" documentation tool — it's a documentation lifecycle management system:

Initialize: the agent reads source code and generates a structured Markdown wiki
Track: binds each material fact (Grounded Claims) to precise code lines
Update: when code changes, checks which facts' "evidence" changed and rewrites only affected pages
Integrate: maintains AGENTS.md and CLAUDE.md at the repo root so coding agents can find and read the wiki

Author / Team

Organization: LangChain AI
Primary language: TypeScript
License: MIT
Created: 2026-06-22

Project Stats

⭐ GitHub Stars: 15,600+
🍴 Forks: 1,133+
📄 License: MIT
📅 Created: 2026-06-22

Quick Start

Installation

# Requires Node.js 22 or newer
npm install -g openwiki

Generate a Wiki for the Current Repository

cd your-project
openwiki --init

The first run walks you through selecting a model provider (default: OpenAI + gpt-5.6-terra), entering your API key, and choosing a model. Documentation is written to openwiki/.

Update an Existing Wiki

# Detect code changes, update only affected pages
openwiki --update

Launch the Visualizer

# Open an interactive node graph in your browser
openwiki visualize

Two Modes

Mode	Documents	Writes to	Command
code (default)	Current repository source	`openwiki/` (inside the repo)	`openwiki --init`
personal	Connected sources (Notion/Gmail/Slack, etc.)	`~/.openwiki/wiki/`	`openwiki personal --init`

Code mode is built for teams: the wiki is committed alongside code, CI keeps it current, and AGENTS.md + CLAUDE.md give coding agents a direct entry point to the documentation.

Personal mode is built for individuals: aggregates knowledge scattered across tools (Notion, email, X/Twitter, Hacker News…) into a local wiki that an AI agent structures into a knowledge graph.

Core Mechanism: Grounded Claims

This is what sets OpenWiki apart from ordinary documentation generators.

A typical documentation tool only tracks "when was this page last generated." OpenWiki goes further: it tracks every material fact in the documentation, down to the exact source lines that back it up.

// A Claim's structure (stored under openwiki/.claims/)
{
  id: "claim-abc123",
  proposition: "AuthMiddleware returns 401 on failure without rethrowing",
  evidence: "repo://src/middleware/auth.ts#L40-L82",
  evidence_version: "git-sha-of-that-commit"
}

Claims cover:

Function and module behavior and responsibilities
Architectural relationships and data flows
Invariants and failure semantics
Configuration requirements and security boundaries

Before an update, OpenWiki checks the evidence for every Claim before even deciding whether anything needs regenerating:

openwiki --update execution order:
1. Check whether each Claim's evidence (to exact line numbers) has changed
2. If evidence changed → that page needs rewriting, regardless of how much code changed
3. If evidence unchanged → the page's content probably doesn't need updating
4. After a page completes, Claims are atomically persisted (no partial updates)

This handles a subtle correctness problem: lots of code can change while a key fact stays correct, or a small refactor can silently invalidate a documented invariant. Line-level evidence tracking makes update decisions much more precise.

Resumable Page-Job Architecture

OpenWiki's generation flow is not a single monolithic batch — it's a stateful page queue:

begin → submit_plan → next_page → submit_page → ... → finish

begin: starts a new generation run, records run state to openwiki/.run.json
submit_plan: submits the page plan (which topics need documentation)
next_page / submit_page: works page by page, atomically persisting Claims on each completion
finish: final verification, deletes .run.json, marks the run complete

Resumability: if generation is interrupted mid-run (e.g., CI timeout), rerunning on the same checkout picks up where it left off without redoing completed pages.

CI note: ephemeral CI runners that start fresh on every run don't retain .run.json after failure, so a failed run restarts from the beginning. Resume capability only works on persistent checkouts.

Coding-Agent Integrations: Let Claude Code Write the Docs

OpenWiki can run inside Claude Code, Codex, or OpenCode, delegating repository research and writing to the coding agent while OpenWiki manages the Claims lifecycle and persistence.

# Install the integration for your coding agent
openwiki integrations install claude
openwiki integrations install codex
openwiki integrations install opencode

After installation, restart the coding agent, open the repository, and ask:

Initialize this repository's OpenWiki from the current source and tests.

Or to update an existing wiki:

Update this repository's OpenWiki for changes since its last successful run.

The integration exposes five operations: openwiki_begin, openwiki_submit_plan, openwiki_next_page, openwiki_submit_page, and openwiki_finish. The coding agent submits a complete intended Claim set with each page; OpenWiki internally creates, updates, preserves, and retracts Claims — and refuses to call finish until the final state is fully durable.

The advantage of this integration: it uses the coding agent's already-authenticated model session, so separate OpenWiki provider credentials are not required.

9 Knowledge-Source Connectors (personal mode)

Connector	Source	Auth
`custom-mcp`	Any HTTP/stdio MCP server	MCP config
`git-repo`	Local git repositories	None
`notion`	Notion pages	OAuth (hosted MCP)
`gmail`	Gmail	Google OAuth
`slack`	Slack conversations	Slack OAuth
`x`	X/Twitter timeline, bookmarks	X OAuth 2.0 (PKCE)
`web-search`	Web search via Tavily	`TAVILY_API_KEY`
`hackernews`	HN feed + search APIs	None
`langsmith`	LangSmith run traces	`OPENWIKI_LANGSMITH_API_KEY`

The LangSmith connector is an exception: it's for code mode, not personal. It pulls recent LangSmith traces (tool calls, outcomes, latency) for chosen projects and injects that runtime context into the codebase wiki — so documentation reflects how the code actually behaves at runtime, not just what the source says.

The same connector can be configured multiple times as separate instances (e.g., two Web Search sources: one tracking AI research, another tracking NBA news), stored as web-search-1 and web-search-2.

13 Model Providers

Provider	Credential
OpenAI (default)	`OPENAI_API_KEY`
OpenAI (ChatGPT login)	Browser OAuth, uses ChatGPT plan
Anthropic	`ANTHROPIC_API_KEY`
Gemini (AI Studio)	`GEMINI_API_KEY`
Gemini Enterprise (Vertex AI)	Google ADC, keyless
AWS Bedrock	IAM credentials
GitHub Copilot	GitHub CLI session
OpenRouter	`OPENROUTER_API_KEY`
Nebius / Fireworks / Baseten / NVIDIA NIM	Provider API key
OpenAI-compatible (LiteLLM/Ollama/LM Studio)	Base URL + key

Local model example with Ollama:

OPENWIKI_PROVIDER=openai-compatible
OPENAI_COMPATIBLE_API_KEY=ollama
OPENAI_COMPATIBLE_BASE_URL=http://localhost:11434/v1
OPENWIKI_MODEL_ID=llama3.2

Interactive Visualization

# Open an interactive node graph in your local browser
openwiki visualize

# Export as a static site (GitHub Pages, MkDocs, etc.)
openwiki visualize openwiki --export docs/openwiki-visualizer

The visualization is a live node graph + Markdown reader side by side: nodes are wiki pages, edges are links between them, clicking a node loads that page in the reader on the right.

The static export contains index.html, client.js, styles.css, and graph.json — deploy directly to any static host.

CI Auto-Update

Copy the example workflow file into your repository:

# .github/workflows/openwiki-update.yml
# Runs on a schedule, checks for code changes,
# opens a documentation PR if the wiki needs updating

Official examples for GitHub Actions, GitLab CI, and Bitbucket Pipelines are all in the examples/ directory. The auto-update flow: detect changes → update wiki → open a PR. It doesn't merge automatically — the documentation change stays in a PR for human review.

Your Wiki Stays Yours

OpenWiki's core design principle: your documentation belongs to you.

The wiki is plain Markdown files committed in your repository — no external service dependency
openwiki/INSTRUCTIONS.md is user-authored; normal update runs never overwrite it
OpenWiki only touches its own block (…) inside AGENTS.md and CLAUDE.md, leaving the rest untouched
Claims files live in openwiki/.claims/, versioned alongside Markdown in git history
No-op runs don't churn docs — only .last-update.json is updated, page content is left alone

Resources

🌟 GitHub: langchain-ai/openwiki
📦 npm: openwiki
🔧 Deep Agents: langchain-ai/deepagentsjs
📋 OKF spec: Open Knowledge Format v0.2

Summary

OpenWiki represents a different way of thinking about documentation engineering: the documentation quality problem isn't that the writing is bad — it's that nothing maintains what was written.

Three things worth noting:

Grounded Claims is the key innovation. Traditional documentation tools have "last updated" timestamps at the page level. OpenWiki takes granularity down to "proposition + code line": each fact knows which lines are its evidence, and when those lines change, the system knows exactly what to rewrite. This transforms updates from "regenerate the whole file" to "only rewrite the propositions whose evidence changed" — far more precise.

Using a coding agent as a writer, not a Q&A tool. OpenWiki's Claude Code/Codex integration puts the coding agent in the role of "repository researcher and documentation author," not a one-shot text generator. The agent has full repository access, works through a page queue incrementally, persists each result, and the whole process is resumable.

The OKF format is a bet on interoperability. Emitting Google Open Knowledge Format v0.2 means the wiki isn't locked to this tool — any OKF-aware system can read it. This is an explicit anti-vendor-lock-in design choice.

If you're building an AI-native engineering team and need living documentation that automatically tracks code changes, OpenWiki is the most complete open-source solution available today.

Explore PrimeSkills — a curated marketplace of AI agents and skills, each validated against real enterprise workflows. No hype, just what actually works.

Visit my personal site for more insights and interesting products.