DEV Community: Harish Kotra (he/him)

Mycelium: A Multi-Agent Memory Mesh with Provenance, Trust, and Contradiction Resolution

Harish Kotra (he/him) — Wed, 01 Jul 2026 18:24:48 +0000

TL;DR — I built an open-source TypeScript framework where AI agents have persistent, shared memory backed by a knowledge graph. Agents can sync facts with provenance tracking, detect and resolve contradictions via LLM, and build trust over time through a sync protocol. The engine is pluggable across five LLM providers and uses Cognee's Rust-native graph/vector backend.

The Problem

Most AI agents today are stateless. They get a prompt, produce an answer, and forget everything the moment the conversation ends. Even agents that use RAG or chat history lack:

Multi-agent shared memory — two agents working on the same domain have no way to share or reconcile their knowledge.
Provenance — if Agent A learns something from Agent B, there's no chain-of-custody.
Contradiction handling — conflicting facts quietly overwrite each other with no signal.
Trust evolution — agents have no mechanism to decide which peers to believe.

Mycelium was built to solve these four problems.

Architecture

                      ┌──────────────────┐
                      │    Dashboard     │  Next.js 15 / React 19
                      │  d3-force viz   │
                      └────────┬─────────┘
                               │ HTTP / data
               ┌───────────────┼────────────────┐
               │               │                │
         ┌─────▼──────┐ ┌──────▼──────┐ ┌──────▼──────┐
         │   Agent    │ │   Agent     │ │   Agent     │
         │ (planner)  │ │ (assistant) │ │   (cli)     │
         │ dataset_a  │ │ dataset_b   │ │  dataset_c  │
         └─────┬──────┘ └──────┬──────┘ └──────┬──────┘
               │               │                │
               ▼               ▼                ▼
         ┌──────────────────────────────────────────┐
         │            Sync Engine                   │
         │  provenance · subscription · trust       │
         │  structural-diff · contradiction         │
         └──────────────────┬───────────────────────┘
                            │
         ┌──────────────────▼───────────────────────┐
         │         Cognee Backend (Rust)            │
         │  Kuzu graph · brute-force vectors        │
         │  mock embeddings · 5 LLM providers       │
         └──────────────────────────────────────────┘

Every agent maps to one Cognee dataset. The sync engine brokers cross-dataset fact exchange with provenance tagging. The diff engine runs on every improve() call to detect structural changes, embedding drift, and logical contradictions.

Tech Stack

Layer	Technology	Why
Language	TypeScript (ES2022)	Type safety for graph data structures
Runtime	Node.js via `tsx`	Fast TypeScript execution without build step
Monorepo	pnpm workspaces	Shared types, independent packages
Graph DB	Kuzu (embedded, via Cognee)	Native graph storage, no server needed
Vector DB	Brute-force (via Cognee)	Zero-config for prototyping
LLM Config	OpenAI / Groq / LM Studio / Ollama / custom	Pluggable via env var
Graph Memory	Cognee Rust addon (`@cognee/cognee-ts`)	Graph extraction, triplet indexing
Dashboard	Next.js 15 / React 19 / d3-force / Recharts	Visualization layer
CLI	Node.js via `tsx`	Ad-hoc agent spawning and sync simulation

Core Concepts

1. Agent

Each agent wraps a Cognee dataset and exposes a high-level API:

const planner = new Agent("travel_planner", "dataset_travel", client);

// Ingest facts (with optional provenance)
await planner.remember({ type: "text", text: "Alice loves flying." });

// Query the graph
const results = await planner.recall("What does Alice think about flying?");

// Improve memory: Cognee re-indexes + diff + contradiction detection
const improved = await planner.improve({ autoResolve: true });

2. CogneeClient

A configurable wrapper around the Cognee Rust SDK that handles the messy env-var setup (Cognee only accepts "openai" as LLM_PROVIDER internally, even when routing to Groq/Ollama):

const client = await CogneeClient.create({
  llmProvider: "groq",
  llmModel: "llama-3.1-8b-instant",
});
// client.llmConfig → { provider: "groq", model: "llama-3.1-8b-instant", endpoint: "...", apiKey: "..." }

3. Diff Engine (Three-Pass)

Every improve() runs three diff passes:

Pass 1 — Structural Diff (structuralDiff.ts):

const structural = diffSnapshots(before, after);
// → { nodes: { added, removed, modified }, edges: { added, removed, modified } }

Pure set-diff on node/edge IDs. A node is "modified" if its properties or type changed. No I/O, no LLM call.

Pass 2 — Embedding Drift (driftDetector.ts):

function cosineDistance(a: number[], b: number[]): number {
  // 1 - (a·b / (|a|·|b|))
}
const drifts = detectDrift(before, after, { threshold: 0.15 });

Flags nodes whose embedding shifted beyond a threshold — useful for detecting semantic drift after re-indexing.

Pass 3 — Contradiction Detection (contradictionDetector.ts):

const contradictions = await detectContradictions(client, dataset, after, before);
// Sends each differing node pair to an LLM with a structured prompt
// → [{ nodeLabel, existingStatement, incomingStatement, isContradiction, confidence }]

The combined result:

export interface MemoryDiffResult {
  before: GraphSnapshot;
  after: GraphSnapshot;
  structural: StructuralDiff;
  drifts: DriftResult[];
  contradictions: ContradictionResult[];
  summary: {
    nodesAdded, nodesRemoved, nodesModified,
    edgesAdded, edgesRemoved, edgesModified,
    driftsDetected, contradictionsDetected,
  };
}

4. Contradiction Resolution

Once contradictions are detected, a pure decision engine resolves them:

const resolved = resolveContradictions(contradictions, {
  strategy: "keep_newer",     // or "keep_higher_trust" | "flag_all"
  confidenceThreshold: 0.8,
});
// → [{ nodeLabel, resolution: "kept_incoming", confidence }]

The resolver is stateless — it returns a decision. The caller is responsible for executing it (e.g., forgetting the losing fact from the graph).

5. Sync Protocol

The sync engine is where agents share knowledge:

const engine = new SyncEngine({
  trustStore,
  autoMergeThreshold: 0.6,  // trust ≥ 0.6 → auto-merge
});

const run = await engine.syncFromSource(
  client,                    // Cognee backend
  plannerDataset,            // subscriber
  plannerId,                 // subscriber agent ID
  assistantDataset,          // source dataset
  assistantId,               // source agent ID
  assistantFacts,            // string[]
);

// Accept or reject post-sync
await acceptSync(run.id, engine, trustStore, plannerId);

Trust model: Starting at 0.5, accepting a sync adds +0.05, rejecting deducts -0.2. This asymmetry nudges agents toward caution — rejection is weighted 4x more than acceptance.

6. Provenance

Every fact that crosses agent boundaries is tagged:

tagWithProvenance(text, { sourceAgentId, factId, timestamp })
// → '__provenance__:{"sourceAgentId":"assistant","factId":"src_123","timestamp":…}\n fact text here'

This prefix survives Cognee's text storage, so extractProvenance() can retrieve the chain of custody on recall.

LLM Provider Configuration

LLM_PROVIDER=groq                          # openai | groq | lm-studio | ollama | custom-openai-compatible
LLM_ENDPOINT=https://api.groq.com/openai/v1
LLM_API_KEY=gsk_...
LLM_MODEL=llama-3.1-8b-instant

Swap providers with one env var — no code changes:

# OpenAI
USE_LLM_PROVIDER=openai pnpm demo

# Local LM Studio (requires running LM Studio server)
USE_LLM_PROVIDER=lm-studio pnpm demo

# Local Ollama
USE_LLM_PROVIDER=ollama pnpm demo

The resolveLlmConfig() function merges explicit input > env vars > provider presets > hard defaults. Legacy aliases OPENAI_URL / OPENAI_TOKEN still work as fallbacks.

The Travel Assistant Demo

The end-to-end demo at examples/travel-assistant-demo/:

pnpm --filter travel-assistant-demo demo

Creates two agents: Travel Planner and Personal Assistant
Seeds the Planner with "Alice enjoys flying" and the Assistant with "Alice is afraid of flying" — a deliberate contradiction
Runs contradiction detection via LLM call
Syncs the Assistant's facts into the Planner's dataset with provenance tags
Runs improve() which captures before/after snapshots, runs the full diff pipeline, and optionally auto-resolves contradictions
Accepts the sync, bumping trust from 0.5 → 0.55

Directory Structure

packages/
  core/          src/    — Agent, CogneeClient, diff, LLM config, types
  cli/           src/    — inspectDiff, simulateSync, spawnAgent
  dashboard/     src/    — Next.js 15 app with d3-force network graph
  diff-engine/           — (planned: standalone npm package)
  sync-protocol/         — (planned: standalone npm package)
  trust/                 — (planned: standalone npm package)
examples/
  travel-assistant-demo/ — End-to-end demo script

How to Fork and Contribute

Clone and install:

   git clone https://github.com/harishkotra/mycelium
   cd mycelium
   pnpm install

Set up your LLM provider:

   cp packages/core/.env.example packages/core/.env
   # Edit .env with your API key

Run the demo:

   pnpm --filter travel-assistant-demo demo

Run tests:

   npx tsx packages/core/test-llm.mjs           # 36 LLM config tests
   npx tsx packages/core/test-improve.ts         # 32 improve/resolver tests

Feature Ideas for Contributors

Feature	Difficulty	Description
Persist sync run history	Easy	Currently in-memory; write to JSON or SQLite
Agent.sync() method	Medium	Add `agent.sync(peerAgent, engine)` that exchanges diffs with peer
WebSocket live sync	Medium	Replace polling with real-time sync events between agents
Dashboard → real backend	Medium	Swap the mock data layer with `CogneeClient` API calls, one screen at a time
Diff engine as npm package	Easy	Extract `@mycelium/diff-engine` as a standalone package
Sync protocol as npm package	Easy	Extract `@mycelium/sync-protocol` as a standalone package
Cross-dataset contradiction scan	Medium	Scan all connected datasets for contradictions, not just before/after improve
Vector DB persistence	Medium	Brute-force vector DB loses index after restart; add proper persistence
CLI: `mycelium agent spawn`	Medium	Interactive CLI to create, seed, and inspect agents
Ollama embedding support	Hard	Cognee only supports OpenAI-compatible embeddings; add Ollama path
Trust decay	Medium	Trust scores decay over time if no recent sync activity
Fact expiry / TTL	Medium	Facts auto-expire after a configurable TTL, with notification
Git-like merge conflict UI	Hard	Show diff three-pane UI for sync review in dashboard

Design Principles

No hidden side-effects — Pure functions (diffSnapshots, resolveContradictions, cosineDistance) are explicitly marked and easily testable
Env-var-first config — One env var (LLM_PROVIDER) changes the entire LLM stack; no code changes needed
Provider abstraction — Contradiction detector calls raw fetch() to OpenAI-compatible APIs rather than importing per-provider SDKs
Cognee as implementation detail — The Agent class wraps Cognee internally; consumers don't import Cognee types directly

Code & more: https://www.dailybuild.xyz/project/180-mycelium

Hover now supports Custom Actions

Harish Kotra (he/him) — Tue, 30 Jun 2026 18:32:28 +0000

Hover already made it easy to ask AI to explain, rewrite, summarize, translate, or fix text without leaving the app you were using. This update adds something bigger: reusable custom actions that let you turn your own prompts into one-tap workflows.

With Custom Actions, Hover becomes less of a generic assistant and more of a personal command palette for writing, editing, and thinking.

What changed

We added a full Custom Actions system across Hover:

A new Actions tab in Settings for creating, editing, duplicating, and deleting saved prompts.
Reusable action cards that can be named, given an icon, and paired with any instruction you want.
Local persistence through the macOS defaults system, so your actions stay on your Mac.
Integration into the floating Hover panel, where custom actions appear alongside the built-in presets.
A streamlined editor for defining prompt text and choosing an icon.

Why this matters

Built-in presets are useful, but they only cover the common cases. Custom Actions make Hover adaptable to the way you actually work.

You can now save prompts for tasks like:

"Translate this into natural French"
"Turn this into a concise commit message"
"Rewrite this for a more professional tone"
"Extract action items from this text"
"Review this code for bugs and edge cases"
"Summarize this email and draft a reply"

The point is not just convenience. It is consistency. Instead of rewriting the same prompt every time, you create a durable action once and reuse it whenever you need it.

How it works

Custom Actions are stored locally and loaded into Hover at launch. In the floating panel, they appear alongside Hover's built-in actions like Explain, Rewrite, Summarize, Reply, Translate, Fix grammar, and Ask.

The flow is simple:

Select text anywhere on your Mac.
Trigger Hover with Command-Shift-X or the optional triple right-click gesture.
Choose a built-in action or one of your saved Custom Actions.
Hover sends the selected text plus your saved prompt to the model.
Review the result, then copy, paste, or replace the original selection.

That keeps the interaction fast while still giving you room to build your own workflows.

What we built

This update touched the app from top to bottom:

Settings experience

The Settings window now has a dedicated Custom Actions section. From there, you can:

Add a new action
Edit an existing one
Duplicate an action as a starting point for a variant
Delete actions you no longer need
Pick from a set of SF Symbol icons for easier scanning

Floating panel integration

The floating response panel now merges built-in actions and saved custom actions into a single action grid. That means your personal workflows are available exactly where you expect them: at the moment you trigger Hover.

Storage

Custom Actions are stored locally in UserDefaults under a dedicated key. No account is required, and no Hover server is involved.

App wiring

The feature is wired through the app environment so the settings window, the floating panel, and the action store all share the same state cleanly.

Privacy and control

Hover is built to stay local-first. This update does not change that.

Your custom prompts are saved on-device, and the app still only sends whatever text or context you explicitly choose to the provider you configured. If you use a local model through LM Studio or Ollama, your workflows can remain entirely local.

Who this helps

This feature is useful if you:

Reuse the same AI prompts every day
Want a faster workflow for writing or editing
Use Hover for multiple work modes, not just quick rewrites
Prefer saving prompt templates instead of keeping them in notes or snippets
Want a menu bar AI app that adapts to your habits rather than forcing a fixed set of actions

Example workflows

Here are a few good first Custom Actions:

Rewrite as a concise product update
Translate to French, keep the tone natural
Summarize in three bullets
Write a code review comment with risks and fixes
Draft a polite reply that confirms next steps
Extract tasks, owners, and deadlines

If you work in support, product, engineering, sales, or writing, those templates can remove a surprising amount of repetition.

What comes next

Custom Actions are the first step toward a more flexible Hover. The goal is to keep the app simple at the surface while letting power users shape it around their own workflows.

If you want to try Hover with reusable prompts, update to the latest build and open Settings to create your first action.

Get Hover here: https://hoverformac.com

Building Phone Cleaner: A Desktop App to Manage WhatsApp Media via ADB

Harish Kotra (he/him) — Mon, 29 Jun 2026 05:44:44 +0000

WhatsApp groups are a black hole of phone storage. Family chats, office groups, neighborhood watch — everyone sends photos and videos, and they pile up silently in your phone's hidden WhatsApp directories. Before you know it, you've got 5,000+ files eating up 15 GB of space, and there's no good way to triage them on the phone itself.

Scrolling through a grid of thumbnails on a 6-inch screen is painful. So I built Phone Cleaner — a desktop Electron app that connects to your Android phone via ADB, scans your WhatsApp media, and lets you review everything on your Mac in a proper grid with filtering, sorting, and one-click deletion.

This post walks through the architecture, the key engineering decisions, and the lessons learned.

The Architecture

The app has three layers:

┌─────────────────────────────────────┐
│        Renderer (Chromium)          │
│   Vanilla JS + CSS + Intersection   │
│   Observer for lazy thumbnails      │
├─────────────────────────────────────┤
│     IPC Bridge (preload.js)         │
│     contextBridge for security      │
├─────────────────────────────────────┤
│      Main Process (Node.js)         │
│   ADB subprocess, file cache,       │
│   persistent history (JSON)         │
└──────────────────┬──────────────────┘
                   │ adb (USB)
            ┌──────┴──────┐
            │   Phone     │
            │  Android    │
            └─────────────┘

The renderer is vanilla HTML/CSS/JS — no framework. The main process runs Node.js and spawns ADB subprocesses. Communication happens through Electron's IPC with contextIsolation: true for security.

Why Electron?

I chose Electron over Tauri for one reason: ADB is already a CLI tool. Electron's Node.js main process can call execFile on adb just like any other system command. Tauri would've required wrapping ADB in Rust bindings, which was overkill for a utility app.

The trade-off is bundle size (~200MB with Electron), but for a developer tool that runs on a Mac with a phone connected via USB, that's acceptable.

Scanning WhatsApp Media

The most interesting engineering challenge was scanning. WhatsApp stores media at one of two paths depending on Android version:

Android 10 and below: /sdcard/WhatsApp/Media/WhatsApp Images/
Android 11+ (scoped storage): /sdcard/Android/media/com.whatsapp/WhatsApp/Media/WhatsApp Images/

My app probes both paths, then for each directory runs a single ls -laR (recursive list) via ADB. This is a single round-trip per top-level directory, regardless of how many files are inside:

const out = await adb(["-s", deviceId, "shell",
  `ls -laR "${dir}" 2>/dev/null || true`,
]);

The output looks like this:

/sdcard/WhatsApp/Media/WhatsApp Images:
-rw-rw---- 1 u0_a147 media_rw 123456 2024-01-15 10:30 IMG-20240115-WA0001.jpg
-rw-rw---- 1 u0_a147 media_rw 654321 2024-01-14 09:20 IMG-20240114-WA0002.jpg

/sdcard/WhatsApp/Media/WhatsApp Images/Sent:
-rw-rw---- 1 u0_a147 media_rw 98765 2024-01-13 08:00 IMG-20240113-WA0003.jpg

Parsing this is straightforward: lines starting with / are directory headers, lines starting with total are summaries, and everything else is a file entry. The parseLsLine() function handles two date formats (modern YYYY-MM-DD HH:MM and legacy Mon DD YYYY) since Android's toybox ls varies across versions.

I initially tried using find + per-file stat calls, but that meant one ADB round-trip per file — completely unusable at scale. With ls -laR, it's just 2-4 ADB calls total, regardless of file count.

Lazy-Loaded Thumbnails

Pulling 1,000+ images from a phone over USB is slow. Pulling them all upfront would make the UI unusable. The solution: IntersectionObserver with a 300px root margin.

Only images that scroll into view (or are about to) get pulled from the phone:

const thumbObs = new IntersectionObserver((entries) => {
  for (const entry of entries) {
    if (!entry.isIntersecting) continue;
    loadThumb(entry.target);
    thumbObs.unobserve(entry.target);
  }
}, { rootMargin: '300px' });

When a thumbnail is needed, the app runs adb pull to copy the file to a local cache directory, then sets it as the <img> source. Subsequent visits load instantly from the cache.

Video files never pull a thumbnail — they render a play-button icon over a dark placeholder. Full video files are only pulled on double-click (preview).

The Keep/Delete Workflow

A key UX insight: users don't know what they want to delete until they know what they want to keep. So the app has two distinct actions:

Ctrl+Click → marks a file as Keep (green border + badge)
Regular Click → selects a file for Deletion (red border)

This lets users quickly protect important files, then invert the selection to delete everything else.

Deletion happens via adb shell rm -f — one command per batch. I tried batching all paths into one rm call, but shell argument length limits on Android can be an issue with 1000+ files, so there's a per-file fallback.

Delete History

Every deletion session is logged to a local JSON file at:

~/Library/Application Support/phone-cleaner/delete-history.json

Each entry tracks:

Timestamp
Number of files deleted
Total bytes recovered
Image/video breakdown

This gives users a sense of accomplishment and a record of what was cleaned.

Design Decisions

No Build Step

The renderer is vanilla JS with no framework, bundler, or transpiler. This keeps the dev loop instant: edit file → reload Electron. No webpack, no vite, no babel.

No Semicolons

Personal preference. The codebase is consistent — no semicolons, 2-space indent, single quotes for strings.

Dark Theme Only

A utility tool for developers and power users. Dark mode reduces eye strain during long scanning sessions. A theme toggle would be a good first PR.

Possible Improvements

If you want to contribute, here's what's next:

Video thumbnails — use ffmpeg to extract a frame from videos for proper thumbnail previews
Duplicate detection — hash-based grouping to find identical files
Wireless ADB — support adb connect over the network
Export before delete — pull selected files to a local folder as a backup
Undo/trash — soft-delete to a trash directory instead of permanent removal

Try It

The complete source code is on GitHub: github.com/harishkotra/phone-cleaner

git clone https://github.com/harishkotra/phone-cleaner.git
cd phone-cleaner
npm install
npm start

All you need is Node.js, a USB cable, and an Android phone with USB Debugging enabled.

Screenshots

Code & more: https://www.dailybuild.xyz/project/178-phone-cleaner

Built with OpenCode

Consumed Context: 136,163 tokens (68% used)
Total Spent: $0.00 spent
_Model: _ DeepSeek V4 Flash

Branch Agent: Git-Style Branching for LLM Conversations

Harish Kotra (he/him) — Sun, 28 Jun 2026 15:04:55 +0000

Fork, branch, and merge AI conversations like code — with different models, prompts, and providers in each parallel timeline.

The Problem

When experimenting with LLMs, you've probably done this: tweak the system prompt, rerun the conversation, compare outputs side-by-side in two browser tabs, manually copy-paste the better response, and repeat. It's messy, non-reproducible, and there's no version history.

What if LLM conversations had the same branching model as Git?

The Idea

A conversation is a tree, not a list. Every message can be a fork point. New branches inherit the full context up to that point via pointer references (O(1), no copying). Each branch gets its own agent configuration — system prompt, model, provider, temperature, tools. Branches can be compared side-by-side and merged back via an AI "Judge Agent."

Architecture

┌──────────────┐      Convex hooks       ┌──────────────────┐
│  Next.js UI  │ ◄─────────────────────► │  Convex Database │
│  (React 19)  │     useQuery/mutation   │  (workspaces,    │
└──────┬───────┘                         │   branches, msgs)│
       │ HTTP POST /chat (SSE stream)    └──────────────────┘
       ▼
┌──────────────────┐
│  Python FastAPI  │  ← Agno Agent SDK
│  Agno Service    │     Creates agent per request with branch config
└──────┬───────────┘
       │ OpenAI-compatible API
       ▼
┌──────────────────┐
│  Any LLM Provider│  ← OpenAI, Together, Groq, Ollama...
└──────────────────┘

Why Convex?

Convex provides:

Reactive queries — UI subscribes to data via WebSockets; when the agent streams a response token-by-token, the UI re-renders automatically
ACID mutations — forkBranch is atomic and isolated
Deterministic actions — the chatWithAgent action runs outside the transaction but can call internal queries/mutations safely
Type-safe — full TypeScript types generated from the schema

Why Agno?

Agno is a Python-native agent framework that supports:

Multiple model providers through a unified interface
Tool integration (web search, calculator, file I/O, etc.)
Streaming responses with intermediate events (tool calls, reasoning steps)
Structured output via Pydantic schemas

The Branch Model (Database Schema)

The schema is deliberately relational to support tree traversal:

// convex/schema.ts
export const agentConfigSchema = v.object({
  systemPrompt: v.optional(v.string()),
  model: v.optional(v.string()),
  baseUrl: v.optional(v.string()),   // per-branch provider URL
  apiKey: v.optional(v.string()),    // per-branch API key
  tools: v.optional(v.array(v.string())),
  temperature: v.optional(v.number()),
  maxTokens: v.optional(v.number()),
});

export const branches = defineTable({
  workspaceId: v.id("workspaces"),
  name: v.string(),
  parentBranchId: v.optional(v.id("branches")),
  snapshotMessageId: v.optional(v.id("messages")),
  agentConfig: v.optional(agentConfigSchema),
  isMerged: v.optional(v.boolean()),
  mergeSummary: v.optional(v.string()),
})
  .index("by_workspace", ["workspaceId"])
  .index("by_parent", ["parentBranchId"]);

export const messages = defineTable({
  branchId: v.id("branches"),
  parentMessageId: v.optional(v.id("messages")),
  role: v.union(v.literal("user"), v.literal("assistant"), v.literal("system")),
  content: v.string(),
  metadata: v.optional(messageMetadataSchema),
})
  .index("by_branch_created", ["branchId", "createdAt"]);

The key insight: snapshotMessageId on a branch points to the message where it forked. History reconstruction walks parentBranchId → snapshotMessageId pointers recursively. This makes forks O(1) in storage — no message duplication.

Forking is O(1)

export const forkBranch = mutation({
  args: {
    sourceBranchId: v.id("branches"),
    snapshotMessageId: v.id("messages"),
    newBranchName: v.string(),
    agentConfig: v.optional(agentConfigSchema),
  },
  handler: async (ctx, args) => {
    // Just create a new branch with pointer references
    // No messages are copied
    return await ctx.db.insert("branches", {
      workspaceId: sourceBranch.workspaceId,
      name: args.newBranchName,
      parentBranchId: args.sourceBranchId,
      snapshotMessageId: args.snapshotMessageId,
      agentConfig: args.agentConfig ?? sourceBranch.agentConfig,
      createdAt: Date.now(),
    });
  },
});

History Traversal

When a user sends a message, the action fetches the full context by walking the branch tree:

export const internalGetBranchHistory = internalQuery({
  handler: async (ctx, args) => {
    const branch = await ctx.db.get(args.branchId);
    const myMessages = await ctx.db
      .query("messages")
      .withIndex("by_branch_created", (q) => q.eq("branchId", args.branchId))
      .order("asc")
      .collect();

    if (!branch.parentBranchId || !branch.snapshotMessageId) {
      return myMessages;  // root branch, just our messages
    }

    // Walk up the parent tree to the snapshot point
    const parentMessages = await traverseToSnapshot(
      ctx, branch.parentBranchId, branch.snapshotMessageId
    );
    return [...parentMessages, ...myMessages];
  },
});

Streaming Agent Responses

The chatWithAgent action sends the full history to the Python Agno service, which streams tokens back via SSE:

// Convex action reads branch config, sends to Agno service
const agnoPayload = {
  messages: conversationMessages,
  system_prompt: branch.agentConfig?.systemPrompt,
  model: branch.agentConfig?.model,
  base_url: branch.agentConfig?.baseUrl,
  api_key: branch.agentConfig?.apiKey,
  tools: branch.agentConfig?.tools,
  temperature: branch.agentConfig?.temperature,
  stream: true,
};

// Parse SSE events and update message content in real-time
for await (const sseEvent of sseReader) {
  if (parsed.type === "content" && parsed.content) {
    fullContent += parsed.content;
    await ctx.runMutation(internalUpdateMessageStream, {
      messageId: assistantMessageId,
      content: fullContent,
    });
  }
}

Each token delta updates the Convex document, which triggers the reactive useQuery hook on the frontend — the UI streams the response smoothly.

The Python Agno Service

# agno_service/agent_handler.py
def create_agent(
    system_prompt: str = None,
    model_name: str = None,
    base_url: str = None,
    api_key: str = None,
    tool_names: list[str] = None,
    temperature: float = None,
    max_tokens: int = None,
) -> Agent:
    model = _resolve_model(
        model_name or AGNO_DEFAULT_MODEL,
        temperature, max_tokens,
        base_url=base_url, api_key=api_key,
    )
    tools = _resolve_tools(tool_names or [])
    return Agent(
        model=model,
        tools=tools or None,
        instructions=[system_prompt] if system_prompt else None,
    )

The service creates a fresh Agent per request — no state leakage between branches. Each branch can point to a completely different provider.

UI: Side-by-Side Compare

The compare view lets you see two branches at the same time. This is particularly useful when testing different system prompts or models against the same conversation history:

<CompareView
  leftBranch={{ name: "main (GPT-4o)", messages }}
  rightBranch={{ name: "fork (Llama)", messages: compareMessages }}
/>

Both panels scroll independently, and messages from forked history are tagged with their source branch name.

Merging with a Judge Agent

When a fork has answered its questions, you can merge it back into the parent branch. The mergeWithJudge action sends both branches' histories to an Agno agent with a "judge" prompt that summarizes the key learnings:

You are a merge judge. Summarize the key learnings, differences, and insights
from the SOURCE branch, and produce a concise merge summary.

The summary is inserted as a system message in the target branch, and the source branch is marked as merged.

Running the Project

# Three terminals:
npx convex dev                    # Convex backend
cd agno_service && python3 main.py  # Agno Python service
npm run dev                       # Next.js frontend

Or one command: ./start.sh

What's Next

This project is a cookbook / reference implementation. Here are ideas for extending it:

Diff view — highlight diverging messages between branches
Prompt playground — preview how all branches would respond to a draft message
Export/import branches — share experiments as JSON files
Auto-resolve merge conflicts — when both branches modified the same context, use the Judge Agent to reconcile
Branch templates — pre-configured branches for specific tasks (research, code review, creative writing)

Tech Stack Summary

Component	Technology	Why
Database	Convex	Reactive WebSockets, ACID, type-safe
Agent SDK	Agno (Python)	Multi-provider, streaming, tools
Frontend	Next.js 15 + React 19	App Router, server components
Styling	Tailwind CSS v4 + shadcn/ui	Utility-first, dark mode
AI Provider	Any OpenAI-compatible API	BYO provider

Code & more: https://www.dailybuild.xyz/project/177-convex-branch-agent

slacktag-oss Part 2: Three Features That Make Memory Actually Useful

Harish Kotra (he/him) — Sat, 27 Jun 2026 18:12:30 +0000

A follow-up to Building a Slack Bot That Actually Remembers. We shipped the foundation — a Slack bot with persistent semantic memory via Mem0. Now we're going deeper: three new features that turn memory from a novelty into something your team will rely on daily.

What we shipped in Part 1

Quick recap: slacktag-oss is an open-source Slack bot backed by Mem0 for persistent semantic memory. Instead of a rolling message window, it extracts facts from every conversation and surfaces them when relevant — even weeks later. Channel memory, thread memory, DM memory, all correctly scoped.

In Part 1 we covered the core architecture. This post is about what we added next, and why each one required Mem0 specifically — these aren't features you could bolt onto a stateless bot.

Feature 1: `!summarize`

The simplest addition, but surprisingly useful.

!memory already existed — it dumps a raw numbered list of every fact stored in the current scope. That's fine for debugging, but nobody wants to read a raw fact dump in a Slack channel. !summarize takes the same data and asks the LLM to synthesize it into a readable paragraph.

def _summarize_memories(memories: list, scope_label: str) -> str:
    if not memories:
        return "Nothing stored yet for this scope."
    raw = _format_memories_for_display(memories)
    prompt = [
        SystemMessage(content="You are a helpful assistant."),
        HumanMessage(content=(
            f"Here are all the facts stored in memory for {scope_label}:\n\n{raw}\n\n"
            "Write a concise, readable summary (3–6 sentences) of what has been discussed "
            "and what key facts are worth remembering."
        )),
    ]
    return llm.invoke(prompt).content

The only new thing here is the prompt. The heavy lifting — fact extraction, deduplication, persistent storage — was already being done by Mem0 on every message. !summarize is just a better read path on top of mem0.get_all().

This is a recurring pattern: Mem0 works in the background accumulating structured knowledge, and you add new commands as different views onto that knowledge. No extra storage. No extra pipelines.

Feature 2: Reaction-to-memory (📌 / 🗑️)

This one changes the dynamic from "the bot decides what to remember" to "humans curate what's worth remembering."

React 📌 (pushpin) to any message → it's explicitly saved as a memory fact for that channel.

React 🗑️ (wastebasket) to any message → the closest matching memory entry is found and deleted.

┌─────────────────────────────────────┐
│  User reacts 📌 to a message        │
│                                     │
│  reaction_added event fires         │
│       │                             │
│       ▼                             │
│  conversations_history API call     │
│  (fetch original message text)      │
│       │                             │
│       ▼                             │
│  mem0.add([{role: user,             │
│             content: message_text}],│
│            user_id=channel_scope)   │
│       │                             │
│       ▼                             │
│  Bot confirms in channel            │
└─────────────────────────────────────┘

The save path is straightforward — mem0.add() with the raw message text. Mem0 handles entity extraction and deduplication the same way it does for regular conversations.

The delete path is more interesting. Mem0 doesn't have a "delete by text content" API — it deletes by memory ID. So we do a semantic search first, take the top result, and delete by its ID:

def handle_reaction(channel_id: str, user_id: str, text: str, action: str) -> str:
    scope = channel_memory.scope_id(channel_id)
    if action == "save":
        channel_memory.add([{"role": "user", "content": text}], scope)
        return f"📌 Saved to memory: _{text[:120]}_"
    elif action == "remove":
        matches = channel_memory.search(text, scope)
        if not matches:
            return "Nothing matching that message found in memory."
        memory_id = matches[0].get("id")
        if memory_id:
            channel_memory.client.delete(memory_id)
            return f"🗑️ Removed from memory: _{matches[0].get('memory', text)[:120]}_"

Why this matters: Auto-extraction is imperfect. The bot might miss a critical decision buried in a long message, or over-index on something throwaway. Reaction triggers give your team a lightweight control surface — no commands to learn, no special syntax. Just use emoji the way Slack teams already do.

Wiring it up in slack-bolt is clean:

@app.event("reaction_added")
def on_reaction_added(event, client, say):
    route_reaction(event, client, say)

The client injected by slack-bolt is used to fetch the original message via conversations_history. One thing to remember: you need reaction_added in your Slack app's bot event subscriptions, and reactions:read in your OAuth scopes.

Feature 3: `!catchmeup` — personalized channel digests

This is the feature that most directly shows what becomes possible when you have two memory scopes to cross-reference.

The problem: !summarize gives everyone the same summary of a channel. But a frontend engineer and a backend engineer don't need the same catch-up. The frontend engineer wants to know about design decisions and API contract changes. The backend engineer wants to know about schema migrations and performance issues.

!catchmeup solves this by combining two Mem0 calls:

channel_memory.get_all(channel_scope) — everything discussed in this channel
dm_memory.get_all(dm_scope) — everything known about the person asking (their role, their past questions, their areas of ownership)

def handle_catchmeup(channel_id: str, user_id: str) -> str:
    channel_scope = channel_memory.scope_id(channel_id)
    channel_facts = channel_memory.get_all(channel_scope)
    if not channel_facts:
        return "No channel history in memory yet — nothing to catch you up on."

    user_scope = dm_memory.scope_id(user_id)
    user_facts = dm_memory.get_all(user_scope)

    channel_context = _format_memories_for_display(channel_facts)
    user_context = (
        _format_memories_for_display(user_facts)
        if user_facts
        else "No personal context available — give a generic summary."
    )

    prompt = [
        SystemMessage(content="You are a helpful assistant summarizing a Slack channel for a returning teammate."),
        HumanMessage(content=(
            f"Here is what has been discussed in this channel:\n\n{channel_context}\n\n"
            f"Here is what I know about the person asking:\n\n{user_context}\n\n"
            "Write a personalized catch-up digest (4–8 bullet points) highlighting what's most "
            "relevant and important for this specific person. Lead with a one-sentence intro. "
            "Skip anything they already clearly know or wouldn't care about."
        )),
    ]
    return llm.invoke(prompt).content

Here's what this looks like in practice. Say Alice (frontend) and Bob (backend) both type !catchmeup after a week away from #platform:

Alice gets:

Here's what happened in #platform while you were away — focus on the API contract changes and the new design system tokens.

The REST API now returns created_at as ISO 8601 instead of Unix timestamp — frontend date parsing will need updating

Design tokens for the new component library were merged; migration guide is in Notion

...

Bob gets:

Here's what happened in #platform while you were away — mostly schema and performance work relevant to your services.

The users table added a composite index on (tenant_id, created_at) — query patterns for your billing service should improve

Rate limiter config moved to Redis; the old in-memory approach was causing inconsistency across pods

...

Same channel. Same Mem0 data. Different people, different summaries.

This only works because DM memory has been accumulating context about each user from their private conversations with the bot. The longer someone uses the bot in DMs, the more personalized their !catchmeup becomes. The memory compounds.

The pattern across all three features

Looking at these three features together, there's a consistent pattern:

Feature	Mem0 operation	What it adds
`!summarize`	`get_all()` → LLM synthesis	Better read path on existing data
📌 reaction	`add()` with explicit content	Human-curated write path
🗑️ reaction	`search()` → `delete(id)`	Human-curated delete path
`!catchmeup`	`get_all()` × 2 scopes → LLM	Cross-scope personalization

Mem0 is doing the hard work in all four cases: storing, indexing, retrieving. The application code is just deciding when to call which operation and how to frame the result for the LLM.

This is the real argument for using a managed memory layer instead of rolling your own vector store: you get add, search, get_all, and delete as primitives, and you can build surprisingly rich behavior by composing them.

What's next

A few directions we're thinking about:

Graph memory — Mem0 supports a graph mode that tracks relationships between entities. You could model "Alice owns the auth service", "auth service depends on user-db", and surface that graph when someone asks about a deployment.

Per-channel LLM config — store a channel's preferred model in Mem0 itself (alongside conversation memory). #architecture uses a powerful reasoning model; #random uses a fast cheap one.

!whoknows <topic> — search across all user DM scopes to find who has the most relevant memories about a given topic. Instant expert routing.

Building a Slack Bot That Actually Remembers: slacktag-oss

Harish Kotra (he/him) — Fri, 26 Jun 2026 18:21:41 +0000

How I built an open-source Slack assistant with persistent semantic memory, powered by any LLM and Mem0's managed memory layer — no vector database required.

The problem with Slack bots and memory

Most AI Slack bots have the memory of a goldfish. Every conversation starts from scratch. You ask it about your sprint goals, it gives a great answer, then three days later you ask a follow-up and it has no idea what you're talking about. You end up re-explaining context constantly.

The commercial solution to this is Claude Tag — a Slack integration that maintains genuine conversational continuity. But it's tied to one provider and not open-source.

slacktag-oss is our attempt to replicate that experience: a Slack bot with real, semantic, persistent memory that works with any LLM — including ones running entirely on your laptop.

What I built

A Python Slack bot with:

Socket Mode for local dev (no public URL needed), HTTP-ready for prod
LangChain to abstract LLM calls across any OpenAI-compatible endpoint
Mem0 managed cloud for semantic memory — no Qdrant, no Pinecone, no infra to run
Three memory scopes: per-channel, per-thread, per-DM
Built-in !clear and !memory commands
A clean, extensible architecture you can fork and build on

Architecture

Before diving into code, here's the full request lifecycle:

┌─────────────────────────────────────────────────────────────┐
│                         Slack                               │
│  @mention in channel  ──┐                                   │
│  DM to bot            ──┼──► Slack Events API               │
│  Thread reply         ──┘         │                         │
└───────────────────────────────────│─────────────────────────┘
                                    │ (Socket Mode / HTTP)
                                    ▼
┌─────────────────────────────────────────────────────────────┐
│                      slack-bolt (Python)                     │
│   bot.py  ──►  router.py  ──►  handler.py                  │
│                                    │                        │
│                    ┌───────────────┤                        │
│                    │               │                        │
│                    ▼               ▼                        │
│              Mem0 Client      LangChain                     │
│              (managed)        ChatOpenAI                    │
└────────────────────────────────────────────────────────────-┘
                    │
                    ▼
        ┌───────────────────────┐
        │   Mem0 Managed Cloud  │
        │  Vector Embeddings    │
        │  Entity Extraction    │
        │  Deduplication        │
        └───────────────────────┘

The key design decision: Mem0 is the only stateful dependency. There's no database to manage, no Redis, no Qdrant. The bot process itself is stateless — you can restart it freely without losing any memory.

Project structure

slacktag-oss/
├── main.py
├── config/settings.py       ← Pydantic settings from .env
├── core/
│   ├── bot.py               ← Slack Bolt app + event registration
│   ├── handler.py           ← All orchestration logic lives here
│   └── router.py            ← Dispatches channel mentions vs DMs
├── memory/
│   ├── base.py              ← Abstract interface
│   ├── channel_memory.py    ← Channel + thread scoped memory
│   ├── dm_memory.py         ← Per-user private memory
│   └── mem0_store.py        ← Mem0 client factory
├── llm/client.py            ← ChatOpenAI factory
└── tools/registry.py        ← Tool plugin stub (v2)

The memory layer: why Mem0

The typical approach to bot memory is a rolling window: keep the last N messages in the prompt. This breaks down fast — context gets stale, important things fall out of the window, and token costs grow linearly.

Mem0 takes a different approach. When you store a conversation, it:

Runs an extraction pass to pull out facts, entities, and preferences
Deduplicates them against what's already stored
Indexes them as vector embeddings for semantic retrieval

When you later ask a question, you get back the most relevant past memories — not just the most recent ones. A user's preference mentioned three weeks ago will surface when relevant, even if hundreds of messages happened in between.

Setting up the client

Because we're using Mem0's managed cloud, the entire backend is three lines:

# memory/mem0_store.py
from mem0 import MemoryClient
from config.settings import settings

def get_mem0_client() -> MemoryClient:
    return MemoryClient(api_key=settings.MEM0_API_KEY)

No vector database config. No embedding model to choose. No collection names to manage.

Memory scoping

The key insight for a Slack bot is that different conversations need different memory boundaries:

# channel_memory.py
def scope_id(self, channel_id: str, thread_ts: str = None) -> str:
    if thread_ts:
        return f"thread:{channel_id}:{thread_ts}"   # isolated thread
    return f"channel:{channel_id}"                   # shared channel

# dm_memory.py
def scope_id(self, user_id: str) -> str:
    return f"dm:{user_id}"                           # private per user

Mem0 uses this string as a user_id — anything stored under channel:C12345 is shared by everyone in that channel. Anything under dm:U67890 is private. Thread memory is completely isolated so a debugging session in a thread doesn't pollute the main channel's memory.

BaseMemory interface

Both ChannelMemory and DMMemory implement the same four-method interface:

# memory/base.py
class BaseMemory(ABC):
    @abstractmethod
    def add(self, messages: list[dict], scope_id: str) -> None: ...

    @abstractmethod
    def search(self, query: str, scope_id: str) -> list[dict]: ...

    @abstractmethod
    def get_all(self, scope_id: str) -> list[dict]: ...

    @abstractmethod
    def clear(self, scope_id: str) -> None: ...

This makes it easy to swap backends later — implement BaseMemory, update the factory, done.

The LLM layer: LangChain + any OpenAI-compatible endpoint

# llm/client.py
from langchain_openai import ChatOpenAI
from config.settings import settings

def get_llm() -> ChatOpenAI:
    return ChatOpenAI(
        base_url=settings.LLM_BASE_URL,
        api_key=settings.LLM_API_KEY,
        model=settings.LLM_MODEL,
        temperature=0.7,
        streaming=True,
    )

base_url is the only thing that changes between providers. Ollama, LM Studio, OpenAI, Groq, Together AI — all work without touching any other code.

The handler: where memory meets LLM

handler.py is the heart of the bot. For every request, it:

Checks for built-in commands
Searches Mem0 for semantically relevant past context
Gets recent history for conversational continuity
Builds the LangChain message list
Invokes the LLM
Stores the exchange back in Mem0

# core/handler.py (simplified)
def handle_channel_mention(channel_id, user_id, text, thread_ts=None):
    scope = channel_memory.scope_id(channel_id, thread_ts)

    # Built-in commands short-circuit before hitting the LLM
    if text.strip() == "!clear":
        channel_memory.clear(scope)
        return "Memory cleared."
    if text.strip() == "!memory":
        return format_memories(channel_memory.get_all(scope))

    # Dual retrieval: semantic + recency
    relevant = channel_memory.search(text, scope)
    history  = channel_memory.get_all(scope)

    messages = build_messages(system_prompt, relevant, history, text)
    response = llm.invoke(messages)
    reply    = response.content

    # Store the exchange — Mem0 extracts entities + deduplicates
    channel_memory.add(
        [{"role": "user", "content": text},
         {"role": "assistant", "content": reply}],
        scope,
    )
    return reply

Building the prompt

The message list passed to the LLM is assembled in a specific order:

def build_messages(system_prompt, relevant_memories, recent_history, user_input):
    messages = [SystemMessage(content=system_prompt)]

    # Inject relevant memories as a second system message
    if relevant_memories:
        memory_context = "\n".join(
            m["memory"] for m in relevant_memories if "memory" in m
        )
        messages.append(SystemMessage(
            content=f"Relevant context from earlier:\n{memory_context}"
        ))

    # Append recent history
    for entry in recent_history[-MAX_HISTORY_MESSAGES:]:
        if entry.get("role") == "user":
            messages.append(HumanMessage(content=entry["content"]))
        elif entry.get("role") == "assistant":
            messages.append(AIMessage(content=entry["content"]))

    # Current user message always last
    messages.append(HumanMessage(content=user_input))
    return messages

The two-system-message pattern keeps the bot's persona and instructions separate from the injected memory context — cleaner for the model to reason about.

Wiring up Slack

slack-bolt makes event handling clean:

# core/bot.py
app = App(token=settings.SLACK_BOT_TOKEN, signing_secret=settings.SLACK_SIGNING_SECRET)

@app.event("app_mention")
def on_mention(event, say):
    route_mention(event, say)   # channel / thread flow

@app.event("message")
def on_message(event, say):
    if event.get("channel_type") == "im" and not event.get("bot_id"):
        route_dm(event, say)    # DM flow, ignore bot's own messages

router.py extracts the relevant fields and calls the appropriate handler:

# core/router.py
def route_mention(event, say):
    channel_id = event.get("channel")
    thread_ts  = event.get("thread_ts")
    text       = event.get("text", "")

    reply = handle_channel_mention(channel_id, event["user"], text, thread_ts)
    say(text=reply, thread_ts=thread_ts or event["ts"])

Replies always go back to the same thread — if the mention was in a thread, the bot stays in that thread.

Configuration with Pydantic Settings

All config lives in one place with validation:

# config/settings.py
class Settings(BaseSettings):
    SLACK_BOT_TOKEN: str
    SLACK_APP_TOKEN: str
    SLACK_SIGNING_SECRET: str
    LLM_BASE_URL: str = "http://localhost:11434/v1"
    LLM_API_KEY: str = "ollama"
    LLM_MODEL: str = "llama3.2"
    MEM0_API_KEY: str
    BOT_NAME: str = "Claude"
    MAX_HISTORY_MESSAGES: int = 20
    SYSTEM_PROMPT: str = ""

    class Config:
        env_file = ".env"

Missing required fields (the Slack tokens, the Mem0 key) raise a ValidationError at startup — fail fast before any event processing begins.

Running it locally

# Get dependencies
pip install -r requirements.txt

# Start the bot (Socket Mode — no public URL needed)
python main.py

That's it. No Docker, no Qdrant, no ngrok. Invite the bot to a channel, @mention it, and it starts building memory from the first message.

What "semantic memory" actually looks like in practice

Here's a realistic example. Day 1:

User: @slacktag Our API rate limit is 100 req/min per tenant. Keep that in mind for capacity planning.
Bot: Got it. I'll factor that in for any capacity discussions.

Day 3 (hundreds of messages later in the channel):

User: @slacktag We're about to onboard 5 new enterprise tenants. Any concerns?
Bot: A few things to consider: with your current API rate limit of 100 req/min per tenant, 5 new enterprise tenants could significantly increase peak load. You may want to review your rate limiting strategy before onboarding...

Mem0 surfaced the rate limit fact from Day 1 because it was semantically relevant to the capacity question — even though it was nowhere in the recent message window.

Deployment path

For production, swap SocketModeHandler for a standard HTTP adapter:

# Using Flask
from slack_bolt.adapter.flask import SlackRequestHandler
from flask import Flask, request

flask_app = Flask(__name__)
handler = SlackRequestHandler(app)

@flask_app.route("/slack/events", methods=["POST"])
def events():
    return handler.handle(request)

Point your Slack app's Request URL to https://your-domain/slack/events, deploy anywhere (Fly.io, Railway, Cloud Run — all work), and you're done. No state in the server — Mem0 holds everything.

What's next (v2 ideas)

A few extensions that would make this significantly more powerful:

Pluggable tools — tools/registry.py is stubbed out for LangChain tool integration. Adding web search (Tavily, Brave Search) or a code execution sandbox would turn this into a capable agent.

Mem0 graph memory — Mem0 supports a graph mode that tracks relationships between entities across conversations. You could map out who's on which team, what projects are in flight, and surface that context automatically.

Per-channel LLM config — let admins set a different model per channel (e.g., a powerful model for #architecture, a fast cheap model for #random).

Reaction triggers — react with 🧠 to explicitly add a message to memory; react with 🗑️ to remove a fact. Much more controllable than pure auto-extraction.

!summarize — call mem0.get_all() and ask the LLM to produce a readable summary of everything it knows about this channel.

Getting involved

The codebase is intentionally small. handler.py is ~100 lines. Every module does one thing. If you want to contribute:

git clone https://github.com/harishkotra/slacktag-oss
cd slacktag-oss
python -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
cp .env.example .env

Pick any feature from the table in the README, implement it, and open a PR. The architecture is designed to stay simple — add without entangling.

Building Serverless Detective: An AI-Powered Incident Investigation Agent with AI SDK 7

Harish Kotra (he/him) — Thu, 25 Jun 2026 16:09:29 +0000

How I built a real-time multi-step AI agent that investigates production incidents, showcases 10+ AI SDK 7 features, and streams results live to the browser.

Production incidents are stressful. An alert fires, the dashboard turns red, and someone has to dig through logs, query databases, check service health, and piece together what went wrong — all while the clock is ticking.

What if an AI agent could do that legwork? Not replace the engineer, but handle the grunt work: scanning logs, running queries, checking services, and producing a structured report with evidence.

That's exactly what Serverless Detective does. It's an open-source demo built with AI SDK 7 and Next.js 15 that showcases a new paradigm in LLM application development — durable, multi-step tool-using agents.

What Is Serverless Detective?

Serverless Detective is an interactive AI agent that:

Ingests a production incident log (2MB, 20,000 lines)
Searches for error patterns using regex/keyword matching
Queries a simulated database for affected records
Checks microservice health endpoints
Escalates by paging on-call engineers or rolling back deployments
Reports findings with a full incident summary and performance metrics

All of this happens in an automated loop: the LLM decides what tool to call, processes the result, and plans the next step — until the investigation is complete.

And the user sees every step as it happens, streamed live via Server-Sent Events.

Architecture Overview

┌──────────────────────────────────────────────────────────┐
│                     Browser (Next.js)                     │
│                                                          │
│   page.tsx (React 19)                                    │
│     └─ fetch("/api/detective")                            │
│     └─ ReadableStream.getReader()                         │
│     └─ Parse SSE data: lines                              │
│     └─ React setState updates per step                    │
└──────────────────────────────────────────────────────────┘
                        │ SSE stream
                        ▼
┌──────────────────────────────────────────────────────────┐
│              Next.js API Route (Node.js)                  │
│                                                          │
│   route.ts                                                │
│     ├─ new TransformStream()                              │
│     ├─ agent.generate() in background                     │
│     │   └─ onStepEnd  → writer.write()                   │
│     │   └─ onEnd      → writer.close()                   │
│     └─ return Response(stream.readable)                   │
└──────────────────────────────────────────────────────────┘
                        │
                        ▼
┌──────────────────────────────────────────────────────────┐
│              ToolLoopAgent (AI SDK 7)                     │
│                                                          │
│   Step loop:                                              │
│     LLM call → tool calls → execute → results → repeat    │
│                                                          │
│   Features:                                               │
│     ├── tool() API with Zod schemas                      │
│     ├── reasoning: high                                   │
│     ├── Timeouts (total/step/tool)                        │
│     ├── Tool approvals (human-in-the-loop)                │
│     ├── runtimeContext (typed shared state)                │
│     ├── isStepCount() stop condition                      │
│     ├── onStepEnd / onEnd callbacks                       │
│     ├── File-based snapshot persistence                   │
│     └── OpenTelemetry tracing                             │
└──────────────────────────────────────────────────────────┘
                        │
                        ▼
              ┌─────────────────┐
              │  Groq / OpenAI  │
              └─────────────────┘

Deep Dive: How It Works

1. The Agent Core (`agent/detective.ts`)

The heart of the application is a ToolLoopAgent — one of AI SDK 7's built-in agent implementations. It runs a loop:

Call the LLM with the current context
If the LLM returns tool calls, execute them
Feed results back to the LLM
Repeat until a stop condition is met

export function createDetectiveAgent(callbacks?: AgentCallbacks) {
  return new ToolLoopAgent<never, DetectiveTools, DetectiveRuntimeContext>({
    id: "serverless-detective",
    model: createModel(),
    tools,
    instructions: `You are "The Serverless Detective" — a senior SRE...`,
    runtimeContext: { incidentId, status: "ingesting", ... },
    timeout: { totalMs: 120000, stepMs: 30000, toolMs: 10000 },
    toolApproval: { pageOnCall: "user-approval", ... },
    stopWhen: isStepCount(10),
    maxOutputTokens: 4096,
  });
}

Key AI SDK 7 concepts at play:

tool() API — Declarative tools with Zod-validated input schemas and typed execute functions
runtimeContext — Typed mutable state that persists across all steps. The agent tracks which services it has checked, its current investigation phase, and hypotheses it has formed
timeout — Three-tier timeout: total for the whole investigation, per-step, and per-tool execution. The rollback tool gets a longer timeout since it's a heavy operation
toolApproval — Sensitive actions like paging an on-call engineer require user confirmation. This is set at the agent level but can be overridden per invocation
isStepCount() — A built-in stop condition that caps the agent at 10 steps, preventing runaway investigations

2. The Investigation Tools (`agent/tools.ts`)

Five tools give the detective its capabilities:

searchLogs(pattern, maxLines)     → Lines matching the pattern
queryDatabase(table, where, limit) → Simulated DB rows with error status
checkServiceHealth(service)       → "healthy" or "degraded"
pageOnCall(severity, summary)     → Creates a PagerDuty incident (requires approval)
rollbackDeployment(service, version) → Rolls back to a stable version (requires approval)

Each tool is defined using the tool() API with Zod schemas:

export const searchLogsTool = tool({
  description: "Search the incident log for patterns",
  inputSchema: z.object({
    pattern: z.string().describe("Regex or keyword to search for"),
    maxLines: z.number().optional().default(20),
  }),
  execute: async ({ pattern, maxLines }) => {
    // Read log file, filter lines, return matches
    return { totalMatches, lines, summary };
  },
});

The inputSchema is AI SDK 7's replacement for the older parameters field. It accepts any Zod schema and uses the schema's .describe() hints as prompt context for the LLM.

3. SSE Streaming (`app/api/detective/route.ts`)

This is where the magic of real-time UX meets AI SDK 7's callback system.

The API route creates a TransformStream, wires the agent's lifecycle callbacks to write SSE events, and returns the readable half as the HTTP response:

export async function GET() {
  const stream = new TransformStream();
  const writer = stream.writable.getWriter();

  const onStepEnd = (event) => {
    writer.write(encoder.encode(`data: ${JSON.stringify({
      type: "step",
      stepNumber: event.stepNumber + 1,
      toolNames: event.toolCalls?.map(tc => tc.toolName),
      durationMs: event.performance?.stepTimeMs,
      tokens: event.usage?.outputTokens,
    })}\n\n`));
  };

  // Start agent in background — don't await
  (async () => {
    const agent = createDetectiveAgent({ onStepEnd, onEnd });
    await agent.generate({ prompt: "..." });
  })();

  return new Response(stream.readable, {
    headers: { "Content-Type": "text/event-stream" },
  });
}

The key insight: agent.generate() runs in a background promise, while the response is returned immediately. The agent's onStepEnd callback writes to the same stream that the HTTP response reads from. This creates a live pipeline from the LLM to the browser.

4. The Web UI (`app/page.tsx`)

The client uses the native ReadableStream API to consume the SSE stream:

const res = await fetch("/api/detective");
const reader = res.body!.getReader();
const decoder = new TextDecoder();
let buffer = "";

while (true) {
  const { done, value } = await reader.read();
  if (done) break;
  buffer += decoder.decode(value, { stream: true });

  for (const part of buffer.split("\n\n")) {
    const match = part.match(/^data: (.+)$/m);
    if (match) {
      const event = JSON.parse(match[1]);
      // event.type === "step" → add step card
      // event.type === "complete" → show report + performance
      // event.type === "error" → show error
    }
  }
}

Each SSE event updates React state, which renders a new step card with tool name badges, duration, and token count. The result is a live-updating UI that feels like watching a real investigation unfold.

5. Workflow Snapshots (`agent/workflow-store.ts`)

For durability, each step is persisted to a JSON file:

onStepEnd: async (event) => {
  saveSnapshot({
    id: runtimeContext.incidentId,
    completedSteps: stepNumber + 1,
    runtimeContext,
    // ...
  });
}

This enables the crash-and-resume demo: if the process is killed mid-investigation (e.g., kill -9), the next run can find the snapshot and resume from where it left off.

Challenges & Learnings

Provider Compatibility

AI SDK 7's @ai-sdk/openai v2 defaults to the OpenAI Responses API format. Groq and other OpenAI-compatible providers only support the Chat Completions format. We fixed this by using provider.chat(modelId) instead of provider(modelId).

Type Safety Across Versions

The StreamTextResult.text property in AI SDK 7 is PromiseLike<string> (not plain string), which means JSON.stringify serializes it as {}. This caused a confusing "Objects are not valid as a React child" error. We switched to agent.generate() which returns GenerateTextResult.text as a plain string.

SSE Multiplexing

Streaming agent events through a TransformStream required careful error handling — the background promise must catch all errors and write them to the stream, then close it cleanly. Unhandled rejections in the background promise are silent failures.

Running the Project

npm install
npm run generate-logs
npm run dev
# Open http://localhost:3000

Configure your API key in .env.local:

# OpenAI
OPENAI_API_KEY=sk-...
OPENAI_MODEL=gpt-4o

# Or Groq
OPENAI_BASE_URL=https://api.groq.com/openai/v1
OPENAI_API_KEY=gsk_...
OPENAI_MODEL=llama-3.3-70b-versatile

What's Next

This demo barely scratches the surface of what's possible with AI SDK 7. Here are ideas I'd love to see the community explore:

Multi-agent investigations using WorkflowAgent from @ai-sdk/workflow — one agent for logs, another for DB, a coordinator to synthesize findings
Real tool integrations — PagerDuty, Datadog, CloudWatch, Slack, Jira
Persistent storage — SQLite or Postgres instead of file-based snapshots
Chat interface — Replace the single button with useChat from @ai-sdk/react for interactive follow-up questions
Authentication — User sessions with investigation history

Code & more: https://www.dailybuild.xyz/project/174-serverless-detective

PocketAgent: Building a Self-Hosted Payment Delegation System for AI Agents

Harish Kotra (he/him) — Wed, 24 Jun 2026 18:23:19 +0000

The Problem

AI agents today are powerful but fundamentally limited by one thing: they can't pay for themselves.

Every API call, every compute minute, every database query costs money. Currently, the only way to keep an agent running is to either:

Pre-fund a wallet — and hope it doesn't run out at 3 AM
Give the agent your credit card — a security nightmare
Build your own billing system — months of work

PocketAgent solves this with a self-hosted implementation of the Nevermined x402 Delegation Extension, an open protocol that lets you delegate payment authority to an AI agent with fine-grained limits, automatic top-up, and zero direct card exposure.

What is the x402 Delegation Extension?

The x402 Delegation Extension is a specification that defines how a "delegator" (you) can grant spending authority to a "delegate" (your AI agent) through a delegation — a config object with spending limits, duration, and provider bindings.

The protocol has four phases:

Phase 0: Card Enrollment — attach a payment method to a PSP (Stripe, Braintree)
Phase 1: Delegation Creation — define who can spend, how much, and where
Phase 2: Verification — validate the delegation before each spend
Phase 3: Settlement — charge the card, credit the agent, deduct usage

What makes it powerful: the agent never sees the card number. The facilitator mediates all charges. The agent only holds a JWT with scoped claims.

Architecture Overview

PocketAgent has three components:

┌──────────────┐     create delegation      ┌──────────────────┐
│  Web UI /    │ ─────────────────────────> │  Mock Facilitator │
│  CLI / Demo  │                            │  (port 3020)     │
│              │ <──── x402 access token ── │  · Stripe API     │
│              │                            │  · On-chain via   │
│              │ ── PAYMENT-SIGNATURE ────> │    viem           │
│              │    header + prompt         │  · JWT sign/verify │
│              │                            │  · Delegation CRUD │
│              │ <─── PAYMENT-RESPONSE ──── │  · Auto top-up     │
│              │     + task result          └──────────────────┘
└─────────────┘

Facilitator (port 3020): The core — handles Stripe, on-chain operations, JWT signing, delegation lifecycle
Agent Server (port 3010): Protected x402 endpoint + Web UI. Forwards settlement to facilitator
Client: Web UI (MetaMask-connected), CLI (Commander-based), or automated demo script

On-chain, we use PocketAgentCredit.sol — a simple ERC20 burnable token on Base Sepolia. Credits minted from fiat top-up, burned per agent invocation.

Deep Dive: How It Works

Phase 0: Card Enrollment

The user enrolls a Stripe test card. In production, this goes through VGS Collect (PCI-compliant proxy). For the demo, we attach a Stripe test PaymentMethod directly:

// Client requests a SetupIntent
const setupRes = await fetch(`${facilitatorUrl}/payments/card/setup`, { method: 'POST' });
const { setupIntentId } = await setupRes.json();

// Facilitator creates SetupIntent + attaches test payment method
const enrollRes = await fetch(`${facilitatorUrl}/payments/card/enroll`, {
  method: 'POST',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify({ setupIntentId }),
});
const { customerId, paymentMethodId } = await enrollRes.json();

On the facilitator side, this creates a Stripe Customer (with name and address.country for Indian export compliance) and attaches the payment method:

// mock-server.ts
const customer = await stripe.customers.create({
  name: `PocketAgent User ${Date.now()}`,
  address: { country: 'IN' },
  payment_method: pm.id,
});

Phase 1: Delegation Creation

The user creates a delegation — think of it as a programmable spending card for your agent:

const delRes = await fetch(`${facilitatorUrl}/api/v1/delegation/create`, {
  method: 'POST',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify({
    provider: 'stripe',
    subscriberAddress: '0x...',
    providerCustomerId: 'cus_...',
    providerPaymentMethodId: 'pm_...',
    spendingLimitCents: 10000,    // $100 max
    durationSecs: 604800,         // 7 days
    currency: 'usd',
    merchantAccountId: 'acct_...', // Stripe Connect (Section 6.4)
  }),
});
const { delegationId, sessionKeyHash } = await delRes.json();

DelegationConfig stores everything needed for enforcement:

interface DelegationConfig {
  delegationId: string;
  provider: string;
  subscriberAddress: `0x${string}`;
  providerCustomerId: string;
  providerPaymentMethodId?: string;
  spendingLimitCents: number;
  spentCents: number;
  currency: string;
  durationSecs: number;
  maxTransactions?: number;
  transactionCount: number;
  merchantAccountId?: string;    // Stripe Connect (Section 6.4)
  createdAt: number;
  expiresAt: number;
  status: 'Active' | 'Exhausted' | 'Expired' | 'Revoked';
  sessionKeyHash: string;
}

Phase 2: Getting an Access Token

Before calling the agent, the user requests an x402 access token. The facilitator verifies the delegation exists, is active, and hasn't exceeded limits:

const permRes = await fetch(`${facilitatorUrl}/x402/permissions`, {
  method: 'POST',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify({
    resource: {
      url: '/api/v1/agents/agent-1/tasks',
      description: 'AI agent task execution',
      mimeType: 'application/json',
    },
    accepted: {
      scheme: 'nvm:card-delegation',
      network: 'stripe',
      extra: { version: '1' },
    },
    delegationConfig: { delegationId },
  }),
});
const { accessToken } = await permRes.json();

The returned accessToken is an RS256-signed JWT with nvm-namespaced claims (per spec Section 3.3):

const token = jwt.sign(
  {
    sub: delegation.subscriberAddress,
    iss: 'pocket-agent',
    aud: delegation.provider,
    exp: delegation.expiresAt,
    iat: Math.floor(Date.now() / 1000),
    nvm: {
      delegationId: delegation.delegationId,
      provider: delegation.provider,
      spendingLimitCents: delegation.spendingLimitCents,
      spentCents: delegation.spentCents,
    },
  },
  rsaPrivateKey,
  { algorithm: 'RS256' },
);

Phase 2-3: Agent Invocation → Verification → Settlement

The magic happens in the x402 middleware (src/agent-server/middleware/x402.ts). Every request to the agent server goes through:

Verification (Section 5)

The middleware checks:

PAYMENT-SIGNATURE header present? — INVALID_PAYLOAD if missing
JWT valid? — RS256 signature match, decode claims
JWT expired? — EXPIRED_TOKEN
Delegation exists? — DELEGATION_NOT_FOUND
Delegation active? — DELEGATION_INACTIVE (Revoked/Exhausted/Expired)
Transaction count exceeded? — TRANSACTION_LIMIT_REACHED
Spending limit exceeded? — BUDGET_EXCEEDED

// x402.ts middleware
const sigHeader = request.headers['payment-signature'];
if (!sigHeader) {
  return reply.status(402).send({ error: { code: 'INVALID_PAYLOAD', message: 'Missing PAYMENT-SIGNATURE header' } });
}

// Verify JWT
const decoded = jwt.verify(sigHeader, publicKey);

// Verify delegation
const del = delegations.get(decoded.nvm.delegationId);
if (!del) throw new PaymentError('DELEGATION_NOT_FOUND');
if (del.status !== 'Active') throw new PaymentError('DELEGATION_INACTIVE');
if (del.expiresAt < now) throw new PaymentError('EXPIRED_TOKEN');
if (del.maxTransactions && del.transactionCount >= del.maxTransactions) {
  throw new PaymentError('TRANSACTION_LIMIT_REACHED');
}

Settlement (Section 6.1)

If verification passes, the middleware forwards settlement to the facilitator. This is the dual-rail logic:

// In /settle handler on the facilitator:
async function settle(delegationId, creditsRequested) {
  // 1. Check on-chain credit balance
  let currentBalance = await getCreditBalance(subscriberAddress);

  // 2. If insufficient, auto top-up: charge card → mint credits
  if (currentBalance < creditsRequested) {
    const planPriceCents = creditsRequested * 10; // 10 cents per credit

    // Atomic: increment spend counter BEFORE charge (Section 6.2)
    del.spentCents += planPriceCents;

    // Stripe charge with Connect routing (Section 6.4)
    const piParams: any = {
      amount: planPriceCents,
      currency: del.currency,
      customer: del.providerCustomerId,
      payment_method: del.providerPaymentMethodId,
      off_session: true,
      confirm: true,
      description: 'PocketAgent AI agent task credit top-up',
    };
    if (del.merchantAccountId) {
      piParams.transfer_data = { destination: del.merchantAccountId };
    }

    // Idempotent: use delegationId + transactionCount as key (Section 9.6)
    const idemKey = `${del.delegationId}:${del.transactionCount}:${planPriceCents}`;
    const pi = await stripe.paymentIntents.create(piParams, { idempotencyKey: idemKey });

    // Mint on-chain credits
    await mintCredits(subscriberAddress, planPriceCents / 10);
  }

  // 3. Burn credits for this invocation
  const txHash = await burnCredits(subscriberAddress, creditsRequested);
  del.transactionCount += 1;

  // 4. Return settlement receipt
  return {
    success: true,
    network: del.provider,
    transaction: txHash,
    creditsRedeemed: String(creditsRequested),
    remainingBalance: String(currentBalance),
    orderTx: pi?.id,
  };
}

The settlement receipt is base64-encoded in the PAYMENT-RESPONSE response header, per spec Section 4.3:

reply.header(
  'PAYMENT-RESPONSE',
  Buffer.from(JSON.stringify(receipt)).toString('base64')
);

Error Handling (Section 7.1)

Every error returns a spec-compliant error code:

Code	When
`INVALID_PAYLOAD`	Missing or malformed headers
`INVALID_TOKEN`	JWT signature or claims invalid
`EXPIRED_TOKEN`	JWT or delegation expired
`DELEGATION_NOT_FOUND`	Delegation id doesn't exist
`DELEGATION_INACTIVE`	Delegation revoked / exhausted
`BUDGET_EXCEEDED`	Spending limit hit
`TRANSACTION_LIMIT_REACHED`	Max invocations exhausted
`INSUFFICIENT_BALANCE`	Not enough on-chain credits
`CARD_DECLINED`	Stripe card declined
`PAYMENT_FAILED`	Stripe charge failed generically
`MINT_FAILED`	On-chain mint reverted
`BURN_FAILED`	On-chain burn reverted

Each error is returned as:

{
  error: {
    code: 'MINT_FAILED',
    message: 'Credit minting failed after successful card charge: ...'
  }
}

Dual-Rail: Why Crypto + Fiat?

The dual-rail design solves a real problem:

Crypto credits (on-chain) — fast, cheap, programmable. Burn 5 credits, get 5 credits of agent work. No settlement delay.

Fiat top-up (Stripe) — replenishes crypto credits when they run low. The user's card is charged, new credits are minted on-chain.

The flow:

Balance: 10 credits
Request: 5 credits
→ Sufficient: burn 5, done

Balance: 2 credits
Request: 5 credits
→ Insufficient: charge card $5.00 → mint 50 credits (now 52) → burn 5, done

This means the agent only touches crypto (fast, cheap) while the facilitator handles the fiat bridge (slow, expensive). Best of both worlds.

Security Model

The key insight: the agent never touches the payment card.

The user enrolls their card at the facilitator (Stripe)
The user creates a delegation with limits ($100, 7 days, 100 transactions)
The agent receives a JWT — not the card number
Every agent invocation is verified and settled by the facilitator
The facilitator enforces limits atomically (spend counter before charge, rollback on failure)

Even if the agent's JWT is compromised, the attacker can only spend within the delegation's limits. The card itself is safe.

Running It Yourself

git clone <this-repo>
npm install
cp .env.example .env
# Fill in: STRIPE_SECRET_KEY, RPC_URL, CREDIT_TOKEN_ADDRESS, FACILITATOR_PRIVATE_KEY
npm start
# Open http://localhost:3010

The Web UI walks through the full flow: connect MetaMask → enroll a test card → create delegation → invoke the agent.

Building Your Own Agent with PocketAgent

The project is designed as a template. Swap src/agent-server/mock-executor.ts with your AI logic:

// Before: mock executor
async function executeTask(prompt: string) {
  return { result: `Echo: ${prompt}` };
}

// After: real AI
import OpenAI from 'openai';
const openai = new OpenAI();

async function executeTask(prompt: string) {
  const completion = await openai.chat.completions.create({
    model: 'gpt-4',
    messages: [{ role: 'user', content: prompt }],
  });
  return { result: completion.choices[0].message.content };
}

Agent ideas you can build:

Agent Idea	How PocketAgent Helps
Research agent — fetches web data, summarizes, emails reports	Crypto credits cover per-query costs; Stripe auto-top-up keeps it running without manual refill
Code review bot — reviews PRs in your repo	Per-review credit cost + monthly card limit = predictable pricing for the repo owner
Social media scheduler — posts, monitors replies, generates content	Delegation limits prevent runaway spend; Stripe Connect routes creator revenue
Trading signal generator — analyzes markets, sends alerts	Dual-rail: crypto for fast micro-transactions, fiat for larger monthly settlements
Personal finance assistant — tracks spending, categorizes transactions	User delegates a card with their own spending limit; agent handles variable usage

Nevermined x402 Spec Compliance

The implementation aligns with 14 of 14 key spec sections:

✅ Scheme nvm:card-delegation (Section 3.1)
✅ PaymentPayload / PaymentRequired structures (Section 3.2)
✅ JWT with nvm claims (Section 3.3)
✅ All 4 protocol phases (Section 4.2)
✅ PAYMENT-* HTTP headers (Section 4.3)
✅ 7 verification checks with spec error codes (Section 5)
✅ Atomic settlement with rollback (Sections 6.1, 6.2)
✅ Delegation lifecycle (Section 6.3)
✅ Stripe Connect routing transfer_data.destination (Section 6.4)
✅ Settlement receipt format (Section 6.5)
✅ All 12 critical error codes (Section 7.1)
✅ Idempotency keys on charge (Section 9.6)

What's Next

This is a reference implementation — production-ready improvements would add:

VGS Collect for PCI-compliant card data collection (Phase 0)
Database persistence for delegations instead of in-memory Map
Session key crypto using @noble/ed25519 for real Ed25519 keys
Braintree + Visa TAP provider implementations
Stripe Connect OAuth onboarding flow for merchants
Rate limiting and request queueing for settlement

PocketAgent shows that AI agents don't need direct access to payment instruments. With the x402 Delegation Extension protocol, you can give your agents spending power that's:

Scoped — fixed limits, time-bound, per-transaction
Safe — the agent never touches the card
Automatic — self-top-up when credits run low
Transparent — every charge is a Stripe PaymentIntent, every credit burn is an on-chain transaction

The code is open-source. Fork it, swap in your agent logic, and you have a production-ready billing system for your AI agent in an afternoon.

Code & more: https://www.dailybuild.xyz/project/173-pocket-agent

Building InfiniteLoop: Real-Time Visualization of AI Loop Engineering

Harish Kotra (he/him) — Mon, 22 Jun 2026 17:18:19 +0000

How I built an interactive platform that watches AI agents learn through escape rooms — one loop at a time.

Most AI demos are black boxes. You type a prompt, the model generates text, and the thinking process disappears into the void. You never see why it chose a particular action, how it recovered from failure, or what it learned along the way.

InfiniteLoop inverts this. It's an open-source, interactive web application that visualizes the Observe → Think → Act → Reflect → Repeat cycle of AI agents in real time through AI-powered escape rooms. Think of it as a debugger for emergent intelligence.

You can watch the agent form hypotheses, take actions, fail, reflect on its mistakes, and adapt — all rendered in a three-panel UI with expandable per-loop details.

The Architecture at 10,000 Feet

The system has four layers:

Room Engine — A deterministic, LLM-independent state machine that manages game state
Agent Runtime — Makes decisions by calling an LLM through OpenRouter, then feeds results back to the engine
Memory System — Three-tier memory (short-term, working, long-term) that persists lessons between loops
UI Layer — Three-panel React app showing room state, agent brain state, and the full loop timeline

The key design decision: the Room Engine never talks to an LLM. It only knows rooms, objects, items, connections, locks, codes, and action handlers. The agent interacts with it through a typed action interface.

UI Layer (React / Zustand / Framer Motion)
    ↓ fetch /api/game
API Route (Next.js App Router)
    ↓
Agent Runtime (runSingleAgentLoop)
    ├── 1. Build state prompt
    ├── 2. Call LLM (fetch → OpenRouter)
    ├── 3. Parse structured JSON (Zod schema)
    ├── 4. Execute action on RoomEngine
    ├── 5. Rule-based reflection (no LLM)
    └── 6. Update MemorySystem

The Room Engine: A Deterministic State Machine

The Room Engine is pure logic. It validates every action before executing it, updates state deterministically, and checks win/lose conditions after every loop.

export class RoomEngine {
  private room: RoomDefinition;
  public state: GameState;

  execute(action: ValidAction): ActionResult {
    const validation = this.validate(action);
    if (!validation.valid) {
      return { success: false, message: validation.error, stateChanges: {} };
    }
    const result = executeAction(this.state, this.room,
      action.type, action.target, action.payload);
    this.applyStateChanges(result.stateChanges);
    this.state.loopCount++;
    this.state.isWon = checkWinCondition(this.state, this.room);
    this.state.isLost = checkLoseCondition(this.state, this.room);
    return result;
  }
}

This separation is deliberate. By keeping the engine LLM-free, we get:

Deterministic behavior — same inputs always produce the same outputs
Testable actions — every handler is a pure function
Model-agnostic gameplay — the engine doesn't care which model drives the agent
Offline-capable room design — room JSON can be authored and validated without any LLM

The Action System

There are 12 action handlers, each a pure function with typed inputs and outputs:

export const actionHandlers: Record<string, ActionHandler> = {
  search_object: (state, room, target) => { /* finds items/clues in objects */ },
  inspect_object: (state, room, target) => { /* reveals detailed description */ },
  pickup_item: (state, room, target) => { /* adds item to inventory */ },
  use_item: (state, room, target, payload) => { /* uses item on target */ },
  enter_code: (state, room, target, payload) => { /* checks code against target */ },
  unlock_object: (state, room, target, payload) => { /* unlocks with key */ },
  move_to_room: (state, room, target) => { /* traverses connections */ },
  combine_items: (state, room, target, payload) => { /* merges two items */ },
  examine_room: (state, room) => { /* re-describes the current room */ },
  open_container: (state, room, target) => { /* opens then searches container */ },
  close_container: (state, room, target) => { /* closes container */ },
  wait: (state, room, target) => { /* does nothing */ },
};

Each handler returns an ActionResult with success, message, stateChanges, and optional discovered items. The engine applies stateChanges to update inventory, discovered objects, unlocked items, visited rooms, and more.

The Agent Runtime: One Loop, One LLM Call

The most important design decision in the agent runtime was reducing LLM calls from 3-4 per loop to exactly 1.

The original architecture had separate agents for:

Observer — "What do I see?" (1 LLM call)
Planner — "What should I do?" (1 LLM call)
Actor — "Execute the action" (0 LLM calls — deterministic)
Reflector — "What did I learn?" (1 LLM call)

That's 3 LLM calls per loop. At 6-7 seconds per call on a free-tier model, a single loop could take 20+ seconds. A full game with 20+ loops would be unusably slow.

The fix: Collapse the Observer, Planner, and Actor into a single combined prompt. The LLM returns a structured JSON object with all three in one call. The Reflector is rule-based and uses no LLM at all.

async function callModel(config, systemPrompt, userPrompt, schema) {
  const body = {
    model: config.model,
    messages: [
      { role: "system", content: `${systemPrompt}\n\nRespond with JSON: ${schema}` },
      { role: "user", content: userPrompt },
    ],
    temperature: config.temperature,
    max_tokens: 2000,
  };

  const res = await fetch("https://openrouter.ai/api/v1/chat/completions", {
    method: "POST", headers: headers(), body: JSON.stringify(body),
    signal: controller.signal,
  });

  const data = await res.json();
  const content = data.choices[0].message.content;
  return JSON.parse(content);
}

The combined schema the model must fill:

const STEP_SCHEMA = {
  type: "object",
  properties: {
    observedFacts: { type: "array", items: { type: "string" } },
    potentialLeads: { type: "array", items: { type: "string" } },
    goal: { type: "string" },
    hypothesis: { type: "string" },
    actionType: { type: "string" },
    actionTarget: { type: "string" },
    actionPayload: { type: "object" },
    confidence: { type: "number" },
    reasoning: { type: "string" },
  },
  required: ["observedFacts", "goal", "hypothesis", "actionType", "actionTarget", "confidence", "reasoning"],
};

The Reflection Engine

Reflection is handled by a deterministic function — no LLM call needed:

function generateReflection(action: string, result: string, confidence: number): Reflection {
  const failWords = ["fail", "can't", "cannot", "not found", "don't have", ...];
  const succeeded = !failWords.some(w => result.toLowerCase().includes(w));

  if (succeeded) {
    lessons.push(`${action} was effective.`);
    adjustments.push("Continue along this path.");
  } else {
    lessons.push(`${action} failed: ${result}`);
    if (result.includes("lock")) {
      adjustments.push("Search for keys before attempting locked things.");
    } else if (result.includes("don't have")) {
      adjustments.push("Search containers and furniture first.");
    }
  }

  return {
    lessonsLearned: lessons,
    strategyAdjustments: adjustments,
    updatedConfidence: succeeded ? Math.min(100, confidence + 5) : Math.max(0, confidence - 10),
  };
}

This approach means:

Zero extra token cost for reflection
Instant feedback — no waiting 6-7 seconds for the LLM to reflect
Deterministic behavior — the agent always learns the right lesson from failure
Transparent logic — every reflection rule is visible in code

The Memory System: Three-Tier Architecture

The agent has three memory tiers, inspired by cognitive science:

interface ShortTermMemory {
  currentRoom: string;
  inventory: string[];
  recentObservations: string[];   // Last 5 observations
  currentGoal: string | null;
}

interface WorkingMemory {
  recentLoops: LoopStep[];         // Last 10 loops
  activeHypotheses: string[];      // Currently being tested
  pendingActions: string[];
  mistakes: number;                 // Track failure rate
}

interface LongTermMemory {
  lessonsLearned: string[];         // Deduplicated lessons
  strategyAdjustments: string[];    // What to do differently
  environmentalMap: Record<string, string[]>;  // Known room graph
  failurePatterns: string[];
  successPatterns: string[];
}

Each loop feeds observations into short-term memory, hypotheses into working memory, and lessons into long-term memory. The getContextForPrompt() method serializes all three tiers into the state prompt so the LLM has access to accumulated knowledge.

Zustand v5: State Without Complexity

Zustand was chosen over React Context or Redux for several reasons:

No providers — the store is a module-level singleton, imported directly
Flat selectors — useGameStore(s => s.activeSessionId) re-renders only when that value changes
getState() pattern — critical for event handlers that need the latest state without stale closures

The store manages sessions, loops, commentaries, game state, and room definitions. Each game gets a unique session ID with its own config, loops array, and state.

export const useGameStore = create<GameStore>((set, get) => ({
  createSession: (roomId, config) => {
    const sessionId = `session_${Date.now()}_${Math.random().toString(36).slice(2, 7)}`;
    set(state => ({
      sessions: { ...state.sessions, [sessionId]: { /* ... */ } },
      activeSessionId: sessionId,
    }));
    return sessionId;
  },

  addLoop: (sessionId, loop) => set(state => ({
    sessions: {
      ...state.sessions,
      [sessionId]: { ...state.sessions[sessionId], loops: [...state.sessions[sessionId].loops, loop] },
    },
  })),
}));

Room Definitions: JSON-Driven Game Design

All five rooms are pure JSON, validated against Zod schemas at build time. This means new rooms can be added without touching any TypeScript code:

{
  "id": "tutorial-room",
  "name": "The Tutorial Chamber",
  "rooms": [
    {
      "id": "chamber",
      "objects": [
        { "id": "wooden_desk", "isContainer": true, "contains": ["brass_key"] },
        { "id": "north_door", "isLocked": true, "requiresItem": "brass_key" }
      ]
    },
    { "id": "exit", "objects": [] }
  ],
  "items": [
    { "id": "brass_key", "usableOn": ["north_door"] },
    { "id": "rusty_key", "usableOn": [] }
  ],
  "objectives": [
    { "type": "collect", "target": "brass_key" },
    { "type": "escape", "target": "exit" }
  ]
}

The Zod schema system ensures:

Every room, object, item, clue, and connection has the right shape
Objectives reference valid targets
Items reference valid objects in usableOn
Containers reference valid item IDs in contains

This catches configuration errors at import time rather than at runtime.

Key Technical Challenges

Challenge 1: OpenRouter Free Tier Reliability

The default model (openai/gpt-oss-20b:free on OpenInference) is free but inconsistent. In direct curl tests with the exact same prompt, it returns valid JSON in ~6.7 seconds. But from the Next.js app, it occasionally returns empty content.

Solution: No retries. If the model returns empty, the agent falls back to examine_room immediately rather than retrying 3 times and wasting 45+ seconds. Each loop completes in under 10 seconds regardless of the model state.

const modelResult = await callModel(config, SYSTEM_PROMPT, buildStatePrompt(engine, memory), STEP_SCHEMA);
const agentStep = modelResult ?? {
  actionType: "examine_room",
  actionTarget: "",
  confidence: 50,
  reasoning: "Model did not return a valid response; falling back.",
  // ... defaults
};

Challenge 2: Stale Closures in Zustand v5

Zustand v5 has a different internal scheduler than v4. Event handlers that captured the store value at render time would operate on stale state.

Solution: Always use useGameStore.getState() inside event handlers. Only use useGameStore(s => s.prop) for reactive re-renders.

const runLoop = useCallback(async () => {
  const sid = useGameStore.getState().activeSessionId;
  // ... use sid
}, []);  // No dependencies needed since we get state directly

Challenge 3: Bypassing LangChain's Structured Output

LangChain's withStructuredOutput() was hanging on OpenRouter's free-tier models due to its retry/fallback logic. The model would return a valid response in 6-7 seconds, but LangChain's wrapper would keep retrying.

Solution: Bypass LangChain entirely for the main call. Use direct fetch() to OpenRouter that matches the verified curl behavior exactly. LangChain is still available for tool definitions but the core loop uses raw HTTP.

The Three-Panel UI

The interface is split into three panels:

Room Visualization (left) — Shows the current room's description, visible objects (with container/lock badges), exits (with lock status), and the agent's inventory
Agent Brain (center) — Shows the agent's current goal, hypothesis (with confidence percentage), last action and its result, accumulated lessons, and recent observations. Animated narrator commentary appears at the top with framer-motion transitions.
Loop Timeline (right) — Expandable cards for each loop, sorted newest-first. Each card shows the complete loop cycle: Observation → Plan → Action → Result → Reflection → Narrator Commentary. Success loops have a green left border, failures have red.

On mobile, these collapse into a tabbed view with Room / Brain / Timeline tabs.

The Developer Dashboard

A separate /dashboard page provides:

Benchmark Runner — Select a room and run multiple models against it. Results show loops used, mistakes made, duration, tokens consumed, and estimated cost. Side-by-side comparison.
Agent Traces (planned) — Full tool-call traces and state transitions
Cost Analytics (planned) — Token usage per loop, per room, per model
Memory Inspector (planned) — Live view of the agent's short-term, working, and long-term memory

What's Next

Multi-agent mode is the next big feature — running 2-3 agents in parallel on the same room with different models, comparing their strategies side by side. Other planned features include a custom room builder UI, agent memory persistence across sessions, chess-style replay, and additional room themes.

Try It Yourself

git clone https://github.com/harishkotra/infinite-loop
cd infinite-loop
npm install
# Add OPENROUTER_API_KEY to .env.local
npm run dev

Open http://localhost:3000, select the Tutorial Chamber, and watch an AI agent learn to escape. No GPU required — just a browser and a free OpenRouter API key.

Screenshots

Code & more: https://www.dailybuild.xyz/project/171-infinite-loop

Building Internet Detective AI: A Production-Grade Multi-Agent AI System

Harish Kotra (he/him) — Sun, 21 Jun 2026 18:33:32 +0000

Paste your LinkedIn, GitHub, Twitter, and resume — get a brutally honest AI investigation of your entire internet personality.

Introduction

Internet Detective AI is a Next.js 15 application that accepts a person's digital footprint (LinkedIn URL, GitHub URL, Twitter handle, resume text) and runs it through a 7-agent AI pipeline to produce a detailed, entertaining investigation report. The output includes evidence-based facts, behavioral signals, hidden obsessions, a career prediction, a startup parody pitch, coworker quotes, a brutal roast, internet personality scores, and a "cooked level" meter.

It serves two purposes simultaneously:

A viral consumer app — Shareable, funny, and surprisingly insightful. Users compete for the best roasts and personality scores.
A reference architecture for production AI engineering — Every layer is designed to be educational, extensible, and production-ready.

The entire system was built in 24 hours and spans 14 areas of modern AI engineering, each documented independently in the docs/ folder.

The Vision

Most AI demo projects are either fun but shallow (a single API call wrapped in a nice UI) or educational but boring (a Jupyter notebook with no frontend). Internet Detective AI bridges this gap:

A single codebase that serves as both a genuinely fun consumer app AND a comprehensive reference architecture.

Every engineering decision was made with two audiences in mind: end-users who just want to see their roast, and developers who want to understand how production AI systems work. The result is a project that scales from "clone and run in 5 minutes" to "study every production pattern used by AI engineering teams."

Architecture Overview

The system follows a layered architecture where each layer has a single responsibility:

┌─────────────────────────────────────────────────────┐
│                   Next.js 15 App                     │
│  ┌─────────────┐  ┌──────────────┐  ┌────────────┐ │
│  │  Landing     │  │ Investigation│  │  Developer  │ │
│  │  Page        │  │ Report Page  │  │  Dashboard  │ │
│  └──────┬───────┘  └──────┬───────┘  └──────┬─────┘ │
│         │                 │                  │        │
│  ┌──────┴─────────────────┴──────────────────┴─────┐ │
│  │              API Routes                          │ │
│  └─────────────────────┬───────────────────────────┘ │
├────────────────────────┼─────────────────────────────┤
│  ┌─────────────────────┴───────────────────────────┐ │
│  │          Multi-Agent Orchestrator                │ │
│  │  ┌──────┐ ┌──────┐ ┌──────┐ ┌──────┐ ┌──────┐  │ │
│  │  │Profile│ │Signal│ │Career│ │Start │ │ Roast│  │ │
│  │  │Analyst│ │Detect│ │Predic│ │Gener │ │Agent │  │ │
│  │  └──┬───┘ └──┬───┘ └──┬───┘ └──┬───┘ └──┬───┘  │ │
│  │     └────────┴────────┴────────┴────────┘       │ │
│  │              ┌──────────────┐                    │ │
│  │              │  Governance  │                    │ │
│  │              │    Agent     │                    │ │
│  │              └──────┬───────┘                    │ │
│  │              ┌──────┴───────┐                    │ │
│  │              │   Synthesis  │                    │ │
│  │              │    Agent     │                    │ │
│  │              └──────────────┘                    │ │
│  └─────────────────────┬───────────────────────────┘ │
│  ┌─────────────────────┴───────────────────────────┐ │
│  │              AI Service Layer                     │ │
│  └─────────────────────┬───────────────────────────┘ │
│  ┌─────────────────────┴───────────────────────────┐ │
│  │           Provider Abstraction Layer              │ │
│  │  Zen │ OpenAI │ Anthropic │ Gemini │ OpenRouter  │ │
│  └─────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────┘

The request flow is: HTTP Request → API Route → Context Builder → Orchestrator → 7 Agents → Governance Loop → Synthesis → Response.

Provider Abstraction

The most fundamental architectural decision was decoupling every agent from any specific AI provider. We use the Adapter pattern with a ProviderAdapter interface:

// src/lib/providers/types.ts
export interface ProviderAdapter {
  name: ProviderType;
  chat(request: ChatCompletionRequest): Promise<ChatCompletionResponse>;
  getModels(): Promise<string[]>;
  isAvailable(): Promise<boolean>;
}

Every provider implements this interface. The factory handles instantiation and configuration:

// src/lib/providers/factory.ts
export class ProviderFactory {
  private static registry = loadConfig();

  static createProvider(type: ProviderType): ProviderAdapter {
    const registration = ProviderFactory.registry[type];
    if (!registration) {
      throw new Error(`Unknown provider type: ${type}`);
    }
    if (!registration.config) {
      throw new Error(
        `Provider "${type}" is not configured. Set the required environment variable.`,
      );
    }
    return new registration.constructor(registration.config);
  }

  static getDefaultProvider(): ProviderAdapter {
    const providerType = (process.env.AI_PROVIDER || "zen") as ProviderType;
    return ProviderFactory.createProvider(providerType);
  }
}

The registry supports 7 providers out of the box: Zen, OpenAI, Anthropic, Gemini, OpenRouter, Featherless, and Ollama. Each provider is gated by environment variables — if the key is missing, it's simply excluded from getAllAvailableProviders().

The BaseProvider abstract class adds shared behavior like error handling with retry logic, latency measurement, and cost calculation:

// src/lib/providers/base.ts
export abstract class BaseProvider implements ProviderAdapter {
  abstract name: ProviderType;
  protected config: ProviderConfig;

  protected async executeWithRetry(
    request: ChatCompletionRequest,
  ): Promise<ChatCompletionResponse> {
    return retryWithBackoff(() => this.chat(request), 3);
  }

  protected handleError(error: unknown, context: string): never {
    if (error instanceof AIProviderError) throw error;
    if (error instanceof Error) {
      const statusCode = this.extractStatusCode(error);
      const retryable = statusCode >= 500 || statusCode === 429;
      throw new AIProviderError(`${this.name}: ${context} - ${error.message}`, {
        code: this.errorCodeFromStatus(statusCode),
        statusCode,
        provider: this.name,
        retryable,
        cause: error,
      });
    }
    throw new AIProviderError(`${this.name}: ${context} - Unknown error`, {
      code: "UNKNOWN_ERROR",
      provider: this.name,
      retryable: false,
      cause: error,
    });
  }
}

The result: Any agent can use any provider. Switch the entire app from GPT-4o to Claude to Gemini with one environment variable. Adding a new provider means writing exactly one adapter file.

Multi-Agent Architecture

The orchestrator runs 7 agents in a directed pipeline. Each agent has a single responsibility:

Agent	Input	Output	Responsibility
Profile Analyst	ContextPack	Facts, digital profile summary	Extract directly observable facts
Signal Detector	Context + Facts	Strong signals, hidden obsessions	Find behavioral patterns
Career Predictor	Context + Signals	Career prediction	Predict future trajectory
Startup Generator	Context + Signals	Startup parody	Create VC-pitch satire
Roast Agent	Context + Signals	Roasts, coworker quotes, verdict	Generate playful humor
Governance Agent	All outputs	Governance check	Validate ethical compliance
Final Synthesis	Everything	InvestigationReport	Assemble final report

Each agent extends BaseAgent, which provides shared infrastructure:

// src/lib/agents/base.ts
export abstract class BaseAgent {
  protected config: AgentConfig;
  protected ai: AIService;

  abstract process(input: any): Promise<{ output: any; trace: AgentTrace }>;

  protected async callAIJSON<T>(
    userPrompt: string,
  ): Promise<{ parsed: T; trace: AgentTrace }> {
    return this.ai.chatJSON<T>({
      systemPrompt: this.config.systemPrompt,
      userPrompt,
      model: this.config.model,
      temperature: this.config.temperature,
      responseFormat: "json_object",
      agentName: this.config.name,
    });
  }

  protected async safeProcess<T>(
    processFn: () => Promise<{ output: T; trace: AgentTrace }>,
    fallbackOutput: T,
  ): Promise<{ output: T; trace: AgentTrace }> {
    try {
      return await processFn();
    } catch (error) {
      return {
        output: fallbackOutput,
        trace: { /* error trace with success: false */ },
      };
    }
  }
}

Every agent has safe defaults and fallback outputs. If an agent call fails (timeout, parse error, provider outage), the orchestrator catches it and continues with degraded data rather than crashing the entire pipeline.

The orchestrator runs agents sequentially, passing their outputs forward:

// src/lib/agents/orchestrator.ts
export class InvestigationOrchestrator {
  async investigate(context: ContextPack) {
    // Step 1: Profile Analyst
    const { output: profileOutput, trace: profileTrace } =
      await this.runAgent(this.profileAnalyst, { context }, { facts: [] });

    // Step 2: Signal Detector (receives analyst facts)
    const { output: signalOutput, trace: signalTrace } =
      await this.runAgent(this.signalDetector,
        { context, facts: profileOutput.facts },
        { strongSignals: [], hiddenObsessions: [] });

    // ... Steps 3-5: Career Predictor, Startup Generator, Roast Agent

    // Step 6: Governance Check (with retries)
    for (let attempt = 0; attempt <= MAX_GOVERNANCE_RETRIES; attempt++) {
      const { output: govOutput, trace: govTrace } =
        await this.runAgent(this.governanceAgent, governanceInput, fallback);
      if (govOutput.passed) break;
      // Sanitize inputs and retry
    }

    // Step 7: Final Synthesis
    const { output: report, trace: synthesisTrace } =
      await this.runAgent(this.finalSynthesis, synthesisInput, fallbackReport);

    return { report, traces, governanceCheck };
  }
}

The governance loop is key: if the governance agent finds violations, the orchestrator sanitizes inputs and retries up to 2 times before accepting the best available result. This creates a self-correcting pipeline.

Context Engineering

Raw profile input comes in many shapes: LinkedIn URLs, GitHub usernames, free-text resumes, Twitter bios. The ContextBuilder class normalizes all of this into a structured ContextPack.

The context pipeline has three phases:

Extraction — Parse each input source independently, extracting education, work experience, skills, repos, and stats from LinkedIn, GitHub, Twitter, and resume text
Deduplication — Merge across sources, removing duplicates using composite keys:

   private deduplicateEducation(items: Education[]): Education[] {
     const seen = new Set<string>();
     return items.filter((item) => {
       const key = `${item.institution}|${item.degree}|${item.field}`;
       if (seen.has(key)) return false;
       seen.add(key);
       return true;
     });
   }

Compression — Remove duplicate lines, normalize whitespace, and calculate compression ratio:

   private compressContent(text: string): string {
     if (!text || text.length < 500) return text;
     const lines = text.split("\n");
     const compressed: string[] = [];
     const seen = new Set<string>();
     for (const line of lines) {
       const trimmed = line.trim();
       if (!trimmed) continue;
       const normalized = trimmed.toLowerCase().replace(/\s+/g, " ");
       if (seen.has(normalized)) continue;
       seen.add(normalized);
       compressed.push(trimmed);
     }
     return compressed.join("\n");
   }

The builder also extracts key signals early — things like years of experience, leadership roles, top companies, open-source popularity, trending focus areas — which get passed to every agent as context. This early signal extraction reduces the burden on each agent to rediscover obvious patterns, saving tokens and improving accuracy.

Structured Outputs

Every agent returns typed JSON. The entire report schema is defined in TypeScript:

// src/lib/types.ts
export interface InvestigationReport {
  id: string;
  profileHash: string;
  digitalProfileSummary: string;
  facts: Fact[];
  strongSignals: StrongSignal[];
  hiddenObsessions: HiddenObsession[];
  coworkerQuotes: CoworkerQuote[];
  startupParody: StartupParody;
  careerPrediction: CareerPrediction;
  brutalRoast: Roast[];
  wildGuesses: WildGuess[];
  finalVerdict: string;
  personalityScores: InternetPersonalityScores;
  cookedLevel: CookedLevel;
  metadata: ReportMetadata;
}

Agents use JSON mode (response_format: { type: "json_object" }) and a chatJSON<T> helper that handles parsing and error recovery:

// src/lib/ai.ts
async chatJSON<T>(
  options: AIRequestOptions,
): Promise<{ parsed: T; trace: AgentTrace }> {
  const jsonOptions = { ...options, responseFormat: "json_object" };
  const response = await this.chat(jsonOptions);

  if (!response.trace.success) {
    throw new Error(`AI chat failed: ${response.trace.error}`);
  }

  // Clean markdown code fences from JSON output
  const cleaned = response.content
    .replace(/```
{% endraw %}
json\s*/gi, "")
    .replace(/
{% raw %}
```\s*$/g, "")
    .trim();

  let parsed: T;
  try {
    parsed = JSON.parse(cleaned) as T;
  } catch (parseError) {
    throw new Error(
      `Failed to parse JSON response: ${parseError.message}\nRaw: ${response.content}`
    );
  }
  return { parsed, trace: response.trace };
}

Every JSON response goes through parser recovery — the system strips markdown code fences, trims whitespace, and falls back to partial matching if the output is malformed. Combined with the safeProcess pattern, this means a single agent failure never crashes the entire investigation.

Prompt Engineering

Prompts are stored as separate markdown files in prompts/system/ and loaded by a PromptRegistry:

// src/lib/prompts/index.ts
export class PromptRegistry {
  private prompts: Map<AgentType, string> = new Map();

  async load(): Promise<void> {
    const entries = await fs.promises.readdir(PROMPTS_DIR, { withFileTypes: true });
    for (const entry of entries.filter(f => f.isFile() && f.name.endsWith(".txt"))) {
      const agentType = this.resolveAgentType(entry.name);
      if (!agentType) continue;
      const content = await fs.promises.readFile(
        path.join(PROMPTS_DIR, entry.name), "utf-8"
      );
      this.prompts.set(agentType, content.trim());
    }
  }
}

This means prompts can be edited without touching any code — useful for iterating with non-technical stakeholders or A/B testing prompt variants.

Every prompt file follows a consistent structure: Purpose → Version → Expected Inputs → JSON Schema → Step-by-Step Instructions → Failure Modes → Guardrails → Example Outputs → "Why This Matters".

The "Why This Matters" section is the key educational feature. For example, the Roast Agent prompt explains why its design choices are effective:

🎭 Persona-Driven Prompting: The "roast comedian" persona is a deliberate choice — it constrains the model to a specific tone, vocabulary, and ethical framework. Persona prompts are one of the most effective prompt engineering techniques because they activate the model's understanding of social roles.

📎 Specificity = Funniest: The instruction to always reference real profile data is grounded in comedy theory: specific humor outperforms generic humor. "Their GitHub has 47 repos with 0 stars each" is funnier than "they code a lot."

⚖️ The Kindness Gate: Adding a "kindness check" step is an ethical prompt engineering pattern. Rather than a blunt guardrail ("don't be mean"), it frames the check as a social simulation leveraging the model's theory of mind capabilities.

This documentation-first approach turns the prompts directory into an educational resource.

Governance & Safety

The system has two independent safety layers running at different points in the pipeline:

Input Safety (`SafetyChecker`)

Runs before any agent processes data. Detects four threat categories:

// src/lib/safety/index.ts
export class SafetyChecker {
  checkPrompt(input: string): SafetyCheck {
    const threats: SafetyThreat[] = [
      ...this.detectPromptInjection(input),
      ...this.detectJailbreak(input),
      ...this.detectPII(input),
    ];
    return { passed: threats.length === 0, threats };
  }
}

Prompt injection detection uses 19 regex patterns targeting common escape techniques ("ignore all previous instructions", "you are now...", "DAN", "developer mode"). PII detection catches emails, phone numbers, SSNs, credit cards, addresses, and passport numbers.

Output Governance (`GovernanceValidator`)

Runs after agents generate content. Checks against 7 prohibited attributes: race, ethnicity, religion, sexual orientation, mental health, medical diagnosis, political affiliation, and criminal activity.

// src/lib/governance/index.ts
export class GovernanceValidator {
  validate(report: Partial<InvestigationReport>): GovernanceCheck {
    const violations: GovernanceViolation[] = [];

    for (const [fieldName, text] of this.extractTextFields(report)) {
      violations.push(...this.checkText(text, fieldName));
    }
    // Also check facts, strong signals, roasts, etc.

    return {
      passed: violations.length === 0,
      violations: this.deduplicateViolations(violations),
      checkedAt: new Date().toISOString(),
    };
  }

  sanitize(report, violations): Partial<InvestigationReport> {
    // Redact any text matching violation patterns
    const redacted = this.redactText(text, violations);
    return { ...report, digitalProfileSummary: redacted, ... };
  }
}

The governance agent operates in a validate → retry → sanitize loop within the orchestrator. If violations are found, the orchestrator strips offending facts and retries the governance check. If violations persist after MAX_GOVERNANCE_RETRIES, they're sanitized via redaction and recorded as metadata.

Evaluations

The evaluation framework measures every generated report across 5 metrics:

// src/lib/eval/index.ts
computeMetrics(report: InvestigationReport, context: ContextPack) {
  return {
    json_compliance: this.measureJSONCompliance(reportJson),
    consistency: this.measureConsistency(report),
    hallucination_rate: this.measureHallucinationRate(report, context),
    humor_score: this.measureHumorScore(report),
    accuracy: this.measureAccuracy(report, context),
  };
}

JSON Compliance — Checks all required fields exist and match expected types
Consistency — Measures how well facts align with the summary, career prediction, and timeline
Hallucination Rate — Compares fact text against source context using word-overlap analysis
Humor Score — Evaluates variety, intensity distribution, and quality signals in roasts
Accuracy — Inverse of hallucination rate

The evaluation datasets include 20 profiles across 4 categories (developers, designers, founders, creators), each with expected score ranges and minimum fact counts:

{
  "id": "dev-1",
  "name": "Senior Full-Stack Developer",
  "expectedOutputs": {
    "minFacts": 14,
    "keySignals": ["strong technical background", "open source contributor"],
    "expectedScores": {
      "builderScore": { "min": 75, "max": 100 },
      "operatorScore": { "min": 60, "max": 90 }
    }
  }
}

The compareModelsWithConfigs method runs the same dataset against multiple model/provider pairs, generating side-by-side comparisons of latency, cost, and quality scores — enabling data-driven model selection.

Production Features

Cost Tracking

Every agent call records its model, token usage, and estimated cost per model-specific pricing tables:

export class CostTracker {
  async recordCost(trace: AgentTrace, investigationId: string): Promise<CostRecord> {
    const record = {
      provider: trace.provider,
      model: trace.model,
      promptTokens: trace.tokenUsage.promptTokens,
      completionTokens: trace.tokenUsage.completionTokens,
      estimatedCost: estimateCost(trace.model,
        trace.tokenUsage.promptTokens,
        trace.tokenUsage.completionTokens),
      timestamp: new Date().toISOString(),
    };
  }
  // Query methods: getCostByProvider(), getCostByModel(), getCostByAgent()
}

Observability

The ObservabilityTracker stores all agent traces and investigations in-memory (with configurable size limits) and optionally forwards them to LangSmith for production monitoring:

export class ObservabilityTracker {
  async getTraceStats() {
    return {
      total, successful, failed,
      avgLatency, totalCost
    };
  }
  async getAgentPerformance() {
    return Record<agentName, { totalCalls, avgLatency, totalCost, successRate }>;
  }
}

Developer Dashboard

A full dashboard at /dashboard exposes real-time stats: per-agent latency and cost breakdowns, success rates, investigation history, and trace export (JSON/CSV). This turns the production data into actionable insights.

What's Next

The architecture is designed for extension. Here's what's on the roadmap:

Real API integrations — GitHub GraphQL, LinkedIn API, Twitter API for live data instead of pasted text
Browser extension — One-click investigation from any LinkedIn/GitHub profile page
New agents — Writing style analyzer, network graph agent, tech stack deep dive
New providers — Together AI, Groq, Replicate (each is one file)
OG image generation — Shareable report cards for social media
Mobile app — React Native wrapper for native sharing

Conclusion

Internet Detective AI demonstrates that a production-grade multi-agent AI system doesn't require a massive team or budget. The key patterns are:

Provider abstraction so you're never locked into one model
Safe defaults with graceful degradation so partial failures produce useful results
Structured outputs with typed schemas so agents produce machine-parseable results
Governance and safety as architectural layers, not afterthoughts
Documentation-first prompts that double as educational resources
Evaluation as a first-class feature for objective quality measurement

The entire codebase is open source at github.com/harishkotra/internet-detective-ai. Each of the 14 engineering topics covered by this project has its own documentation file in docs/.

Code & more: https://www.dailybuild.xyz/project/170-internet-detective-ai

Building a Confidential AI Agent on Terminal 3: Weather Agent Inside a TDX Enclave

Harish Kotra (he/him) — Sat, 20 Jun 2026 05:32:49 +0000

How to run Rust code inside a hardware TEE that calls OpenWeatherMap — without the API key ever leaving the enclave.

The Problem

AI agents are useless without access to real APIs, real credentials, and real user data. But every external call introduces a trust question: do we trust the agent's operator not to log our API key? Do we trust the node runner not to peek at the response?

Traditional solutions stack SLAs and compliance paperwork. Terminal 3 (T3N) takes a different approach: hardware-enforced confidentiality. Your code runs inside an Intel TDX Trusted Execution Environment — a hardware black box that even the node operator cannot inspect.

This post walks through building a weather agent that proves the pattern end-to-end.

Architecture Overview

┌──────────────┐     encrypted session      ┌──────────────────────┐
│  User CLI    │ ─────────────────────────→ │  T3N Node            │
│  (tsx + SDK) │                             │  ┌────────────────┐  │
└──────────────┘                             │  │  TDX Enclave   │  │
                                             │  │  ┌──────────┐  │  │
                                             │  │  │ Weather  │  │  │
                                             │  │  │ Contract │──┼──┼──→ OpenWeatherMap
                                             │  │  │ (WASM)   │  │  │
                                             │  │  └────┬─────┘  │  │
                                             │  │       │        │  │
                                             │  │  kv_store::get() │  │
                                             │  │       │        │  │
                                             │  │  ┌────▼─────┐  │  │
                                             │  │  │  Sealed  │  │  │
                                             │  │  │  KV Map  │  │  │
                                             │  │  └──────────┘  │  │
                                             │  └────────────────┘  │
                                             └──────────────────────┘

Three layers of protection:

Transport: All communication is encrypted with ML-KEM (post-quantum) through the T3N SDK's session layer
Compute: The WASM contract executes inside an Intel TDX enclave — memory encrypted by the CPU, attestable remotely
Secrets: API keys are written via the control plane (map-entry-set), bypassing even the map's own ACL, and can only be read by kv_store::get() from inside the enclave

The WIT Interface

WebAssembly Interface Types (WIT) declares the contract's boundary — what it imports from the host and what it exports to callers:

package z:tenant-weather@0.1.0;

world tenant-weather {
    // Host capabilities the contract needs
    import host:tenant/tenant-context@1.0.0;  // → tenant-did, contract-id
    import host:interfaces/logging@2.1.0;     // → info, debug, error
    import host:interfaces/kv-store@2.1.0;    // → get, put, delete, scan
    import host:interfaces/http@2.1.0;        // → call (GET/POST/etc.)

    // What callers can invoke
    export contracts;
}

interface contracts {
    record generic-input {
        input: option<list<u8>>,
        user-profile: option<list<u8>>,
        context: option<list<u8>>,
    }

    get-weather: func(req: generic-input) -> result<list<u8>, string>;
}

Every capability the contract has is declared explicitly here. No implicit access to filesystem, network, or secrets. The host enforces this at the Wasm boundary.

Contract Implementation

Reading Secrets Inside the TEE

The contract resolves the tenant DID at runtime, constructs the map name, and reads the API key:

fn get_api_key() -> Result<String, String> {
    let tid = tenant_context::tenant_did();
    let map_name = format!("z:{}:secrets", hex::encode(&tid));
    let bytes = kv_store::get(&map_name, b"weather_api_key")
        .map_err(|e| format!("kv read: {e}"))?
        .ok_or("weather_api_key not found")?;
    String::from_utf8(bytes).map_err(|e| e.to_string())
}

The key was seeded via the SDK's control plane (tenant.executeControl("map-entry-set", ...)) — it writes directly to the underlying storage, bypassing the map's writers ACL. This is a one-time operation done during deployment. After that, no external observer can read it back.

Making HTTP from Inside the Enclave

let resp = http_iface::call(&http_iface::Request {
    method: http_iface::Verb::Get,
    url: format!("{OWM_BASE}/data/2.5/weather?q={}&appid={}&units=metric",
                 city, api_key),
    headers: None,
    payload: None,
})?;

The HTTP call goes through the host's egress proxy. Before the request leaves the enclave, the host checks: does the calling user's authorization grant allow this contract to dial api.openweathermap.org? If not — the request is denied with egress_denied.

Parsing and Returning

let weather = WeatherResp {
    city: req.city,
    temperature: data["main"]["temp"].as_f64().unwrap_or(0.0),
    feels_like: data["main"]["feels_like"].as_f64().unwrap_or(0.0),
    humidity: data["main"]["humidity"].as_u64().unwrap_or(0) as u32,
    description: data["weather"][0]["description"]
        .as_str().unwrap_or("unknown").to_string(),
    wind_speed: data["wind"]["speed"].as_f64().unwrap_or(0.0),
};

serde_json::to_vec(&weather)

The JSON-serialized response is encrypted through the session and returned to the caller — who never saw the API key, never touched OpenWeatherMap directly, and whose only interaction with the TEE was a signed invocation.

Deployment Script

The TypeScript SDK orchestrates the full lifecycle:

// 1. Authenticate
setEnvironment("testnet");
const t3n = new T3nClient({
  wasmComponent: await loadWasmComponent(),
  handlers: { EthSign: metamask_sign(address, undefined, key) },
});
await t3n.handshake();
const did = await t3n.authenticate(createEthAuthInput(address));

// 2. Register the WASM component
const tenant = new TenantClient({ t3n, baseUrl: getNodeUrl(), tenantDid });
const { contract_id } = await tenant.contracts.register({
  tail: "weather-agent", version: "0.1.0", wasm: wasmBytes,
});

// 3. Create sealed KV map (only this contract can access)
await tenant.maps.create({
  tail: "secrets", visibility: "private",
  writers: { only: [contract_id] },
  readers: { only: [contract_id] },
});

// 4. Seed the API key (control-plane bypasses ACL)
await tenant.executeControl("map-entry-set", {
  map_name: tenant.canonicalName("secrets"),
  key: "weather_api_key", value: OWM_API_KEY,
});

// 5. Grant the contract permission to call OpenWeatherMap
await t3n.execute({
  script_name: "tee:user/contracts",
  function_name: "agent-auth-update",
  input: {
    agents: [{
      agentDid: tenantDid,
      scripts: [{
        scriptName, functions: ["get-weather"],
        allowedHosts: ["api.openweathermap.org"],
      }],
    }],
  },
});

// 6. Invoke
const result = await t3n.executeAndDecode({
  script_name: scriptName, function_name: "get-weather",
  input: { city: "Tokyo" },
});

The script is designed to be idempotent — if the contract version already exists, registration is skipped, the map ACL is updated in place, and only the invocation costs tokens on re-runs.

Live Output

$ npm run deploy

authenticated as did:t3n:2c9d71730c17e69e394afbffb973d1a6444f4423
registered z:...:weather-agent as contract id 260
seeded weather_api_key into z:<tid>:secrets
authorized outbound HTTP to api.openweathermap.org

weather result: {
  "city": "Tokyo",
  "temperature": 23.18,
  "feels_like": 23.85,
  "humidity": 88,
  "description": "moderate rain",
  "wind_speed": 4.78
}

Why This Matters

The weather agent is deliberately minimal, but the pattern generalizes to any confidential agent workflow:

Scenario	Secret	External API	Pattern
Weather	OpenWeatherMap key	api.openweathermap.org	This demo
Payroll	Bank API credentials + PII	bank.disbursement.com	`http-with-placeholders` for PII substitution
Travel booking	Duffel/Amadeus API key	api.duffel.com	Same pattern + PII placeholders
KYC/Identity	Veriff/Onfido key + user docs	api.veriff.com	PII substitution; user profile never in WASM memory
Healthcare	EHR API token + patient data	fhir.ehr.com	PII substitution; HIPAA-relevant audit trail

The key insight: the contract holds credentials, not the agent. The agent calls the contract; the contract uses the credentials inside the TEE. If the agent is compromised, the attacker gets a function call — not the API key.

The Full Stack

Layer	Technology
Contract language	Rust
Target	`wasm32-wasip2` (WASI Preview 2 component)
Code generation	`wit-bindgen` 0.49
Host interfaces	`kv-store`, `http`, `logging`, `tenant-context`
Client SDK	`@terminal3/t3n-sdk` 3.9.0
TEE technology	Intel TDX (trusted execution environment)
Network	T3N testnet (public sandbox with test tokens)
External data	OpenWeatherMap API
Serialization	`serde` + `serde_json` (no-std, alloc-only)

Try It Yourself

Get a T3N API key at terminal3.io/claim-page (20,000 free test tokens)
Get an OpenWeatherMap API key
Clone the repo, run npm install && cargo build --release && npm run deploy

Full source code and documentation: https://github.com/harishkotra/weather-agent

Built with Terminal 3 — the agent developer kit for confidential, auditable AI agents.

RAG vs HippoRAG: Solving a Detective Mystery with Knowledge Graphs

Harish Kotra (he/him) — Fri, 19 Jun 2026 18:04:29 +0000

A hands-on comparison of vector search retrieval and graph-based Personalized PageRank on a multi-hop reasoning task — running fully locally.

Standard RAG (Retrieval-Augmented Generation) is everywhere. Embed your documents, store them in a vector database, search by cosine similarity, feed the top results to an LLM. It works well for simple fact lookups. But what happens when your query requires connecting information across multiple documents?

"Who had both a motive and the opportunity to steal the pearl necklace?"

This question appears simple, but it requires the system to:

Know who was in the mansion during the theft (D4: Victoria Crane's visit)
Know who had financial troubles (D8: Victoria lost her job)
Connect that the figure in the security footage (D7) could be Victoria

Standard RAG retrieves individual chunks independently. It might get D8 (financial trouble) and D7 (security footage) but has no explicit mechanism to connect them. The LLM has to infer the link from the text alone.

HippoRAG, introduced in this paper, takes a different approach: build a knowledge graph from your documents, then use Personalized PageRank to retrieve passages. Entities in the question act as "seed nodes" that spread probability through the graph, naturally aggregating related evidence.

Let's build both systems from scratch and see where each shines.

The Setup

We created "The Purloined Pearl" — a detective case with 10 witness statements, alibi records, and evidence logs about a stolen $2M necklace. Here's a sample of the evidence:

{
  "id": "D4",
  "title": "Victoria Crane's Visit",
  "text": "Mrs. Blackwood's sister, Victoria Crane, visited the mansion at 8 PM for dinner. She left at 9:30 PM, visibly upset after an argument with Mr. Blackwood."
},
{
  "id": "D8",
  "title": "Victoria's Financial Trouble",
  "text": "Victoria Crane had recently lost her job and was facing foreclosure on her home. Mr. Blackwood had refused to lend her money."
}

Notice that "Victoria Crane" appears in multiple documents — D4, D6, D8, D9, D10. A vector DB stores each document independently. A knowledge graph can explicitly connect all documents mentioning "Victoria Crane" through a shared entity node.

Architecture

Both systems share a common LLM backend and embedding pipeline. The difference is purely in retrieval strategy.

                    ┌─────────────────────┐
                    │   LM Studio Server   │
                    │  (OpenAI-compatible)  │
                    └─────────┬───────────┘
                              │
              ┌───────────────┼───────────────┐
              │               │               │
              ▼               ▼               ▼
        ┌──────────┐   ┌──────────┐   ┌──────────┐
        │llm_complete│  │  embed() │   │  embed() │
        └─────┬─────┘   └────┬─────┘   └────┬─────┘
              │              │              │
     ┌────────┴────────┐    │              │
     │                 │    │              │
     ▼                 ▼    ▼              ▼
┌──────────┐   ┌──────────┐   ┌───────────────────┐
│Standard  │   │ OpenIE   │   │  ChromaDB         │
│RAG QA    │   │ Triple   │   │  (Vector Index)    │
│          │   │Extraction│   └───────────────────┘
└──────────┘   └────┬─────┘
                    │
                    ▼
              ┌──────────┐   ┌───────────────────┐
              │Knowledge │   │  PPR Retrieval     │
              │ Graph    │──►│  (NetworkX)        │
              │(NetworkX)│   └─────────┬─────────┘
              └──────────┘            │
                                      ▼
                               ┌──────────┐
                               │HippoRAG  │
                               │QA        │
                               └──────────┘

Building the Systems

The LLM Client

Both systems talk to LM Studio through an OpenAI-compatible wrapper:

def llm_complete(prompt, model="qwen/qwen3.5-9b", max_tokens=2048):
    resp = requests.post(
        "http://127.0.0.1:1234/v1/chat/completions",
        json={"model": model, "messages": [{"role": "user", "content": prompt}],
              "temperature": 0, "max_tokens": max_tokens}
    )
    msg = resp.json()["choices"][0]["message"]
    return msg.get("content", "").strip()

def embed(texts, model="nomic-embed-text-v1.5:2"):
    resp = requests.post(
        "http://127.0.0.1:1234/v1/embeddings",
        json={"input": texts}
    )
    return [d["embedding"] for d in resp.json()["data"]]

This works with LM Studio, Ollama, or any OpenAI-compatible server with a single URL change.

Standard RAG: ChromaDB + Vector Search

Standard RAG is straightforward — embed everything, store in ChromaDB, search by cosine similarity:

def index_docs(docs):
    collection = chromadb_client.create_collection("detective_case")
    embeddings = embed([d["text"] for d in docs])
    collection.add(ids=ids, documents=texts, embeddings=embeddings)
    return collection

def retrieve(collection, query, top_k=3):
    q_emb = embed([query])[0]
    results = collection.query(query_embeddings=[q_emb], n_results=top_k)
    return extract_results(results)

The pain point: each document is an isolated vector. There's no link between D4 (Victoria's visit) and D8 (her financial trouble) even though they describe the same person. The LLM must infer this connection from the raw text.

HippoRAG: Knowledge Graph + Personalized PageRank

HippoRAG adds two steps before retrieval:

Step 1: Open Information Extraction

We prompt a small LLM to extract (subject, relation, object) triples from each document:

TRIPLE_PROMPT = """
Return a JSON array of triples, each with "subject", "relation", "object" keys.

Text: {text}
"""

def extract_triples(text, doc_id):
    raw = llm_complete(TRIPLE_PROMPT.format(text=text),
                       model="google/gemma-4-e4b")
    triples = json.loads(extract_json(raw))
    return [t | {"doc_id": doc_id} for t in triples]

For D4, the LLM might extract:

[
  {"subject": "Victoria Crane", "relation": "visited", "object": "mansion"},
  {"subject": "Victoria Crane", "relation": "left_at", "object": "9:30 PM"},
  {"subject": "Victoria Crane", "relation": "argued_with", "object": "Mr. Blackwood"}
]

Step 2: Build the Knowledge Graph

We use NetworkX to create a graph with two node types:

Entity nodes (entity:Victoria Crane) — people, places, objects
Passage nodes (passage:D4) — the original documents

Edges connect entities to passages they appear in, and entities to other entities via triple relations:

class KnowledgeGraph:
    def add_triple(self, subj, rel, obj, doc_id):
        passage_id = f"passage:{doc_id}"
        self.graph.add_edge(f"entity:{subj}", passage_id)
        self.graph.add_edge(f"entity:{obj}", passage_id)
        self.graph.add_edge(f"entity:{subj}", f"entity:{obj}",
                            triple=f"{subj} --[{rel}]-> {obj}")

This creates a graph like:

entity:Victoria Crane ──mentioned_in── passage:D4
entity:Victoria Crane ──mentioned_in── passage:D8
entity:Victoria Crane ──argued_with── entity:Mr. Blackwood
entity:Victoria Crane ──lost_job── entity:job

Step 3: Personalized PageRank Retrieval

When a query arrives, we extract its entities and use them as seed nodes for Personalized PageRank:

def retrieve_hipporag(graph, question, top_k=3):
    entities = extract_query_entities(question)  # e.g., ["Victoria Crane"]

    personalization = {node: 0.0 for node in graph.graph.nodes()}
    for entity in entities:
        node = f"entity:{entity}"
        if node in personalization:
            personalization[node] = 1.0

    # Normalize and run PPR
    pr = nx.pagerank(graph.graph, alpha=0.5,
                     personalization=personalization)

    # Aggregate scores to passage nodes
    passage_scores = defaultdict(float)
    for node, score in pr.items():
        if node.startswith("passage:"):
            passage_scores[node] += score
        elif node.startswith("entity:"):
            for neighbor in graph.graph.neighbors(node):
                if neighbor.startswith("passage:"):
                    passage_scores[neighbor] += score * 0.5

    return top_k passages by score

The alpha=0.5 damping factor controls how much probability stays at seed nodes vs. propagates through the graph. Lower values = more aggressive spread.

Results: Where Each System Excels

Simple Queries (single-document fact lookups)

Q: "What time did Detective Miller arrive at the mansion?"

Both systems retrieve D1 and answer correctly: "10 PM". Standard RAG does it faster (no graph build needed).

Winner: Standard RAG (speed, simplicity).

Multi-hop Queries (cross-document reasoning)

Q: "Who had both a motive and the opportunity to steal the pearl necklace?"

This requires:

D4: Victoria was at the mansion (opportunity)
D7: A figure was in the hallway at 9:15 PM (opportunity timeline)
D8: Victoria lost her job, Mr. Blackwood refused money (motive)

Standard RAG retrieves documents by embedding similarity. "Motive and opportunity" is a semantic query — it might retrieve D8 (motive), D4 (opportunity), and D7 (footage). But each is retrieved independently; there's no structural reason they should be connected. The LLM must piece it together from the text.

HippoRAG extracts entity "Victoria Crane" from the question. PPR spreads probability from entity:Victoria Crane to all connected passages (D4, D6, D8, D9, D10) and their neighboring entities. The result: all evidence about Victoria surfaces together, regardless of which document it's in.

Winner: HippoRAG (structural evidence aggregation).

Sense-making Queries

Q: "Given the timing of events, the suspect's circumstances, and the physical evidence, who is the most likely thief?"

Standard RAG retrieves the top-3 documents by vector similarity. If D4 (visit), D8 (finances), and D10 (coat) rank highest, D7 (footage) and D9 (photo ID) might be missed.

HippoRAG's PPR naturally distributes probability across the entire subgraph connected to "Victoria Crane", potentially surfacing all relevant documents.

Winner: HippoRAG (broader evidence coverage).

Key Implementation Details

Handling Reasoning Models

Modern LLMs like Qwen and Gemma include chain-of-thought reasoning that can consume output tokens. Two strategies for clean extraction:

Separate models — Use a smaller, non-reasoning model (google/gemma-4-e4b) for structured extraction tasks where you need clean JSON output.
Reasoning content handling — OpenAI-compatible APIs return reasoning in a separate reasoning_content field for supported models:

msg = response["choices"][0]["message"]
content = msg.get("content", "")
if not content and "reasoning_content" in msg:
    content = msg["reasoning_content"]

PPR Damping Factor

The damping factor α controls exploration vs. exploitation:

High (0.85): Probability stays near seed entities — good for precision
Low (0.15): Probability spreads widely — good for recall
Medium (0.50): Balanced — worked best in our tests

Fallback Entity Extraction

If the LLM fails to extract JSON entities from the query, we fall back to a simpler comma-separated extraction:

if not entities:
    raw_keywords = llm_complete("Extract entities as comma-separated list: " + question)
    entities = [e.strip() for e in raw_keywords.split(",") if e.strip()]

Running It Yourself

# Setup
python3 -m venv venv && source venv/bin/activate
pip install chromadb networkx requests streamlit

# Start LM Studio with models loaded:
# - qwen/qwen3.5-9b (QA)
# - google/gemma-4-e4b (extraction)
# - nomic-embed-text-v1.5:2 (embeddings)

# CLI comparison
python run_comparison.py

# Streamlit dashboard
streamlit run streamlit_app.py

Limitations & Future Work

LLM-based triple extraction is slow — Every document requires an LLM call. For 10 documents it's manageable; for 10,000 you'd want a dedicated OpenIE system.
Small graph — Our 57-node, 90-edge graph is trivial. At scale, you'd need to partition the graph or use approximate PPR.
Chunk independence — Each document is a single passage. In production, documents would be chunked, with each chunk as a separate passage node.
No IDF weighting — The real HippoRAG paper weights entities by inverse document frequency. Common entities like "the" or "necklace" would be downweighted.
Deterministic OpenIE — Tools like Stanford OpenIE or SPICE extract triples without an LLM, making the pipeline faster and more predictable.

Standard RAG is fast, simple, and effective for fact lookup. But when your questions require connecting information across documents — the kind of multi-hop reasoning that comes naturally to a human detective — a knowledge graph approach like HippoRAG provides structural advantages that vector similarity alone cannot match.

Screenshots

Code & more: https://www.dailybuild.xyz/project/168-detective-demo

DEV Community: Harish Kotra (he/him)

Mycelium: A Multi-Agent Memory Mesh with Provenance, Trust, and Contradiction Resolution

The Problem

Architecture

Tech Stack

Core Concepts

1. Agent

2. CogneeClient

3. Diff Engine (Three-Pass)

4. Contradiction Resolution

5. Sync Protocol

6. Provenance

LLM Provider Configuration

The Travel Assistant Demo

Directory Structure

How to Fork and Contribute

Feature Ideas for Contributors

Design Principles

Hover now supports Custom Actions

What changed

Why this matters

How it works

What we built

Settings experience

Floating panel integration

Storage

App wiring

Privacy and control

Who this helps

Example workflows

What comes next

Building Phone Cleaner: A Desktop App to Manage WhatsApp Media via ADB

The Architecture

Why Electron?

Scanning WhatsApp Media

Lazy-Loaded Thumbnails

The Keep/Delete Workflow

Delete History

Design Decisions

No Build Step

No Semicolons

Dark Theme Only

Possible Improvements

Try It

Screenshots

Branch Agent: Git-Style Branching for LLM Conversations

The Problem

The Idea

Architecture

Why Convex?

Why Agno?

The Branch Model (Database Schema)

Forking is O(1)

History Traversal

Streaming Agent Responses

The Python Agno Service

UI: Side-by-Side Compare

Merging with a Judge Agent

Running the Project

What's Next

Tech Stack Summary

slacktag-oss Part 2: Three Features That Make Memory Actually Useful

What we shipped in Part 1

Feature 1: !summarize

Feature 2: Reaction-to-memory (📌 / 🗑️)

Feature 3: !catchmeup — personalized channel digests

The pattern across all three features

What's next

Links

Building a Slack Bot That Actually Remembers: slacktag-oss

The problem with Slack bots and memory

What I built

Architecture

Project structure

The memory layer: why Mem0

Setting up the client

Memory scoping

BaseMemory interface

The LLM layer: LangChain + any OpenAI-compatible endpoint

The handler: where memory meets LLM

Feature 1: `!summarize`

Feature 3: `!catchmeup` — personalized channel digests

1. The Agent Core (`agent/detective.ts`)

2. The Investigation Tools (`agent/tools.ts`)

3. SSE Streaming (`app/api/detective/route.ts`)

4. The Web UI (`app/page.tsx`)

5. Workflow Snapshots (`agent/workflow-store.ts`)