Branch Agent: Git-Style Branching for LLM Conversations

Fork, branch, and merge AI conversations like code — with different models, prompts, and providers in each parallel timeline.

---

The Problem

When experimenting with LLMs, you've probably done this: tweak the system prompt, rerun the conversation, compare outputs side-by-side in two browser tabs, manually copy-paste the better response, and repeat. It's messy, non-reproducible, and there's no version history.

What if LLM conversations had the same branching model as Git?

The Idea

A conversation is a tree, not a list. Every message can be a fork point. New branches inherit the full context up to that point via pointer references (O(1), no copying). Each branch gets its own agent configuration — system prompt, model, provider, temperature, tools. Branches can be compared side-by-side and merged back via an AI "Judge Agent."

Architecture

┌──────────────┐      Convex hooks       ┌──────────────────┐
│  Next.js UI  │ ◄─────────────────────► │  Convex Database │
│  (React 19)  │     useQuery/mutation   │  (workspaces,    │
└──────┬───────┘                         │   branches, msgs)│
       │ HTTP POST /chat (SSE stream)    └──────────────────┘
       ▼
┌──────────────────┐
│  Python FastAPI  │  ← Agno Agent SDK
│  Agno Service    │     Creates agent per request with branch config
└──────┬───────────┘
       │ OpenAI-compatible API
       ▼
┌──────────────────┐
│  Any LLM Provider│  ← OpenAI, Together, Groq, Ollama...
└──────────────────┘

Why Convex?

Convex provides:

• Reactive queries — UI subscribes to data via WebSockets; when the agent streams a response token-by-token, the UI re-renders automatically
• ACID mutations — forkBranch is atomic and isolated
• Deterministic actions — the chatWithAgent action runs outside the transaction but can call internal queries/mutations safely
• Type-safe — full TypeScript types generated from the schema

Why Agno?

Agno is a Python-native agent framework that supports:

• Multiple model providers through a unified interface
• Tool integration (web search, calculator, file I/O, etc.)
• Streaming responses with intermediate events (tool calls, reasoning steps)
• Structured output via Pydantic schemas

The Branch Model (Database Schema)

The schema is deliberately relational to support tree traversal:

// convex/schema.ts
export const agentConfigSchema = v.object({
  systemPrompt: v.optional(v.string()),
  model: v.optional(v.string()),
  baseUrl: v.optional(v.string()),   // per-branch provider URL
  apiKey: v.optional(v.string()),    // per-branch API key
  tools: v.optional(v.array(v.string())),
  temperature: v.optional(v.number()),
  maxTokens: v.optional(v.number()),
});

export const branches = defineTable({
  workspaceId: v.id("workspaces"),
  name: v.string(),
  parentBranchId: v.optional(v.id("branches")),
  snapshotMessageId: v.optional(v.id("messages")),
  agentConfig: v.optional(agentConfigSchema),
  isMerged: v.optional(v.boolean()),
  mergeSummary: v.optional(v.string()),
})
  .index("by_workspace", ["workspaceId"])
  .index("by_parent", ["parentBranchId"]);

export const messages = defineTable({
  branchId: v.id("branches"),
  parentMessageId: v.optional(v.id("messages")),
  role: v.union(v.literal("user"), v.literal("assistant"), v.literal("system")),
  content: v.string(),
  metadata: v.optional(messageMetadataSchema),
})
  .index("by_branch_created", ["branchId", "createdAt"]);

The key insight: snapshotMessageId on a branch points to the message where it forked. History reconstruction walks parentBranchId → snapshotMessageId pointers recursively. This makes forks O(1) in storage — no message duplication.

Forking is O(1)

export const forkBranch = mutation({
  args: {
    sourceBranchId: v.id("branches"),
    snapshotMessageId: v.id("messages"),
    newBranchName: v.string(),
    agentConfig: v.optional(agentConfigSchema),
  },
  handler: async (ctx, args) => {
    // Just create a new branch with pointer references
    // No messages are copied
    return await ctx.db.insert("branches", {
      workspaceId: sourceBranch.workspaceId,
      name: args.newBranchName,
      parentBranchId: args.sourceBranchId,
      snapshotMessageId: args.snapshotMessageId,
      agentConfig: args.agentConfig ?? sourceBranch.agentConfig,
      createdAt: Date.now(),
    });
  },
});

History Traversal

When a user sends a message, the action fetches the full context by walking the branch tree:

export const internalGetBranchHistory = internalQuery({
  handler: async (ctx, args) => {
    const branch = await ctx.db.get(args.branchId);
    const myMessages = await ctx.db
      .query("messages")
      .withIndex("by_branch_created", (q) => q.eq("branchId", args.branchId))
      .order("asc")
      .collect();

    if (!branch.parentBranchId || !branch.snapshotMessageId) {
      return myMessages;  // root branch, just our messages
    }

    // Walk up the parent tree to the snapshot point
    const parentMessages = await traverseToSnapshot(
      ctx, branch.parentBranchId, branch.snapshotMessageId
    );
    return [...parentMessages, ...myMessages];
  },
});

Streaming Agent Responses

The chatWithAgent action sends the full history to the Python Agno service, which streams tokens back via SSE:

// Convex action reads branch config, sends to Agno service
const agnoPayload = {
  messages: conversationMessages,
  system_prompt: branch.agentConfig?.systemPrompt,
  model: branch.agentConfig?.model,
  base_url: branch.agentConfig?.baseUrl,
  api_key: branch.agentConfig?.apiKey,
  tools: branch.agentConfig?.tools,
  temperature: branch.agentConfig?.temperature,
  stream: true,
};

// Parse SSE events and update message content in real-time
for await (const sseEvent of sseReader) {
  if (parsed.type === "content" && parsed.content) {
    fullContent += parsed.content;
    await ctx.runMutation(internalUpdateMessageStream, {
      messageId: assistantMessageId,
      content: fullContent,
    });
  }
}

Each token delta updates the Convex document, which triggers the reactive useQuery hook on the frontend — the UI streams the response smoothly.

The Python Agno Service

# agno_service/agent_handler.py
def create_agent(
    system_prompt: str = None,
    model_name: str = None,
    base_url: str = None,
    api_key: str = None,
    tool_names: list[str] = None,
    temperature: float = None,
    max_tokens: int = None,
) -> Agent:
    model = _resolve_model(
        model_name or AGNO_DEFAULT_MODEL,
        temperature, max_tokens,
        base_url=base_url, api_key=api_key,
    )
    tools = _resolve_tools(tool_names or [])
    return Agent(
        model=model,
        tools=tools or None,
        instructions=[system_prompt] if system_prompt else None,
    )

The service creates a fresh Agent per request — no state leakage between branches. Each branch can point to a completely different provider.

UI: Side-by-Side Compare

The compare view lets you see two branches at the same time. This is particularly useful when testing different system prompts or models against the same conversation history:

<CompareView
  leftBranch={{ name: "main (GPT-4o)", messages }}
  rightBranch={{ name: "fork (Llama)", messages: compareMessages }}
/>

Both panels scroll independently, and messages from forked history are tagged with their source branch name.

Merging with a Judge Agent

When a fork has answered its questions, you can merge it back into the parent branch. The mergeWithJudge action sends both branches' histories to an Agno agent with a "judge" prompt that summarizes the key learnings:

You are a merge judge. Summarize the key learnings, differences, and insights
from the SOURCE branch, and produce a concise merge summary.

The summary is inserted as a system message in the target branch, and the source branch is marked as merged.

Running the Project

# Three terminals:
npx convex dev                    # Convex backend
cd agno_service && python3 main.py  # Agno Python service
npm run dev                       # Next.js frontend

Or one command: ./start.sh

What's Next

This project is a cookbook / reference implementation. Here are ideas for extending it:

• Diff view — highlight diverging messages between branches
• Prompt playground — preview how all branches would respond to a draft message
• Export/import branches — share experiments as JSON files
• Auto-resolve merge conflicts — when both branches modified the same context, use the Judge Agent to reconcile
• Branch templates — pre-configured branches for specific tasks (research, code review, creative writing)

Tech Stack Summary

Component	Technology	Why
Database	Convex	Reactive WebSockets, ACID, type-safe
Agent SDK	Agno (Python)	Multi-provider, streaming, tools
Frontend	Next.js 15 + React 19	App Router, server components
Styling	Tailwind CSS v4 + shadcn/ui	Utility-first, dark mode
AI Provider	Any OpenAI-compatible API	BYO provider

Code & more: https://www.dailybuild.xyz/project/177-convex-branch-agent

Comments

[Alex Shev]

Branching conversations is a strong idea because agents explore possibilities faster than humans can review them. The missing piece is usually merge discipline: what evidence from a branch survives, what gets discarded, and how the final answer proves which path actually changed the artifact.