DEV Community

Cover image for Building a Lightweight AI Agent in Go: Baize's Architecture and Trade-offs
rebornace
rebornace

Posted on

Building a Lightweight AI Agent in Go: Baize's Architecture and Trade-offs

#ai #go #agents #opensource

Preface

Baize is a "sidecar" AI assistant runtime: a single process that sits beside services you already run, turns API documentation into tools the assistant can call, and pauses important writes until a person approves them. Stop it and it leaves almost nothing behind.

One sentence positioning: it's a runtime, not a framework. That positioning drives every architecture decision below.

1. Why Go

Single binary, zero dependencies

The core requirement of sidecar deployment is "one process, copy it over, run it." A Go binary needs no interpreter, no dependency installation, no virtualenv on the target machine. For an assistant meant to live in enterprise environments, that's the lowest-cost delivery form.

Goroutine concurrency

One assistant process serves multiple entry points at once: the web console, signed alert/ticket ingress, and IM channels. Goroutines make "one process handling many sessions concurrently" straightforward — and parallel tool calls later become almost free.

Cross-compilation

Enterprise environments run everything: Windows, Linux, macOS, ARM. A single GOOS=linux GOARCH=arm64 go build produces a binary for the target platform, with no toolchain setup on the destination machine.

Static typing and tool contracts

Tool input schemas come from OpenAPI documents; mapping them onto Go's strong types catches many errors at compile time. For a long-running daemon, that saves a lot of operational pain.

2. Architecture Overview

A clean three-layer structure: core loop → tool router → executors.

Users / Channels ──► Agent Core Loop (think → pick tool → run → report)
                           │
                           ▼
                     Tool Router (Registry)
                           │
          ┌────────────────┼────────────────┐
    OpenAPI connector    HTTP plugin      MCP connector
          └────────────────┼────────────────┘
                           ▼
              Invoker closures (registered in Registry)
                           │
                     [ HITL approval gate ]
                           │
             ┌─────────────┴─────────────┐
       Direct execution          HTTP callback executor
     (plugin / proxy)            (callback to your side)
Enter fullscreen mode Exit fullscreen mode

  • Core loop (internal/run): LLM thinks → picks a tool → executes → reports. The event stream (llm.thinking, llm.tool_call, tool.result) is persisted, so the console can follow along in real time.

  • Tool router (internal/tool): a mutex-protected map that registers not functions but "tool contracts + invoker closures".

  • Executors (internal/connector): tools enter through three sources — OpenAPI docs, HTTP plugins, MCP tool servers — plus a "callback execution" mode that hands execution back to your side.

3. Key Design Decisions

Why HTTP callback instead of a plugin protocol

This is Baize's most important trade-off.

The problem with plugin protocols: in-process plugins (Go plugins, shared libraries, language-binding SDKs) require the plugin to compile with the host process — language, version, and ABI must all align. Most enterprise systems aren't written in Go: legacy systems, Java/.NET/Python services can't load an in-process plugin at all. And even when they could, upgrading a plugin means restarting the process — "sidecar in, clean out" is gone.

What Baize does instead: it doesn't execute the tool itself; it POSTs the invocation to your own endpoint:

{
  "tool": "create_ticket",
  "arguments": { ... },
  "run_id": "run_xxx",
  "agent_id": "agent_xxx",
  "idempotency_key": "uuid-xxx",
  "callback_urls": { "event": "https://your-service/baize-events" }
}
Enter fullscreen mode Exit fullscreen mode

Your side executes it and returns the result. Benefits: language-agnostic, process-isolated, auditable; idempotency_key makes retries safe (no duplicate execution); callback_urls lets your side keep driving follow-up actions.

The cost: one extra network round-trip, the callback endpoint must be reachable, and to prevent forged callbacks you need signed requests (Baize uses callback signing with a TTL against replay).

Dynamic tool registration and discovery

The registry (tool.Registry) is the core data structure: a sync.RWMutex guarding a map, with runtime register/unregister and per-connector bulk unregister — adding a tool or disabling a connector never requires a restart.

Three tool sources share one registration path:

  • OpenAPI docs: import Swagger/OpenAPI/Postman-style docs, each operation becomes a tool;

  • HTTP plugins: a small companion service declares "what tools exist and how to invoke them";

  • MCP tool servers: connect to external tool ecosystems as an MCP client.

Security policy is baked into each entry at registration time: require_approval (needs a human), require_login (needs a session), security_schemes (which auth scheme to use). Security policy is decided at registration, not asked at execution time — this is the precondition for letting the assistant actually act.

Discovery is trivial: Registry.List() / Registry.Specs() feed the model's tool list, visible live in the console.

Graceful degradation on failure

Failures are the norm in AI agents, so degradation design matters more than the happy path:

  • Timeout guardrail: every tool invocation is bound to context.WithTimeout (default 60s, configurable);

  • Failure is content: Invoker returns (content, isError, err) — err is an infrastructure failure (timeout, network), isError is a business-side failure. Both flow back to the model as structured content, so the model can retry, switch tools, or explain to the user — instead of crashing the whole session;

  • Approval rejection is not a crash: when a human rejects a write, the run settles into an explicit "rejected" terminal state with a trail, no panic;

  • Everything is observable: the llm.tool_call → tool.result event stream is persisted, so any problem can be traced step by step;

  • Context compaction: long sessions get rolling summaries so quality doesn't degrade as the thread grows.

4. Go vs Python / Node.js for Agent Scenarios

Let's be honest first: Python is the best choice in the AI/Agent ecosystem. LangChain, LlamaIndex and most reference implementations live there. If your goal is fast experimentation and deep reuse of the LLM ecosystem, Python has no rival.

Baize chose Go because its positioning is different:

Dimension Go Python Node.js
Deployment Single binary, zero deps Interpreter + deps/venv Node runtime + node_modules
Resource footprint Low; one resident process is cheap Higher; resident processes need care Medium
Concurrency Native goroutines GIL-limited; multi-process/async Event loop
Type safety Static, compile-time checks Dynamic, found at runtime Dynamic / TypeScript
LLM ecosystem Newer, catching up fast Richest Rich
Cross-platform Cross-compile to all platforms Needs interpreter on target Needs Node on target

The conclusion isn't "Go is better than Python" — it's positioning decides the language:

  • Goal: framework / fast experimentation → Python;

  • Goal: sidecar, resident, one-command deployment to enterprise environments, running on modest hardware for a long time → Go's advantages in deployment and resource usage are hard to replace.

5. Core Code Snippets (Go)

All snippets are from the project source, lightly trimmed. Each comes with one line on what problem it solves.

1. A tool = contract + invoker closure

Modeling a "tool" as "a contract the model sees (Spec) + an invoker closure injected by the connector" fully decouples routing from execution:

type Invoker func(ctx context.Context, args map[string]any) (
    content map[string]any, isError bool, err error)
type Meta struct {
    Spec            llm.ToolSpec
    ConnectorID     string
    Method          string
    Path            string
    RequireLogin    bool
    SecuritySchemes []string
}
Enter fullscreen mode Exit fullscreen mode

2. Dynamic registration with policy baked in

require_approval / require_login are written into the entry at registration; the tool list is "hot" — adding/removing connectors never requires a restart:

func (r *Registry) RegisterMeta(meta Meta, inv Invoker, requireApproval bool) {
    r.mu.Lock()
    defer r.mu.Unlock()
    r.tools[meta.Spec.Name] = entry{
        spec:            meta.Spec,
        invoker:         inv,
        requireApproval: requireApproval,
        requireLogin:    meta.RequireLogin,
        connectorID:     meta.ConnectorID,
        method:          meta.Method,
        path:            meta.Path,
    }
}
Enter fullscreen mode Exit fullscreen mode

3. HTTP callback executor

Hands execution back to your side; the idempotency key makes network retries safe:

payload := map[string]any{
    "tool":            tool,
    "arguments":       args,
    "run_id":          meta.RunID,
    "agent_id":        meta.AgentID,
    "idempotency_key": meta.IdempotencyKey,
}
if strings.TrimSpace(meta.CallbackEventURL) != "" {
    payload["callback_urls"] = map[string]any{
        "event": meta.CallbackEventURL,
    }
}
rawPayload, _ := json.Marshal(payload)
req, _ := http.NewRequestWithContext(ctx, http.MethodPost, c.URL, bytes.NewReader(rawPayload))
Enter fullscreen mode Exit fullscreen mode

4. Writes automatically enter the approval gate

Non-GET/HEAD/OPTIONS operations are automatically flagged "needs approval" at registration, and only run after a human clicks approve/reject in the console:

needApproval := t.RequireApproval
if ctx.requireApprovalMutating && isMutatingMethod(t.Method) && t.Source == store.ToolSourceSpec {
    needApproval = true
}
Enter fullscreen mode Exit fullscreen mode

5. Timeouts and failure degradation

A timeout guardrail plus "failure is content" semantics keeps one bad tool call from blowing up the whole session:

toolCtx, toolCancel := context.WithTimeout(ctx, e.toolTimeout())
defer toolCancel()
content, isError, invErr := e.Tools.Invoke(toolCtx, payload.ToolName, payload.Arguments)
if invErr != nil {
    // Infrastructure failure (timeout/network): persist and close the round
    return e.finalizeFailedRun(runID, invErr)
}
// When isError is true, the failure flows back to the model as content;
// the model decides whether to retry or explain.
Enter fullscreen mode Exit fullscreen mode

Closing

Baize is still early. The trade-offs above are far from "optimal" — especially the approval UX, channel adapters, and executor extensibility. If you have real-world scenarios, I'd love to hear them.

Top comments (0)