DEV Community: rebornace

Baize v0.4.0: Tool Matching Upgraded — Optional Vector Retrieval, Lexical Baseline Unchanged

rebornace — Thu, 08 Oct 2026 02:27:30 +0000

What this release is about

Baize is a team-facing AI assistant runtime: you point it at an existing service, upload an OpenAPI / Swagger-style doc, and the endpoints become tools the assistant can call. A real internal backend, once wired up, usually brings hundreds of tools into the catalog.

That scale creates a problem: stuffing hundreds of tool schemas into every prompt is expensive, and the model has to pick from a very long list. Baize already solved the first part with a decision layer — before the main model call, a cheap deterministic step narrows the candidate tools. Published numbers: against 37 real read-only requests across 3 backends (390 tools total, DeepSeek-Flash), narrowing to the default 16 candidates cut turn-0 prompt tokens by about 34%, and 185 runs across 5 loops finished at 99.5% success.

The lexical baseline, however, has a known ceiling: it ranks by tokens. Same meaning, different wording, and a tool may simply not surface. v0.4.0 enhances that step without touching the baseline.

What v0.4.0 adds

Baize's tool retrieval has always been Tool-RAG in spirit: each tool is indexed as a small document (name, description, HTTP method and path, parameter names).

The lexical baseline stays exactly as it was. Every tool document gets a BM25 index plus a sparse TF-IDF index; at query time the two scores are fused with RRF. It ships on by default, needs zero dependencies, and stays the zero-friction path.

What's new is the dense channel. Once an embedder is configured, each tool document also gets a vector. At query time, the query is embedded and cosine similarity is computed — and in the fusion, the dense channel is the primary one, with BM25 demoted to an auxiliary channel. The two are merged again with RRF before being handed to the model. Division of labor: BM25 catches exact terms — proper nouns and fixed phrases in your business domain; the dense channel catches paraphrases, so a user asking the same thing with different words still surfaces the right tool. The mixed mode and the pure lexical mode are distinct internal states, observable and reversible.

You can pick either embedding source:

Local Ollama. For setups that don't want to send text to a cloud embedder. The settings page walks through the whole flow: one-click Ollama install , pulling the matching model, showing install and model paths, a custom models directory, and a clean uninstall when you don't need it. No command line required.
Any OpenAI-compatible Embedding API. If you already have a compatible endpoint, fill in the base URL and key — the same habit Baize uses for chat models.

Design tradeoffs

Making this an optional enhancement rather than the default was deliberate:

Zero-cost onboarding. Fresh install, lexical matching just works. You only opt into the dense channel when you feel lexical recall isn't enough.
Fail open to the lexical path. If the dense channel misbehaves — Ollama not ready, embedding API timeout — retrieval automatically falls back to the pure lexical index and the conversation continues. Consistent with the runtime's fail-open philosophy: speedups should never become the new failure point.
Narrowing never disables tools. Pruning only decides which schemas land in this step's prompt; any registered tool still runs if the model names it directly. When nothing matches, it falls back to the full set, and system / login tools are always retained.
Easier to find. The runtime settings page is now organized into tabs: common, speedups, memory & compaction, security. Tool matching lives under "speedups".

Reproducibility

The release also ships an evidence evaluation script: it replays and scores tool-call traces. After a change to matching, one run tells you whether recall regressed, instead of relying on gut feel. Corpus and script are in the repo.

(The previous v0.3.x line already brought the web workspace, long-horizon projection — model-facing hints only, saved chats never rewritten — and a built-in self-help skill. This post focuses on the v0.4.0 matching upgrade; the README covers everything else.)

Try it

You'll need Go 1.25+, or grab a prebuilt binary from GitHub Releases (which also ships the optional WeChat channel adapter). Unpack, fill in your .env file, and start — the console is at /ui. Out of the box you get lexical matching; to try the dense channel, open the tool matching page in settings and pick Ollama or an Embedding endpoint.

Wrapping up

The direction is straightforward: the more tools you wire in, the more retrieval at this step matters. Baize keeps it "zero-cost by default, enhancement on demand, fail open" — users who want no new dependencies see no change, and users willing to spend a little on embeddings get more reliable tool recall.

v0.4.0 is out: https://github.com/rebornace/baize

Feedback on matching quality, or on the Ollama one-click flow, is very welcome — I'm following the issues and will keep shipping.

Context as a File: Baize's CLM-Inspired Long-Conversation Projection

rebornace — Sun, 04 Oct 2026 02:40:48 +0000

Project: Baize — an AI assistant runtime for your team (Go 1.25+, MIT)
Repo:
https://github.com/rebornace/baize

1. The short version

Baize v0.3.x takes inspiration from the recent Context Language Models (CLM) papers and ships a constrained take: keep important facts in long chats. It treats the context that reaches the model as something that can be curated — pin the essentials, roll up rolling summaries, drop oversized tool results — but it never rewrites the saved chat messages, and on failure it automatically falls back to the existing compaction logic.

One line for the tradeoff: CLM hands editing rights over the context to the model; Baize puts the editing in a system-side projection layer, trading some freedom for reliability.

2. Why long conversations are a hard problem for agents

An agent's context is the classic append-only structure: every new message, tool result and reasoning trace gets appended to the history. Once a conversation gets long, the problems show up:

Attention dilution: with thousands of lines of history, one critical configuration item can get lost; the model may not reliably pick it up anymore.
Truncation is irreversible: cutting old messages silently drops early agreements and context the user never notices.
Compaction is irreversible: rolling summaries save space, but a summary is a lossy, one-way transformation; key details can simply disappear.

The traditional answer is to hand context management to an external framework: compress on a threshold, offload, retrieve. Those approaches work, but they are "the framework defines the actions, the model can only pick one".

3. What CLM brings

CLM is a paper from late September 2026 (arXiv:2609.37725, by the University of Washington, Meta Superintelligence Labs, MIT and others). It turns the problem around: context is not an append-only log — it is a file the model has write permission on. The model can append, but it can also edit, delete and rewrite that file, and every change is synced into the next turn's context.

The observations in the paper are direct:

Models can learn on their own what is most worth keeping in context, and behaviors show up that previous frameworks never had — maintaining a tracker table for multi-agent collaboration, or defining reusable context-management functions.
On the deep-research benchmark BrowseComp-Plus, out-of-the-box CLM beats the strongest baseline by 11.4% in accuracy while using 21.5% fewer inference FLOPs; on a 24-hour, multi-repo agent-swarm task it achieves 65% higher end-to-end speedup at the same compute.
Because context management becomes model behavior rather than an external harness policy, the strategy itself can keep improving through reinforcement learning.

The appeal is that context management stops being a hard-coded rule set and becomes a product the model can understand and learn. But handing the model full editing rights has real production gates: is the editing reliable, does the archive stay trustworthy, and what happens when it fails?

4. Baize's constrained take: keep important facts in long chats

Baize did not copy CLM wholesale. It extracted the core idea — context is curatable output, not an append-only log — and added three constraints:

1. The projection only lives on the model input side; saved chats are never rewritten.

When a thread gets long, the assistant keeps a model-facing projection of the facts it still needs: it pins what actually matters, summarizes the secondary stuff, and drops oversized tool results. The chat the user sees stays untouched, ready to review, fork or roll back at any time. This matters for team use: the archive is the basis of audit and trust. The model input can be curated aggressively; the archive must stay true.

2. Fail-open.

If the projection fails, the assistant falls back to the original compaction and the conversation keeps going — context curation must never become an availability bottleneck. This matches the philosophy Baize's decision layer already follows: optimization must not affect whether the tools themselves still work.

3. No promise of lower cloud token usage.

The projection itself costs tokens; the curated result is not necessarily cheaper than plain compaction. The docs say it plainly. What this feature solves is key-fact retention and context usability — not the bill.

The knob is optional (default off), hot-reloadable, and costs nothing when unused.

Why "constrained"? Baize talks to OpenAI-compatible APIs: it cannot read logits, and there is no native model-side context-editing capability. Rather than letting the model rewrite history in an uncontrolled place, the editing lives in the system-side projection layer, where it can be made reliable and reversible. CLM is the radical route of native model editing; Baize's projection is the pragmatic route of framework-style editing. Same goal, different cost and reliability.

5. A little design philosophy

Looking back at this upgrade, a few decisions are consistent:

Optional, rather than deciding for the user by default: the projection and the decision layer are both runtime knobs, off by default, taking effect via hot reload when enabled.
Fail-open first: projection falls back to compaction, routing falls back to full tools — availability is never harmed by an optimization.

If you are also working on long-conversation handling for agents, come discuss your tradeoffs at the repo: https://github.com/rebornace/baize. Everything mentioned here (the projection and workspaces) already exists in the public repository; docs live under docs/developers in both English and Chinese. I'll keep following up on issues and suggestions.

Reference: Context Language Models (arXiv:2609.37725): https://arxiv.org/abs/2609.37725

After Jev Blew Up, I Brought Its "System One Judgment" Idea Into My Open-Source AI Agent

rebornace — Wed, 23 Sep 2026 15:22:04 +0000

Project: Baize — an AI assistant runtime for your team (Go 1.25+, MIT)
Repo:

https://github.com/rebornace/baize
This post is not a tutorial for "integrating Jev". It shares one judgment:

Jev is worth learning for more than its model — the engineering idea that high-frequency decisions shouldn't ask a generative model to write an essay.

And that idea lands without depending on Jev's service at all.

1. Decision layer design: three principles + one red line

Baize is a sidecar AI assistant runtime: a single process deployed next to your business systems, turning OpenAPI / MCP / HTTP plugins into callable tools, with human approval required for write operations. It calls models through OpenAI-compatible APIs, so it cannot read logits — and therefore cannot get calibrated probabilities. The decision layer was designed from the start with no probability-dependent logic.

Three hard principles:

Pluggable — one decision interface; local rules, a local small model, or a remote decision service are all implementations. Default to the cheapest. No dependency on any external service.
Degradable — every decision point fails open to the existing rules path; the decision layer is never a single point of failure for a run.
Observable — every judgment records its source and any degradation (decide.degraded, decide.tool_narrow events).

One red line: no confidence numbers of any kind in judgment results. Without calibrated logits, a number that looks like a probability but is not calibrated is more dangerous than no number at all — it invites branches like "auto-execute when p > 0.9".

Interface shape: verdicts only

The return shape is deliberately minimal: verdicts are yes / no enums; multi-select returns the chosen items; every answer carries its source (rules, remote, or fallback) and a degraded flag. There is no float confidence field in the interface — a type-level guarantee against treating uncalibrated numbers as confidence.

The Chain: errors must be swallowed by degradation

All implementations are chained and tried in order; the first success returns. If all fail, the chain returns the fallback verdict hard-coded by the call site, marked degraded. The chain itself never returns an error — errors are consumed by degradation, never propagate into the main flow. Callers always get a legal enum value; a decision-layer failure looks like "old behavior", never like a crash.

Easy to miss: the fallback verdict is hard-coded at the call site, not a config item. Failure directions differ per decision point — memory extraction fails open to Yes (spend a call rather than lose a memory), tool narrowing fails open to the full set (spend prefill rather than miss a tool). As a config knob, someone would eventually flip it under cost pressure and silently lose data. Failure direction is a safety property, not an ops parameter.

Implementations today

Two tiers on the chain, remote first: when a dedicated decision-model profile is configured, a small model answers under a prompt that forces enum-only output, validated by regex / JSON parsing; unparseable replies abstain and let the chain degrade. Multi-select answers accept only names from the offered set; out-of-set values are dropped. Because structured-output support varies across OpenAI-compatible endpoints, the contract is enforced by prompt + parsing, not by endpoint features.

With no profile configured (or remote unavailable), the zero-latency rules backstop handles memory extraction only — it abstains everywhere else: short chitchat with no fact cue returns No; explicit fact cues ("remember that", "my number", passwords, addresses, phone numbers), a run of three or more digits, or substantial length returns Yes.

Want to plug in a self-hosted inference endpoint that can read logits later? Add one more tier to the chain; callers don't change.

2. Four decision points: moving "judgment" out of "essay-writing"

Four places in Baize were classic waste: paying a generative model to write a paragraph just to get a "yes / no / which one" answer. The decision layer moves each one down.

2.1 DP-1: memory-extraction pre-check (the best cost/benefit)

Baize used to fire an LLM call after every successful conversation to extract memory. Most conversations contain nothing worth remembering and return an empty array — but each one paid a full call plus a JSON-parse risk.

Now, before paying for that call, the engine asks the layer "is this turn worth extracting?": a No skips extraction and records a skip event (marked as decided by the layer) together with the estimated input tokens saved, so the win is measurable; a Yes — or an unavailable layer — extracts exactly as before.

The probe itself is capped at 1500 characters: the probe must never cost more than the call it tries to save.

2.2 DP-2: two-level tool-candidate narrowing (the structural one)

The most worthwhile change in Baize. Its positioning is "give it OpenAPI, get tools automatically"; the more connectors you attach, the more full tool schemas go to the main model every turn — prefill cost grows linearly with connectors. That is a structural cost that scales with your users.

Narrowing has two levels:

Level 1: system routing (System Targets). The decision model picks which backend systems — a handful of connector ids — this turn needs; it may pick several. Choosing a tool among hundreds is hard; choosing among three to five systems is easy. System descriptions are not hand-written: they are auto-derived from each system's tool names and descriptions via within-system term frequency times cross-system IDF, so adding a connector needs no routing-corpus maintenance.
Level 2: in-system keyword prefilter. Within the chosen systems, deterministic IDF keyword scoring (tool names weighted 2x, descriptions 1x, stopwords removed, CJK tokenized as bigrams), merged round-robin across systems, narrowed to 16 candidates by default.

Narrowing is not one-size-fits-all; three floors prevent over-pruning:

Category-tool floor: for "list the users", a generic pagination tool ranks poorly under IDF — but a list request needs exactly that tool. List / detail intent forces the matching admin read tool into the set.
Auth floor: once a system is chosen, its login / current-session primitives are always kept — on a 401 the model must recover the session on the spot, and the user will never say "login".
Hybrid routing: discriminative domain words in the query (e.g. pets, orders) force their systems and can never be removed by the model — the small model can only add on top, so it cannot vote away the only correct system.

Also, main-model calls support a choice constraint (tool_choice=required) via optional interfaces, without breaking existing provider signatures; providers that don't support it fail open to an ordinary call. A constraint is an optimization, not a hard gate.

2.3 DP-3: tool-result pruning

In long conversations, a big JSON returned by an earlier tool call keeps bloating the context. When compaction triggers, the layer judges each of the bulkiest tool results "worth keeping verbatim?"; unworthy ones become a short placeholder that preserves the message / tool-call pairing — the model can re-invoke the tool, and the original result stays in the run log.

Three bounds keep the judgment cheap: only tool results are judged (not ordinary turns), only those over roughly 500 estimated tokens, at most 8 per turn. All failures mean keep everything.

2.4 DP-4: tier arbitration

Baize used a pure heuristic (image present? code fences?) to route each turn to light / standard / power tiers. The problem: "help me refactor this function" contains no code fences, gets classified as light, and the strong model never runs.

DP-4 does not replace the heuristic. It only consults the layer when the heuristic lands on the ambiguous standard tier, the turn is long enough (at least 400 characters), and routing is Auto — asking for a light / power choice. Short turns are not worth asking: "hello" barely differs across tiers, and every consult costs latency.

DP-4's judgment goes through the same chain: a dedicated decision tier makes the two-way pick when available; on degradation, the heuristic's standard tier stands — the layer advises, never rewrites the routing outcome.

3. Results and validation

Reproducible benchmark: 37 real read-only business requests across 3 connected backends (390 tools total), model DeepSeek-Flash, stability run of 5 rounds (185 requests). With the default top-16: run success rate 99.5% (184/185), average turn-0 prompt ~3,090 tokens, about 34% lower than top-32; sending all 390 tools directly measured ~85k turn-0 prompt tokens. The dataset and scripts live in scripts/tool-routing-eval — reproducible.
Default narrowing: convergence kicks in above the threshold (default 12 tools); 16 candidates is the default cap — the measured cost / success sweet spot.
Regression harness: sweep (width), stability, and analyze (offline) scripts that restore your previous configuration when they finish. An optimization that changes behavior should only go live with shadow data first, then flip shadow off.

Everything ships with the decision layer off by default and enabled per point: the master switch (decide_enabled) defaults to off, each decision point has its own knob, all controlled through the existing hot-reload settings — no wiring changes.

4. Honest boundaries

No calibrated probabilities. Baize calls OpenAI-compatible APIs and cannot read logits. The layer only ever returns enums, and there is no "auto-execute above a probability threshold" feature. Write approval stays deterministic rules + human approval. That red line does not bend for Jev.
This is not "integrating Jev". The decision layer depends on no external decision service. If you later point your inference at a local vLLM running a decision plugin, the remote implementation works as-is — a free win, not an architectural prerequisite.

5. Closing

Jev made one thing click for me: models matter, of course — but plenty of high-frequency little decisions shouldn't ask a generative model to write an essay every time. Pulling "judgment" out of "generation" into a pluggable, degradable, observable layer is engineering any agent project can do — it does not depend on any company's window.

Baize's decision layer is live in the main branch (internal/decide + four decision points). Come take a look, run it, or open an issue:

GitHub: https://github.com/rebornace/baize
More: the README's decision-layer section, and the bilingual developer docs under docs/developers

If you have hit the same wall — tool schemas going to the model every turn while prefill gets more expensive — I'd love to hear your solution. I'll keep following up on feedback and issues.

Building a Lightweight AI Agent in Go: Baize's Architecture and Trade-offs

rebornace — Thu, 17 Sep 2026 02:06:02 +0000

Preface

Baize is a "sidecar" AI assistant runtime: a single process that sits beside services you already run, turns API documentation into tools the assistant can call, and pauses important writes until a person approves them. Stop it and it leaves almost nothing behind.

One sentence positioning: it's a runtime, not a framework. That positioning drives every architecture decision below.

1. Why Go

Single binary, zero dependencies

The core requirement of sidecar deployment is "one process, copy it over, run it." A Go binary needs no interpreter, no dependency installation, no virtualenv on the target machine. For an assistant meant to live in enterprise environments, that's the lowest-cost delivery form.

Goroutine concurrency

One assistant process serves multiple entry points at once: the web console, signed alert/ticket ingress, and IM channels. Goroutines make "one process handling many sessions concurrently" straightforward — and parallel tool calls later become almost free.

Cross-compilation

Enterprise environments run everything: Windows, Linux, macOS, ARM. A single GOOS=linux GOARCH=arm64 go build produces a binary for the target platform, with no toolchain setup on the destination machine.

Static typing and tool contracts

Tool input schemas come from OpenAPI documents; mapping them onto Go's strong types catches many errors at compile time. For a long-running daemon, that saves a lot of operational pain.

2. Architecture Overview

A clean three-layer structure: core loop → tool router → executors.

Users / Channels ──► Agent Core Loop (think → pick tool → run → report)
                           │
                           ▼
                     Tool Router (Registry)
                           │
          ┌────────────────┼────────────────┐
    OpenAPI connector    HTTP plugin      MCP connector
          └────────────────┼────────────────┘
                           ▼
              Invoker closures (registered in Registry)
                           │
                     [ HITL approval gate ]
                           │
             ┌─────────────┴─────────────┐
       Direct execution          HTTP callback executor
     (plugin / proxy)            (callback to your side)

Core loop (internal/run): LLM thinks → picks a tool → executes → reports. The event stream (llm.thinking, llm.tool_call, tool.result) is persisted, so the console can follow along in real time.
Tool router (internal/tool): a mutex-protected map that registers not functions but "tool contracts + invoker closures".
Executors (internal/connector): tools enter through three sources — OpenAPI docs, HTTP plugins, MCP tool servers — plus a "callback execution" mode that hands execution back to your side.

3. Key Design Decisions

Why HTTP callback instead of a plugin protocol

This is Baize's most important trade-off.

The problem with plugin protocols: in-process plugins (Go plugins, shared libraries, language-binding SDKs) require the plugin to compile with the host process — language, version, and ABI must all align. Most enterprise systems aren't written in Go: legacy systems, Java/.NET/Python services can't load an in-process plugin at all. And even when they could, upgrading a plugin means restarting the process — "sidecar in, clean out" is gone.

What Baize does instead: it doesn't execute the tool itself; it POSTs the invocation to your own endpoint:

{
  "tool": "create_ticket",
  "arguments": { ... },
  "run_id": "run_xxx",
  "agent_id": "agent_xxx",
  "idempotency_key": "uuid-xxx",
  "callback_urls": { "event": "https://your-service/baize-events" }
}

Your side executes it and returns the result. Benefits: language-agnostic, process-isolated, auditable; idempotency_key makes retries safe (no duplicate execution); callback_urls lets your side keep driving follow-up actions.

The cost: one extra network round-trip, the callback endpoint must be reachable, and to prevent forged callbacks you need signed requests (Baize uses callback signing with a TTL against replay).

Dynamic tool registration and discovery

The registry (tool.Registry) is the core data structure: a sync.RWMutex guarding a map, with runtime register/unregister and per-connector bulk unregister — adding a tool or disabling a connector never requires a restart.

Three tool sources share one registration path:

OpenAPI docs: import Swagger/OpenAPI/Postman-style docs, each operation becomes a tool;
HTTP plugins: a small companion service declares "what tools exist and how to invoke them";
MCP tool servers: connect to external tool ecosystems as an MCP client.

Security policy is baked into each entry at registration time: require_approval (needs a human), require_login (needs a session), security_schemes (which auth scheme to use). Security policy is decided at registration, not asked at execution time — this is the precondition for letting the assistant actually act.

Discovery is trivial: Registry.List() / Registry.Specs() feed the model's tool list, visible live in the console.

Graceful degradation on failure

Failures are the norm in AI agents, so degradation design matters more than the happy path:

Timeout guardrail: every tool invocation is bound to context.WithTimeout (default 60s, configurable);
Failure is content: Invoker returns (content, isError, err) — err is an infrastructure failure (timeout, network), isError is a business-side failure. Both flow back to the model as structured content, so the model can retry, switch tools, or explain to the user — instead of crashing the whole session;
Approval rejection is not a crash: when a human rejects a write, the run settles into an explicit "rejected" terminal state with a trail, no panic;
Everything is observable: the llm.tool_call → tool.result event stream is persisted, so any problem can be traced step by step;
Context compaction: long sessions get rolling summaries so quality doesn't degrade as the thread grows.

4. Go vs Python / Node.js for Agent Scenarios

Let's be honest first: Python is the best choice in the AI/Agent ecosystem. LangChain, LlamaIndex and most reference implementations live there. If your goal is fast experimentation and deep reuse of the LLM ecosystem, Python has no rival.

Baize chose Go because its positioning is different:

Dimension	Go	Python	Node.js
Deployment	Single binary, zero deps	Interpreter + deps/venv	Node runtime + node_modules
Resource footprint	Low; one resident process is cheap	Higher; resident processes need care	Medium
Concurrency	Native goroutines	GIL-limited; multi-process/async	Event loop
Type safety	Static, compile-time checks	Dynamic, found at runtime	Dynamic / TypeScript
LLM ecosystem	Newer, catching up fast	Richest	Rich
Cross-platform	Cross-compile to all platforms	Needs interpreter on target	Needs Node on target

The conclusion isn't "Go is better than Python" — it's positioning decides the language:

Goal: framework / fast experimentation → Python;
Goal: sidecar, resident, one-command deployment to enterprise environments, running on modest hardware for a long time → Go's advantages in deployment and resource usage are hard to replace.

5. Core Code Snippets (Go)

All snippets are from the project source, lightly trimmed. Each comes with one line on what problem it solves.

1. A tool = contract + invoker closure

Modeling a "tool" as "a contract the model sees (Spec) + an invoker closure injected by the connector" fully decouples routing from execution:

type Invoker func(ctx context.Context, args map[string]any) (
    content map[string]any, isError bool, err error)
type Meta struct {
    Spec            llm.ToolSpec
    ConnectorID     string
    Method          string
    Path            string
    RequireLogin    bool
    SecuritySchemes []string
}

2. Dynamic registration with policy baked in

require_approval / require_login are written into the entry at registration; the tool list is "hot" — adding/removing connectors never requires a restart:

func (r *Registry) RegisterMeta(meta Meta, inv Invoker, requireApproval bool) {
    r.mu.Lock()
    defer r.mu.Unlock()
    r.tools[meta.Spec.Name] = entry{
        spec:            meta.Spec,
        invoker:         inv,
        requireApproval: requireApproval,
        requireLogin:    meta.RequireLogin,
        connectorID:     meta.ConnectorID,
        method:          meta.Method,
        path:            meta.Path,
    }
}

3. HTTP callback executor

Hands execution back to your side; the idempotency key makes network retries safe:

payload := map[string]any{
    "tool":            tool,
    "arguments":       args,
    "run_id":          meta.RunID,
    "agent_id":        meta.AgentID,
    "idempotency_key": meta.IdempotencyKey,
}
if strings.TrimSpace(meta.CallbackEventURL) != "" {
    payload["callback_urls"] = map[string]any{
        "event": meta.CallbackEventURL,
    }
}
rawPayload, _ := json.Marshal(payload)
req, _ := http.NewRequestWithContext(ctx, http.MethodPost, c.URL, bytes.NewReader(rawPayload))

4. Writes automatically enter the approval gate

Non-GET/HEAD/OPTIONS operations are automatically flagged "needs approval" at registration, and only run after a human clicks approve/reject in the console:

needApproval := t.RequireApproval
if ctx.requireApprovalMutating && isMutatingMethod(t.Method) && t.Source == store.ToolSourceSpec {
    needApproval = true
}

5. Timeouts and failure degradation

A timeout guardrail plus "failure is content" semantics keeps one bad tool call from blowing up the whole session:

toolCtx, toolCancel := context.WithTimeout(ctx, e.toolTimeout())
defer toolCancel()
content, isError, invErr := e.Tools.Invoke(toolCtx, payload.ToolName, payload.Arguments)
if invErr != nil {
    // Infrastructure failure (timeout/network): persist and close the round
    return e.finalizeFailedRun(runID, invErr)
}
// When isError is true, the failure flows back to the model as content;
// the model decides whether to retry or explain.

Closing

Baize is still early. The trade-offs above are far from "optimal" — especially the approval UX, channel adapters, and executor extensibility. If you have real-world scenarios, I'd love to hear them.

Repo: https://github.com/rebornace/baize (MIT)
Issues: open a discussion with your scenario — I'll follow up.

Add AI to Your Legacy System in 5 Minutes: A Baize Hands-On Tutorial (Spring Boot Example)

rebornace — Wed, 16 Sep 2026 04:34:42 +0000

Got a backend system that's been running for years — piles of APIs, complex business logic, and the thought of adding AI feels like a massive undertaking? This tutorial shows you how to hook up an AI assistant to your existing system in 5 minutes using Baize — no refactoring, no core code changes, at most one new endpoint.

Baize is an open-source AI Agent runtime written in Go. Its core ideas: sidecar deployment, APIs-as-tools, and human-in-the-loop approval for critical operations. It connects to business systems over HTTP, so it doesn't matter what your backend is — Spring Boot, Django, Express, Gin, ThinkPHP — if it has APIs, it can connect. The AI understands user intent, calls the right tools automatically, and pauses for human confirmation on sensitive operations.

Language-agnostic, but we'll use Spring Boot as the main example. The principles are fully universal — for other frameworks, just swap the Controller code in Step 3 for your language of choice. The Baize config file stays exactly the same.

Prerequisites

To follow along, you'll need two things:

A backend service (we'll use a Spring Boot ticket system as an example, but feel free to use your own project)
The Baize binary (a single executable, zero dependencies)

You can clone the repo which includes a demo ticket system and Baize configs:

git clone https://github.com/rebornace/baize.git
cd baize

The Spring Boot code examples in this tutorial use Java 17 + Spring Boot 3.x. Readers using other languages or frameworks can focus on the configuration and protocol sections, and map the code examples to their own stack.

Step 1: Download Baize and Start It with One Command

Baize is a single static binary written in Go — no runtime to install.

Option 1: Download the Binary (Recommended)

Grab the latest release for your platform from GitHub Releases, unzip it, and you're ready to go.

Option 2: Build from Source

If you have Go 1.25+ locally, you can build it yourself:

git clone https://github.com/rebornace/baize.git
cd baize
go build -o baize ./cmd/baize

Start in Demo Mode

Baize comes with a demo mode that uses a Mock LLM (no API key needed) and a built-in demo ticket system — perfect for getting the full flow working first.

# Windows
.\baize demo

# macOS / Linux
./baize demo

When you see output like this, it's running:

baize demo listening on :8080
mock ticket server listening on :18080

Open your browser and go to http://localhost:8080/ui — you'll see Baize's web dashboard.

For production, use baize start with a config file and a real LLM API key. We'll cover that later.

No Config File? No Problem — Configure via the Web UI

Baize supports zero-config startup — just run baize start without any config file, then open the Web UI and set everything up interactively: models, connectors, tools, approval rules — all of it. Great if you prefer a graphical interface over editing YAML by hand.

# Windows PowerShell
$env:BAIZE_API_KEY = "your-api-key"
.\baize start

# macOS / Linux
export BAIZE_API_KEY="your-api-key"
./baize start

Once it's running, go to http://localhost:8080/ui → Settings in the sidebar. You can configure your LLM provider, add connectors, and manage tools directly from the page. Changes take effect immediately — no restart needed.

This tutorial uses YAML config files for clarity in the step-by-step flow. Both approaches produce the same result — pick whichever you prefer.

Step 2: Declare Tools in a Config File

The core of how Baize connects to business systems is the Connector. The most common approach is the OpenAPI Connector — if you have an OpenAPI/Swagger spec, Baize automatically turns every endpoint into a tool the AI can use.

2.1 Prepare the OpenAPI Spec

Let's say your Spring Boot ticket system has these endpoints:

Method	Path	Description	operationId
GET	/tickets	List all tickets	`list_tickets`
POST	/tickets	Create a ticket	`create_ticket`
GET	/tickets/{id}	Get ticket details	`get_ticket`
PATCH	/tickets/{id}	Update ticket status	`update_ticket_status`

Here's the corresponding OpenAPI spec (ticket-api.yaml):

openapi: 3.0.3
info:
  title: Ticket System API
  version: 1.0.0
paths:
  /tickets:
    get:
      operationId: list_tickets
      summary: List all tickets
      responses:
        "200":
          description: List of tickets
    post:
      operationId: create_ticket
      summary: Create a ticket
      requestBody:
        required: true
        content:
          application/json:
            schema:
              type: object
              required: [title]
              properties:
                title: { type: string, description: Ticket title }
                priority: { type: string, description: Priority level }
      responses:
        "201":
          description: Created successfully
  /tickets/{id}:
    get:
      operationId: get_ticket
      summary: Get ticket details
      parameters:
        - name: id
          in: path
          required: true
          schema: { type: string }
      responses:
        "200":
          description: Ticket details
    patch:
      operationId: update_ticket_status
      summary: Update ticket status
      parameters:
        - name: id
          in: path
          required: true
          schema: { type: string }
      requestBody:
        required: true
        content:
          application/json:
            schema:
              type: object
              required: [status]
              properties:
                status: { type: string, description: New status }
      responses:
        "200":
          description: Updated successfully

If your Spring Boot project already has springdoc-openapi or Swagger integrated, you can export the JSON directly from /v3/api-docs — Baize understands both YAML and JSON.

2.2 Write the Baize Config File

Create baize-config.yaml:

listen: ":8080"
store:
  driver: sqlite
  sqlite_path: ./data/baize.db
ui:
  enabled: true

llm:
  provider: openai_compatible
  base_url: https://api.deepseek.com
  model: deepseek-chat
  api_key_env: BAIZE_API_KEY

agent:
  id: ticket-agent
  system: "You are a ticket assistant. You can only access the ticket system through tools."

connector:
  id: ticket-api
  type: openapi
  spec: ./ticket-api.yaml
  base_url: http://localhost:8081   # Your Spring Boot service URL
  require_approval:
    - create_ticket
    - update_ticket_status

run:
  max_steps: 16
  tool_timeout_sec: 60

Key configuration points:

connector.type: openapi — Auto-discovers tools from the OpenAPI spec
connector.spec — Path to your OpenAPI spec file
connector.base_url — Your backend service URL; the AI calls this address when using tools
require_approval — Tools listed here require human confirmation before execution (recommended for all write operations)

2.3 Start with the Config File

# Set your LLM API Key (DeepSeek example)
# Windows PowerShell
$env:BAIZE_API_KEY = "your-api-key"

# macOS / Linux
export BAIZE_API_KEY="your-api-key"

# Start
./baize start -c baize-config.yaml

Open http://localhost:8080/ui → click "Tools" in the sidebar, and you'll see all 4 ticket endpoints listed as AI-usable tools.

At this step, you haven't changed a single line of your business code — Baize is just another caller hitting your APIs, no different from a frontend app.

Step 3: Add a Callback Endpoint for Full Control (Spring Boot Example)

The OpenAPI approach above works great when "the APIs already exist and the AI can call them directly." But sometimes you need more control:

You want to add custom logic before/after AI calls (audit logging, rate limiting, parameter validation)
The tool logic is complex and not suitable for exposing as a direct API
You don't want the AI touching internal services directly

This is where Execution Callback mode comes in: instead of calling your business APIs directly, Baize sends a message saying "which tool to call and with what arguments" to an endpoint you specify. You decide how to execute it. The protocol is dead simple — any language that can serve HTTP can implement it.

Below is a Spring Boot example; the same pattern applies to Django, Express, Gin, and any other framework.

3.1 Add an `/execute` Endpoint (Spring Boot Version)

Add one Controller with one method to your Spring Boot project:

package com.example.ticket.controller;

import com.example.ticket.service.TicketService;
import org.springframework.web.bind.annotation.*;

import java.util.Map;

@RestController
@RequestMapping("/baize")
public class BaizeCallbackController {

    private final TicketService ticketService;

    public BaizeCallbackController(TicketService ticketService) {
        this.ticketService = ticketService;
    }

    @PostMapping("/execute")
    public Map<String, Object> execute(
            @RequestHeader("X-Baize-Protocol") String protocol,
            @RequestBody Map<String, Object> body) {

        // Protocol check
        if (!"v0".equals(protocol)) {
            return Map.of(
                "content", Map.of("error", "unsupported protocol"),
                "is_error", true
            );
        }

        String tool = (String) body.get("tool");
        @SuppressWarnings("unchecked")
        Map<String, Object> args = (Map<String, Object>) body.get("arguments");

        // Dispatch to your business logic based on tool name
        Object result = switch (tool) {
            case "list_tickets" -> ticketService.listTickets();
            case "create_ticket" -> ticketService.createTicket(
                (String) args.get("title"),
                (String) args.get("priority")
            );
            case "get_ticket" -> ticketService.getTicket((String) args.get("id"));
            case "update_ticket_status" -> ticketService.updateStatus(
                (String) args.get("id"),
                (String) args.get("status")
            );
            default -> null;
        };

        if (result == null) {
            return Map.of(
                "content", Map.of("error", "unknown tool: " + tool),
                "is_error", true
            );
        }

        return Map.of(
            "content", result,
            "is_error", false
        );
    }
}

That's it — one Controller, one method, one switch statement for dispatch. You're in full control of the execution logic. Add logging, add authorization checks, add caching — whatever you need.

3.2 Update Baize Config to Point to the Callback URL

Change the connector in baize-config.yaml to callback mode:

connector:
  id: ticket-api
  type: openapi
  spec: ./ticket-api.yaml
  base_url: ""                           # Can be empty in callback mode
  execution_callback_url: http://localhost:8081/baize/execute
  require_approval:
    - create_ticket
    - update_ticket_status

The key addition is execution_callback_url — with this field set, Baize won't call base_url directly. Instead, it sends tool invocation requests to this callback address.

You still need the OpenAPI spec file because Baize uses it to discover what tools exist and what parameters they take — that's tool discovery. The actual tool execution is handled by your Spring Boot service.

After restarting Baize, the user experience looks the same, but the execution path is completely different — every tool call now goes through your /baize/execute endpoint.

Step 4: Send a Message and Watch the AI Call Tools

Once Baize is running, you can talk to the AI in two ways: via the Web UI or the HTTP API.

Option 1: Web UI (Recommended for Getting Started)

Open http://localhost:8080/ui and type in the chat box:

Show me all the tickets

You'll see the AI's reasoning process step by step, including the full list_tickets tool call — both the request parameters and the returned result.

Try this next:

Create a ticket with title "Login page loads slowly" and high priority

Because create_ticket is in the require_approval list, the AI will pause and wait for your confirmation. Click Approve on the approval card that pops up, and only then will it actually execute the create operation. This is HITL (Human-in-the-Loop) — sensitive operations are never executed blindly.

Option 2: HTTP API (Great for Integration)

Baize exposes a full REST control plane that you can call from scripts or other systems.

Start a new conversation run:

curl -X POST http://localhost:8080/v0/runs \
  -H "Content-Type: application/json" \
  -d '{
    "agent_id": "ticket-agent",
    "messages": [
      {
        "role": "user",
        "content": "Show me all the tickets"
      }
    ]
  }'

The response includes a run_id. Use it to check status and stream events:

# Get run result
curl http://localhost:8080/v0/runs/{run_id}

# Stream events (SSE)
curl http://localhost:8080/v0/runs/{run_id}/stream

Going Further

Adding More Tools

Tools come from the OpenAPI spec. Adding a tool = adding an endpoint to the spec, then restarting Baize (or hot-loading through the settings page).

For example, if you want to add a "close ticket" tool, add this path to ticket-api.yaml:

  /tickets/{id}/close:
    post:
      operationId: close_ticket
      summary: Close a ticket
      parameters:
        - name: id
          in: path
          required: true
          schema: { type: string }
      responses:
        "200":
          description: Closed successfully

Then add close_ticket to require_approval, and you're done.

Timeouts and Retries

Baize has a default timeout for each tool call, which you can adjust in the config:

run:
  max_steps: 16
  tool_timeout_sec: 120    # Timeout per tool call, default is 60 seconds

If your business APIs are occasionally flaky, you can handle retries inside your Spring Boot callback endpoint — Spring Retry or Resilience4j both work great. Baize just "sends the request and waits for the result" — retry strategy is up to you.

Handling Login / Auth

If your system requires login before calling APIs, Baize has you covered. Configure auth.capture:

connector:
  id: ticket-api
  type: openapi
  spec: ./ticket-api.yaml
  execution_callback_url: http://localhost:8081/baize/execute
  auth:
    mode: passthrough
    capture:
      tool_name_glob: "*login*"
      token_json_paths: ["accessToken", "data.accessToken"]
      label_json_paths: ["email"]
      header_template: "Bearer {{token}}"

When the AI detects that login is required, it calls the login tool first, automatically extracts the token from the response, and attaches it to subsequent calls. Login info is stored in the session identity and persists across conversations.

FAQ

Q: Does Baize have to run in Kubernetes?

No. Baize is a single binary — it runs on bare metal, VMs, Docker, and Kubernetes alike. Put it on the same internal network as your legacy system, or even deploy it on the public network and call back over VPN (security will be weaker, of course).

Q: What if my OpenAPI spec is outdated or doesn't cover all endpoints?

Two options:

Use Execution Callback mode — the OpenAPI spec just declares "what tools exist," and you write the actual execution logic yourself
Use the HTTP Plugin v0 protocol — write a small standalone service to add capabilities your original system doesn't have

Q: Could the AI call APIs randomly and mess up my data?

No. Two layers of protection:

Human approval — tools in the require_approval list pause before execution and wait for someone to click approve in the dashboard
Read-only first — it's good practice to start with read-only endpoints open and put all write operations behind approval, then gradually open things up as you gain confidence

Q: Which LLMs are supported?

Any model compatible with the OpenAI API format works — including OpenAI, DeepSeek, Anthropic (via compatible gateways), local Ollama, and more. Just change llm.base_url and llm.model in the config.

Q: Can it connect to messaging platforms like Slack or Discord?

Yes. Baize's channels are designed as an extensible capability. You can use the Webhook channel to adapt any platform. One assistant, one set of tools, one approval flow — reachable from multiple channels.

Q: What's the difference between demo mode and production mode?

Demo mode (baize demo): Uses Mock LLM, no API key needed, includes a built-in demo ticket system — great for trying it out quickly
Production mode (baize start): Connects to a real LLM, works with real business systems — full feature set

Wrapping Up

Let's recap the whole flow — really just four steps:

Download Baize → single binary, zero dependencies
Write an OpenAPI spec + configure the connector → APIs become tools automatically
(Optional) Add an /execute endpoint → full control over execution logic
Chat → the AI calls tools automatically, with human approval for sensitive operations

Baize's design philosophy is "sidecar deployment, clean removal" — it doesn't intrude on your business system, it just adds an intelligent caller. Turn it on when you want to try it, turn it off when you don't. Your legacy system stays untouched.

Open Source Info

Baize · MIT License · Go 1.25 · Zero C dependencies

GitHub: https://github.com/rebornace/baize

Core features: ReAct Agent · OpenAPI Connector · HTTP Plugin v0 · Execution Callback · MCP Bridge/Export · HITL (Human-in-the-Loop) · Built-in Web UI

Supported LLMs: OpenAI / DeepSeek / any OpenAI-compatible API

Stars, issues, and PRs are all welcome.