DEV Community: Izu Tolandona

#8 - Shipping a Go AI Agent to Production: 18MB Docker Image, 5 Services, Zero Downtime

Izu Tolandona — Sat, 21 Feb 2026 17:00:53 +0000

Part 8 of the "Building Production-Ready AI Agent APIs in Go" series

The most common advice for deploying Go applications is: "compile a binary and run it." That is true, but production requires more: automated database migrations, a proxy for LLM providers, a cache layer, health checks, graceful shutdown, and resource limits.

This article walks through every aspect of deploying this AI agent API to production. By the end, you will understand the multi-stage Dockerfile that produces an 18MB image, the 5-service Docker Compose stack, the graceful shutdown pattern, and how LiteLLM lets you swap GPT-4o for Claude with a config file change.

The Multi-Stage Dockerfile: Builder → Scratch

The deployments/Dockerfile uses two stages:

# Stage 1: Build
FROM golang:1.24-alpine AS builder

RUN apk add --no-cache git ca-certificates tzdata

WORKDIR /app

# Cache dependencies first (Docker layer caching)
COPY go.mod go.sum ./
RUN go mod download

# Then copy source
COPY . .

# Build a statically-linked binary
ARG VERSION=dev
RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build \
    -ldflags="-w -s -X main.version=${VERSION}" \
    -o /app/server ./cmd/api

# Stage 2: Final image
FROM scratch

# Copy TLS certificates (needed for HTTPS calls to LiteLLM, PostgreSQL SSL)
COPY --from=builder /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/

# Copy timezone data (needed for correct time.Now() behavior)
COPY --from=builder /usr/share/zoneinfo /usr/share/zoneinfo

# Copy the compiled binary
COPY --from=builder /app/server /server

# Copy migrations — the binary and its migrations ship as one unit
COPY --from=builder /app/internal/infrastructure/persistence/postgres/migrations /migrations

EXPOSE 8080
ENTRYPOINT ["/server"]

Why `FROM scratch`?

FROM scratch is Docker's empty base image. No shell, no utilities, no package manager, no OS userspace. The final image contains exactly:

CA certificates (for TLS)
Timezone data
The compiled Go binary
The SQL migration files

That is it. The result is an 18MB image.

Compare this to a typical Dockerfile using FROM golang:1.24 as the final stage — that image is 1.2GB because it includes the entire Go toolchain and Alpine Linux. Using FROM alpine as the final stage gets you to ~15MB for the OS layer + your binary. FROM scratch gets you to just the binary.

Why `CGO_ENABLED=0`?

Go can link against C libraries (CGO), but that creates a binary that depends on the system's C standard library being present at runtime. FROM scratch has no C library.

CGO_ENABLED=0 compiles a fully static binary — all code, including any normally-CGO parts, is compiled into the binary itself. The resulting binary runs anywhere with no dynamic library dependencies.

The flags on the build command:

GOOS=linux GOARCH=amd64 — explicit cross-compilation target (important if you build on macOS/ARM)
-w — strip DWARF debug information (reduces binary size)
-s — strip symbol table (reduces binary size further)
-X main.version=${VERSION} — embed the version string at compile time

The Migration Trick

COPY --from=builder /app/internal/infrastructure/persistence/postgres/migrations /migrations

The migration SQL files are copied into the final image alongside the binary. This means the binary and its database migrations are always deployed together as a single atomic unit. You cannot accidentally deploy a binary that requires migrations that were not deployed.

In the Docker Compose setup, a separate migrate container uses golang-migrate to run these files at startup. In a Kubernetes deployment, you would run this as an init container before the API pod starts.

The 5-Service Docker Compose Stack

services:
  api:          # The Go agent API
  postgres:     # PostgreSQL 16 (persistent storage)
  redis:        # Redis 7 (rate limiting, caching)
  litellm:      # LLM proxy (GPT-4o, Claude, Gemini)
  migrate:      # golang-migrate (one-shot migration runner)

Startup Order with Health Checks

The most important aspect of the Compose configuration is startup ordering:

api:
  depends_on:
    postgres:
      condition: service_healthy
    redis:
      condition: service_healthy

migrate:
  depends_on:
    postgres:
      condition: service_healthy

Each depends_on with condition: service_healthy means Docker waits for the health check to pass before starting the dependent service. Without this, the API might start before PostgreSQL is ready to accept connections — leading to startup failures.

The health checks:

postgres:
  healthcheck:
    test: ["CMD-SHELL", "pg_isready -U postgres"]
    interval: 5s
    timeout: 5s
    retries: 5

redis:
  healthcheck:
    test: ["CMD", "redis-cli", "ping"]
    interval: 5s
    timeout: 5s
    retries: 5

api:
  healthcheck:
    test: ["CMD", "wget", "-q", "--spider", "http://localhost:8080/health"]
    interval: 10s
    timeout: 5s
    retries: 3

Startup sequence:

Postgres and Redis start simultaneously
Once both are healthy, migrate runs migrations
Once Postgres is healthy (migrate could have altered tables), api starts
api becomes healthy once /health returns 200

Memory Limits

api:
  deploy:
    resources:
      limits:
        memory: 128M
      reservations:
        memory: 64M

The API is limited to 128MB RAM. This is feasible because:

The Go binary itself is ~18MB on disk, ~30MB in memory with stack/heap
Connection pools (PostgreSQL, Redis) use minimal memory
The main variable cost is concurrent request processing — the Eino workflow state per request is small

In practice, the API uses 30-60MB at moderate load. The 128MB limit provides headroom for spikes while preventing a single runaway process from consuming all available memory.

Redis has a memory limit configured in its startup command:

redis:
  command: redis-server --appendonly yes --maxmemory 128mb --maxmemory-policy allkeys-lru

allkeys-lru means when Redis hits 128MB, it evicts the least-recently-used keys. For rate limiting counters that expire after 60 seconds or 24 hours anyway, this is acceptable — a missed eviction just means a slightly less accurate rate limit count, not data corruption.

LiteLLM: Swap LLM Providers With a Config File

LiteLLM is the secret weapon. It is an OpenAI-compatible proxy that translates requests to any LLM provider's API format. The Go code sends OpenAI-format requests to LiteLLM. LiteLLM forwards them to whatever provider you configure.

The config in deployments/litellm_config.yaml:

model_list:
  - model_name: gpt-4o-mini
    litellm_params:
      model: openai/gpt-4o-mini
      api_key: os.environ/OPENAI_API_KEY

  - model_name: claude-haiku
    litellm_params:
      model: anthropic/claude-3-haiku-20240307
      api_key: os.environ/ANTHROPIC_API_KEY

  - model_name: gpt-4o
    litellm_params:
      model: openai/gpt-4o
      api_key: os.environ/OPENAI_API_KEY

To switch the default model from GPT-4o-mini to Claude Haiku: change LLM_DEFAULT_MODEL=gpt-4o-mini to LLM_DEFAULT_MODEL=claude-haiku in your .env file. Restart the API container. Done. Zero code changes.

The Go code sends to http://litellm:4000/chat/completions with model: "claude-haiku". LiteLLM looks up claude-haiku in its config, translates to Anthropic's API format, forwards the request, and translates the response back to OpenAI format. The Go code never knows the difference.

This also enables:

Cost optimization: use a cheaper model for general tasks, expensive model for complex coding
Fallback routing: if OpenAI is down, route to Anthropic automatically
A/B testing: route a percentage of traffic to different models
Local models: point LiteLLM at an Ollama instance for self-hosted models

Graceful Shutdown: The Complete Pattern

The shutdown logic in cmd/api/main.go follows the standard Go pattern:

func main() {
    // ... setup ...

    // Server starts in a goroutine
    go func() {
        slog.Info("starting server", "addr", server.Addr)
        if err := server.ListenAndServe(); err != nil && err != http.ErrServerClosed {
            slog.Error("server error", "error", err)
            os.Exit(1)
        }
    }()

    // Block until shutdown signal
    quit := make(chan os.Signal, 1)
    signal.Notify(quit, syscall.SIGINT, syscall.SIGTERM)
    <-quit  // Wait here until Ctrl+C or SIGTERM

    slog.Info("shutting down server...")

    // Give in-flight requests up to 30 seconds to complete
    shutdownCtx, shutdownCancel := context.WithTimeout(context.Background(), cfg.Server.ShutdownTimeout)
    defer shutdownCancel()

    if err := server.Shutdown(shutdownCtx); err != nil {
        slog.Error("server shutdown error", "error", err)
    }

    slog.Info("server stopped")
    // defer db.Close() and defer redisClient.Close() run here
}

server.Shutdown(ctx) does the following:

Stops accepting new connections immediately
Waits for all in-flight requests to complete
If the context times out, forcefully closes remaining connections

This means a user in the middle of a streaming LLM response gets to finish. A 30-second timeout means a user who just started a long generation gets up to 30 seconds of grace. After that, the server forcefully closes.

signal.Notify(quit, syscall.SIGINT, syscall.SIGTERM) listens for:

SIGINT — Ctrl+C in development
SIGTERM — what Kubernetes and Docker send when stopping a container

The defer db.Close() and defer redisClient.Close() at the top of main() run after server.Shutdown() completes, ensuring connections are properly closed in the right order.

Environment Configuration

All configuration is environment-variable driven. The full set:

# Server
SERVER_PORT=8080
SERVER_HOST=0.0.0.0
SERVER_READ_TIMEOUT=30s
SERVER_WRITE_TIMEOUT=30s
SERVER_SHUTDOWN_TIMEOUT=30s

# Database
DB_HOST=postgres
DB_PORT=5432
DB_USER=postgres
DB_PASSWORD=change-in-production
DB_NAME=aiagent
DB_SSL_MODE=disable  # Use "require" in production

# Redis
REDIS_HOST=redis
REDIS_PORT=6379
REDIS_PASSWORD=

# JWT
JWT_SECRET=change-me-in-production-must-be-32-chars
JWT_ACCESS_TTL=15m
JWT_REFRESH_TTL=168h  # 7 days

# Rate Limiting
RATE_LIMIT_ENABLED=true
RATE_LIMIT_PER_MINUTE=60
RATE_LIMIT_PER_DAY=10000

# LLM
LLM_PROVIDER=litellm
LLM_BASE_URL=http://litellm:4000
LLM_API_KEY=sk-1234  # LiteLLM master key
LLM_DEFAULT_MODEL=gpt-4o-mini

# Tools
WEB_SEARCH_API_KEY=your-brave-search-api-key
TOOLS_MAX_CONCURRENT=10

Every variable has a default. config.Load() reads from environment variables with fallbacks:

func getEnv(key, defaultValue string) string {
    if value := os.Getenv(key); value != "" {
        return value
    }
    return defaultValue
}

This means go run ./cmd/api works out of the box in development (with sensible defaults), and production is configured purely through environment variables — no config file parsing, no YAML, no TOML.

Production Checklist

Before going to production:

Security:
[ ] Set strong JWT_SECRET (32+ random chars: openssl rand -hex 32)
[ ] Set DB_SSL_MODE=require (encrypt DB connections)
[ ] Set Redis password (REDIS_PASSWORD=...)
[ ] Change LiteLLM master key from "sk-1234"
[ ] Set DB_PASSWORD to something strong

Observability:
[ ] Configure log aggregation (the structured JSON logs are ready for Datadog, Loki, etc.)
[ ] Set up /health and /ready monitoring
[ ] Set DB backup schedule

Scale:
[ ] Increase DB_MAX_OPEN_CONNS for higher concurrency (default 25)
[ ] Increase memory limits if needed
[ ] Enable Redis sentinel/cluster for HA

LLM:
[ ] Set real OPENAI_API_KEY and/or ANTHROPIC_API_KEY
[ ] Configure LiteLLM fallback routing

The most critical: JWT_SECRET. Any JWT signed with the default value can be forged by anyone who reads this article. Always override it.

Running the Full Stack

# 1. Clone and configure
git clone https://github.com/wyuneed/go-agent-api
cd go-agent-api
cp .env.example .env
# Edit .env: add API keys, change JWT_SECRET

# 2. Start all services
make docker-up
# or: docker-compose -f deployments/docker-compose.yml up -d

# 3. Run migrations (if not already run via the migrate service)
make migrate-up

# 4. Verify
curl http://127.0.0.1:8080/health
# {"success":true,"data":{"status":"ok",...}}

# 5. Open Swagger UI
open http://127.0.0.1:8080/swagger/index.html

# 6. Register and test
curl -X POST http://127.0.0.1:8080/v1/auth/register \
  -H "Content-Type: application/json" \
  -d '{"email":"you@example.com","password":"secret","name":"You"}'

curl -X POST http://127.0.0.1:8080/v1/auth/login \
  -H "Content-Type: application/json" \
  -d '{"email":"you@example.com","password":"secret"}'
# → returns access_token

curl -X POST http://127.0.0.1:8080/v1/chat/completions \
  -H "Authorization: Bearer <access_token>" \
  -H "Content-Type: application/json" \
  -d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"What is 15% of 847?"}]}'

What We Just Learned

FROM scratch produces an 18MB image — no OS, no shell, just the binary and its assets
CGO_ENABLED=0 produces a fully static binary that runs in the scratch container
Migrations ship inside the image (COPY --from=builder /migrations) — binary and schema changes are atomic
depends_on: condition: service_healthy ensures startup order is correct, not just "service started"
LiteLLM decouples LLM providers from application code — swap GPT-4o for Claude with a config change
Graceful shutdown uses signal.Notify(quit, SIGINT, SIGTERM) + server.Shutdown(ctx) with a timeout
All configuration is environment-variable driven with sensible defaults for local development

Series Complete

You have now read through the entire architecture:

Article	Topic
1	Architecture Overview: DDD, five pillars, explicit DI
2	Domain Layer: Entities with business rules, repository interfaces
3	Tool System: 15-line tools, OpenAI format, per-token ACLs
4	Eino Workflow: 6-node DAG, ReAct loop, type-safe routing
5	Dual Auth: JWT + API keys, hashing, fire-and-forget tracking
6	SSE Streaming: Channel pipeline, Flusher, context cancellation
7	Human-in-the-Loop: Pause, approve, resume with `*bool`
8	Deployment: 18MB scratch image, health checks, graceful shutdown

The codebase is at github.com/wyuneed/go-agent-api. It is production-ready and designed to be extended — add tools, add agents, add new LLM providers, replace components. The architecture makes extension easy and regression unlikely.

#7 - Human-in-the-Loop AI: How to Pause, Approve, and Resume an Agentic Workflow

Izu Tolandona — Sat, 21 Feb 2026 16:58:40 +0000

Part 7 of the "Building Production-Ready AI Agent APIs in Go" series

The hardest problem in deploying AI agents is not intelligence — it is control.

When an AI agent can take real actions in the world (send an email, modify a database record, make a purchase, delete a file), you often need a human to review and approve before the action runs. The agent is not wrong to propose the action — but the consequences are irreversible, and you want a human in the approval chain.

This is called human-in-the-loop (HITL). Implementing it correctly requires:

The workflow pausing mid-execution without losing state
The paused state being persisted (so a restart does not lose it)
An API endpoint for the human to approve or reject
The workflow resuming from exactly where it stopped

All four are implemented in this project. Let me show you how.

The Problem: Irreversible Tool Actions

Consider a hypothetical DeleteFileTool. If the AI agent calls this tool and it runs without review, files are gone. Even if the AI was right, users need to feel in control.

The standard approach in this codebase: any tool can mark itself as requiring approval by overriding one method.

// Default: tool runs immediately
type BaseTool struct{}
func (BaseTool) RequiresApproval() bool { return false }

// Override to require approval
type DeleteFileTool struct {
    tool.BaseTool
}

func (t *DeleteFileTool) RequiresApproval() bool {
    return true  // This is the ENTIRE change needed to pause the workflow
}

That single method override triggers the entire approval flow. Everything else — pausing the workflow, persisting state, resuming on approval — is handled automatically.

Step 1: The actNode Detects RequiresApproval

In internal/infrastructure/eino/graphs/chatbot_graph.go, the actNode processes each tool call. Before executing, it checks RequiresApproval():

func (g *ChatbotGraph) actNode(ctx context.Context, s *state.AgentState) (*state.AgentState, error) {
    s.ToolResults = []port.ChatMessage{}

    for _, tc := range s.PendingTools {
        args, err := tc.ParseArguments()
        if err != nil {
            s.AddToolResult(tc.ID, tc.Function.Name, map[string]string{"error": "failed to parse arguments"})
            continue
        }

        t, ok := g.toolRegistry.Get(tc.Function.Name)
        if !ok {
            s.AddToolResult(tc.ID, tc.Function.Name, map[string]string{"error": "tool not found"})
            continue
        }

        // THE APPROVAL CHECK
        if t.RequiresApproval() {
            s.RequiresApproval = true
            s.ApprovalReason = fmt.Sprintf("Tool '%s' requires approval", tc.Function.Name)
            s.ApprovalData = map[string]any{
                "tool":      tc.Function.Name,
                "arguments": args,
                "tool_call": tc,
            }
            return s, nil  // Return immediately — do not execute this tool
        }

        // Normal execution
        result, err := t.Execute(ctx, args)
        if err != nil {
            s.AddToolResult(tc.ID, tc.Function.Name, map[string]string{"error": err.Error()})
        } else {
            s.AddToolResult(tc.ID, tc.Function.Name, result)
        }
        s.ToolCallsCount++
    }

    s.PendingTools = nil
    return s, nil
}

When RequiresApproval() returns true:

s.RequiresApproval = true — signals the routing function
s.ApprovalData — stores exactly what was going to be executed (for display to the human)
return s, nil immediately — the loop stops; pending tools are NOT cleared

The routing function after actNode sends the graph to human_approval:

// Actually, the routing happens after think, not act.
// The actNode sets RequiresApproval, then returns.
// The routing after observe checks it:
func (g *ChatbotGraph) routeAfterObserve(_ context.Context, s *state.AgentState) (string, error) {
    if s.ShouldStop {
        return "response", nil
    }
    if s.RequiresApproval {
        return "human_approval", nil  // ← Goes here
    }
    return "think", nil
}

Step 2: The humanApprovalNode Pauses the Workflow

func (g *ChatbotGraph) humanApprovalNode(_ context.Context, s *state.AgentState) (*state.AgentState, error) {
    // This node pauses the workflow. State will be persisted and
    // workflow resumed via the approve API endpoint.
    return s, nil
}

This function does nothing. It returns immediately. But its routing function determines what happens next:

func (g *ChatbotGraph) routeAfterApproval(_ context.Context, s *state.AgentState) (string, error) {
    if s.IsApproved == nil {
        return compose.END, nil  // No decision yet → end this graph run
    }
    if *s.IsApproved {
        return "act", nil     // Approved → resume tool execution
    }
    return "response", nil    // Rejected → generate rejection response
}

IsApproved *bool — a pointer to bool. This is the core of the three-state design:

Value	Meaning
`nil`	No decision yet (first time through)
`true`	Human approved
`false`	Human rejected

When the workflow first reaches humanApprovalNode, IsApproved is nil. The routing returns compose.END — the graph run terminates here.

The entire AgentState at this point (including the pending tool calls that weren't executed) is serialized and stored in the database. The HTTP response to the client contains the conversation's pending_approval status, telling the UI to ask for user input.

Step 3: Conversation.RequestApproval() — Storing the Pause State

In the SendMessage use case, after the graph run ends with RequiresApproval == true, the conversation status is updated:

// After graph.Invoke() returns with s.RequiresApproval == true:
conv.RequestApproval(s.CurrentNode, s.ApprovalData)

From internal/domain/entity/conversation.go:

func (c *Conversation) RequestApproval(node string, data map[string]any) {
    c.Status = ConversationStatusPending  // "pending_approval"
    c.CurrentNode = node                 // Which Eino node triggered this
    c.Metadata["pending_approval"] = data // What data the human needs to review
    c.UpdatedAt = time.Now()
}

After calling RequestApproval() and saving the conversation to the database, the response to the client includes:

{
    "conversation": {
        "id": "...",
        "status": "pending_approval",
        "metadata": {
            "pending_approval": {
                "tool": "delete_file",
                "arguments": {"path": "/important/data.csv"},
                "tool_call": {...}
            }
        }
    }
}

The UI reads status: "pending_approval" and shows the human a dialog: "The AI wants to delete /important/data.csv. Approve?"

Step 4: The ApproveActionUseCase — Validating and Transitioning

When the human clicks Approve or Reject, they hit POST /v1/conversations/{id}/approve. The use case in internal/application/usecase/chat/approve_action.go:

func (uc *ApproveActionUseCase) Execute(ctx context.Context, input ApproveInput) (*ApproveOutput, error) {
    conv, err := uc.convRepo.FindByID(ctx, input.ConversationID)
    if err != nil {
        return nil, fmt.Errorf("conversation not found: %w", err)
    }

    // Authorization: only the conversation owner can approve
    if conv.UserID != input.UserID {
        return nil, fmt.Errorf("unauthorized access to conversation")
    }

    // Guard: must actually be waiting for approval
    if !conv.IsPendingApproval() {
        return nil, fmt.Errorf("conversation is not pending approval")
    }

    // Domain method handles the state transition
    if input.Approved {
        conv.Approve()   // Status → "active", clears pending_approval metadata
    } else {
        conv.Reject(input.Reason)  // Status → "active", records rejection reason
    }

    // Persist the updated conversation
    if err := uc.convRepo.Update(ctx, conv); err != nil {
        return nil, fmt.Errorf("update conversation: %w", err)
    }

    return &ApproveOutput{Conversation: conv, Status: "approved"}, nil
}

Three validations happen in sequence:

Does the conversation exist? If not, 404
Does the user own it? If not, 403 (prevents user A from approving user B's conversation)
Is it actually waiting for approval? If not, 400 (prevents double-approval)

Then the domain entity handles the transition: conv.Approve() or conv.Reject(reason). These methods clean up the pending_approval metadata and update the status.

Step 5: Resuming the Workflow

After the human approves, the next message from the client (or the approval response itself) needs to resume the workflow. The workflow is resumed by loading the saved AgentState from the conversation's WorkflowState field, setting IsApproved = &true, and running the graph again.

The key routing logic that makes resumption work:

func (g *ChatbotGraph) routeAfterApproval(_ context.Context, s *state.AgentState) (string, error) {
    if s.IsApproved == nil {
        return compose.END, nil   // First run: no decision → stop
    }
    if *s.IsApproved {
        return "act", nil         // Resumed with approval → execute tools
    }
    return "response", nil        // Resumed with rejection → respond to user
}

When the graph is resumed with IsApproved = &true:

START → router → think → act (but wait — we are resuming)
Actually: the saved state is loaded with PendingTools still populated (they were not cleared on the first run)
The graph is re-entered, IsApproved is already set to &true
humanApprovalNode runs, routeAfterApproval returns "act"
actNode executes the pending tools
Workflow continues normally

The fact that PendingTools were not cleared when RequiresApproval was detected is intentional. The pending tool calls are preserved in the serialized state, ready to be executed when the human approves.

The Approval Handler

The HTTP handler in internal/infrastructure/http/handler/chat_handler.go:

// ApproveAction godoc
// @Summary      Approve or reject a pending action
// @Description  Resumes a conversation that is paused waiting for human-in-the-loop approval
// @Tags         conversations
// @Param        id    path  string               true "Conversation ID (UUID)"
// @Param        body  body  request.ApprovalRequest  true "Approval decision"
// @Router       /v1/conversations/{id}/approve [post]
func (h *ChatHandler) ApproveAction(w http.ResponseWriter, r *http.Request) {
    user := middleware.GetUserFromContext(r.Context())

    convID, err := uuid.Parse(chi.URLParam(r, "id"))
    if err != nil {
        response.Error(w, http.StatusBadRequest, "INVALID_ID", "Invalid conversation ID")
        return
    }

    var req request.ApprovalRequest
    if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
        response.Error(w, http.StatusBadRequest, "INVALID_REQUEST", "Invalid request body")
        return
    }

    result, err := h.approveAction.Execute(r.Context(), chat.ApproveInput{
        ConversationID: convID,
        UserID:         user.ID,
        Approved:       req.Approved,
        Reason:         req.Reason,
    })
    if err != nil {
        response.Error(w, http.StatusInternalServerError, "APPROVAL_ERROR", err.Error())
        return
    }

    response.JSON(w, http.StatusOK, result)
}

The request body:

{
    "approved": true,
    "reason": ""
}

Or to reject:

{
    "approved": false,
    "reason": "I don't want to delete that file"
}

The Complete Flow, End to End

Let us trace the complete human-in-the-loop cycle:

1. User: "Delete the old backup file"
   POST /v1/conversations/{id}/messages
   {"message": "Delete the old backup file"}

2. Workflow runs: router → think → act
   LLM calls delete_file tool
   actNode: t.RequiresApproval() == true
   AgentState: RequiresApproval = true, ApprovalData = {tool: "delete_file", args: {path: "/backup.csv"}}
   humanApprovalNode: IsApproved == nil → routeAfterApproval returns END
   Graph terminates

3. Server: conv.RequestApproval("act", s.ApprovalData)
   conv.Status = "pending_approval"
   Saves conversation + AgentState to DB
   Response: {conversation: {status: "pending_approval", metadata: {pending_approval: {...}}}}

4. UI: shows "The AI wants to delete /backup.csv — Approve?"
   User clicks Approve
   POST /v1/conversations/{id}/approve
   {"approved": true}

5. ApproveActionUseCase:
   - Verifies user owns conversation
   - Verifies status is "pending_approval"
   - conv.Approve() → status = "active"
   - Saves conversation
   Response: {status: "approved"}

6. Workflow resumes with IsApproved = &true
   humanApprovalNode: IsApproved = &true → routeAfterApproval returns "act"
   actNode: executes delete_file
   File deleted, result added to messages
   observe → think → response
   Final answer: "I've deleted /backup.csv"

7. User sees: "I've deleted /backup.csv"

The user was in control throughout. The AI proposed, the human decided, the AI executed.

Why *bool Instead of bool

The IsApproved *bool field is worth dwelling on. Why not just IsApproved bool?

With bool:

false means "not approved" OR "not yet decided" — ambiguous
You need a second field like ApprovalDecisionMade bool to distinguish "no decision" from "rejected"

With *bool:

nil = "no decision yet" — the workflow just reached this node for the first time
true = "approved" — human said yes
false = "rejected" — human said no

This is a fundamental Go idiom: use pointer types to add a null/absent state. It appears repeatedly in the codebase wherever three-value semantics are needed (LastUsedAt *time.Time, CompletedAt *time.Time on conversations).

The routing function explicitly checks for nil:

if s.IsApproved == nil {
    return compose.END, nil
}
if *s.IsApproved {
    return "act", nil
}
return "response", nil

This is safer than if !s.IsApproved — you cannot accidentally dereference a nil pointer because you check for nil first.

What We Just Learned

Any tool marks itself as requiring approval with a single method: func (t *YourTool) RequiresApproval() bool { return true }
The Eino actNode checks RequiresApproval() before executing — if true, it sets RequiresApproval = true on AgentState and returns without executing
IsApproved *bool uses pointer semantics for three states: nil (waiting), true (approved), false (rejected)
Conversation.RequestApproval() persists the pause state to the DB
ApproveActionUseCase validates ownership and uses domain methods (conv.Approve() / conv.Reject()) for state transitions
The workflow resumes by loading saved state and re-running the graph with IsApproved set

#6 - Server-Sent Events in Go: Building Real-Time AI Streaming Without WebSockets

Izu Tolandona — Sat, 21 Feb 2026 16:55:33 +0000

Part 6 of the "Building Production-Ready AI Agent APIs in Go" series

Every AI chat interface you have used — ChatGPT, Claude, Gemini — shows you text as it is generated, word by word. The LLM does not wait until the full response is complete before sending it. It streams.

Implementing this in Go involves four layers working together: the LLM provider sends SSE (Server-Sent Events) → we parse it into a typed Go channel → the use case sends events downstream → the HTTP handler writes them to the client with http.Flusher.

The good news: Go's standard library handles all of this. No WebSockets, no third-party streaming library, no special runtime. Just bufio.Reader, channels, and http.Flusher.

Why SSE Over WebSockets for LLM Streaming

WebSockets provide full-duplex communication — both client and server can send messages at any time. For LLM streaming, that capability is unnecessary. The request flow is:

Client → sends full message (request)
Server → streams tokens back (response)

This is still fundamentally request-response. The server streams the response, but the client is not sending data during streaming. SSE is designed for exactly this pattern: the server pushes data to the client over a persistent HTTP connection. The client reads it as a stream.

SSE advantages over WebSockets for this use case:

Works through HTTP/2 proxies and load balancers without special configuration
Automatic reconnection built into browser clients
Uses standard HTTP headers — no protocol upgrade negotiation
Simpler to implement and debug (it is just text over HTTP)
Compatible with existing HTTP middleware (auth, rate limiting, logging all work unchanged)

The LLMProvider Interface: Streaming as a Contract

The streaming contract is defined in internal/application/port/llm_provider.go:

type StreamEvent struct {
    Type     string             `json:"type"`
    Delta    string             `json:"delta,omitempty"`   // Text chunk
    ToolCall *toolspec.ToolCall `json:"tool_call,omitempty"`
    Usage    *UsageInfo         `json:"usage,omitempty"`
    Error    error              `json:"-"`   // Not serialized to JSON
    Done     bool               `json:"done,omitempty"`
}

type LLMProvider interface {
    Chat(ctx context.Context, req ChatRequest) (*ChatResponse, error)
    ChatStream(ctx context.Context, req ChatRequest) (<-chan StreamEvent, error)
}

ChatStream returns <-chan StreamEvent — a receive-only channel. The caller reads events from this channel until event.Done == true or event.Error != nil. The channel is closed by the implementor when the stream ends.

StreamEvent has a Type field with these possible values:

"content" — a text chunk, Delta contains the token(s)
"tool_call" — the LLM is requesting a tool call, ToolCall is populated
"finish" — the stream has ended, Delta is the finish reason, Usage has token counts
Plus Done: true and Error: err for terminal states

The Error field has json:"-" — it is never serialized to JSON. When we forward a StreamEvent to the client, errors go through a separate error event rather than being exposed as an internal Go error.

The LiteLLM Client: Parsing SSE

The implementation is in internal/infrastructure/llm/litellm/client.go. ChatStream makes the HTTP request and starts a goroutine:

func (c *Client) ChatStream(ctx context.Context, req port.ChatRequest) (<-chan port.StreamEvent, error) {
    req.Stream = true   // Tell the LLM to stream

    body, _ := json.Marshal(req)
    httpReq, _ := http.NewRequestWithContext(ctx, "POST",
        c.baseURL+"/chat/completions", bytes.NewReader(body))

    httpReq.Header.Set("Content-Type", "application/json")
    httpReq.Header.Set("Authorization", "Bearer "+c.apiKey)
    httpReq.Header.Set("Accept", "text/event-stream")

    resp, err := c.httpClient.Do(httpReq)
    if err != nil {
        return nil, fmt.Errorf("do request: %w", err)
    }

    if resp.StatusCode != http.StatusOK {
        body, _ := io.ReadAll(resp.Body)
        resp.Body.Close()
        return nil, fmt.Errorf("API error (status %d): %s", resp.StatusCode, string(body))
    }

    events := make(chan port.StreamEvent)
    go c.readSSE(ctx, resp, events)  // Parse SSE in a goroutine

    return events, nil
}

Notice: we check the HTTP status before starting the goroutine. If the request fails (401 Unauthorized, 400 Bad Request, etc.), we return an error synchronously. The goroutine only starts if the connection was established successfully.

The SSE parser:

func (c *Client) readSSE(ctx context.Context, resp *http.Response, events chan<- port.StreamEvent) {
    defer close(events)        // Always close the channel when done
    defer resp.Body.Close()    // Always close the response body

    reader := bufio.NewReader(resp.Body)

    for {
        // Check for context cancellation (client disconnect)
        select {
        case <-ctx.Done():
            events <- port.StreamEvent{Error: ctx.Err()}
            return
        default:
        }

        line, err := reader.ReadString('\n')
        if err != nil {
            if err != io.EOF {
                events <- port.StreamEvent{Error: err}
            }
            events <- port.StreamEvent{Done: true}
            return
        }

        line = strings.TrimSpace(line)

        if line == "" {
            continue  // Empty lines are SSE event separators — skip
        }

        if !strings.HasPrefix(line, "data: ") {
            continue  // Ignore SSE comment lines (": keep-alive") and headers
        }

        data := strings.TrimPrefix(line, "data: ")

        if data == "[DONE]" {
            events <- port.StreamEvent{Done: true}
            return
        }

        // Parse the JSON chunk
        var chunk struct {
            Choices []struct {
                Delta struct {
                    Content   string              `json:"content"`
                    ToolCalls []toolspec.ToolCall `json:"tool_calls"`
                } `json:"delta"`
                FinishReason string `json:"finish_reason"`
            } `json:"choices"`
            Usage *port.UsageInfo `json:"usage"`
        }

        if err := json.Unmarshal([]byte(data), &chunk); err != nil {
            continue  // Skip malformed chunks (can happen with some providers)
        }

        if len(chunk.Choices) > 0 {
            choice := chunk.Choices[0]

            if choice.Delta.Content != "" {
                events <- port.StreamEvent{Type: "content", Delta: choice.Delta.Content}
            }

            for _, tc := range choice.Delta.ToolCalls {
                events <- port.StreamEvent{Type: "tool_call", ToolCall: &tc}
            }

            if choice.FinishReason != "" {
                events <- port.StreamEvent{Type: "finish", Delta: choice.FinishReason, Usage: chunk.Usage}
            }
        }
    }
}

A few implementation details worth noting:

bufio.NewReader(resp.Body) — The standard bufio.NewReader wraps the HTTP response body with buffering. ReadString('\n') reads until a newline, which is how SSE lines are delimited.

The [DONE] sentinel — OpenAI's SSE format uses data: [DONE] to signal stream completion. This is not JSON — it is a literal string. We check for it explicitly before attempting JSON parsing.

The ctx.Done() select — Between each line, we check if the context is cancelled. This is how client disconnects propagate. When the HTTP client closes its connection, the request context is cancelled. The select fires, we send an error event, and return — which closes the channel and the response body. No goroutine leak.

Silently skip malformed JSON — json.Unmarshal failures are ignored with continue. Some SSE providers send occasional keep-alive messages or other non-JSON data. Silently skipping them is more resilient than returning an error.

The SSE Response Helpers

The internal/pkg/response/response.go package provides helpers for the HTTP handler side:

// Stream sets the SSE headers on the response
func Stream(w http.ResponseWriter) {
    w.Header().Set("Content-Type", "text/event-stream")
    w.Header().Set("Cache-Control", "no-cache")
    w.Header().Set("Connection", "keep-alive")
    w.Header().Set("X-Accel-Buffering", "no")   // Disable nginx buffering
}

// SSEEvent sends a named SSE event
func SSEEvent(w http.ResponseWriter, event, data string) {
    if event != "" {
        fmt.Fprintf(w, "event: %s\n", event)
    }
    fmt.Fprintf(w, "data: %s\n\n", data)  // Double newline = event separator
    if f, ok := w.(http.Flusher); ok {
        f.Flush()
    }
}

// SSEData sends a JSON-serialized data event
func SSEData(w http.ResponseWriter, data any) {
    b, _ := json.Marshal(data)
    fmt.Fprintf(w, "data: %s\n\n", string(b))
    if f, ok := w.(http.Flusher); ok {
        f.Flush()
    }
}

X-Accel-Buffering: no is the nginx-specific header that tells the reverse proxy not to buffer the response. Without this, nginx waits for the full response before forwarding it — defeating the purpose of streaming. Other proxies have equivalent headers.

http.Flusher — Go's http.ResponseWriter optionally implements http.Flusher with a single method: Flush(). This forces buffered data to be written to the client immediately. Without calling Flush() after each fmt.Fprintf, Go's HTTP layer might batch multiple writes and send them together — also defeating the purpose of streaming.

The ok check (if f, ok := w.(http.Flusher); ok) is defensive: not all ResponseWriter implementations support flushing (test recorders, for example). In production, net/http's built-in ResponseWriter always implements Flusher.

The Chat Handler: Putting It Together

The complete streaming handler in internal/infrastructure/http/handler/chat_handler.go:

func (h *ChatHandler) ChatCompletions(w http.ResponseWriter, r *http.Request) {
    user := middleware.GetUserFromContext(r.Context())
    token := middleware.GetTokenFromContext(r.Context())

    var req request.ChatCompletionRequest
    if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
        response.Error(w, http.StatusBadRequest, "INVALID_REQUEST", "Invalid request body")
        return
    }

    // Branch: streaming or non-streaming
    if req.Stream {
        h.handleStreamingChat(w, r, user, token, req)
        return
    }

    // Non-streaming: wait for full response
    result, err := h.sendMessage.Execute(r.Context(), chat.SendMessageInput{
        UserID: user.ID, Messages: req.Messages, Model: req.Model,
    })
    if err != nil {
        response.Error(w, http.StatusInternalServerError, "CHAT_ERROR", err.Error())
        return
    }
    response.JSON(w, http.StatusOK, result)
}

func (h *ChatHandler) handleStreamingChat(
    w http.ResponseWriter,
    r *http.Request,
    user *entity.User,
    _ *entity.Token,
    req request.ChatCompletionRequest,
) {
    response.Stream(w)  // Set SSE headers

    events, err := h.sendMessage.ExecuteStream(r.Context(), chat.SendMessageInput{
        UserID: user.ID, Messages: req.Messages, Model: req.Model,
    })
    if err != nil {
        response.SSEData(w, map[string]string{"error": err.Error()})
        return
    }

    for event := range events {
        if event.Error != nil {
            response.SSEData(w, map[string]string{"error": event.Error.Error()})
            break
        }
        response.SSEData(w, event)  // Forward each event to the client
        if event.Done {
            break
        }
    }

    response.SSEEvent(w, "", "[DONE]")  // OpenAI-compatible termination
}

The streaming handler:

Sets SSE headers
Gets an event channel from the use case
Range-loops over the channel, writing each event
Breaks on error or Done
Sends [DONE] as the final event

The for event := range events loop is elegant — it naturally terminates when the channel is closed. The goroutine inside the LiteLLM client closes the channel when it is done, which unblocks the range loop.

Context Cancellation: The Full Pipeline

What happens when the client disconnects mid-stream?

Go's net/http server detects client disconnection and cancels the request's context.Context. Here is how that propagates through our stack:

Client disconnects
  → net/http cancels r.Context()
  → handleStreamingChat is using r.Context() via the range loop
  → r.Context() is passed to sendMessage.ExecuteStream()
  → ExecuteStream passes ctx to litellm.ChatStream()
  → ChatStream creates httpReq with http.NewRequestWithContext(ctx, ...)
  → The HTTP request to LiteLLM is also cancelled
  → readSSE's select{case <-ctx.Done():} fires
  → readSSE sends an error event and returns
  → events channel is closed
  → for event := range events loop in handleStreamingChat terminates

The entire pipeline cleans up through a single context cancellation. No goroutine leaks, no open HTTP connections to LiteLLM, no wasted tokens being generated for a disconnected client.

This is why context.Context is so important in Go. It is not bureaucracy — it is the mechanism that lets you thread cancellation signals through an entire call stack.

Testing Streaming Handlers

Testing SSE handlers requires a custom response recorder that implements http.Flusher:

type flushableRecorder struct {
    *httptest.ResponseRecorder
    flushed int
}

func (f *flushableRecorder) Flush() {
    f.flushed++
}

func TestChatHandler_Streaming(t *testing.T) {
    // Create a mock LLM that returns streaming events
    mockLLM := &MockLLMProvider{}
    events := make(chan port.StreamEvent, 3)
    events <- port.StreamEvent{Type: "content", Delta: "Hello"}
    events <- port.StreamEvent{Type: "content", Delta: " world"}
    events <- port.StreamEvent{Done: true}
    close(events)
    mockLLM.On("ChatStream", mock.Anything, mock.Anything).Return(events, nil)

    // Setup handler and recorder
    handler := NewChatHandler(/* ... */)
    rec := &flushableRecorder{ResponseRecorder: httptest.NewRecorder()}

    req := httptest.NewRequest("POST", "/v1/chat/completions",
        strings.NewReader(`{"stream": true, "messages": [{"role":"user","content":"hi"}]}`))

    handler.ChatCompletions(rec, req)

    // Verify SSE headers were set
    assert.Equal(t, "text/event-stream", rec.Header().Get("Content-Type"))

    // Verify events were written and flushed
    assert.Contains(t, rec.Body.String(), `"delta":"Hello"`)
    assert.Contains(t, rec.Body.String(), "[DONE]")
    assert.Greater(t, rec.flushed, 0)
}

The flushableRecorder wraps httptest.ResponseRecorder and adds a no-op Flush() method. This satisfies the http.Flusher interface check in SSEData() without actually flushing anything (since it is a test recorder, not a real connection).

What We Just Learned

SSE is simpler than WebSockets for LLM streaming — it is just HTTP with persistent connections and text events
LLMProvider.ChatStream() returns <-chan StreamEvent — a typed, closeable channel
bufio.NewReader + ReadString('\n') parses SSE line by line
Context cancellation propagates from client disconnect → request context → HTTP request → goroutine shutdown
http.Flusher.Flush() must be called after each write — otherwise Go's HTTP layer buffers events
X-Accel-Buffering: no disables nginx proxy buffering
The [DONE] sentinel is checked as a literal string before JSON parsing
Testing requires a custom recorder that implements http.Flusher

#5 - JWT Meets API Keys: Building Dual-Auth in Go Without the Usual Mistakes

Izu Tolandona — Sat, 21 Feb 2026 16:52:55 +0000

Part 5 of the "Building Production-Ready AI Agent APIs in Go" series

Most authentication tutorials make you choose: JWT tokens or API keys. User-facing web apps usually want JWTs. Machine-to-machine integrations usually want long-lived API keys. If your product serves both — humans using a web app and developers integrating via code — you need both.

This project supports both in a single authentication pipeline. The decision point is one line: does the token start with "sk-"? If yes, it is an API key. Otherwise, it is a JWT. Everything downstream — rate limiting, tool permissions, user lookup — is identical.

Let us walk through how it is built.

The Two Authentication Flows

Before code, the high-level picture:

JWT Flow (for web apps):

User logs in → POST /v1/auth/login
→ bcrypt verify password
→ create Token record in DB
→ generate JWT (access token, 15 min) + JWT (refresh token, 7 days)
→ return both tokens to client
→ client sends: Authorization: Bearer eyJhbGci...

API Key Flow (for developers):

User creates key → POST /v1/auth/tokens (authenticated)
→ generate "sk-{uuid}" raw key
→ SHA-256 hash the raw key
→ store Token record with hash (NEVER store raw)
→ return raw key once (user saves it)
→ developer sends: Authorization: Bearer sk-abc123...

Both result in a Token entity and a User entity being available in the request context. The rest of the codebase uses them identically.

The JWT Manager: Generating and Validating Tokens

The JWT utility lives in internal/pkg/jwt/jwt.go:

type Claims struct {
    UserID    uuid.UUID `json:"uid"`
    TokenID   uuid.UUID `json:"tid"`  // References the Token record in DB
    TokenType string    `json:"typ"`  // "access" or "refresh"
    jwt.RegisteredClaims
}

type JWTManager struct {
    secret          []byte
    accessTokenTTL  time.Duration   // 15 minutes
    refreshTokenTTL time.Duration   // 7 days
    issuer          string
}

Why does the JWT carry a TokenID (UUID) in addition to UserID?

Because JWTs are stateless by design. Once issued, you cannot invalidate a JWT before its expiry time — unless you maintain a blocklist. Instead of a blocklist, we take a different approach: every JWT references a Token record in the database. When we validate the JWT, we load the Token record and check token.IsValid(). If the token has been revoked, IsValid() returns false, and the JWT is effectively invalidated.

This gives us the best of both worlds: JWTs are fast (no DB lookup for validity in theory) but can be revoked by updating the DB record.

func (m *JWTManager) GenerateAccessToken(userID, tokenID uuid.UUID) (string, error) {
    claims := Claims{
        UserID:    userID,
        TokenID:   tokenID,
        TokenType: "access",
        RegisteredClaims: jwt.RegisteredClaims{
            ExpiresAt: jwt.NewNumericDate(time.Now().Add(m.accessTokenTTL)),
            IssuedAt:  jwt.NewNumericDate(time.Now()),
            Issuer:    m.issuer,
        },
    }
    token := jwt.NewWithClaims(jwt.SigningMethodHS256, claims)
    return token.SignedString(m.secret)
}

The validation is standard HMAC-SHA256:

func (m *JWTManager) ValidateToken(tokenString string) (*Claims, error) {
    token, err := jwt.ParseWithClaims(tokenString, &Claims{}, func(token *jwt.Token) (interface{}, error) {
        if _, ok := token.Method.(*jwt.SigningMethodHMAC); !ok {
            return nil, errors.New("unexpected signing method")
        }
        return m.secret, nil
    })
    if err != nil {
        return nil, err
    }
    claims, ok := token.Claims.(*Claims)
    if !ok || !token.Valid {
        return nil, errors.New("invalid token")
    }
    return claims, nil
}

The signing method check (if _, ok := token.Method.(*jwt.SigningMethodHMAC)) is important — it prevents the "algorithm confusion" attack where an attacker crafts a JWT with alg: none or alg: RS256 and your server accidentally accepts it. Always verify the algorithm explicitly.

Hashing API Keys: Why You Never Store the Raw Value

// HashToken hashes a raw token for storage.
func HashToken(rawToken string) string {
    hash := sha256.Sum256([]byte(rawToken))
    return hex.EncodeToString(hash[:])
}

// GenerateAPIKey generates a random API key.
func GenerateAPIKey() string {
    return "sk-" + uuid.New().String()
}

When a user creates an API key, GenerateAPIKey() produces something like sk-550e8400-e29b-41d4-a716-446655440000. We:

Return the raw key to the user — this is the only time they will see it
Hash it with SHA-256
Store only the hash in the database

This mirrors how password hashing works (except we use SHA-256 instead of bcrypt, because API keys are already high-entropy random values). If someone breaches your database, they get only hashes — worthless without the raw keys.

When the user later sends the raw key, we hash it again and look it up:

func (uc *ValidateTokenUseCase) validateAPIKey(ctx context.Context, rawKey string) (*ValidateTokenResult, error) {
    tokenHash := jwt.HashToken(rawKey)
    token, err := uc.tokenRepo.FindByHash(ctx, tokenHash)
    // ...
}

The sk- prefix is important. It makes API keys visually distinct from JWTs, enables tooling to detect accidental exposure (GitHub's secret scanning looks for sk- patterns), and most importantly, it enables the single-line routing in the ValidateTokenUseCase.

The ValidateTokenUseCase: One Branching Point

The entire dual-auth routing logic lives in internal/application/usecase/auth/validate_token.go:

func (uc *ValidateTokenUseCase) Execute(ctx context.Context, rawToken string) (*ValidateTokenResult, error) {
    // Check if it's an API key (starts with "sk-")
    if strings.HasPrefix(rawToken, "sk-") {
        return uc.validateAPIKey(ctx, rawToken)
    }
    // Otherwise treat as JWT
    return uc.validateJWT(ctx, rawToken)
}

One line. That is the entirety of the routing decision. The API key path goes to validateAPIKey(), the JWT path goes to validateJWT(). Both return *ValidateTokenResult with the same shape.

The API key path:

func (uc *ValidateTokenUseCase) validateAPIKey(ctx context.Context, rawKey string) (*ValidateTokenResult, error) {
    tokenHash := jwt.HashToken(rawKey)

    token, err := uc.tokenRepo.FindByHash(ctx, tokenHash)
    if err != nil {
        return nil, fmt.Errorf("invalid API key")  // Don't leak "not found" vs "wrong key"
    }

    if !token.IsValid() {
        return nil, fmt.Errorf("API key is expired or revoked")
    }

    user, err := uc.userRepo.FindByID(ctx, token.UserID)
    if err != nil {
        return nil, fmt.Errorf("user not found")
    }

    if !user.IsActive {
        return nil, fmt.Errorf("user account is inactive")
    }

    // Update last used (fire and forget — don't add latency to every request)
    go uc.tokenRepo.UpdateLastUsed(context.Background(), token.ID)

    return &ValidateTokenResult{User: user, Token: token}, nil
}

Notice the error message for a missing token: "invalid API key" instead of "token not found". This prevents enumeration attacks — an attacker cannot distinguish between "wrong key" and "nonexistent key" from the error response.

The JWT path:

func (uc *ValidateTokenUseCase) validateJWT(ctx context.Context, tokenString string) (*ValidateTokenResult, error) {
    claims, err := uc.jwtMgr.ValidateToken(tokenString)
    if err != nil {
        return nil, fmt.Errorf("invalid token: %w", err)
    }

    // Load the Token record from DB to check IsValid() (revocation check)
    token, err := uc.tokenRepo.FindByID(ctx, claims.TokenID)
    if err != nil {
        return nil, fmt.Errorf("token not found")
    }

    if !token.IsValid() {
        return nil, fmt.Errorf("token is expired or revoked")
    }

    user, err := uc.userRepo.FindByID(ctx, claims.UserID)
    if err != nil {
        return nil, fmt.Errorf("user not found")
    }

    if !user.IsActive {
        return nil, fmt.Errorf("user account is inactive")
    }

    go uc.tokenRepo.UpdateLastUsed(context.Background(), token.ID)

    return &ValidateTokenResult{User: user, Token: token}, nil
}

The JWT path does one extra DB lookup compared to the API key path: it loads Token by the TokenID from claims. This is the revocation check — even if the JWT's expiry time has not passed, if the DB record is revoked, token.IsValid() returns false.

The Fire-and-Forget UpdateLastUsed Pattern

Both validation paths end with:

go uc.tokenRepo.UpdateLastUsed(context.Background(), token.ID)

This goroutine fires and is never awaited. It updates the last_used_at timestamp on the token in the database — useful for identifying stale tokens that should be cleaned up, and for audit trails.

Why fire-and-forget? Because adding a synchronous database write to every authenticated request would increase latency for every request. The last_used_at update is best-effort audit data. If the goroutine fails occasionally (DB hiccup, server shutdown), that is acceptable — the request still succeeded, and the timestamp is only approximate anyway.

Note the context.Background() instead of passing the request context. If we passed the request context, the goroutine might be cancelled when the request completes, before the DB write finishes. context.Background() ensures the goroutine has a fresh, uncancelled context.

The Auth Middleware: Context Injection

The auth middleware in internal/infrastructure/http/middleware/auth.go wraps the ValidateTokenUseCase and injects results into the request context:

func (m *AuthMiddleware) Authenticate(next http.Handler) http.Handler {
    return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        authHeader := r.Header.Get("Authorization")
        if authHeader == "" {
            response.Error(w, http.StatusUnauthorized, "UNAUTHORIZED", "Missing authorization header")
            return
        }

        // Strip "Bearer " prefix if present
        token := strings.TrimPrefix(authHeader, "Bearer ")
        token = strings.TrimSpace(token)

        result, err := m.validateToken.Execute(r.Context(), token)
        if err != nil {
            response.Error(w, http.StatusUnauthorized, "INVALID_TOKEN", err.Error())
            return
        }

        ctx := context.WithValue(r.Context(), UserContextKey, result.User)
        ctx = context.WithValue(ctx, TokenContextKey, result.Token)
        next.ServeHTTP(w, r.WithContext(ctx))
    })
}

After the middleware runs, every handler downstream can call:

user := middleware.GetUserFromContext(r.Context())   // *entity.User
token := middleware.GetTokenFromContext(r.Context()) // *entity.Token

The helper functions use typed context keys to avoid the interface{} collision risk:

type contextKey string

const (
    UserContextKey  contextKey = "user"
    TokenContextKey contextKey = "token"
)

func GetUserFromContext(ctx context.Context) *entity.User {
    user, ok := ctx.Value(UserContextKey).(*entity.User)
    if !ok {
        return nil
    }
    return user
}

Using a custom type (contextKey string) for context keys prevents key collisions with other middleware or libraries that might use "user" as a plain string key. The type assertion ctx.Value(UserContextKey).(*entity.User) returns nil if the value was stored with a different key type or is a different concrete type.

How Per-Token ACLs Flow Through the System

Once the token is in context, it is used downstream to filter what the user can do:

In the tool handler, available tools are filtered by the token:

func (h *ToolHandler) ListTools(w http.ResponseWriter, r *http.Request) {
    token := middleware.GetTokenFromContext(r.Context())

    var tools []toolspec.Tool
    if token != nil {
        tools = h.registry.ListForToken(token.AllowedTools)
    } else {
        tools = h.registry.List()
    }
    response.JSON(w, http.StatusOK, map[string]any{"tools": tools})
}

In the rate limiter middleware, per-token rate limits are enforced:

func (rl *RateLimiter) Limit(next http.Handler) http.Handler {
    return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        token := GetTokenFromContext(r.Context())
        if token == nil {
            next.ServeHTTP(w, r)
            return
        }

        minuteKey := fmt.Sprintf("ratelimit:%s:minute:%d", token.ID, time.Now().Unix()/60)
        count, _ := rl.redis.Incr(ctx, minuteKey).Result()

        if count > int64(token.RateLimitPerMinute) {
            response.Error(w, http.StatusTooManyRequests, "RATE_LIMIT", "Rate limit exceeded")
            return
        }
        // ...
    })
}

The rate limit key includes token.ID, so each token has its own counter. A token with RateLimitPerMinute: 1000 and a token with RateLimitPerMinute: 10 are tracked separately, even for the same user account.

The Login Use Case: Where Tokens Are Created

When a user logs in (POST /v1/auth/login), the LoginUseCase creates a Token record and issues a JWT:

func (uc *LoginUseCase) Execute(ctx context.Context, input LoginInput) (*LoginOutput, error) {
    user, err := uc.userRepo.FindByEmail(ctx, input.Email)
    if err != nil {
        return nil, apperrors.ErrInvalidCredentials
    }

    // bcrypt comparison (constant-time, prevents timing attacks)
    if err := bcrypt.CompareHashAndPassword([]byte(user.PasswordHash), []byte(input.Password)); err != nil {
        return nil, apperrors.ErrInvalidCredentials
    }

    // Create a Token record in the DB (this is what the JWT's TokenID references)
    tokenID := uuid.New()
    accessToken, _ := uc.jwtMgr.GenerateAccessToken(user.ID, tokenID)
    refreshToken, _ := uc.jwtMgr.GenerateRefreshToken(user.ID, tokenID)

    token := entity.NewAccessToken(user.ID, jwt.HashToken(accessToken), "session")
    token.ID = tokenID  // Use the same ID that's baked into the JWT
    uc.tokenRepo.Create(ctx, token)

    return &LoginOutput{
        AccessToken:  accessToken,
        RefreshToken: refreshToken,
        User:         user,
    }, nil
}

The TokenID is generated once and embedded in both the JWT claims and the DB record. This is the link that enables revocation: when you call ValidateJWT, the TokenID from claims lets us load the DB record and check IsValid().

What We Just Learned

Dual auth routes on a single strings.HasPrefix(rawToken, "sk-") check in ValidateTokenUseCase
JWTs embed TokenID to enable revocation via a DB IsValid() check
API keys are SHA-256 hashed before storage — the raw value is shown once and never stored
The sk- prefix distinguishes API keys from JWTs, prevents enumeration, and enables secret scanning
go uc.tokenRepo.UpdateLastUsed(context.Background(), token.ID) is a fire-and-forget pattern — best-effort audit without adding latency
Auth middleware injects *entity.User and *entity.Token into context for downstream use
Rate limits and tool permissions are per-token, tracked in Redis by token.ID

#4 - Eino: ByteDance's LangGraph for Go — Building a 6-Node Agentic Workflow

Izu Tolandona — Sat, 21 Feb 2026 16:50:21 +0000

Part 4 of the "Building Production-Ready AI Agent APIs in Go" series

If you build AI agents in Python, you have probably used LangGraph. It lets you define stateful, cyclical workflows as a directed graph — nodes that process state, edges that connect them, and conditional branches that let the agent decide where to go next. It is the foundation of the "ReAct" (Reason + Act) agent loop.

Eino is ByteDance's Go equivalent. Same concept, same graph-based architecture — but compiled, type-safe, and running as a single binary at ~5MB. Almost nothing has been written about it in English.

This article is the deep dive. We will build the 6-node chatbot graph that powers the agent in this project, step by step. By the end, you will understand how a Go LLM call can loop through tool execution and return a final response.

What Eino Is (and Is Not)

Eino is a workflow orchestration library from ByteDance (github.com/cloudwego/eino). It lets you:

Define nodes — functions that take a state object and return a modified version
Connect nodes with edges — simple A→B connections
Add branches — conditional routing functions that decide which node to visit next
Compile the graph into a Runnable that can be invoked with a state object

What it is not: a replacement for an LLM provider, a tool framework, or a database. It is purely the orchestration layer.

The key difference from LangGraph: Eino is fully type-parameterized. compose.NewGraph[I, O]() takes generic type parameters for input and output. The compiler enforces that your state type flows correctly through every node in the graph. In LangGraph, state shape errors only surface at runtime.

The AgentState: One Struct That Carries Everything

Before building the graph, we need the state that flows through it. Every node receives a pointer to AgentState and returns a (potentially modified) pointer. Here is the complete struct from internal/infrastructure/eino/state/agent_state.go:

type AgentState struct {
    // Identifiers
    ConversationID uuid.UUID `json:"conversation_id"`
    UserID         uuid.UUID `json:"user_id"`
    RequestID      string    `json:"request_id"`

    // Messages (the conversation history)
    Messages     []port.ChatMessage `json:"messages"`
    SystemPrompt string             `json:"system_prompt"`

    // Current turn
    UserInput     string `json:"user_input"`
    CurrentOutput string `json:"current_output"`

    // Model configuration
    Model       string  `json:"model"`
    Temperature float64 `json:"temperature"`
    MaxTokens   int     `json:"max_tokens"`

    // Tool execution
    AvailableTools []toolspec.Tool     `json:"available_tools"`
    PendingTools   []toolspec.ToolCall `json:"pending_tools"`
    ToolResults    []port.ChatMessage  `json:"tool_results"`

    // Multi-agent routing
    CurrentAgent string   `json:"current_agent"`
    AgentHistory []string `json:"agent_history"`

    // Human-in-the-loop
    RequiresApproval bool           `json:"requires_approval"`
    ApprovalReason   string         `json:"approval_reason"`
    ApprovalData     map[string]any `json:"approval_data"`
    IsApproved       *bool          `json:"is_approved,omitempty"`

    // Control flow
    Iteration     int    `json:"iteration"`
    MaxIterations int    `json:"max_iterations"`
    ShouldStop    bool   `json:"should_stop"`
    Error         string `json:"error,omitempty"`

    // Metrics
    StartTime      time.Time `json:"start_time"`
    TokensUsed     int       `json:"tokens_used"`
    ToolCallsCount int       `json:"tool_calls_count"`
}

Key design choices here:

PendingTools []toolspec.ToolCall — when the LLM responds with tool calls, they land here. The act node processes them. After processing, PendingTools is cleared.

ToolResults []port.ChatMessage — tool execution results are temporarily staged here, then moved to Messages by FlushToolResults(). This two-phase approach means the act→observe nodes can be tested independently.

IsApproved *bool — a pointer, not a bool. This is a deliberate three-state design:

nil = waiting for approval (workflow paused)
true = approved (workflow resumes to execute tools)
false = rejected (workflow routes to response node)

A regular bool cannot represent "not yet decided." The pointer makes the third state explicit and compiler-enforced.

ShouldStop bool — set by either the observe node (when Iteration >= MaxIterations) or the response node. The routing functions check this to decide when to terminate the loop.

The state also implements helpful mutation methods:

func (s *AgentState) AddAssistantMessage(content string, toolCalls []toolspec.ToolCall) {
    msg := port.ChatMessage{Role: "assistant", Content: content}
    if len(toolCalls) > 0 {
        msg.ToolCalls = toolCalls
    }
    s.Messages = append(s.Messages, msg)
}

func (s *AgentState) AddToolResult(toolCallID, name string, result any) {
    content, _ := json.Marshal(result)
    s.ToolResults = append(s.ToolResults, port.ChatMessage{
        Role: "tool", Content: string(content),
        ToolCallID: toolCallID, Name: name,
    })
}

func (s *AgentState) FlushToolResults() {
    s.Messages = append(s.Messages, s.ToolResults...)
    s.ToolResults = []port.ChatMessage{}
}

These methods enforce consistent state transitions. Any node that needs to add messages goes through these methods — there is no ad-hoc append scattered across nodes.

The Graph Structure

Here is the full workflow we are building, exactly as it appears in the source code's ASCII diagram:

    ┌─────────┐
    │  START  │
    └────┬────┘
         │
    ┌────▼────┐
    │  Router │   (selects agent type + system prompt + model)
    └────┬────┘
         │
    ┌────▼────┐
    │  Think  │   (calls the LLM, gets response + tool calls)
    └────┬────┘
    ┌────┴─────┬──────────┐
    │          │          │
┌───▼───┐ ┌───▼────┐ ┌───▼────┐
│  Act  │ │Approve │ │Response│
└───┬───┘ └────┬───┘ └───┬────┘
    │          │          │
┌───▼───┐      │          │
│Observe│      │          │
└───┬───┘      │          │
    │          │          │
    └──────────┴──────────┘
                │
           ┌────▼────┐
           │   END   │
           └─────────┘

There are 6 nodes: router, think, act, observe, human_approval, response.

The key insight: think → act → observe is a loop. After observe, the routing function checks whether to loop back to think (more tool calls needed) or go to response (done). This is the ReAct loop — the core of any tool-using agent.

Building the Graph: AddLambdaNode + compose.InvokableLambda

The graph is built in Build(ctx context.Context) in internal/infrastructure/eino/graphs/chatbot_graph.go. Here is how you add nodes:

func (g *ChatbotGraph) Build(ctx context.Context) (compose.Runnable[*state.AgentState, *state.AgentState], error) {
    graph := compose.NewGraph[*state.AgentState, *state.AgentState]()

    // Add nodes using AddLambdaNode + compose.InvokableLambda
    if err := graph.AddLambdaNode("router", compose.InvokableLambda(g.routerNode)); err != nil {
        return nil, fmt.Errorf("add router node: %w", err)
    }
    if err := graph.AddLambdaNode("think", compose.InvokableLambda(g.thinkNode)); err != nil {
        return nil, fmt.Errorf("add think node: %w", err)
    }
    if err := graph.AddLambdaNode("act", compose.InvokableLambda(g.actNode)); err != nil {
        return nil, fmt.Errorf("add act node: %w", err)
    }
    if err := graph.AddLambdaNode("observe", compose.InvokableLambda(g.observeNode)); err != nil {
        return nil, fmt.Errorf("add observe node: %w", err)
    }
    if err := graph.AddLambdaNode("human_approval", compose.InvokableLambda(g.humanApprovalNode)); err != nil {
        return nil, fmt.Errorf("add human_approval node: %w", err)
    }
    if err := graph.AddLambdaNode("response", compose.InvokableLambda(g.responseNode)); err != nil {
        return nil, fmt.Errorf("add response node: %w", err)
    }

compose.InvokableLambda() wraps a Go function with the signature func(ctx context.Context, in I) (O, error) into an Eino-compatible node. Because our graph is typed as *state.AgentState → *state.AgentState, every node function has the signature:

func (g *ChatbotGraph) routerNode(ctx context.Context, s *state.AgentState) (*state.AgentState, error)

The compiler enforces this. If you accidentally return a different type, it fails at compile time.

Edges and Branches

Simple edges connect nodes unconditionally:

// START → router (always)
graph.AddEdge(compose.START, "router")

// router → think (always, after routing decision is stored in state)
graph.AddEdge("router", "think")

// act → observe (always)
graph.AddEdge("act", "observe")

// response → END (always)
graph.AddEdge("response", compose.END)

Conditional branches use compose.NewGraphBranch():

// think → one of: act, response, human_approval
thinkBranch := compose.NewGraphBranch(g.routeAfterThink, map[string]bool{
    "act":            true,
    "response":       true,
    "human_approval": true,
})
graph.AddBranch("think", thinkBranch)

The routing function receives the current state and returns the name of the next node:

func (g *ChatbotGraph) routeAfterThink(_ context.Context, s *state.AgentState) (string, error) {
    if len(s.PendingTools) > 0 {
        return "act", nil       // LLM wants to call tools
    }
    if s.RequiresApproval {
        return "human_approval", nil  // A tool needs human sign-off
    }
    return "response", nil      // LLM gave a final answer
}

The map map[string]bool{"act": true, "response": true, "human_approval": true} tells Eino which node names are valid return values. This is a static declaration that enables the graph compiler to validate routes at compile time rather than at runtime.

Node Implementations

Router Node: Keyword-Based Agent Selection

func (g *ChatbotGraph) routerNode(ctx context.Context, s *state.AgentState) (*state.AgentState, error) {
    input := s.UserInput
    s.CurrentAgent = "general"

    codeKeywords := []string{"code", "function", "program", "debug", "fix", "implement", "class", "api"}
    for _, kw := range codeKeywords {
        if containsIgnoreCase(input, kw) {
            s.CurrentAgent = "coder"
            break
        }
    }

    researchKeywords := []string{"search", "find", "research", "what is", "who is", "latest", "news", "current"}
    for _, kw := range researchKeywords {
        if containsIgnoreCase(input, kw) {
            s.CurrentAgent = "researcher"
            break
        }
    }

    s.AgentHistory = append(s.AgentHistory, s.CurrentAgent)

    if model, ok := g.modelConfig[s.CurrentAgent]; ok {
        s.Model = model   // "coder" → "gpt-4o", others → "gpt-4o-mini"
    }
    if prompt, ok := g.systemPrompts[s.CurrentAgent]; ok {
        s.SystemPrompt = prompt
    }

    return s, nil
}

The router selects between three agent personas: general, coder, researcher. Each gets a different system prompt and model. "coder" gets gpt-4o because code generation benefits from the more capable model. "general" and "researcher" get gpt-4o-mini for cost efficiency.

This is a simple keyword router. In a production system, you might replace this with a small LLM call that classifies intent more accurately. The graph structure does not change — just the router node implementation.

Think Node: The LLM Call

func (g *ChatbotGraph) thinkNode(ctx context.Context, s *state.AgentState) (*state.AgentState, error) {
    messages := g.buildMessages(s)  // system prompt + conversation history + user input
    tools := g.toolRegistry.List()  // all available tools in OpenAI format
    s.AvailableTools = tools

    resp, err := g.llm.Chat(ctx, port.ChatRequest{
        Model:       s.Model,
        Messages:    messages,
        Tools:       tools,
        Temperature: s.Temperature,
        MaxTokens:   s.MaxTokens,
    })
    if err != nil {
        s.Error = err.Error()
        return s, err
    }

    choice := resp.Choices[0]
    s.TokensUsed += resp.Usage.TotalTokens
    s.CurrentOutput = choice.Message.Content
    s.PendingTools = choice.Message.ToolCalls  // If LLM called tools, they land here

    s.AddAssistantMessage(choice.Message.Content, choice.Message.ToolCalls)

    return s, nil
}

The think node calls the LLM with the full conversation history and tool definitions. The LLM responds with either:

A text answer (no tool calls) → PendingTools is empty → routing goes to response
Tool call requests → PendingTools is populated → routing goes to act

Act Node: Tool Execution

func (g *ChatbotGraph) actNode(ctx context.Context, s *state.AgentState) (*state.AgentState, error) {
    s.ToolResults = []port.ChatMessage{}

    for _, tc := range s.PendingTools {
        args, err := tc.ParseArguments()
        if err != nil {
            s.AddToolResult(tc.ID, tc.Function.Name, map[string]string{"error": "failed to parse arguments"})
            continue
        }

        t, ok := g.toolRegistry.Get(tc.Function.Name)
        if !ok {
            s.AddToolResult(tc.ID, tc.Function.Name, map[string]string{"error": "tool not found"})
            continue
        }

        // Check if this tool needs human approval before running
        if t.RequiresApproval() {
            s.RequiresApproval = true
            s.ApprovalReason = fmt.Sprintf("Tool '%s' requires approval", tc.Function.Name)
            s.ApprovalData = map[string]any{
                "tool": tc.Function.Name, "arguments": args, "tool_call": tc,
            }
            return s, nil  // Pause here
        }

        result, err := t.Execute(ctx, args)
        if err != nil {
            s.AddToolResult(tc.ID, tc.Function.Name, map[string]string{"error": err.Error()})
        } else {
            s.AddToolResult(tc.ID, tc.Function.Name, result)
        }
        s.ToolCallsCount++
    }

    s.PendingTools = nil
    return s, nil
}

The act node is where tools run. For each pending tool call from the LLM:

Parse the JSON arguments
Look up the tool in the registry
Check if it requires human approval (if yes, pause)
Execute and store the result

Observe Node: Loop Control

func (g *ChatbotGraph) observeNode(_ context.Context, s *state.AgentState) (*state.AgentState, error) {
    s.FlushToolResults()  // Move tool results into Messages for the next LLM call
    s.Iteration++

    if s.Iteration >= s.MaxIterations {
        s.ShouldStop = true  // Prevent infinite loops
    }

    return s, nil
}

The observe node does two things: moves tool results into the message history (so the LLM can see them next iteration), and increments the iteration counter. If we hit MaxIterations (default: 10), we force a stop. This prevents runaway agent loops that would rack up LLM costs.

The Observe Branch: Loop or Stop?

func (g *ChatbotGraph) routeAfterObserve(_ context.Context, s *state.AgentState) (string, error) {
    if s.ShouldStop {
        return "response", nil    // Force a final response
    }
    if s.RequiresApproval {
        return "human_approval", nil  // Pause for approval
    }
    return "think", nil           // Loop back for another LLM call with tool results
}

When the graph returns to think after observe, the LLM now sees the tool results in its message history. It can either call more tools or give a final answer with the information it gathered.

Compiling and Running the Graph

After defining all nodes and edges, compile the graph:

return graph.Compile(ctx, compose.WithGraphName("chatbot"))

Compile() validates the graph structure — every branch target must be a registered node, there must be a path from START to END, etc. It returns a compose.Runnable[*state.AgentState, *state.AgentState]:

type Runnable[I, O any] interface {
    Invoke(ctx context.Context, input I, opts ...Option) (O, error)
    Stream(ctx context.Context, input I, opts ...Option) (*StreamReader[O], error)
}

To run the agent:

runnable, _ := chatbotGraph.Build(ctx)

initialState := state.NewAgentState(convID, userID, "What is the weather in Tokyo?")
finalState, err := runnable.Invoke(ctx, initialState)

fmt.Println(finalState.CurrentOutput) // The agent's final response
fmt.Println(finalState.TokensUsed)    // Total tokens consumed
fmt.Println(finalState.ToolCallsCount) // How many tools were called

The Invoke() call runs the full graph synchronously. For streaming, you would use Stream() to get a StreamReader that emits state updates as they occur.

A Complete ReAct Loop Traced

Let us trace what happens when a user asks "What is 15% of 847?":

START → Router: Input contains no code or research keywords → s.CurrentAgent = "general", model = "gpt-4o-mini"
Router → Think: LLM receives the question with calculator tool available. Responds with a tool call: {"name": "calculator", "arguments": "{\"expression\": \"847 * 0.15\"}"}
Think → Act (routeAfterThink: PendingTools is not empty): Act node parses arguments, calls calculator.Execute(ctx, {"expression": "847 * 0.15"}), gets {"expression": "847 * 0.15", "result": 127.05}, stores in ToolResults
Act → Observe: Observe moves result from ToolResults into Messages. Iteration = 1. ShouldStop = false.
Observe → Think (routeAfterObserve: not stopped, no approval needed): LLM now sees the tool result in its message history. Responds: "15% of 847 is 127.05"
Think → Response (routeAfterThink: PendingTools is empty): Response node stores s.CurrentOutput = "15% of 847 is 127.05", sets ShouldStop = true
Response → END: Final state returned. finalState.CurrentOutput contains the answer. Total: 2 LLM calls, 1 tool execution, 2 graph iterations.

Why This Architecture Works

Testability. Every node is a pure function: (*AgentState) → (*AgentState, error). You can unit test actNode by constructing an AgentState with specific PendingTools and asserting the state after execution — no mocking of the graph itself needed.

Observability. The complete state after each node is available. You can log s.TokensUsed, s.ToolCallsCount, s.Iteration, and s.AgentHistory at the end of every request.

Safety. The MaxIterations counter prevents infinite loops. The RequiresApproval check prevents unauthorized tool execution. The compile-time graph validation prevents invalid routing.

Extensibility. Add a new node by implementing a function with the right signature and calling graph.AddLambdaNode(). Add a new agent persona by adding an entry to systemPrompts and modelConfig. The graph structure handles the routing.

What We Just Learned

Eino is ByteDance's Go LangGraph equivalent — type-safe, compiled, single-binary
AgentState is a single struct carrying all workflow state, passed by pointer through every node
IsApproved *bool uses pointer semantics to represent three states (nil/true/false)
compose.InvokableLambda() wraps any func(ctx, in) (out, error) into an Eino node
compose.NewGraphBranch() with a routing function handles conditional edges
The think→act→observe loop is the ReAct pattern: reason, act, observe, repeat
MaxIterations prevents runaway loops; ShouldStop signals the graph to terminate

#3 - Add a New AI Tool in 15 Lines of Go: The OpenAI-Compatible Tool System Explained

Izu Tolandona — Sat, 21 Feb 2026 16:47:52 +0000

Part 3 of the "Building Production-Ready AI Agent APIs in Go" series

Every AI agent framework needs a tool system. Python's LangChain has @tool decorators. LangGraph has ToolNode. In Go, we built ours around a clean interface pattern — and the result is that adding a new AI tool to a production agent takes about 15 lines of code and 2 lines to register it.

This is not an exaggeration. By the end of this article, you will have added a real, working tool that your AI agent can call. Let me show you exactly how it works.

The Tool Interface: 6 Methods, 2 Are Free

Everything starts with this interface in internal/application/usecase/tool/tool_registry.go:

// Tool interface that all tools must implement.
type Tool interface {
    Name() string
    Description() string
    Definition() toolspec.Tool
    Execute(ctx context.Context, args map[string]any) (any, error)
    RequiresApproval() bool
    Validate(args map[string]any) error
}

Six methods. But here is the trick — you never implement RequiresApproval() or Validate() yourself unless you need to. There is a BaseTool struct you embed that gives you both for free:

// BaseTool provides default implementations.
type BaseTool struct{}

func (BaseTool) RequiresApproval() bool            { return false }
func (BaseTool) Validate(args map[string]any) error { return nil }

So in practice, every tool you write needs exactly four methods: Name(), Description(), Definition(), and Execute(). Three of those are trivial one-liners. The only method with real logic is Execute().

The CalculatorTool: A 15-Line Deep Dive

Here is the entire CalculatorTool from internal/application/usecase/tool/builtin/calculator.go:

type CalculatorTool struct {
    tool.BaseTool  // Embeds RequiresApproval() and Validate()
}

func NewCalculatorTool() *CalculatorTool {
    return &CalculatorTool{}
}

func (t *CalculatorTool) Name() string { return "calculator" }

func (t *CalculatorTool) Description() string {
    return "Perform mathematical calculations. Supports basic arithmetic (+, -, *, /), parentheses, and common functions."
}

func (t *CalculatorTool) Definition() toolspec.Tool {
    return toolspec.NewTool(t.Name(), t.Description(), &toolspec.JSONSchema{
        Type: "object",
        Properties: map[string]toolspec.PropertySchema{
            "expression": {
                Type:        "string",
                Description: "Mathematical expression to evaluate. Example: '(2 + 3) * 4' or '15 / 3'",
            },
        },
        Required: []string{"expression"},
    })
}

func (t *CalculatorTool) Execute(ctx context.Context, args map[string]any) (any, error) {
    expr, _ := args["expression"].(string)
    if expr == "" {
        return nil, fmt.Errorf("expression is required")
    }
    result, err := t.evaluate(expr)
    if err != nil {
        return map[string]any{"error": err.Error(), "expression": expr}, nil
    }
    return map[string]any{"expression": expr, "result": result}, nil
}

That is the entire public surface of the tool. The 15 lines that matter are the Execute() function and the struct definition. Everything else is metadata that tells the LLM what the tool does and what parameters it expects.

The Secret: Go's AST Parser as a Safe Evaluator

The evaluate() method underneath is the genuinely interesting part:

func (t *CalculatorTool) evaluate(expr string) (float64, error) {
    node, err := parser.ParseExpr(expr)
    if err != nil {
        return 0, fmt.Errorf("invalid expression: %w", err)
    }
    return t.eval(node)
}

func (t *CalculatorTool) eval(node ast.Expr) (float64, error) {
    switch n := node.(type) {
    case *ast.BasicLit:
        return strconv.ParseFloat(n.Value, 64)
    case *ast.BinaryExpr:
        left, _ := t.eval(n.X)
        right, _ := t.eval(n.Y)
        switch n.Op {
        case token.ADD: return left + right, nil
        case token.SUB: return left - right, nil
        case token.MUL: return left * right, nil
        case token.QUO:
            if right == 0 { return 0, fmt.Errorf("division by zero") }
            return left / right, nil
        }
    case *ast.ParenExpr:
        return t.eval(n.X)
    case *ast.UnaryExpr:
        val, _ := t.eval(n.X)
        if n.Op == token.SUB { return -val, nil }
        return val, nil
    }
    return 0, fmt.Errorf("unsupported expression type")
}

We use Go's own go/parser and go/ast packages — the same infrastructure the Go compiler uses — to parse and evaluate mathematical expressions. This gives us:

Safety: No eval(), no string execution. Just AST tree walking.
Correctness: Operator precedence handled automatically by the parser.
Zero dependencies: It is all in the Go standard library.

parser.ParseExpr("(2 + 3) * 4") returns an *ast.BinaryExpr where the left side is another *ast.BinaryExpr (the parenthesized 2 + 3) and the right side is *ast.BasicLit (the number 4). We walk the tree recursively and evaluate each node. No external math library needed.

The toolspec Package: Why OpenAI Format Matters

The Definition() method returns a toolspec.Tool from pkg/toolspec/openai_format.go. This is a public package because the format is shared across the entire stack:

// Tool definition compatible with OpenAI, Anthropic (via LiteLLM), and other providers.
type Tool struct {
    Type     string   `json:"type"`     // Always "function"
    Function Function `json:"function"`
}

type Function struct {
    Name        string      `json:"name"`
    Description string      `json:"description"`
    Parameters  *JSONSchema `json:"parameters,omitempty"`
    Strict      bool        `json:"strict,omitempty"`
}

type JSONSchema struct {
    Type       string                    `json:"type"`
    Properties map[string]PropertySchema `json:"properties,omitempty"`
    Required   []string                  `json:"required,omitempty"`
}

When the LLM (GPT-4o, Claude, Gemini via LiteLLM) receives this definition, it knows exactly what calculator does, what parameters it accepts, and which are required. When the LLM decides to call calculator, it returns a ToolCall:

type ToolCall struct {
    ID       string       `json:"id"`       // e.g. "call_abc123"
    Type     string       `json:"type"`     // "function"
    Function FunctionCall `json:"function"`
}

type FunctionCall struct {
    Name      string `json:"name"`       // "calculator"
    Arguments string `json:"arguments"`  // JSON string: {"expression": "(2+3)*4"}
}

// ParseArguments parses the JSON arguments into a map.
func (tc *ToolCall) ParseArguments() (map[string]any, error) {
    var args map[string]any
    return args, json.Unmarshal([]byte(tc.Function.Arguments), &args)
}

This is the OpenAI tool calling format. Because we use LiteLLM as a proxy, the exact same JSON works with GPT-4o, Claude 3.5 Sonnet, and Gemini Pro. You change your model name in a config file; the tool calling code does not change at all.

The ToolRegistry: Thread-Safe Tool Management

All tools are stored in a ToolRegistry — a thread-safe map protected by sync.RWMutex:

type ToolRegistry struct {
    tools map[string]Tool
    mu    sync.RWMutex
}

func (r *ToolRegistry) Register(tool Tool) {
    r.mu.Lock()
    defer r.mu.Unlock()
    r.tools[tool.Name()] = tool
}

func (r *ToolRegistry) Get(name string) (Tool, bool) {
    r.mu.RLock()
    defer r.mu.RUnlock()
    return r.tools[name], ok
}

func (r *ToolRegistry) List() []toolspec.Tool {
    r.mu.RLock()
    defer r.mu.RUnlock()
    defs := make([]toolspec.Tool, 0, len(r.tools))
    for _, tool := range r.tools {
        defs = append(defs, tool.Definition())
    }
    return defs
}

The write lock is only held during Register(). The much more frequent read operations (Get(), List()) use the read lock, which allows concurrent reads without blocking each other. In a high-throughput agent API, this matters.

Per-Token Tool Permissions

There is one more method that makes this registry genuinely production-ready:

// ListForToken returns tools filtered by token permissions.
func (r *ToolRegistry) ListForToken(allowedTools []string) []toolspec.Tool {
    if len(allowedTools) == 0 {
        return r.List() // All tools allowed
    }

    r.mu.RLock()
    defer r.mu.RUnlock()

    allowedMap := make(map[string]bool)
    for _, t := range allowedTools {
        if t == "*" {
            return r.List() // Wildcard: all allowed
        }
        allowedMap[t] = true
    }

    defs := make([]toolspec.Tool, 0)
    for name, tool := range r.tools {
        if allowedMap[name] {
            defs = append(defs, tool.Definition())
        }
    }
    return defs
}

Each API token in the database has an AllowedTools []string field. When the agent is invoked, the tool list passed to the LLM is filtered to only the tools that token is permitted to use. An API key for a customer who purchased "basic" might only see calculator. An admin key sees everything.

The LLM never sees tools it is not allowed to call. No extra validation needed.

Adding Your Own Tool: A Practical Example

Let's add a WeatherTool that calls a hypothetical weather API. Here is everything you need to write:

// internal/application/usecase/tool/builtin/weather.go
package builtin

import (
    "context"
    "fmt"
    "net/http"
    "encoding/json"

    "github.com/wyuneed/go-agent-api/internal/application/usecase/tool"
    "github.com/wyuneed/go-agent-api/pkg/toolspec"
)

type WeatherTool struct {
    tool.BaseTool
    apiKey string
}

func NewWeatherTool(apiKey string) *WeatherTool {
    return &WeatherTool{apiKey: apiKey}
}

func (t *WeatherTool) Name() string { return "get_weather" }

func (t *WeatherTool) Description() string {
    return "Get the current weather for a city. Returns temperature, conditions, and humidity."
}

func (t *WeatherTool) Definition() toolspec.Tool {
    return toolspec.NewTool(t.Name(), t.Description(), &toolspec.JSONSchema{
        Type: "object",
        Properties: map[string]toolspec.PropertySchema{
            "city": {
                Type:        "string",
                Description: "The city name, e.g. 'Tokyo' or 'New York'",
            },
        },
        Required: []string{"city"},
    })
}

func (t *WeatherTool) Execute(ctx context.Context, args map[string]any) (any, error) {
    city, _ := args["city"].(string)
    if city == "" {
        return nil, fmt.Errorf("city is required")
    }
    // Call your weather API here...
    return map[string]any{
        "city":        city,
        "temperature": "22°C",
        "conditions":  "Partly cloudy",
    }, nil
}

Then register it in cmd/api/main.go — two lines:

toolRegistry.RegisterAll(
    builtin.NewCalculatorTool(),
    builtin.NewWebSearchTool(cfg.Tools.WebSearchAPIKey),
    builtin.NewWeatherTool(cfg.Tools.WeatherAPIKey), // ← Add this
)

That is genuinely it. The next time a user asks "What is the weather in Tokyo?", the LLM will call get_weather with {"city": "Tokyo"}, your Execute() method runs, and the result goes back to the LLM for a final response.

The Execution Pipeline

When the Eino workflow calls your tool, it goes through execute_tool.go which handles the full pipeline:

Look up the tool in the registry by name
Check permissions — does this token's AllowedTools include this tool name?
Parse arguments — tc.ParseArguments() unmarshals the JSON string
Validate — tool.Validate(args) (default: always passes)
Execute — tool.Execute(ctx, args)

func (uc *ExecuteToolUseCase) Execute(ctx context.Context, input ExecuteToolInput) (*ExecuteToolOutput, error) {
    t, ok := uc.registry.Get(input.ToolCall.Function.Name)
    if !ok {
        return nil, apperrors.ErrNotFound.Wrap(fmt.Errorf("tool not found: %s", input.ToolCall.Function.Name))
    }

    // Check token permission
    if input.Token != nil && !input.Token.CanUseTool(t.Name()) {
        return nil, apperrors.ErrToolNotAllowed
    }

    args, err := input.ToolCall.ParseArguments()
    if err != nil {
        return nil, fmt.Errorf("parse arguments: %w", err)
    }

    if err := t.Validate(args); err != nil {
        return nil, fmt.Errorf("validate: %w", err)
    }

    start := time.Now()
    result, err := t.Execute(ctx, args)
    duration := time.Since(start)

    return &ExecuteToolOutput{
        ToolCallID: input.ToolCall.ID,
        ToolName:   t.Name(),
        Result:     result,
        Duration:   duration,
    }, err
}

Notice that the Eino graph's actNode (which we will cover in Article 4) also checks t.RequiresApproval() before executing. If a tool needs human approval, the graph pauses instead of executing — no changes needed to the ExecuteToolUseCase.

What Makes This Design Work

The elegance here comes from a few specific decisions:

Interface over inheritance. Go does not have class hierarchies. The Tool interface, combined with BaseTool embedding, gives you optional overrides without forcing you to implement everything.

Separation of definition from execution. Definition() describes the tool to the LLM in a format the LLM understands. Execute() is pure Go logic. These two concerns never mix.

Registry as the single source of truth. The LLM, the HTTP endpoint (GET /v1/tools), and the workflow graph all use the same registry. There is no duplication of tool definitions.

OpenAI format means portability. By targeting the OpenAI tool calling spec, the same tool definitions work with any LLM provider that LiteLLM supports — which today includes GPT-4o, Claude 3.5, Gemini 1.5 Pro, Llama 3, Mistral, and dozens more.

What We Just Learned

The Tool interface has 4 required methods; 2 are free via BaseTool embedding
CalculatorTool uses Go's go/ast package for safe, dependency-free expression evaluation
The toolspec package defines OpenAI-compatible tool types that work with any LLM via LiteLLM
ToolRegistry is a thread-safe map with sync.RWMutex that supports per-token filtering
Adding a new tool requires: one new file implementing the interface + two lines in main.go

Try This Now

git clone https://github.com/wyuneed/go-agent-api
make docker-up
make migrate-up
make run
curl http://127.0.0.1:8080/v1/tools  # See registered tools

#2 - Domain-Driven Design for Go Developers: Build Entities That Actually Enforce Business Rules

Izu Tolandona — Sat, 21 Feb 2026 16:43:58 +0000

Part 2 of the "Building Production-Ready AI Agent APIs in Go" series

Here is a question to consider: where does the rule "an expired token cannot be used" live in your codebase?

In many Go applications, that check exists in three or four different places — a middleware function, a repository method, a handler guard. When the rule changes, you have to find all four. When you write a test, you have to test all four.

In Domain-Driven Design, that rule lives in exactly one place: the Token entity. The method is called IsValid(), it is 5 lines, and every piece of code that needs to check token validity calls it. There is one test for the rule, and it is a plain Go test with no database, no HTTP server, no mocks.

This article shows you how the four domain entities in this project — User, Token, Conversation, and Message — encode their business rules as methods, and how the repository interface pattern makes the domain layer completely independent of PostgreSQL.

What the Domain Layer Is (and What It Cannot Import)

The domain layer lives in internal/domain/. Open the go.mod imports in any file there and you will find exactly two external packages:

import (
    "time"
    "github.com/google/uuid"
)

That is it. No pgx, no redis, no chi, no eino. The domain layer is pure Go logic. It can be compiled, tested, and reasoned about without any external infrastructure.

This is not an accident. It is the Dependency Rule: source code dependencies point inward. Domain ← Application ← Infrastructure. The inner layers cannot import the outer layers.

The practical consequence: you can run every domain test with go test ./internal/domain/... and it completes in milliseconds. No database connection needed. No containers. No ports. Just Go.

The User Entity: Roles and Business Methods

// internal/domain/entity/user.go
type UserRole string

const (
    UserRoleAdmin UserRole = "admin"
    UserRoleUser  UserRole = "user"
    UserRoleAgent UserRole = "agent"  // For AI agent service accounts
)

type User struct {
    ID           uuid.UUID
    Email        string
    PasswordHash string
    Name         string
    Role         UserRole
    IsActive     bool
    Metadata     map[string]any
    CreatedAt    time.Time
    UpdatedAt    time.Time
}

The User struct is plain data. No ORM tags, no JSON annotations (those belong in the infrastructure layer). But notice the constructor:

func NewUser(email, passwordHash, name string) *User {
    return &User{
        ID:        uuid.New(),
        Role:      UserRoleUser,    // New users are regular users by default
        IsActive:  true,            // Active by default
        Metadata:  make(map[string]any),
        CreatedAt: time.Now(),
        UpdatedAt: time.Now(),
    }
}

The constructor enforces invariants. You cannot create a User with no ID, no role, or uninitialized metadata. The zero value of User{} is invalid — the constructor is the only correct way to create one.

Then the business methods:

func (u *User) CanUseTools() bool {
    return u.IsActive && (u.Role == UserRoleAdmin || u.Role == UserRoleUser)
}

func (u *User) IsAdmin() bool {
    return u.Role == UserRoleAdmin
}

CanUseTools() encodes a business rule: only active admin and user accounts can execute tools. Agent accounts cannot. This rule lives in the entity because it is a business concern, not a technical one. The HTTP handler asks if !user.CanUseTools() — it does not duplicate the role logic.

The Token Entity: A Permission Carrier

The Token entity is where this project gets interesting. A token is not just an authentication credential — it is a permission carrier with rate limits and access control lists built in:

type Token struct {
    ID                 uuid.UUID
    UserID             uuid.UUID
    TokenHash          string      // Never store raw tokens
    TokenType          TokenType   // api_key, access, refresh
    Name               string      // "Production API Key", "Dev Testing"
    ExpiresAt          time.Time
    LastUsedAt         *time.Time  // Pointer: nil until first use
    RateLimitPerMinute int         // Per-token rate limits
    RateLimitPerDay    int
    AllowedTools       []string    // nil = all tools; ["calculator"] = calculator only
    AllowedModels      []string    // nil = all models
    IsRevoked          bool
    Metadata           map[string]any
    CreatedAt          time.Time
    UpdatedAt          time.Time
}

The business methods on Token are where the real value is:

func (t *Token) IsExpired() bool {
    return time.Now().After(t.ExpiresAt)
}

func (t *Token) IsValid() bool {
    return !t.IsRevoked && !t.IsExpired()
}

func (t *Token) CanUseTool(toolName string) bool {
    if len(t.AllowedTools) == 0 {
        return true  // nil = all tools allowed
    }
    for _, allowed := range t.AllowedTools {
        if allowed == toolName || allowed == "*" {
            return true
        }
    }
    return false
}

func (t *Token) CanUseModel(modelName string) bool {
    if len(t.AllowedModels) == 0 {
        return true  // nil = all models allowed
    }
    for _, allowed := range t.AllowedModels {
        if allowed == modelName || allowed == "*" {
            return true
        }
    }
    return false
}

IsValid() — one line that combines two conditions. Every piece of code that needs to check token validity calls this method. The rule is defined once.

CanUseTool() — the access control logic for tools. If AllowedTools is empty, all tools are allowed (open access). Otherwise, it checks for an exact match or a wildcard "*". This means you can issue API keys that are scoped to a specific subset of tools.

For example, a customer on a "Basic" plan gets an API key with AllowedTools: ["calculator"]. A "Pro" customer gets AllowedTools: nil (all tools). An enterprise customer gets AllowedTools: ["calculator", "web_search", "database_query"].

The rate limits per token mean different keys can have different throttling — your internal admin key has no limit, third-party integrations have conservative limits.

The Conversation Entity: A Go State Machine

The Conversation entity is the aggregate root for chat sessions. It has five possible statuses:

const (
    ConversationStatusActive    ConversationStatus = "active"
    ConversationStatusPending   ConversationStatus = "pending_approval"
    ConversationStatusCompleted ConversationStatus = "completed"
    ConversationStatusFailed    ConversationStatus = "failed"
    ConversationStatusArchived  ConversationStatus = "archived"
)

And the transition methods enforce the state machine:

func (c *Conversation) RequestApproval(node string, data map[string]any) {
    c.Status = ConversationStatusPending
    c.CurrentNode = node
    c.Metadata["pending_approval"] = data
    c.UpdatedAt = time.Now()
}

func (c *Conversation) Approve() {
    c.Status = ConversationStatusActive
    delete(c.Metadata, "pending_approval")
    c.UpdatedAt = time.Now()
}

func (c *Conversation) Reject(reason string) {
    c.Status = ConversationStatusActive
    c.Metadata["last_rejection"] = map[string]any{
        "reason": reason,
        "at":     time.Now(),
    }
    c.UpdatedAt = time.Now()
}

func (c *Conversation) Complete() {
    c.Status = ConversationStatusCompleted
    now := time.Now()
    c.CompletedAt = &now
    c.UpdatedAt = now
}

What makes this a proper state machine:

Transitions have names — RequestApproval(), not c.Status = "pending_approval"
Transitions carry data — RequestApproval(node, data) stores what was pending and where
Transitions have side effects — Approve() clears the pending approval metadata; Reject() records the rejection reason
UpdatedAt is always maintained — every mutation updates the timestamp

If you wrote this as raw field assignments scattered across use cases and handlers:

// BAD: business logic leaking into application layer
conv.Status = "pending_approval"
conv.CurrentNode = node
conv.Metadata["pending_approval"] = data
conv.UpdatedAt = time.Now()

...you would have to remember those four lines every time. With the domain method, it is one call.

The Conversation entity also carries Eino workflow state:

type Conversation struct {
    // ...
    CurrentNode   string         // Which Eino node is waiting
    WorkflowState map[string]any // Serialized AgentState for resumption
    // ...
}

When a workflow pauses for human approval, the entire AgentState is serialized to JSON and stored in WorkflowState. When the user approves, the state is deserialized and the workflow resumes from where it left off. The entity is the persistence boundary.

The Message Entity: Protocol Translation at Domain Level

The Message entity stores chat messages with OpenAI-compatible fields:

type Message struct {
    ID             uuid.UUID
    ConversationID uuid.UUID
    Role           MessageRole    // system, user, assistant, tool
    Content        string
    Name           string         // For tool messages
    ToolCalls      []ToolCall     // Assistant → tool calls
    ToolCallID     string         // Tool response → back-reference
    Model          string
    PromptTokens   int
    CompletionTokens int
    Latency        time.Duration
    SequenceNumber int
    CreatedAt      time.Time
}

Notice ToolCalls []ToolCall and ToolCallID string. These are the two sides of the tool calling protocol:

When the assistant wants to call a tool, it produces a message with Role = "assistant" and ToolCalls populated
When the tool returns a result, it produces a message with Role = "tool", a matching ToolCallID, and the result in Content

The factory constructors enforce correct construction:

func NewToolMessage(conversationID uuid.UUID, toolCallID string, name string, result any) *Message {
    content, _ := json.Marshal(result)
    return &Message{
        ID:             uuid.New(),
        ConversationID: conversationID,
        Role:           RoleTool,
        Name:           name,
        Content:        string(content),  // result serialized to JSON
        ToolCallID:     toolCallID,
        CreatedAt:      time.Now(),
    }
}

The result any parameter gets JSON-serialized into Content. The domain entity knows that tool results are always JSON strings. The caller passes any Go value; the entity handles the serialization.

The ToOpenAIFormat() method handles protocol translation:

func (m *Message) ToOpenAIFormat() map[string]any {
    msg := map[string]any{
        "role":    string(m.Role),
        "content": m.Content,
    }
    if len(m.ToolCalls) > 0 {
        msg["tool_calls"] = m.ToolCalls
    }
    if m.ToolCallID != "" {
        msg["tool_call_id"] = m.ToolCallID
    }
    if m.Name != "" {
        msg["name"] = m.Name
    }
    return msg
}

This method converts the domain entity into the map format expected by the OpenAI chat completions API. The domain entity knows about OpenAI's format — not as a framework dependency, but as a protocol specification that the entity is responsible for producing correctly.

Repository Interfaces: Why They Live in the Domain Layer

The repository interfaces are defined in internal/domain/repository/, not in internal/infrastructure/. This is the critical architectural decision.

// internal/domain/repository/conversation_repository.go
type ConversationFilter struct {
    UserID  *uuid.UUID
    Status  *entity.ConversationStatus
    Limit   int
    Offset  int
    OrderBy string
    Order   string
}

type ConversationRepository interface {
    Create(ctx context.Context, conv *entity.Conversation) error
    FindByID(ctx context.Context, id uuid.UUID) (*entity.Conversation, error)
    FindByUserID(ctx context.Context, userID uuid.UUID, filter ConversationFilter) ([]*entity.Conversation, error)
    Update(ctx context.Context, conv *entity.Conversation) error
    UpdateWorkflowState(ctx context.Context, id uuid.UUID, state map[string]any) error
    Delete(ctx context.Context, id uuid.UUID) error
    CountByUserID(ctx context.Context, userID uuid.UUID) (int64, error)
}

The interface lives in the domain because it expresses what the domain needs from its persistence mechanism — not what PostgreSQL can provide. The domain layer defines the contract; the infrastructure layer fulfills it.

This means:

The SendMessage use case depends on ConversationRepository (an interface) — not postgres.ConversationRepository (a concrete type)
To test SendMessage, you pass a mock implementation of ConversationRepository — no PostgreSQL needed
To swap PostgreSQL for MySQL or DynamoDB, you implement the interface with a new concrete type — the domain and application layers do not change

Testing Pure Domain Logic

The lack of external dependencies in the domain layer means tests are instant and have no setup:

// internal/domain/entity/token_test.go
func TestToken_IsValid(t *testing.T) {
    t.Run("valid token", func(t *testing.T) {
        token := entity.NewAPIKey(uuid.New(), "hash123", "test", time.Now().Add(time.Hour))
        assert.True(t, token.IsValid())
    })

    t.Run("expired token", func(t *testing.T) {
        token := entity.NewAPIKey(uuid.New(), "hash123", "test", time.Now().Add(-time.Hour))
        assert.False(t, token.IsValid())
        assert.True(t, token.IsExpired())
    })

    t.Run("revoked token", func(t *testing.T) {
        token := entity.NewAPIKey(uuid.New(), "hash123", "test", time.Now().Add(time.Hour))
        token.IsRevoked = true
        assert.False(t, token.IsValid())
    })
}

No database. No mock setup. No context. Just: construct an entity, call a method, assert the result. These tests run in under a millisecond.

The same pattern applies to the conversation state machine:

func TestConversation_ApprovalFlow(t *testing.T) {
    conv := entity.NewConversation(uuid.New(), "general")
    assert.True(t, conv.IsActive())
    assert.False(t, conv.IsPendingApproval())

    conv.RequestApproval("act", map[string]any{"tool": "dangerous_tool"})
    assert.True(t, conv.IsPendingApproval())
    assert.Equal(t, "act", conv.CurrentNode)

    conv.Approve()
    assert.True(t, conv.IsActive())
    assert.NotContains(t, conv.Metadata, "pending_approval")
}

Every business rule in the entity has a corresponding test. The tests are exhaustive because the entities are small and focused.

What Makes This Design Work

Business rules live with the data they protect. The rule "a revoked token cannot be used" lives on the Token struct, not in a validator, not in a middleware, not in a handler. The method IsValid() is the single source of truth.

Constructors are factories, not just {}.** NewConversation(), NewAPIKey(), NewUserMessage() — every entity has a constructor that sets invariants and defaults. The zero value of any entity struct is intentionally incomplete.

State transitions are named methods. RequestApproval(), Approve(), Reject(), Complete() — you read the code and understand what is happening at the business level, not just at the field level.

Protocol translation at the boundary. Message.ToOpenAIFormat() is the only place in the domain layer that knows about the OpenAI message structure. This method is the boundary between "what a message means to our domain" and "what a message looks like to an LLM API."

What We Just Learned

The domain layer imports only stdlib and uuid — zero framework dependencies
NewUser(), NewToken(), NewConversation() constructors enforce invariants that the zero value cannot
Token carries rate limits, tool allowlists, and model allowlists as first-class business data
Conversation is a proper state machine with named transition methods
Message.ToOpenAIFormat() handles protocol translation at the domain boundary
Repository interfaces live in the domain layer, not the infrastructure layer — they define what the domain needs, not what the database provides
Domain tests run in milliseconds with no external dependencies

#1 - I Built a Production-Ready AI Agent API in Go — Here's the Architecture That Makes It Actually Work

Izu Tolandona — Sat, 21 Feb 2026 16:38:42 +0000

Part 1 of the "Building Production-Ready AI Agent APIs in Go" series

Most AI agent tutorials give you a Python script. Some give you a FastAPI app with a few routes. Almost none give you something you would actually run in production.

I built one in Go. It has JWT authentication, API key management, rate limiting, a full workflow engine, streaming responses, human-in-the-loop approvals, and an 18MB Docker image. It runs on a $6/month VPS with plenty of headroom.

This article is the overview. I will walk you through every architectural decision and why we made it. The rest of the series goes deep on each piece.

Why Go for AI Agents?

Python is dominant in AI. The tooling is excellent. But if you are building an API that serves AI agents in production, Go has concrete advantages:

Memory footprint. A typical Go HTTP server uses 10-20MB of RAM at idle. A Python FastAPI equivalent uses 80-150MB. For an API handling concurrent agent sessions, this matters.

Cold start. Go binaries start in milliseconds. No interpreter to spin up, no imports to resolve at runtime. If you are running in containers that scale to zero, this is the difference between a 50ms cold start and a 2-second one.

Single binary. The entire application, with all dependencies compiled in, is one executable. No requirements.txt, no virtual environments, no pip install in your Dockerfile. Copy the binary, run it.

Concurrency model. Goroutines are cheap (2KB stack initially) and Go's runtime scheduler handles thousands of them efficiently. Running 100 concurrent agent sessions with streaming responses is straightforward.

Type safety. When the LLM returns a tool call, you know exactly what fields it has. When you pass state through a workflow graph, the compiler catches shape mismatches at compile time.

None of this means Python is wrong for AI. The model inference, fine-tuning, and research ecosystem is irreplaceable in Python. But the API serving layer? Go is an excellent fit.

The Five Pillars

The codebase is built on five architectural pillars:

┌─────────────────────────────────────────────────────────────┐
│ 1. DDD — Clean layer separation keeps code maintainable     │
│ 2. Eino — Workflow engine manages agentic reasoning loops   │
│ 3. OpenAI Format — Portable tool calling across LLM providers│
│ 4. Dual Auth — JWT sessions + API keys with per-key ACLs    │
│ 5. SSE Streaming — Real-time responses without WebSockets   │
└─────────────────────────────────────────────────────────────┘

These are not optional features you can swap out. They compose together. The auth system controls which tools a token can use. The tool system feeds into the Eino workflow. The workflow streams its output via SSE. The DDD layers keep each piece independently testable.

Walking the Directory Tree

Here is the project structure with the "why" for each layer:

go-agent-api/
├── cmd/api/main.go              ← DI container: wires everything together
│
├── internal/
│   ├── domain/                  ← Pure Go. Zero external dependencies.
│   │   ├── entity/              ← User, Token, Conversation, Message
│   │   ├── repository/          ← Interfaces only (no implementations here)
│   │   ├── service/             ← Domain services (password hashing logic)
│   │   └── event/               ← Domain events (MessageCreated, etc.)
│   │
│   ├── application/             ← Orchestration. Depends only on domain.
│   │   ├── usecase/             ← auth/, chat/, tool/, user/
│   │   ├── dto/                 ← Request/response shapes
│   │   └── port/                ← Interfaces for external services
│   │
│   └── infrastructure/          ← The outside world. Depends on application.
│       ├── eino/                ← Workflow graph and state
│       ├── http/                ← Handlers, middleware, router
│       ├── llm/                 ← LiteLLM client
│       ├── persistence/         ← PostgreSQL repos + Redis
│       └── config/              ← Environment variable loading
│
├── pkg/toolspec/                ← Public: OpenAI-compatible tool types
├── deployments/                 ← Dockerfile + docker-compose.yml
└── docs/                        ← Swagger spec + UI

The most important thing to understand is the direction of dependencies:

Domain ← Application ← Infrastructure

The domain layer does not know about PostgreSQL, Redis, HTTP, or LiteLLM. The application layer does not know about chi, pgx, or docker-compose. The infrastructure layer implements the interfaces that the inner layers define.

This is not theoretical purity — it has practical consequences. When you want to write a unit test for the SendMessage use case, you mock the repository interface and the LLM provider interface. No database, no HTTP server, no running LiteLLM instance needed. The test runs in milliseconds.

The Dependency Rule in 30 Seconds

The dependency rule states that source code dependencies must point inward. Nothing in the inner circles knows anything about the outer circles.

// DOMAIN: knows nothing about PostgreSQL
// internal/domain/repository/user_repository.go
type UserRepository interface {
    FindByEmail(ctx context.Context, email string) (*entity.User, error)
}

// INFRASTRUCTURE: implements the domain interface
// internal/infrastructure/persistence/postgres/user_repository.go
type userRepository struct {
    pool *pgxpool.Pool
}
func (r *userRepository) FindByEmail(ctx context.Context, email string) (*entity.User, error) {
    // pgx query here
}

The domain defines UserRepository as an interface. The infrastructure provides userRepository as a concrete PostgreSQL implementation. The domain has zero knowledge that PostgreSQL exists.

This pattern is applied everywhere. port.LLMProvider in the application layer is an interface. litellm.Client in the infrastructure layer implements it. Swap LiteLLM for a direct OpenAI client by writing a new struct that satisfies the interface.

main.go: The Entire DI Container, Explicit and Readable

The entry point in cmd/api/main.go is 200 lines of explicit dependency injection. No magic, no framework, no annotations. Here is the core of it:

// Database
db, err := postgres.NewConnection(ctx, cfg.Database)

// Redis
redisClient, err := redis.NewConnection(ctx, cfg.Redis)

// Repositories (infrastructure layer)
userRepo := postgres.NewUserRepository(db)
tokenRepo := postgres.NewTokenRepository(db)
convRepo := postgres.NewConversationRepository(db)
msgRepo := postgres.NewMessageRepository(db)

// JWT Manager
jwtMgr := jwt.NewJWTManager(cfg.JWT.Secret, cfg.JWT.AccessTokenTTL, cfg.JWT.RefreshTokenTTL)

// LLM Provider (infrastructure layer)
llmProvider := litellm.NewClient(cfg.LLM.BaseURL, cfg.LLM.APIKey)

// Tool Registry (application layer)
toolRegistry := tool.NewToolRegistry()
toolRegistry.RegisterAll(
    builtin.NewCalculatorTool(),
    builtin.NewWebSearchTool(cfg.Tools.WebSearchAPIKey),
)

// Use Cases (application layer)
validateTokenUC := auth.NewValidateTokenUseCase(tokenRepo, userRepo, jwtMgr)
loginUC := auth.NewLoginUseCase(userRepo, tokenRepo, jwtMgr)
sendMessageUC := chat.NewSendMessageUseCase(convRepo, msgRepo, llmProvider, cfg.LLM.DefaultModel)
approveActionUC := chat.NewApproveActionUseCase(convRepo)

// Middleware + Handlers (infrastructure layer)
authMiddleware := middleware.NewAuthMiddleware(validateTokenUC)
chatHandler := handler.NewChatHandler(sendMessageUC, getConversationUC, listConversationsUC, approveActionUC)

Every dependency is explicit. You can read from top to bottom and understand exactly what depends on what. There are no hidden singletons, no global variables, no container magic.

This explicitness makes debugging trivial. If chatHandler misbehaves, you can see exactly what sendMessageUC depends on, what llmProvider it uses, and which convRepo implementation backs it. The entire dependency graph is right there in one file.

docker-compose up: 5 Services, One Command

The deployments/docker-compose.yml gives you a complete development and production stack:

services:
  api:         # Your Go binary (128MB memory limit)
  postgres:    # PostgreSQL 16 (primary database)
  redis:       # Redis 7 (rate limiting, caching)
  litellm:     # LLM proxy (GPT-4o, Claude, Gemini — config-driven)
  migrate:     # golang-migrate (runs DB migrations on startup)

The services are wired with proper health checks:

depends_on:
  postgres:
    condition: service_healthy
  redis:
    condition: service_healthy

The API container will not start until both Postgres and Redis are accepting connections. The migrate container runs migrations before the API processes any traffic. This is a production-grade startup sequence, not just "hope the database is ready."

What You Can Build With 10 New Lines of Code

Once the infrastructure is running, extending it is intentionally easy. Here is what adding a new capability looks like:

Add a new tool (file + 2 lines in main.go):

// New file: internal/application/usecase/tool/builtin/weather.go
type WeatherTool struct{ tool.BaseTool }
func (t *WeatherTool) Name() string { return "get_weather" }
// ... implement Execute()

// In main.go:
toolRegistry.RegisterAll(builtin.NewWeatherTool(cfg.Tools.WeatherAPIKey))

Add a new API key with restricted tools:

# Create a token that can ONLY use calculator
POST /v1/auth/tokens
{"name": "limited-key", "allowed_tools": ["calculator"]}

Switch from GPT-4o to Claude:

# deployments/litellm_config.yaml — change one line
- model_name: gpt-4o-mini
  litellm_params:
    model: anthropic/claude-3-haiku-20240307
    api_key: os.environ/ANTHROPIC_API_KEY

No code changes. LiteLLM handles the API format translation.

The Full API Surface

The server exposes 18 endpoints across 4 domains:

Health:
  GET  /health                          # Liveness check
  GET  /ready                           # Readiness check

Auth (public):
  POST /v1/auth/register
  POST /v1/auth/login
  POST /v1/auth/refresh

Chat (protected):
  POST /v1/chat/completions             # OpenAI-compatible, streaming supported
  POST /v1/conversations
  GET  /v1/conversations
  GET  /v1/conversations/{id}
  POST /v1/conversations/{id}/messages
  POST /v1/conversations/{id}/approve   # Human-in-the-loop

Tools (protected):
  GET  /v1/tools
  POST /v1/tools/execute
  POST /v1/tools/batch

There is also a Swagger UI at http://127.0.0.1:8080/swagger/index.html with interactive documentation for every endpoint.

The Dependency Stack

The full go.mod has 16 direct dependencies:

Package	Purpose
`github.com/cloudwego/eino`	Workflow engine (ByteDance)
`github.com/go-chi/chi/v5`	HTTP router
`github.com/golang-jwt/jwt/v5`	JWT tokens
`github.com/jackc/pgx/v5`	PostgreSQL driver
`github.com/redis/go-redis/v9`	Redis client
`github.com/google/uuid`	UUID generation
`golang.org/x/crypto`	bcrypt password hashing
`github.com/stretchr/testify`	Test assertions
`github.com/swaggo/swag`	Swagger generation

No ORMs. No dependency injection frameworks. No "enterprise" middleware. Every dependency does one specific thing.

Series Roadmap

Here is what each article in this series covers:

#	Title	What You Learn
1	Architecture Overview (this article)	Why these choices, how the pieces fit
2	DDD Domain Layer	Entities with business logic, repository interfaces
3	The Tool System	Add a tool in 15 lines, OpenAI format, per-token ACLs
4	Eino Workflow Engine	6-node DAG, ReAct loop, conditional routing
5	Dual Auth	JWT + API keys, token-as-permission-carrier
6	SSE Streaming	Real-time responses, channel pipeline, context cancellation
7	Human-in-the-Loop	Pause/approve/resume agentic workflows
8	Deployment	18MB Docker image, graceful shutdown, LiteLLM

Each article stands alone but is richer in context when read in sequence.

What We Just Learned

Go's memory efficiency, cold start speed, and type safety make it well-suited for AI agent APIs
The project uses DDD with strict inward dependency: Domain ← Application ← Infrastructure
cmd/api/main.go is an explicit DI container — every dependency is visible and readable
One docker-compose up brings up 5 services with health checks and automatic migrations
Adding new tools, switching LLM providers, and creating restricted API keys all take single-digit lines of code

Try This Now

git clone https://github.com/wyuneed/go-agent-api
cd go-agent-api
cp .env.example .env
make docker-up
make migrate-up
make run
# Open http://127.0.0.1:8080/swagger/index.html

DEV Community: Izu Tolandona

#8 - Shipping a Go AI Agent to Production: 18MB Docker Image, 5 Services, Zero Downtime

The Multi-Stage Dockerfile: Builder → Scratch

Why FROM scratch?

Why CGO_ENABLED=0?

The Migration Trick

The 5-Service Docker Compose Stack

Startup Order with Health Checks

Memory Limits

LiteLLM: Swap LLM Providers With a Config File

Graceful Shutdown: The Complete Pattern

Environment Configuration

Production Checklist

Running the Full Stack

What We Just Learned

Series Complete

#7 - Human-in-the-Loop AI: How to Pause, Approve, and Resume an Agentic Workflow

The Problem: Irreversible Tool Actions

Step 1: The actNode Detects RequiresApproval

Step 2: The humanApprovalNode Pauses the Workflow

Step 3: Conversation.RequestApproval() — Storing the Pause State

Step 4: The ApproveActionUseCase — Validating and Transitioning

Step 5: Resuming the Workflow

The Approval Handler

The Complete Flow, End to End

Why *bool Instead of bool

What We Just Learned

#6 - Server-Sent Events in Go: Building Real-Time AI Streaming Without WebSockets

Why SSE Over WebSockets for LLM Streaming

The LLMProvider Interface: Streaming as a Contract

The LiteLLM Client: Parsing SSE

The SSE Response Helpers

The Chat Handler: Putting It Together

Context Cancellation: The Full Pipeline

Testing Streaming Handlers

What We Just Learned

#5 - JWT Meets API Keys: Building Dual-Auth in Go Without the Usual Mistakes

The Two Authentication Flows

The JWT Manager: Generating and Validating Tokens

Hashing API Keys: Why You Never Store the Raw Value

The ValidateTokenUseCase: One Branching Point

The Fire-and-Forget UpdateLastUsed Pattern

The Auth Middleware: Context Injection

How Per-Token ACLs Flow Through the System

The Login Use Case: Where Tokens Are Created

What We Just Learned

#4 - Eino: ByteDance's LangGraph for Go — Building a 6-Node Agentic Workflow

What Eino Is (and Is Not)

The AgentState: One Struct That Carries Everything

The Graph Structure

Building the Graph: AddLambdaNode + compose.InvokableLambda

Edges and Branches

Node Implementations

Router Node: Keyword-Based Agent Selection

Think Node: The LLM Call

Act Node: Tool Execution

Observe Node: Loop Control

The Observe Branch: Loop or Stop?

Compiling and Running the Graph

A Complete ReAct Loop Traced

Why This Architecture Works

What We Just Learned

#3 - Add a New AI Tool in 15 Lines of Go: The OpenAI-Compatible Tool System Explained

The Tool Interface: 6 Methods, 2 Are Free

The CalculatorTool: A 15-Line Deep Dive

The Secret: Go's AST Parser as a Safe Evaluator

The toolspec Package: Why OpenAI Format Matters

The ToolRegistry: Thread-Safe Tool Management

Per-Token Tool Permissions

Adding Your Own Tool: A Practical Example

The Execution Pipeline

What Makes This Design Work

What We Just Learned

Try This Now

#2 - Domain-Driven Design for Go Developers: Build Entities That Actually Enforce Business Rules

What the Domain Layer Is (and What It Cannot Import)

The User Entity: Roles and Business Methods

The Token Entity: A Permission Carrier

The Conversation Entity: A Go State Machine

The Message Entity: Protocol Translation at Domain Level

Why `FROM scratch`?

Why `CGO_ENABLED=0`?