Kantemir Satibalov

Posted on Jul 26

Building an MCP Gateway in Go: Bridging AI Agents and JSON-RPC Tools

#go #ai #gateway #infrastructure

The Model Context Protocol (MCP) from Anthropic is becoming the standard for connecting LLM agents to external tools. But there's a gap: MCP servers speak JSON-RPC over stdio, while agents and orchestrators usually speak HTTP.

I built mcp-gateway to bridge that gap once — a production-minded Go service that registers MCP servers, proxies tool calls over HTTP, and exposes Prometheus metrics. Here's how it works and why I made the architectural choices I did.

The Problem

MCP is great for tool interoperability. You install an MCP server (filesystem, database, web fetch), and any MCP-compatible agent can use it. But:

Every agent needs an MCP client embedded
JSON-RPC over stdio is hard to debug and monitor
No centralized health checks, retries, or metrics
Running 10 MCP servers means 10 subprocesses to manage

I wanted a single HTTP entry point: agents send JSON, gateway routes to the right MCP server, and I get observability for free.

Architecture

Key Design Decisions

1. In-Memory Registry (for MVP)

I chose an in-memory registry over PostgreSQL for the MVP. Why?

MCP servers are subprocesses — their state lives in memory anyway
Adding a database doesn't solve failover (if the gateway crashes, subprocesses die)
The real bottleneck is stdio IPC, not registry lookups
PostgreSQL is in the roadmap for multi-tenant deployments

2. stdio Transport

MCP servers are typically distributed as CLI tools (npx, uvx). stdio is the most compatible transport. The gateway:

Spawns the process on register
Sends JSON-RPC via stdin
Reads responses from stdout
Kills the process on shutdown (with 3s grace period)

3. Error Semantics

MCP distinguishes between transport errors (server down, JSON-RPC broken) and tool errors (file not found, invalid query):

Transport/RPC errors → HTTP 502 + log
Tool execution errors (isError: true) → HTTP 200 + metric mcp_tool_calls_total{status="error"}

This mirrors how HTTP proxies work: the gateway is healthy even if the upstream tool fails.

The Code

Declarative Configuration

# config/servers.yaml
servers:
  - name: filesystem
    command: npx
    args: ["-y", "@modelcontextprotocol/server-filesystem", "/data"]
    enabled: true

  - name: fetch
    command: npx
    args: ["-y", "@modelcontextprotocol/server-fetch"]
    enabled: true

HTTP Proxy Handler

func (h *Handler) CallTool(w http.ResponseWriter, r *http.Request) {
    server := chi.URLParam(r, "name")
    tool := chi.URLParam(r, "tool")

    client, err := h.registry.Get(server)
    if err != nil {
        http.Error(w, "server not found", http.StatusNotFound)
        return
    }

    var req CallToolRequest
    if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
        http.Error(w, "invalid JSON", http.StatusBadRequest)
        return
    }

    result, err := client.CallTool(r.Context(), tool, req.Args)
    if err != nil {
        // RPC/transport error → 502
        http.Error(w, err.Error(), http.StatusBadGateway)
        return
    }

    w.Header().Set("Content-Type", "application/json")
    json.NewEncoder(w).Encode(result)
}

Graceful Shutdown

ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
defer stop()

// Start HTTP server
srv := &http.Server{Addr: ":" + port, Handler: router}
go func() { srv.ListenAndServe() }()

// Wait for signal
<-ctx.Done()

// 1. Stop accepting new connections
shutdownCtx, cancel := context.WithTimeout(context.Background(), 15*time.Second)
defer cancel()
srv.Shutdown(shutdownCtx)

// 2. Stop health loop
reg.StopHealthChecks()

// 3. Close all MCP clients (kills subprocesses)
reg.Close()

Observability

# Tool call volume
curl -s http://localhost:8080/metrics | grep mcp_tool_calls_total

# Server health
curl -s http://localhost:8080/metrics | grep mcp_server_up

# Request latency
curl -s http://localhost:8080/metrics | grep mcp_tool_call_duration_seconds

Every request gets a request ID injected into slog and propagated through context. Logs are structured JSON — no parsing regex in production.

Running It

git clone https://github.com/kantik001/mcp-gateway.git
cd mcp-gateway
docker compose up --build -d

# Test it
curl -s http://localhost:8080/v1/servers
curl -s -X POST http://localhost:8080/v1/servers/filesystem/tools/read_file   -H "Content-Type: application/json"   -d '{"args":{"path":"/data/README.md"}}'

Testing

The MCP client layer has 70.6% test coverage, including mock subprocess tests. The CI gate requires:

All tests passing
golangci-lint clean
Coverage ≥ 60% on internal/mcp

make test
make coverage
make lint

What's Next

Postgres-backed registry for multi-tenant deployments
Redis tool-result cache
OpenTelemetry traces
SSE streaming for long-running tools

DEV Community