Smart MCP Proxy — Hot-Swap MCP Aggregation + AI Concierge

I got tired of restarting my agent every time I added an MCP server. Edit a config, restart the gateway, hope the desktop app picks it up... away from your desk, that's a dealbreaker. So I built a proxy that hot-swaps MCP servers live and shares their subprocess pools across every agent you run.

It's one endpoint for all your MCP servers. Add or remove them at runtime, no restart needed, no API keys embedded. The whole thing is a single Python process — no database, no web UI, no Docker. Clone, run, and it works offline.

Why bother

Your agent might be smart, but your MCP servers are dumb tools. And dumb tools burn context, waste memory, and demand restarts every time you touch a config. I kept running into the same three things:

1. Restarts on every config change. Edit a YAML file and you're restarting the gateway, maybe the app too. Not workable if you're managing servers remotely, even off a Telegram message.
2. Resource waste. Three agents each hooking straight into seven servers means 21 subprocesses eating memory. Heavy servers like Playwright or windows-mcp make that unsustainable fast.
3. Context drain. Agents juggling raw tool calls, parameter lists, and multi-step chains burn tokens and reasoning cycles just to get a simple result out of a complex server.

The proxy fixes all three. There are two builds.

Build 1 — the hot-swap proxy

One subprocess pool per server, shared across every connected agent. So three agents plus seven servers is seven pools, not 21. If a pool gets busy, it spawns an extra subprocess on demand and kills it after it goes idle. Crash recovery tries three times with backoff.

The good part is the hot-swap. A file watcher watches proxy-config.yaml. On a change, it diffs the old server list against the new one, closes pools for servers you removed, and spins up pools for ones you added. No restart, either side.

What you set up looks like this:

proxy:
  host: "127.0.0.1"
  port: 9876

servers:
  my-server:
    type: stdio
    command: "~/.mcp_servers/xxx/cmd"
    args: ["--flag"]
    timeout: 120

Each downstream tool keeps its real name and full parameter schema — no generic arguments: object garbage. Images and binary content come through as JSON.

Build 2 — the AI concierge (optional)

This is the part I actually run daily. Instead of the agent fumbling with raw tools, it gets a second way in: just talk.

mcp_proxy_ask("compare grok, claude, and gemini on this topic")

The smart layer figures out which server to hit, loads the right skill template if one fits, pulls the parameters out of your plain English, runs the tool, and chains follow-ups if the skill asks for them. Then it hands back only the final answer. All the intermediate noise never touches the agent's context.

Routing runs off MCP Sampling, so it borrows the connected agent's own LLM. No API key embedded anywhere. If the client doesn't support Sampling, it falls back to keyword matching.

The skill templates are just markdown files in skills/<server-name>/. Drop an .md in, it works. No code changes.

Also worth noticing: every response tells you which server was used and how confident the match was.

What's it good for

The README has a longer take on this, but the short version: point one proxy per machine and you get a cascade where an org-level agent can see every box while each team's agent only sees its own. Screenshots, commands, files — local hands, remote brain, talking over MCP. I'll leave that vision to the docs, but honestly that direction is the fun part of this thing.

Run it

pip install mcp fastmcp pydantic pyyaml watchdog click uvicorn httpx
python -m src --enable-smart

Then point any Hermes profile at it:

mcp_servers:
  smart-mcp-proxy:
    url: "http://localhost:9876/mcp"

There are bin/smart-mcp-proxy.cmd and .sh wrappers for start/stop/restart/status if you'd rather not call it directly.

Caveats

It's v1.0.0 and MIT licensed. Authentication and HTTPS are planned but not shipped yet, so don't put it on a public port. The multi-step chain is capped at four hops and strips image data from follow-up context so you don't blow up your context window. For single-user local setups it's been solid for me, but treat it as new software until you've watched it a while.

That's the whole pitch. The repo is at github.com/MilkyWay008/Smart-MCP-Proxy if you want to poke at it or tell me what's awkward.