DEV Community

Cover image for Why I Stopped Using JSON Tool-Calling for My Coding Agent
Jahanzeb Ahmed
Jahanzeb Ahmed

Posted on

Why I Stopped Using JSON Tool-Calling for My Coding Agent

#agents #ai #llm #python

A few months ago I needed an agentic coding assistant inside a internal CI repair tool I was building. The obvious move was to embed something that already exists — Claude Code, or Google's Antigravity SDK. Both work well. Neither one gets you what you actually need once you're shipping a product on top of it.

Claude Code isn't embeddable as infrastructure. You can shell out to it, but you don't own the runtime, the protocol, or the failure modes. Antigravity gets you closer, it's an SDK you can embed, but you're still building on someone else's closed layer. If that layer changes pricing, rate limits, or behavior, your product changes with it and you have no say in it.

So I built CodePilot: an open source, embeddable Python runtime for coding agents. pip install codepilot-ai and it's yours. No vendor layer between you and the model.

Ditching JSON tool calls (and markdown too)

Early versions of CodePilot used JSON tool calling, standard stuff, the model emits a JSON blob, you parse it, you dispatch a function. It broke constantly on large code payloads. Every quote and newline in a code block has to be escaped, and models reliably mangle that escaping once the payload gets long enough.

My first fix was a markdown based protocol, fenced code blocks instead of JSON strings. Better, but still not right. It's not how you actually want a model editing files, and it turned out to be a stale idea I outgrew fast.

What CodePilot uses now is a plain text search and replace protocol. The model writes something like this:

src/main.py
<<<<<<< SEARCH
=======
def main():
    print("hello")
>>>>>>> REPLACE
Enter fullscreen mode Exit fullscreen mode

No JSON escaping, no markdown parsing ambiguity. Just a file path and a diff-shaped block the runtime applies directly.

Where it gets interesting: codepilot.py

Every CodePilot runtime has one special file: codepilot.py, an ephemeral action file that lives at ~/.codepilot/runtime/codepilot.py. This is where the model writes and executes its own tool calls.

Instead of the model being limited to a fixed set of tools you predefined, it can write arbitrary Python into codepilot.py using the same search/replace protocol:

codepilot.py
<<<<<<< SEARCH
=======
execute("main", "python3 main.py", timeout=10)
>>>>>>> REPLACE
Enter fullscreen mode Exit fullscreen mode

The model isn't choosing from a menu of tools. It's writing the logic it needs, in the moment, and the runtime runs it. That's a meaningfully different capability than a fixed tool schema gives you, and it only works because the protocol is cheap enough (no JSON, no escaping overhead) that the model can afford to write real code instead of a constrained function call.

The rest of the runtime

  • MCP client support, built from scratch without the SDK, with semantic tool retrieval over voyage-code-3 embeddings so the agent isn't drowning in tool descriptions once you connect more than a couple of MCP servers
  • A headless VT100 terminal emulator, so the agent drives a real interactive shell instead of one-shot subprocess calls
  • Explicit archive and reveal tools for context management, instead of silently truncating history when the window fills up

Try it

CodePilot is MIT licensed, multi-provider (works with DeepSeek, Qwen, GPT and Claude), and installs with pip install codepilot-ai.

If you've built something similar, I'm curious how you handled letting the model write and run its own logic safely, that's the part I'm still hardening.

Repo: https://github.com/Jahanzeb-git/codepilot

Top comments (0)