DEV Community

Jimmy
Jimmy

Posted on

I built a manifest standard for agent-callable CLI tools – 93.6% routing accuracy, zero fine-tuning

I've been building agentic systems and kept running into the same problem:
agents either stuff entire tool docs into every prompt, or hallucinate
the wrong tool entirely.

CLI tools are 10–32× cheaper than MCP and nearly 100% reliable — but
there's no standard way for agents to discover and invoke them.

So I built CLIbrary: an open standard where each CLI tool has a
manifest.json with intent_triggers — natural language phrases
describing when to use it.

A routing layer does nearest-neighbor search over these embeddings
and returns a ready-to-execute tool call.

Results

Validated on 1,380 test cases using multilingual-e5-base, zero fine-tuning:

  • Overall accuracy: 93.6%
  • All 8 categories above 90%

Current state

69 manifests across 8 categories (ai-ml, devops, data, web, security,
media, productivity, finance). Schema and metadata only right now —
no CLI implementations yet. This is the standard layer first.

Manifest example

{
  "name": "sql-runner",
  "category": "data",
  "intent_triggers": [
    "query a database",
    "run a SQL statement",
    "get data from postgres"
  ]
}
Enter fullscreen mode Exit fullscreen mode

Looking for feedback

  • Does the manifest schema make sense for your use case?
  • What CLIs or categories would you want to see first?
  • Is anyone else building tooling for agent tool discovery? Would love to connect.

GitHub: https://github.com/clibrary-hub/manifests

Top comments (0)