How I built deferred tool discovery for my desktop AI assistant (no embeddings needed)

I'm building Ankita, an open-source desktop AI assistant (Electron + terminal CLI) that can run shell commands, edit files with approval diffs, search the live web, manage scheduled routines, and talk hands-free. It's powered by GitHub Copilot models, and one design constraint has shaped almost everything in its architecture:

The CLI has zero runtime npm dependencies.

That constraint is really about context, not packaging. When your assistant calls web_search or git_diff, the model needs the full parameter schema for every tool it might pick — and those schemas are context you pay for on every single request. Ship dozens of tools with everything loaded upfront and you burn tokens before the user has typed anything.

So I built a deferred tool-discovery system. Here's how it works, and the three small decisions that made it feel invisible.

The problem: schemas are context

Ankita's tool catalogue covers web search/fetch/scrape, Git, filesystem, process management, scheduling, page watches, GitHub notifications, project memory, MCP servers, image generation, voice… Each tool is one ESM module under tools/, exporting name, description, parameters, and run(). If I loaded all of them into every request, the model would see hundreds of lines of JSON schema before the conversation even starts.

Decision 1: one find_tools tool, not fifty

The assistant's default toolset is deliberately small. When it needs something outside that set, it calls a single tool:

export const name = "find_tools";
export const description =
  "Load extra tools that are not in your default set: searching the internet and scraping pages, " +
  "Git, port/process management, scheduled routines and page watches, project management and memory, GitHub notifications, and " +
  "directory creation. Call this first whenever a task needs one of those; the tools become callable " +
  "immediately afterwards.";

It takes a plain-language query — "search the web", "remind me daily", "where does this project stand" — and returns the matched tool schemas. Those tools then become callable in the same session. The rest stay unloaded.

Decision 2: categories with keyword summaries, no embeddings

The matching lives in tools/catalog.mjs. Tools are grouped into categories, and each category carries a short summary plus a keyword list:

{
  id: 'process',
  summary: 'find port owners and terminate an approved process or port listener',
  keywords: ['port', 'process', 'pid', 'address in use', 'eaddrinuse', 'kill', 'listener', 'taskkill'],
  tools: [portStatus, killProcess],
},

matchCategories() in tools/find-tools.mjs then matches the user's query against category ids, keywords, and even tool names — with word-boundary regexes, not naive substring checks (so "port" doesn't fire on "transport"). It's pure and synchronous, which makes it trivially testable with Node's built-in test runner.

No embeddings, no vector index, no extra dependency. This is a deliberate trade-off: embeddings would handle paraphrase better, but a curated keyword list is predictable, debuggable, and costs nothing at runtime. For a desktop app where I control both ends, predictability wins.

There are also pragmatic disambiguation rules baked in — e.g. a "github notifications" query loads the built-in GitHub inbox category and not the connectors category, so one request doesn't pull in an unrelated schema. Real usage is full of these collisions, and a comment in the code documenting why beats cleverness every time.

Decision 3: skills are just markdown, loaded on demand

Tools cover capabilities; skills cover procedures. A skill is a SKILL.md file with frontmatter (name, description, suggested-tools) and markdown instructions below. The skill tool loads one by name and returns the body, capped at 8000 characters:

export function run(args = {}, ctx = {}) {
  const requested = String(args.name ?? '').trim().toLowerCase();
  // ...
  return renderSkill(found); // "# Skill: commit-review\n...---\n" + body, capped
}

Suggested tools are hints only — the skill never forces a tool call. This keeps procedural knowledge (like the repo's own conventions in ankita-dev: read before editing, zero runtime deps, run tests before reporting) out of the system prompt and loaded only when the task actually matches the skill's description.

What I'd do differently next

The keyword lists are hand-maintained, and they drift: every time I add a tool I have to think about which synonyms users will type. I've considered generating candidate keywords at build time from tool descriptions and reviewing the diff — human curation with machine assistance, rather than either extreme. And the always-on categories (personal memory, skills) deserve a periodic review, because "always loaded" is a cost I should keep auditing.

Try it and tell me what breaks

Ankita is open source — akyourowngames/A.N.K.I.T.A — with portable Windows builds in every release. The code discussed here is in tools/catalog.mjs, tools/find-tools.mjs, and tools/skills/skill.mjs.

If you've built lazy tool-loading for an agent of your own — embeddings, keyword maps, something weirder — I'd genuinely like to hear what worked and what bit you. I'm 16, building this in public, and the feedback loop is the best part.