How We Built an Agentic-Task Detector for LLM Routing

Disclosure: I maintain Lynkr, the open-source LLM router whose agentic detector this post dissects. Every snippet below is real, shipping code — read the whole file here, it's 350 lines.

"Fix the auth bug in session.js."

Eight words. Every token-count heuristic on earth routes this to the small, cheap model — it's short. And every one of them is wrong, because those eight words are about to unleash a grep → read → edit → test loop with exact-string file edits, the precise workload where small models fumble tool calls and kill sessions.

The inverse request — three paragraphs asking for a detailed comparison of locking strategies — looks expensive and routes safely to a free local model, because it's pure text generation. Size and stakes are nearly uncorrelated in coding-agent traffic. So the router's real job is detecting agentic intent, and this post is a tour of how Lynkr's detector does it: the signals, the weights, the classification ladder — and the embarrassing false positive that almost made the whole thing useless.

Not "agentic: yes/no" — a ladder

The first design decision: agentic-ness isn't boolean. The detector classifies requests into four types, each with a minimum tier floor and a score boost fed into the complexity scorer:

const AGENT_TYPES = {
  SINGLE_SHOT: { minTier: 'SIMPLE',    scoreBoost: 0 },   // request-response, no tools
  TOOL_CHAIN:  { minTier: 'MEDIUM',    scoreBoost: 15 },  // read -> edit -> test
  ITERATIVE:   { minTier: 'COMPLEX',   scoreBoost: 25 },  // retry loops, debugging cycles
  AUTONOMOUS:  { minTier: 'REASONING', scoreBoost: 35 },  // "figure it out", full autonomy
};

The minTier is a floor, not a suggestion: even if every other dimension scores low, an ITERATIVE request cannot route below the COMPLEX tier. Mid-debugging-loop is the worst possible moment to hand the session to a 7B model.

The six signals

Each request accumulates a score from six independent signals. The interesting part is why each one exists:

1. Tool count (up to +25). Many tools attached usually means the client is prepared for multi-step work. Usually. This signal is also the source of the great false positive — hold that thought.

2. Agentic tools specifically (up to +25). Not all tools are equal evidence. Bash, Write, Edit, Task, git and test runners form an explicit set — these mutate state, and their presence signals mutation work. A request that can only Read/Grep/WebSearch sits in a separate read-only set and earns nothing here. Two requests with five tools each can be night and day.

3. Prior tool results (up to +30 — the heaviest signal). If the conversation already contains tool_result blocks, you're not predicting an agentic loop — you're inside one. More than five results means a deep loop with accumulated exact state (file contents, error strings); downgrading the model now throws away the context discipline keeping that loop convergent.

4. Language patterns (up to +25 each). Regexes over the last user message:

• tool-chain: "then use", "after that", "step 2"
• iterative: "keep trying", "until", "retry", "debug"
• autonomous: "figure out", "make it work", "on your own", "whatever it takes"
• multi-file: "across the codebase", "refactor entire", "everywhere"

Plus a combination rule: "implement" alone is +10-ish planning noise, but "implement" and "test/verify/make sure" in the same request is +15 — build-and-verify phrasing is a reliable tell of real work.

5. Conversation depth (up to +20). Fifteen-plus messages means established context and momentum.

6. Prompt length (+10). The weakest signal, deliberately — see the opening paragraph.

Score ≥ 25 → the request is agentic. The classification ladder then applies both thresholds and signal combinations — AUTONOMOUS needs score ≥ 60, or an explicit autonomous phrase with score ≥ 40. A phrase alone doesn't do it; a high score without autonomous language doesn't either, unless it's overwhelming.

The false positive that almost sank it

Early versions had a humiliating problem: every single Claude Code request scored agentic. Including "hello."

Why? Claude Code attaches its full tool loadout — Read, Write, Edit, Bash, Grep, Glob, Task, and friends — to every request, even a greeting. Signals 1 and 2 saw 11+ tools, four of them mutating, on everything. Every request cleared the threshold, every request routed to expensive tiers, and the router's entire value proposition — savings — evaporated. The detector was technically working and practically useless.

The fix ships as client profiles: known harnesses (Claude Code, Cursor, Codex CLI) have documented baseline loadouts, and the tool-count signals score only the tools beyond that baseline:

// Signals 1 & 2 score only tools BEYOND the harness's baseline loadout —
// Claude Code's 11 always-attached tools shouldn't count as "agentic
// intent" on their own.
toolsForScoring = clientProfiles.effectiveTools(payload, profile);

Crucially, signals 3–6 still use the full payload — prior tool results and conversational language are genuine evidence regardless of which harness sent them. Only the tool-presence signals get the subtraction, because only they are polluted by the harness's constant.

And for traffic from harnesses we've never seen? A guard: if every attached tool looks like a standard baseline and there are 10+ of them, the tool-count signals zero out rather than fire:

} else if (clientProfiles.allToolsAreBaseline(payload) && rawTools.length >= 10) {
  // Unknown harness that looks like Claude Code / Cursor / Codex —
  // zero out the tool-count signals to avoid the same trap.
  toolsForScoring = [];
  scoringNote = 'unknown_harness_guard';
}

Better to under-detect and lean on the five uncorrupted signals than to re-create the everything-is-agentic bug for unknown clients.

One subtle consequence, preserved as a comment in the source: with the baseline subtracted, tool counts rarely reach the AUTONOMOUS threshold on their own — so the autonomous phrase pattern becomes the primary path to the top classification. The signal design acknowledges its own post-fix physics.

What it still gets wrong

Honesty section. Known limitations, from the code itself:

• It reads only the last user message. "Do what I described above" carries the intent of an earlier message the regexes never see. Conversation-depth and tool-result signals partially compensate — but pattern detection is myopic by design (scanning full history was too noisy).
• Regexes can't tell mention from intent. "Why did the retry loop break?" trips the iterative pattern despite being a read-only question. In practice this fails safe — over-routing a question up-tier costs cents, under-routing an edit session down-tier costs the session — but it's still a false positive.
• English only. The patterns are English regexes; agentic intent in other languages leans entirely on the structural signals.

Every detection returns its full evidence — score, signal list with weights, classification, and a scoringNote explaining any baseline subtraction — so when the router misjudges, the telemetry shows exactly which signal lied. Debuggability was a design requirement: a routing layer you can't interrogate is a routing layer you'll eventually rip out.

Takeaways if you're building anything similar

1. Subtract the constant before reading the signal. Whatever your equivalent of "the harness always attaches 11 tools" is — find it and remove it, or every request looks the same.
2. Separate "prepared for tools" from "already using tools." Attached tools are weak evidence; tool_result blocks in the conversation are near-proof.
3. Fail toward the expensive model. Asymmetric costs mean your threshold should be calibrated so mistakes over-spend pennies rather than break sessions.
4. Make the detector explain itself. A score without a signal list is a black box you'll never be able to tune.

The whole detector is 350 lines of dependency-free JavaScript: src/routing/agentic-detector.js. Steal it, or tell me which of your prompts it would misjudge — the failure cases are the roadmap.