DEV Community

Cover image for Stop Paying Claude Prices for "Hi" — Route LLM Prompts to the Right Model in 50 Lines
Alexey Sazhin
Alexey Sazhin

Posted on

Stop Paying Claude Prices for "Hi" — Route LLM Prompts to the Right Model in 50 Lines

#ai #cloudflare #tutorial #webdev

Every AI app hits the same fork in the road: pick an expensive frontier model and burn budget on trivial prompts, or pick a cheap open model and ship mediocre answers on hard ones.

You shouldn't have to pick. Not every prompt deserves Claude Sonnet 4.6. "What's the capital of Portugal?" and "Refactor this 400-line React component to use Suspense" are not the same job — and they shouldn't cost the same.

In this tutorial we'll build a prompt classifier that routes each incoming request to the right model automatically:

  • Coding questions → Claude Sonnet 4.6 (best-in-class for code)
  • Everything else → a cheap, fast open model on Workers AI

The whole thing is ~50 lines of TypeScript running on Cloudflare Workers, and the routing logic lives in AI Gateway — no vendor lock-in, unified logs, one endpoint for your app.

The architecture

Three moving parts:

  1. A Worker intercepts each OpenAI-compatible chat request
  2. It uses Workers AI (Llama 4 Scout, free tier) to classify the prompt as coding or simple
  3. It attaches that tag as a cf-aig-metadata header, then forwards to AI Gateway, which reads the tag and routes to the correct upstream model

Your client only ever sees one endpoint. AI Gateway gives you unified logs, caching, and rate-limiting across both providers.

Prerequisites

  • A Cloudflare account (free tier is fine)
  • An Anthropic API key (sign up here)
  • Node.js 20+ and Wrangler installed

Step 1: Create an AI Gateway

  1. In the Cloudflare Dashboard → AI → AI Gateway → Create Gateway. Give it a name (e.g. smart-router).
  2. Make sure Authenticated Gateway option is enabled;
  3. AI Gateway created with Authenticated Gateway enabled;
  4. Copy the API token shown on the next screen — Cloudflare won't show it again. You will need this token to set the secret AI_GATEWAY_TOKEN on a later step.
  5. Then add your Anthropic API key:
  6. Open your gateway → Provider Keys tab
  7. Click Add → select Anthropic → paste your key

AI Gateway will now handle authentication to Anthropic on your behalf — you won't need to expose that key from your Worker.

Step 2: Configure the Dynamic Route

Still in your gateway, go to Dynamic Routes → Add Route → Start from scratch. Name it route1.

Build this logic in the visual editor:

  • If metadata.task == "coding" → route to Anthropic → claude-sonnet-4-6
  • Else → route to Workers AI → @cf/moonshotai/kimi-k2.7-code (or any other cheap Workers AI model)

Save the route. AI Gateway now knows how to fork traffic based on a metadata.task field.

Step 3: Build the classifier Worker

Scaffold a new Worker:

npm create cloudflare@latest -- prompt-router --type hello-world --ts
cd prompt-router
Enter fullscreen mode Exit fullscreen mode

Navigate into the new prompt-router/src folder and replace src/index.ts with:

export interface Env {
  AI: Ai;
  GATEWAY_ACCOUNT_ID: string;
  GATEWAY_NAME: string;
}

export default {
  async fetch(request: Request, env: Env): Promise<Response> {
    const gatewayUrl = `https://gateway.ai.cloudflare.com/v1/${env.GATEWAY_ACCOUNT_ID}/${env.GATEWAY_NAME}/compat/chat/completions`;

    let body: Record<string, any> = {};
    try {
      body = await request.json();
    } catch {
      return new Response("prompt-router is running");
    }

    if (!body.messages?.length) {
      return new Response("prompt-router is running");
    }

    // Extract the latest user message
    const lastMessage = body.messages.at(-1);
    const prompt = typeof lastMessage?.content === "string"
      ? lastMessage.content
      : lastMessage?.content?.find((c: any) => c.type === "text")?.text ?? "";

    // Classify with a small, cheap model
    const classification = await env.AI.run(
      "@cf/meta/llama-4-scout-17b-16e-instruct",
      {
        messages: [
          {
            role: "system",
            content:
              "Classify the user prompt into exactly one word: 'coding' or 'simple'. Reply with only that single word, nothing else.",
          },
          { role: "user", content: prompt },
        ],
      }
    );

    const task = classification.response?.trim().toLowerCase() === "coding"
      ? "coding"
      : "simple";

    // Forward to AI Gateway with the classification as metadata
    const headers = new Headers(request.headers);
    headers.set("cf-aig-metadata", JSON.stringify({ task }));

    return fetch(gatewayUrl, {
      method: "POST",
      headers,
      body: JSON.stringify(body),
    });
  },
};
Enter fullscreen mode Exit fullscreen mode

Update wrangler.jsonc.
1) Replace {GATEWAY_ACCOUNT_ID} by your Cloudflare Account ID:

  • In the Cloudflare dashboard, go to the Account list.
  • Select the menu button at the end of the account row.
  • Select Copy account ID. Image description

2) Update {GATEWAY_NAME} with your values (find them in AI > AI Gateway > Settings).

{
  "name": "prompt-router",
  "main": "src/index.ts",
  "compatibility_date": "2025-01-01",
  "compatibility_flags": ["nodejs_compat"],
  "ai": {
    "binding": "AI"
  },
  "vars": {
    "GATEWAY_ACCOUNT_ID": "your-account-id",
    "GATEWAY_NAME": "smart-router"
  }
}
Enter fullscreen mode Exit fullscreen mode

Store the AI Gateway API token (saved on the creation of AI Gateway) as a Worker secret:

wrangler secret put AI_GATEWAY_TOKEN
# paste the token when prompted
Enter fullscreen mode Exit fullscreen mode

Deploy the worker:

npm run deploy
Enter fullscreen mode Exit fullscreen mode

Step 4: Test it

Fire two prompts at your Worker with curl:

# Simple prompt — should route to Workers AI
curl -X POST https://prompt-router.<your-subdomain>.workers.dev \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer <your-ai-gateway-token>" \
  -d '{
    "model": "dynamic/route1",
    "messages": [{"role": "user", "content": "What is the capital of Portugal?"}]
  }'

# Coding prompt — should route to Claude
curl -X POST https://prompt-router.<your-subdomain>.workers.dev \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer <your-ai-gateway-token>" \
  -d '{
    "model": "dynamic/route1",
    "messages": [{"role": "user", "content": "Write a Python function that reverses a linked list in place."}]
  }'
Enter fullscreen mode Exit fullscreen mode
  • Go to Cloudflare Dashboard > Compute > Workers & Pages prompt-router.
  • Check the Observability / Logs tab. You should see it successfully extract the prompt and log: Task: coding.
  • Go to AI > AI Gateway > Gateways > Your Gateway.
  • Click the Logs tab and make sure that claude-sonnet-4-6 model (slower, expensive, higher quality) is used for coding tasks and Kimi-K2 (fast, cheap) for the rest. Cost delta is visible in the logs.

Step 5: Point your existing app at it
Anything that speaks the OpenAI Chat Completions API will work. For example, in Open WebUI or LibreChat:
Base URL: https://prompt-router..workers.dev
Model: dynamic/route1
API Key: your AI Gateway token

In Continue.dev or Cursor, use the same base URL and model name. From the client's perspective it's a single OpenAI-compatible model — the routing is invisible.

Does this actually save money?

Very rough back-of-envelope, based on current public prices:

Model Input $/1M tokens Output $/1M token
Claude Sonnet 4.6 $3.0 $15.00
Workers AI (Kimi-K2) $0.95 $4.00

If 70% of a typical chat app's prompts are "simple" (conversation, factual questions, summaries) and 30% are "coding" or complex reasoning, routing them separately cuts total LLM cost by roughly 60-70% versus sending everything to Claude — while keeping Claude on the workloads where it earns its price.

Your mileage will vary. Instrument the classifier's output and measure your own traffic mix before quoting numbers.

Where to go from here

A few natural extensions:

  • More categories. Add reasoning, creative, summarization, translation — each routed to a specialist model.
  • Confidence-based fallback. Ask the classifier for a confidence score, and only route to the cheap model if confidence is high; otherwise default to the strong one.
  • Per-user or per-tier routing. Free-tier users always hit the cheap model; paid users get the frontier one. Read a header or JWT claim and inject it into cf-aig-metadata alongside task.
  • Evals-based routing. Score responses from both models offline, then bias the router toward whichever one wins for a given category.

Wrap

  • The interesting bit isn't the classifier — it's the pattern. Once you have a Worker sitting between your app and AI Gateway, injecting metadata, you can express any routing policy you want in the Gateway's visual editor without touching code again.
  • Prompt-based routing is the simplest useful policy. Once you've shipped it, cost-aware, latency-aware, and quality-aware routing are all one Dynamic Route edit away.

Full code and setup notes: GitHub Repo

If you build something with this, drop a comment — especially if you find a smarter classifier prompt than mine.

Top comments (0)