DEV Community: Felix

Kimi K3: What's Actually Verified vs What's Vendor Benchmark

Felix — Thu, 10 Sep 2026 08:07:48 +0000

Moonshot's launch materials for Kimi K3 include a demo where the model spent 48 hours autonomously designing a physical chip — architecture, optimization, and verification, using open-source EDA tools — and ended up with a working 4-square-millimeter design that hit timing convergence at 100MHz. It's a genuinely impressive proof-of-concept. It's also the kind of headline-grabbing demo that tells you almost nothing about whether K3 will actually be good at the task you're evaluating it for, which is a more useful question and a different one than "look what it can do in a controlled showcase."

Every model launch mixes three different kinds of claims together: structural facts that are objectively checkable, vendor-run benchmarks scored on the vendor's own harness, and independent third-party signal from someone with no stake in the outcome. K3's launch is a clean example of all three, and separating them is the actual useful exercise here — both for evaluating this specific model and as a habit worth applying to the next one.

The structural facts

These are the parts that don't depend on trusting anyone's benchmark methodology — verifiable properties of the released model itself.

Kimi K3 shipped July 16, 2026 with API access, followed by full open weights on July 27. It's a 2.8-trillion-parameter Mixture-of-Experts model, reportedly the largest publicly released model of its kind, with 896 experts and roughly 104 billion active parameters per token — the architecture only activates a fraction of the total weights on any given forward pass. Context window is 1,048,576 tokens, four times K2.7 Code's 262K. It reads text, images, and video natively, and runs as a reasoning model with high reasoning effort enabled by default rather than optional.

The architectural change worth understanding, not just naming, is Kimi Delta Attention (KDA) — a hybrid attention mechanism that interleaves linear-attention layers with a smaller number of full-attention layers (reported as roughly a 3:1 ratio across the model's 93 layers), paired with a technique Moonshot calls Attention Residuals. The idea is that linear attention layers handle local sequence structure cheaply, while the full-attention layers preserve the global information flow that pure linear attention tends to lose. This isn't a launch-day invention — Moonshot published the underlying KDA research separately in October 2025, tested at a much smaller 48-billion-parameter scale.

That last detail matters for the next section, because it's exactly where structural fact and performance claim start to blur.

The vendor-reported numbers

Moonshot's technical blog reports K3 at 81.2 on FrontierSWE and 88.3 on Terminal-Bench 2.1, and an Elo of 1686 on GDPval-AA v2 — positioned above Opus 4.8's reported 1593 but below Fable 5's reported 1747 on that specific benchmark. Worth knowing explicitly: every number in that comparison set comes from Moonshot's own published materials, and the accompanying technical report was revised after initial launch, with some scores shifting slightly between versions — not evidence of anything nefarious, just a reminder that "the benchmark number" from a fresh launch is sometimes still settling.

The original KDA research paper reported up to 75% KV-cache memory reduction and up to 6x decoding throughput at 1M-token context compared to full-attention baselines. That's a real, published result — at the 48B-parameter scale it was tested on. Whether those exact multipliers hold at K3's full 2.8T scale is a separate question that, as far as I can find, hasn't been independently reproduced yet. The mechanism is real and the small-scale result is real; the specific numbers carrying over unchanged to a 60x larger model is an assumption worth flagging rather than something to repeat as settled fact.

The one clearly independent signal

Amid all of that, there's a genuinely third-party data point worth weighing more heavily than the vendor's own numbers: K3 debuted at #1 on LMArena's frontend coding leaderboard with a score of 1,679. That's a crowdsourced, blind-comparison ranking run by a party with no stake in K3 specifically looking good, which makes it meaningfully different in kind from a vendor's own benchmark table — not infallible, but a genuinely independent signal in a launch otherwise dominated by self-reported numbers.

On the chip-design demo, specifically

Back to where this started. The 48-hour autonomous chip-design run is real, documented, and genuinely demonstrates something about long-horizon autonomous agent capability — that's a legitimately hard, multi-stage task to sustain coherent progress on for two straight days without human intervention. What it isn't is a benchmark, a reproducible eval, or evidence about how K3 performs on a typical coding or agentic task you'd actually throw at it. Treat vendor showcase demos like this one for what they are: an existence proof that the capability shows up somewhere, not a performance number you can compare against anything else.

Is it actually open-source?

"Open-weight" is doing real work in how this gets described. The weights are public as of July 27, released under a Kimi K3-specific license rather than a standard permissive one — worth reading the actual license terms if your use case has commercial redistribution implications, rather than assuming "open-weight" means unrestricted. Running it yourself is also a genuinely different proposition than downloading a smaller model: 2.8 trillion total parameters with expert sparsity spread across many devices means production self-hosting requires real distributed inference infrastructure and high-bandwidth interconnects between nodes, not a single beefy GPU. "Open-weight" here means "inspectable and self-hostable if you have the infrastructure," not "runnable on a laptop."

Testing it against your own task

Given how much of the launch narrative is vendor-sourced, the reasonable move — consistent with evaluating any new model, not just this one — is running your own comparison on your own representative task rather than trusting either the vendor's benchmark table or a single leaderboard placement as the full picture:

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["MOONSHOT_API_KEY"],
    base_url="https://api.moonshot.ai/v1",
)

def run_task(model, prompt):
    completion = client.chat.completions.create(
        model=model,
        messages=[{"role": "user", "content": prompt}],
        max_completion_tokens=800,
    )
    return completion.choices[0].message.content

test_prompt = "Refactor this function for readability and explain each change: ..."

for model in ["kimi-k3", "kimi-k2.7-code"]:
    print(f"\n--- {model} ---")
    print(run_task(model, test_prompt))

Running the same real task — not a benchmark-style prompt, your actual work — against both K3 and K2.7-code (which the OpenAI-compatible API makes trivial to do, same key, just a different model string) tells you more about which one fits your specific use case than any leaderboard number will, benchmark-topping demo included.

Where this leaves you

K3's structural specs are real and checkable: 2.8T parameters, 1M context, the KDA architecture, native multimodality. Its benchmark position relative to other frontier models is mostly vendor-reported, with LMArena's independent leaderboard placement as the one clearly third-party data point currently available. The chip-design demo is a legitimate but narrow proof-of-concept, not a general performance signal. None of that makes K3 uninteresting — the architecture is a real technical contribution and the independent leaderboard result is genuinely good — it just means the honest version of "how good is K3" is "better than most of the launch coverage lets you conclude on its own, worth testing directly for your specific use case."

If you're accessing it via API rather than self-hosting, it's available directly through Moonshot, and also through gateways that list it alongside other model families — RouteAI, for instance, includes Kimi K3 in its catalog alongside DeepSeek, Qwen, GLM, and MiniMax models, which is worth knowing if you're already routing other providers through one key and would rather not open a separate account just to run the comparison above.

TL;DR: Kimi K3's structural specs (2.8T parameters, 1M context, the Kimi Delta Attention architecture) are verifiable facts, but most of its benchmark story — including the headline chip-design demo — comes from Moonshot's own materials. LMArena's independent frontend-coding leaderboard placement is the one clearly third-party signal so far, and the honest move before relying on any of it is running your own task against the model directly.

Website: https://www.fastrouteai.com

MiniMax Agent (Mavis): What the Hosted Product Actually Does vs Building Your Own on the API

Felix — Wed, 09 Sep 2026 07:21:32 +0000

I opened MiniMax Agent expecting a chatbot with some extra buttons. What I found instead was a genuinely full-featured hosted agent product — desktop app, mobile companion, multi-agent collaboration, its own credit-based subscription — that's iterated fast enough this year that a guide written six months ago would already be describing an out-of-date version of it. It's also, and this matters if you're a developer rather than just an end user, not something you can drop into your own application. That distinction is the actual useful thing to understand before deciding whether it's the tool you want.

What MiniMax Agent actually is right now

MiniMax Agent launched publicly in mid-2025 as a general-purpose agent aimed at long, multi-step tasks — things like full-stack web development, presentation creation, and in-depth research, built around what MiniMax describes as a hierarchical collaborative agent framework with long-range memory. It's moved fast since then: an M2-series model integration and mobile app in October 2025, a desktop app and an "Expert Agent" mode for building customized specialist assistants in January 2026, and then a bigger overhaul in May 2026 that gave the product a new name — Mavis, styled as "MiniMax as a Jarvis" — alongside a genuinely significant feature: Agent Teams, which lets you run multiple agents in parallel, each assigned a different role, collaborating on a task too large for a single agent to finish alone. That same release merged the separate token and agent subscription plans into one, so API usage and Agent usage now draw from a shared credit pool under one subscription rather than two separate billing relationships.

Most recently, MiniMax's M3 model shipped in August 2026 alongside an updated pairing agent product called MiniMax Code, tuned specifically to use M3's long-context and coding-agent capabilities, and a remote-control feature landed in early September letting you monitor and interact with a running desktop Agent session from your phone while the actual project files, terminal, and runtime stay on the desktop machine.

If you're evaluating this as a hosted tool for actually getting long-running tasks done — research reports, draft documents, coding sessions you can check in on from your phone — this is a legitimately capable and fast-moving product, and the Agent Teams feature specifically fills a real gap: a single agent producing its own final output is, as MiniMax's own release notes put it, both the judge and the contestant on its own work, and having separate role-assigned agents collaborate is a reasonable answer to that.

The catch: it's a product, not an API

Here's where it stops being relevant to "build something with it" the way an API model is. MiniMax Agent is a hosted application with its own UI, its own credit-based subscription, and its own workflow — there's no way to call "MiniMax Agent" as a component inside your own app the way you'd call a model endpoint. If what you actually want is agent-style behavior embedded in something you're building — a support tool, an internal automation, a feature in your own product — the Agent product itself isn't the thing to integrate. The thing to integrate is the underlying model, called directly through MiniMax's API, with your own agent loop built around it.

That's a meaningfully different task than using the hosted product, and it's worth being clear-eyed about the scope difference: MiniMax's hierarchical multi-agent framework, long-range memory system, and Agent Teams orchestration represent real engineering investment that a weekend project isn't going to replicate. What you can reasonably build is a scoped-down version — a single agent with a tool-calling loop suited to your specific task — using the same underlying model.

A minimal agent loop on the raw API

MiniMax's API is OpenAI-compatible, which means the same request shape you'd use for any chat completion applies, with tool-calling handled the standard way:

// minimax-agent-loop.js — a minimal single-agent tool-calling loop,
// not a replacement for MiniMax's own multi-agent framework, just a
// scoped starting point for embedding agent-style behavior in your own app

const TOOLS = [
  {
    type: "function",
    function: {
      name: "search_internal_docs",
      description: "Search internal documentation for relevant context",
      parameters: {
        type: "object",
        properties: { query: { type: "string" } },
        required: ["query"],
      },
    },
  },
];

async function callModel(baseURL, apiKey, messages) {
  const response = await fetch(`${baseURL}/chat/completions`, {
    method: "POST",
    headers: {
      "Content-Type": "application/json",
      Authorization: `Bearer ${apiKey}`,
    },
    body: JSON.stringify({
      model: "minimax-m2.7",
      messages,
      tools: TOOLS,
    }),
  });
  return response.json();
}

async function runAgentLoop(baseURL, apiKey, task, { maxSteps = 5 } = {}) {
  const messages = [{ role: "user", content: task }];

  for (let step = 0; step < maxSteps; step++) {
    const result = await callModel(baseURL, apiKey, messages);
    const message = result.choices[0].message;
    messages.push(message);

    if (!message.tool_calls) {
      return message.content; // model finished without needing another tool call
    }

    for (const call of message.tool_calls) {
      const args = JSON.parse(call.function.arguments);
      const toolResult = await executeTool(call.function.name, args);
      messages.push({
        role: "tool",
        tool_call_id: call.id,
        content: JSON.stringify(toolResult),
      });
    }
  }

  throw new Error(`Agent loop exceeded ${maxSteps} steps without finishing`);
}

async function executeTool(name, args) {
  if (name === "search_internal_docs") {
    // plug in your actual retrieval logic here
    return { results: [`Stub result for query: ${args.query}`] };
  }
  throw new Error(`Unknown tool: ${name}`);
}

module.exports = { runAgentLoop };

This is intentionally the simplest version of the pattern — a linear loop, one model, a hard step cap so it can't run away. It's the right scope for a task like "look up context and answer a question," and the wrong scope for anything resembling what MiniMax's Agent Teams feature is built for — genuinely long, multi-role, parallel work. Knowing which of those two categories your actual task falls into is most of the decision between using the hosted product and building your own loop.

On the M3 benchmark numbers

Worth a specific note since MiniMax has published its own evaluation results for M3: in an internally run, fully autonomous benchmark covering math reasoning, tool calling, science knowledge, and code generation, MiniMax reported M3 scoring 0.37, against Opus 4.7 at 0.42 and GPT-5.5 at 0.39, describing M3 as ahead of other models in the comparison. That's a vendor's own self-reported number on a benchmark of their own design, run without independent verification in what I could find — worth treating the same way as any single-vendor benchmark claim: a data point suggesting the model is at least competitive, not a settled ranking, and worth checking against independent evals if the specific comparison matters for a real decision.

Where the API access piece fits

If you land on "I want the model, not the hosted Agent product," MiniMax's own API is the direct route, and it's also one of several ways to reach the same M2.5/M2.7/M3 model family — gateways like RouteAI list MiniMax's models alongside DeepSeek, Qwen, Kimi, and GLM, which is worth knowing if you're already routing calls to other providers through one key and would rather not open a separate MiniMax account and billing relationship just for this one model family.

The actual takeaway

MiniMax Agent (now Mavis) is a legitimately fast-moving, capable hosted agent product if what you want is a tool you interact with directly for long-running tasks — and the Agent Teams multi-role collaboration feature is a genuinely useful answer to the single-agent judge-and-contestant problem. If what you actually want is agent-style behavior inside your own application, that's a different build entirely: a scoped tool-calling loop against the raw model API, not an integration with the Agent product itself. Knowing which one you're actually asking for is the first question worth answering, before evaluating either.

TL;DR: MiniMax Agent (rebranded Mavis in May 2026) is a hosted, subscription-based agent product with a genuinely useful multi-agent "Teams" feature — but it's not embeddable in your own app. Developers who want agent-style behavior inside their own product need to build a scoped tool-calling loop against MiniMax's raw API instead, using the same underlying model.

Website: https://www.fastrouteai.com

Seedance 2.0 Prompt Lessons from Building a Real Product Video Generator

Felix — Mon, 07 Sep 2026 09:42:43 +0000

I built a small internal tool to generate short product demo clips from a text description and a couple of reference product photos — the kind of thing a small e-commerce team could use instead of booking a video shoot for every SKU. The video model doing the actual generation was ByteDance's Seedance 2.0, and the first week of building this taught me more about prompt structure than any guide I read beforehand, mostly because my first dozen prompts produced technically fine but oddly generic clips.

Here's what actually moved the needle, and the small API client I ended up with once I stopped fighting the async job pattern these video generation APIs use.

Seedance 2.0, briefly

Seedance 2.0 is ByteDance's multimodal video generation model, released earlier this year, built on a dual-branch diffusion transformer that generates video and audio together rather than bolting sound on afterward. It accepts text, image, video, and audio as inputs, which means you can describe a scene in words, hand it a reference image for style or a starting frame, hand it a reference video for a specific camera move or action you want copied, and hand it an audio clip to set mood or rhythm — combined in a single generation request. It ships as a small family of models — Mini, Fast, and the full Seedance 2.0 — that share the same feature set and trade off quality against cost and speed, which turns out to matter a lot for how you actually work with it day to day.

Mistake one: writing one paragraph instead of three instructions

My early prompts read like a single flowing description: "a wireless earbuds case opening slowly on a marble surface, soft studio lighting, cinematic feel." That produced clips that were fine but generic — competent b-roll, nothing that looked directed.

The fix was separating what I was actually asking for into three distinct things the model treats differently: the camera and subject action, any dialogue or on-screen text, and the sound. Seedance 2.0 wants dialogue written in double quotes so it can voice it with matching lip movement, and it treats audio essentially as a separate sound brief rather than a mood adjective tacked onto the visual description. Once I split my prompts into explicit camera-move-then-action-then-sound structure instead of one adjective-heavy paragraph, the difference was immediate — the model stopped guessing which words were describing the shot versus describing the mood and started following each instruction more literally.

For multi-shot clips, labeling shots explicitly — Shot 1, Shot 2, Shot 3, in order — and describing each with the same camera-then-action-then-position-then-sound structure gave noticeably more consistent results than one long paragraph trying to cover a 10-second clip in a single breath.

Mistake two: not using reference assets to do the describing

The model's multimodal reference support is the actual headline feature, and I ignored it for the first several attempts out of habit — I was used to text-to-image workflows where you describe everything in words. Seedance 2.0 supports up to nine reference images, three video clips, and three audio files per generation, but the practical guidance (which held up in my testing) is to use far fewer than the ceiling allows. Once I started handing it one clean product photo as a style and color reference instead of describing the product's exact appearance in prose, both consistency and detail improved — skin-pore and material-texture level detail is genuinely something the model handles well when it has a real reference to work from rather than an adjective describing it.

One combination worth knowing before you hit it as a confusing failure: audio alone, or text plus audio with no visual reference, won't generate anything. Every job needs a visual anchor — text-to-video, image-to-video, or a reference video — and audio only works layered on top of one of those, not standalone.

The workflow that actually worked: draft cheap, finalize once

Because the model family spans Mini, Fast, and full Seedance 2.0 at different cost and quality points, the workflow that stopped wasting my API budget was drafting on the cheaper, faster tier until the prompt and reference combination was actually behaving the way I wanted, then running the final pass on the full model once. Iterating on the full-quality tier while still figuring out shot structure was the single biggest source of wasted spend early on.

Handling the async job pattern

Video generation isn't a single request-response call the way a chat completion is — you submit a job with your prompt and references, then poll until it's done, then fetch the result. That shape is common across most video generation APIs regardless of provider, so it's worth building the polling logic once instead of writing it inline every time:

// seedance-client.js — submit a generation job and poll until it's ready

async function submitVideoJob(baseURL, apiKey, { prompt, imageRefs = [], model }) {
  const response = await fetch(`${baseURL}/video/generations`, {
    method: "POST",
    headers: {
      "Content-Type": "application/json",
      Authorization: `Bearer ${apiKey}`,
    },
    body: JSON.stringify({
      model, // e.g. "seedance-2.0-fast" for drafting, "seedance-2.0" for the final pass
      prompt,
      reference_images: imageRefs,
      duration: "auto",
    }),
  });

  if (!response.ok) {
    throw new Error(`Job submission failed: ${response.status}`);
  }
  const { job_id } = await response.json();
  return job_id;
}

async function pollUntilDone(baseURL, apiKey, jobId, { intervalMs = 5000, timeoutMs = 300000 } = {}) {
  const start = Date.now();

  while (Date.now() - start < timeoutMs) {
    const res = await fetch(`${baseURL}/video/generations/${jobId}`, {
      headers: { Authorization: `Bearer ${apiKey}` },
    });
    const data = await res.json();

    if (data.status === "completed") return data.video_url;
    if (data.status === "failed") throw new Error(`Job ${jobId} failed: ${data.error ?? "unknown error"}`);

    await new Promise((resolve) => setTimeout(resolve, intervalMs));
  }

  throw new Error(`Job ${jobId} timed out after ${timeoutMs}ms`);
}

async function generateDraftThenFinal(baseURL, apiKey, prompt, imageRefs) {
  // Draft on the cheaper tier first
  const draftJobId = await submitVideoJob(baseURL, apiKey, {
    prompt,
    imageRefs,
    model: "seedance-2.0-fast",
  });
  const draftUrl = await pollUntilDone(baseURL, apiKey, draftJobId);
  console.log("Draft ready:", draftUrl);

  // Only run the full-quality pass once the prompt is confirmed working
  const finalJobId = await submitVideoJob(baseURL, apiKey, {
    prompt,
    imageRefs,
    model: "seedance-2.0",
  });
  return pollUntilDone(baseURL, apiKey, finalJobId);
}

module.exports = { submitVideoJob, pollUntilDone, generateDraftThenFinal };

This is deliberately generic — exact request and response field names vary by which provider's endpoint you're calling — but the submit-then-poll shape and the draft-then-finalize pattern is what actually made the tool practical to use instead of burning full-price generations while I was still iterating on prompt structure.

Where RouteAI fit in

I ended up calling Seedance 2.0 through RouteAI, which lists it alongside other current models in its catalog, mainly because it meant I didn't need a separate account and billing relationship just to get access to one video model alongside the text models the rest of the internal tool already used. It didn't change anything about the prompting lessons above — those are properties of the model itself, not the gateway in front of it — but consolidating video and text generation behind one key was a smaller convenience worth mentioning rather than a reason to pick a video model in the first place.

What I'd tell someone starting out

Structure your prompt as camera move, then action, then position, then sound — as separate instructions, not one adjective-heavy paragraph. Use a real reference image or video instead of describing appearance in prose whenever you can; the model is genuinely better at matching a reference than interpreting an adjective. Keep reference counts well below the documented limits rather than maxing them out. Draft on the cheaper model tier and only run your final prompt on the full model once it's actually working. And build the polling logic once — you'll be submitting a lot of jobs before the first one looks the way you want.

TL;DR: Splitting Seedance 2.0 prompts into explicit camera/action/sound instructions instead of one descriptive paragraph, leaning on reference images and video instead of prose descriptions, and drafting on a cheaper model tier before the final pass were what actually improved output quality — the API itself is a standard submit-then-poll async pattern worth wrapping once in a reusable client.

Website: https://www.fastrouteai.com

Why Developers Look for an OpenRouter Alternative (And What Actually Changes When You Switch)

Felix — Thu, 03 Sep 2026 09:19:20 +0000

Search "OpenRouter alternative" and you'll land on a dozen posts, most of them written by the alternative being recommended. That's not automatically wrong — plenty of these products are genuinely worth knowing about — but it does mean the comparisons tend to skip the actual question, which isn't "what's the best gateway" in the abstract. It's "what specifically is OpenRouter not doing for my setup, and which alternative actually fixes that."

I went through this evaluation for a small internal tool that had outgrown its original single-provider setup, and the useful realization was that "OpenRouter alternative" isn't one search intent — it's at least three different ones, and they point to different products.

Reason one: the cost structure at scale

OpenRouter's core pitch is real: one API key, one OpenAI-compatible endpoint, a large catalog of models across providers. For prototyping, that's genuinely hard to beat. Where it gets more complicated is billing mechanics once usage grows. Credit purchases carry a platform fee on top of the underlying model price, and the bring-your-own-key option — using your own provider credentials through OpenRouter's routing — is only fee-free up to a monthly request cap, after which it costs a percentage of what the call would have cost natively. None of that is hidden; it's documented pricing. It just means the "one API, no markup" mental model doesn't hold once you're past prototype-scale traffic.

For teams where this is the actual pain point, the fix isn't usually a fundamentally different architecture — it's a gateway with a simpler or lower fee structure that still speaks the same OpenAI-compatible protocol. This is the category where you'll find products like RouteAI, Requesty, and NanoGPT: same request shape, different billing model, no infrastructure change required on your end beyond swapping a base URL and key.

Reason two: self-hosting and compliance requirements

A different group of teams isn't primarily unhappy about fees — they're blocked by the fact that every request has to route through a third-party managed service they can't inspect, deploy inside their own VPC, or align with data-residency requirements. For regulated industries or larger platform teams, "we can't self-host our AI routing layer" is a hard stop regardless of price.

This is where open-source, self-hostable gateways like LiteLLM and Bifrost come in, along with enterprise-oriented platforms like Portkey, TrueFoundry, and Kong AI Gateway that add virtual keys, per-team budgets, RBAC, and audit logging on top of routing. These solve a genuinely different problem than pricing — they're about who controls the request path and what governance sits around it, not about shaving cents off a token rate.

Reason three: latency and architecture overhead in agentic workflows

The third reason shows up specifically in multi-step, agentic setups — a coding agent or pipeline that makes many chained model calls per task. Every hop through a third-party proxy adds latency, and in a workflow making dozens of calls per session, that overhead compounds in a way it doesn't for a single chat request. Some comparison posts publish specific time-to-first-token benchmarks showing OpenRouter running meaningfully slower than a direct or self-hosted path — worth treating as a data point from a source with its own gateway to sell, not as a settled fact, but the underlying mechanism (an extra network hop costs something) is real regardless of the exact number.

Teams optimizing specifically for this tend to land on either a self-hosted proxy running close to their own infrastructure (again, LiteLLM or Bifrost), or a direct relationship with a single inference provider like Together AI or Fireworks when they've settled on one model family and don't need multi-provider flexibility anymore.

The mistake is picking one "winner"

Almost every comparison post I read while doing this evaluation eventually converges on a single "best" pick, which is a strange conclusion given that the three reasons above don't share a solution. A team blocked by BYOK fees doesn't need RBAC and audit logs. A compliance team that needs self-hosting doesn't care whether the fee structure is 2% cheaper somewhere else. Matching the alternative to the actual reason you're evaluating one is most of the decision — the rest is testing.

Testing candidates instead of trusting the comparison table

Because most of these alternatives are OpenAI-compatible, the fastest way to evaluate one honestly is to hit it with the same request shape and compare latency and response quality directly, rather than relying on a vendor's own benchmark numbers:

// gateway-compare.js — sends the same request to multiple OpenAI-compatible
// gateways and reports latency, so you can compare candidates on your own traffic

const CANDIDATES = [
  {
    name: "openrouter",
    baseURL: "https://openrouter.ai/api/v1",
    apiKey: process.env.OPENROUTER_API_KEY,
    model: "deepseek/deepseek-v4-flash",
  },
  {
    name: "routeai",
    baseURL: process.env.ROUTEAI_BASE_URL,
    apiKey: process.env.ROUTEAI_API_KEY,
    model: "deepseek-v4-flash",
  },
  // add any other OpenAI-compatible candidate here
];

async function timedRequest(candidate, messages) {
  const start = Date.now();
  try {
    const response = await fetch(`${candidate.baseURL}/chat/completions`, {
      method: "POST",
      headers: {
        "Content-Type": "application/json",
        Authorization: `Bearer ${candidate.apiKey}`,
      },
      body: JSON.stringify({ model: candidate.model, messages }),
    });
    const elapsedMs = Date.now() - start;
    if (!response.ok) {
      return { name: candidate.name, ok: false, status: response.status, elapsedMs };
    }
    const data = await response.json();
    return {
      name: candidate.name,
      ok: true,
      elapsedMs,
      tokens: data.usage?.total_tokens ?? null,
    };
  } catch (err) {
    return { name: candidate.name, ok: false, error: err.message, elapsedMs: Date.now() - start };
  }
}

async function compareAll(prompt) {
  const messages = [{ role: "user", content: prompt }];
  const results = await Promise.all(CANDIDATES.map((c) => timedRequest(c, messages)));
  console.table(results);
  return results;
}

compareAll("Explain the difference between a mutex and a semaphore in two sentences.");

Running this against your own real traffic patterns — not a single toy prompt — for a day or two gives you something a comparison post can't: actual latency and reliability numbers for your specific region, model choice, and request volume. Pricing pages and benchmark screenshots are a starting point, not a substitute for this.

Where RouteAI fits, specifically

To be concrete rather than vague about it: RouteAI is relevant to reason one above — teams who like the OpenAI-compatible, multi-model-in-one-key pattern OpenRouter popularized, and whose actual complaint is the fee structure once usage grows, not a need for self-hosting or enterprise governance. It exposes the same /chat/completions shape, supports models across several labs (DeepSeek, Qwen, Kimi, GLM, MiniMax, and others), and bills transparently per call with no monthly fee. It is not an answer to reason two or three — if your blocker is self-hosting or agentic-workflow latency, look at the gateways built specifically for that instead.

The actual takeaway

"OpenRouter alternative" isn't a single product category, and most top-10 lists obscure that by ranking everything against the same criteria regardless of why you're looking. Before picking anything, name the actual reason you're evaluating a switch — fee structure, self-hosting, or latency — because that alone eliminates most of the list. Then run your own traffic through the remaining candidates rather than trusting anyone's benchmark, including this post's.

TL;DR: "OpenRouter alternative" searches usually come from one of three distinct problems — fee structure, self-hosting/compliance needs, or agentic-workflow latency — and each points to a different kind of product, so the right move is naming your actual reason first and then testing candidates against your own traffic instead of trusting a ranked list.

Website: https://www.fastrouteai.com

I Connected Cursor to OpenRouter. Here's a Script to Verify It Before You Trust the Editor.

Felix — Wed, 02 Sep 2026 06:35:12 +0000

Why I Wanted to Verify Outside the Editor First

Cursor's error messages when a custom API connection fails aren't always helpful — sometimes you get a clear error, sometimes a mode just quietly stops producing useful output with no obvious signal about why. Before trusting my Cursor OpenRouter setup inside the editor, I wanted to confirm the connection actually worked from a plain script first, so I'd know whether any future weirdness was Cursor-specific or an actual API problem.

The Cursor-Specific Endpoint

OpenRouter provides a dedicated endpoint built specifically to handle Cursor's request format: https://openrouter.ai/api/v1/cursor — not the standard /api/v1 you'd use for a generic OpenAI compatible API call. This is currently in beta, and it's worth checking OpenRouter's own API documentation for the current state before assuming behavior is fully stable.

The Verification Script

import requests
import json

OPENROUTER_API_KEY = "YOUR_OPENROUTER_API_KEY"
CURSOR_ENDPOINT = "https://openrouter.ai/api/v1/cursor/chat/completions"

def verify_cursor_endpoint(model_id, api_key=OPENROUTER_API_KEY):
    headers = {
        "Authorization": f"Bearer {api_key}",
        "Content-Type": "application/json",
    }
    payload = {
        "model": model_id,
        "messages": [{"role": "user", "content": "Reply with exactly: connection ok"}],
    }
    response = requests.post(CURSOR_ENDPOINT, headers=headers, json=payload, timeout=15)

    print(f"Status: {response.status_code}")
    if response.status_code == 200:
        content = response.json()["choices"][0]["message"]["content"]
        print(f"Response: {content}")
        return True
    else:
        print(f"Error body: {response.text}")
        return False

if __name__ == "__main__":
    models_to_check = [
        "anthropic/claude-3.5-sonnet",
        "openai/gpt-5.6-mini",
    ]
    for model in models_to_check:
        print(f"\n--- Checking {model} ---")
        verify_cursor_endpoint(model)

Running this confirms two things independently of Cursor's UI: that your OpenRouter API key is valid and has access to the models you're planning to use, and that the Cursor-specific endpoint itself is responding correctly for those model IDs. If this script works and Cursor still misbehaves, you've isolated the problem to Cursor's client behavior rather than your credentials or endpoint configuration.

Configuring Cursor Itself

Once the script confirms the connection works:

Open Cursor → Settings → Models
Enable "Override OpenAI Base URL"
Enter https://openrouter.ai/api/v1/cursor (not the plain /api/v1 path)
Paste your OpenRouter key into the API key field
Add a custom model using the same model ID you verified in the script
Click Verify

Where This Actually Got Murky

Chat-based modes worked immediately once configured this way. Agent mode was less consistent — Cursor's support for custom API connections hasn't extended identically across every mode historically, and I found conflicting information even in adjacent documentation about which modes fully support this at any given time. If Agent mode seems to silently stop being useful rather than throwing a clear error, that's a known rough edge in an integration that's actively evolving, not necessarily something wrong with your setup — which is exactly why running the verification script first is worth the five minutes; it tells you definitively whether the problem is upstream of Cursor.

Why Bother With This Instead of Cursor's Native Providers

Cursor supports a handful of providers natively through its own key-based setup. Routing through an AI API gateway like OpenRouter instead gets you access to a much wider set of OpenRouter models through one key, plus automatic failover if a specific provider has an outage, and centralized usage tracking if you're managing this across a team rather than just yourself.

If You're Setting This Up Yourself

Run a verification script against the API directly before assuming a Cursor problem is a connection problem
Use the Cursor-specific endpoint path, not OpenRouter's general base URL — they don't handle the request format identically
Test in a simpler mode (chat/ask) before troubleshooting Agent mode specifically, since support levels aren't guaranteed to match Testing the Same Pattern Elsewhere

Once I had this verification pattern working, I used a similar approach — hitting an API directly with a small script before trusting an editor's UI — to sanity-check a comparable setup through RouteAI for a different project, unrelated to Cursor specifically. That's a separate decision worth checking on its own terms; the verification pattern above is the actually reusable part, regardless of which gateway you're pointing an editor at.

TL;DR: Before trusting Cursor's UI when a custom OpenRouter connection misbehaves, verify the endpoint and key work with a plain script first — that isolates whether a problem is your API configuration or Cursor's client behavior in a specific mode. Endpoint must be /api/v1/cursor, not the standard /api/v1. Verification script above.

Check out the tool I covered here: www.fastrouteai.com

Setting Up My First OpenRouter AI API Key: Working Code and Two Mistakes I Made

Felix — Mon, 31 Aug 2026 08:44:22 +0000

Why I Went Looking for an AI API Gateway

I was comparing three different LLMs for a small side project — a tool that rewrites messy meeting notes into clean summaries — and quickly realized that testing each one meant signing up separately with each provider, generating a separate key, and learning each provider's specific request format. That's a lot of setup before I'd written a single line of the actual feature. This is roughly the point where most people discover what an AI API gateway actually is: a single service that sits in front of multiple LLM providers, so you authenticate once and call whichever model you want through one consistent format.

OpenRouter is one of the more commonly recommended options for this, so that's where I started. Here's the actual setup process, plus the two mistakes that cost me time I didn't need to lose.

Getting an OpenRouter API Key

Sign up on OpenRouter's site, and there's a dedicated Keys section in the dashboard where you generate one. This part is genuinely simple — no unusual configuration, just a string you'll pass in your requests like any other AI API key.

Configuring It in Code

OpenRouter's API follows the OpenAI compatible API format, which meant I didn't need a new SDK — just a different base URL and my OpenRouter key:

from openai import OpenAI

client = OpenAI(
    api_key="YOUR_OPENROUTER_API_KEY",
    base_url="https://openrouter.ai/api/v1"
)

def ask(model, prompt):
    response = client.chat.completions.create(
        model=model,
        messages=[{"role": "user", "content": prompt}],
    )
    return response.choices[0].message.content

meeting_note = "Discussed Q3 roadmap, agreed to delay launch by two weeks, action items assigned to Sam and Priya."

result = ask("anthropic/claude-3.5-sonnet", f"Summarize this meeting note in two sentences: {meeting_note}")
print(result)

The model string is where the actual utility of using this kind of gateway shows up — switching between different OpenRouter models just meant changing that one argument, not rebuilding my client setup for each provider I wanted to test.

Testing Multiple Models Without Repeating Setup

models_to_compare = [
    "anthropic/claude-3.5-sonnet",
    "openai/gpt-5.6-mini",
    "google/gemini-2.5-flash",
]

for model in models_to_compare:
    print(f"--- {model} ---")
    print(ask(model, f"Summarize this meeting note in two sentences: {meeting_note}"))
    print()

This is the actual value for a comparison-heavy workflow like mine: one function, one loop, three models tested in the time it would've taken to configure one provider's SDK from scratch.

Mistake #1: Assuming Auth Errors Mean a Bad Key

My first several requests failed with an authentication error even though I'd copied the key correctly. The actual problem was a leftover Authorization header format in my client config from a different provider I'd tested earlier. If you're migrating code written for another LLM API, double-check your header format against what the OpenRouter API documentation actually shows — don't assume a failed auth request means the key itself is wrong.

Mistake #2: Assuming Flat-Rate Pricing

I initially assumed one API key meant one pricing tier across every model. It doesn't — each model you call carries its own per-token pricing set by the underlying provider, and it varies meaningfully between something like a fast, cheap model and a larger flagship one. Worth checking current pricing for your specific target models before running anything at real volume, rather than assuming a uniform rate.

What I'd Tell Someone Following an OpenRouter API Tutorial for the First Time
Get one model working end-to-end before adding a comparison loop — isolate whether an issue is your setup or a specific model's behavior
Store your key in an environment variable immediately; a leaked gateway key exposes access to every model you've configured it to reach, not just one provider's service
Check per-model pricing explicitly rather than assuming your existing budget assumptions from a single-provider setup still apply
Where This Left Me

Once I had this comparison running cleanly, I also tested the same three models through RouteAI, mainly out of curiosity about how pricing and available models compared across a couple of gateways before settling on one long-term. That's genuinely a "check both, see what fits your specific models and budget" decision rather than something I'd tell you to take on faith — the actual problem worth solving here isn't which gateway you pick, it's not losing an afternoon to four separate provider signups just to run one comparison.

TL;DR: Setting up an OpenRouter API key is mostly a five-minute process — the two things that actually cost me time were a stale auth header from a different provider, and assuming pricing was flat across models when it's per-model. Working code for a multi-model comparison loop is above.

Linking the tool mentioned above: www.fastrouteai.com

GLM 5.3's API Breaks a Common GLM-5.2 Pattern. Here's the Fix.

Felix — Thu, 27 Aug 2026 09:30:53 +0000

The Upgrade That Broke Everything

I swapped glm-5.2 for glm-5.3 in my model string, ran my test suite, and every single call failed. No syntax errors, no auth issues — just requests bouncing back immediately. It took digging through GLM-5.3's release notes to find the actual cause: a parameter my code had been setting for months no longer exists.

What Changed

GLM-5.2 let you explicitly disable thinking mode for tasks that don't need extended reasoning:

# This worked fine on GLM-5.2
response = client.chat.completions.create(
    model="glm-5.2",
    messages=messages,
    extra_body={"thinking": {"type": "disabled"}}
)

GLM-5.3 removed that option entirely. It now exposes three effort levels — low, high, max (max is the default) — but there's no way to turn thinking off. Passing type: disabled doesn't get ignored or silently fall back to something reasonable; the request fails outright.

# This fails on GLM-5.3 — "disabled" is no longer a valid type
response = client.chat.completions.create(
    model="glm-5.3",
    messages=messages,
    extra_body={"thinking": {"type": "disabled"}}
)

The fix is small once you know what's happening:

# This works on GLM-5.3
response = client.chat.completions.create(
    model="glm-5.3",
    messages=messages,
    extra_body={"thinking": {"type": "enabled", "effort": "low"}}
)

What "Low Effort" Actually Costs You

I tested effort: low against my old disabled baseline on the same task (structured code review comments) to see how close it actually gets:

import time

def timed_call(model, extra_body, prompt):
    start = time.time()
    response = client.chat.completions.create(
        model=model,
        messages=[{"role": "user", "content": prompt}],
        extra_body=extra_body,
    )
    elapsed = time.time() - start
    return response.choices[0].message.content, elapsed

# Old baseline
old_output, old_time = timed_call(
    "glm-5.2", {"thinking": {"type": "disabled"}}, my_review_prompt
)

# New setup
new_output, new_time = timed_call(
    "glm-5.3", {"thinking": {"type": "enabled", "effort": "low"}}, my_review_prompt
)

print(f"GLM-5.2 disabled: {old_time:.2f}s")
print(f"GLM-5.3 low effort: {new_time:.2f}s")

On my workload, low effort landed close to the old disabled latency but not identical — GLM-5.3 still generates some reasoning trace even at the lowest setting, so expect slightly more tokens and slightly more time than a true off-switch used to cost. For short, structured tasks this difference was small enough not to matter. If you're running high-volume, latency-sensitive requests, measure this on your own workload before assuming low is a clean substitute.

Where This Actually Improved Things

Once the breaking change was handled, the underlying model gains were real for what I was doing. GLM-5.3 is a post-training update on the same GLM-5.2 base (same 744B parameters, same MoE architecture) — the improvements are concentrated in agentic and coding-specific tasks rather than general capability. Zhipu's own benchmarks show a jump on Terminal-Bench 3.0 from roughly 4.6% to 28.3%. For my multi-step code review comments, that tracked: GLM-5.3 caught logic issues spanning several files noticeably better than 5.2 did. For simpler single-function reviews, I couldn't reliably tell the difference.

A Few Things Worth Checking Before You Migrate
Grep your codebase for thinking.*disabled before touching the model string — if you're not setting it, you're probably unaffected
Benchmark low effort against your old disabled baseline if latency matters for your use case; they're close, not identical
Match the upgrade to your actual workload — GLM-5.3's gains are concentrated in coding and agentic tasks, not general-purpose improvement across the board
Testing Across Providers

I ended up testing my updated GLM-5.3 setup through RouteAI, mostly because I wanted to compare its coding-task output against a couple of other models before fully committing my pipeline to the migration, and that meant not maintaining separate client configs per provider. That part is a convenience detail, not the finding here — the breaking change and its fix are the same whether you call GLM-5.3 directly or through a gateway.

TL;DR: GLM-5.3 removed the ability to fully disable thinking mode — disabled now fails outright, and you need enabled with an effort level instead. low effort gets close to the old latency but isn't identical. Full before/after code above.

Worth exploring if this is relevant to your stack: www.fastrouteai.com

I Built a Tool-Calling Agent on Qwen. Here's the Full Working Loop.

Felix — Wed, 26 Aug 2026 07:12:39 +0000

Tool Calling Is the Easy Part. The Loop Is Where It Gets Real.

My first "agent" wasn't actually an agent. It was one API call that returned a tool call, which I executed, and then I just... stopped, because I hadn't written anything to feed the result back in. It looked like it worked. It didn't — it just did one step and quit.

I was building a small assistant that could search a local notes database and check a to-do list, using Qwen's API. Here's the actual working setup, including the part I got wrong the first time.

Defining Tools

Qwen's function calling uses the same OpenAI-compatible schema most major providers use now — define your functions, pass them with your messages, and the model decides whether to respond with text or a structured tool call.

tools = [
    {
        "type": "function",
        "function": {
            "name": "search_notes",
            "description": "Search the user's saved notes for a keyword and return matching entries. Use this when the user asks to find or recall something they wrote down.",
            "parameters": {
                "type": "object",
                "properties": {
                    "keyword": {"type": "string", "description": "The term to search for"}
                },
                "required": ["keyword"]
            }
        }
    },
    {
        "type": "function",
        "function": {
            "name": "check_todo_list",
            "description": "Return the user's current to-do list. Use this when the user asks what they need to do or what's pending.",
            "parameters": {"type": "object", "properties": {}}
        }
    }
]

The description fields do more work than they look like they should. My first version just said "Searches notes" and "Checks todos" — the model started picking the wrong tool for ambiguous queries because there wasn't enough signal to differentiate intent. Being explicit about when to use each tool, not just what it does, fixed most of the misfires once I had more than two tools competing.

The Part I Got Wrong: No Loop

Here's the broken version — the one that looks like it works because it doesn't error out:

response = client.chat.completions.create(
    model="qwen-plus",
    messages=[{"role": "user", "content": "What's on my to-do list, and do I have notes about Berlin?"}],
    tools=tools,
)

message = response.choices[0].message
if message.tool_calls:
    for call in message.tool_calls:
        result = execute_tool(call.function.name, call.function.arguments)
        print(result)  # dead end — the model never sees this

This runs the tool and prints the result. It never tells the model what happened, so if the query needs more than one tool call, or needs the model to actually respond based on the tool's output, it just stops. For a query needing two tools ("to-do list" and "Berlin notes"), this version might only execute one and call it done.

The Working Version: Actually Looping

messages = [{"role": "user", "content": user_input}]

while True:
    response = client.chat.completions.create(
        model="qwen-plus",
        messages=messages,
        tools=tools,
    )
    message = response.choices[0].message

    if message.tool_calls:
        messages.append(message)
        for call in message.tool_calls:
            result = execute_tool(call.function.name, call.function.arguments)
            messages.append({
                "role": "tool",
                "tool_call_id": call.id,
                "content": str(result)
            })
    else:
        print(message.content)
        break

The difference: tool results get appended back into messages as a new turn, and the loop calls the API again with that updated context. The model sees what the tool returned and decides whether it has enough to answer or needs another tool call. This is the actual mechanism behind "agent" — not a mode you enable, just a loop that keeps going until the model stops asking for tools.

The Failure Mode Nobody Warns You About

Without a cap, this loop can run indefinitely if the model keeps deciding it needs "one more" tool call, especially if a tool returns something ambiguous or a call fails silently. Add a max-iteration limit:

MAX_ITERATIONS = 6
messages = [{"role": "user", "content": user_input}]

for _ in range(MAX_ITERATIONS):
    response = client.chat.completions.create(
        model="qwen-plus",
        messages=messages,
        tools=tools,
    )
    message = response.choices[0].message

    if message.tool_calls:
        messages.append(message)
        for call in message.tool_calls:
            result = execute_tool(call.function.name, call.function.arguments)
            messages.append({
                "role": "tool",
                "tool_call_id": call.id,
                "content": str(result)
            })
    else:
        print(message.content)
        break
else:
    print("Hit max iterations without a final answer.")

I found this the hard way — a test query looped six times on something that should have taken one, because a tool call returned an empty result and the model kept retrying instead of giving up gracefully.

Where This Actually Got Tested Further

Once the loop was solid, I wanted to check whether tool-selection accuracy held up if I swapped the underlying model — not because Qwen was underperforming, but because I was curious whether the loop logic above was actually model-agnostic or whether I'd built something that happened to work for one provider's quirks. I ran the same loop through RouteAI, an OpenAI-compatible gateway, against a couple of other models, and the loop code above didn't need to change — only the model argument did, since the request format stayed consistent. Worth being clear this wasn't the hard part of the project; the loop and tool descriptions were.

If You're Building Your First Agent
Get one tool working end-to-end, including the loop, before adding a second
Write descriptions that say when to use a tool, not just what it does
Always cap your iterations — an ungated loop will eventually run past what you expect

TL;DR: An "agent" is a loop, not a single API call — tool results have to get fed back into the conversation for the model to actually finish the task. Full working loop with an iteration cap included above; the tool descriptions matter more than most tutorials mention.

Worth exploring if this is relevant to your stack: www.fastrouteai.com

I Wrote a Script to Track My DeepSeek API Costs After the Weekend Pricing Change

Felix — Mon, 24 Aug 2026 09:41:45 +0000

The Bill That Made Me Curious

I don't normally check my API dashboard more than once a week. But last Sunday I did, out of boredom, and the number was lower than I expected for how much I'd been running.

Turns out DeepSeek quietly changed its peak/off-peak billing structure. According to their pricing docs, peak hours are now strictly Monday–Friday, 01:00–04:00 and 06:00–10:00 UTC. Everything else — including all of Saturday and Sunday — bills at off-peak rates, which run at roughly half of peak pricing for both input and output tokens.

I run most of my testing on weekends (day job during the week), so in theory this should matter a lot for me. But "in theory" isn't good enough when I'm the one paying the invoice. I wanted actual numbers from my actual usage, not a guess.

So I wrote a small script.

What the Script Does

Nothing fancy — it takes a log of my API calls (timestamp + input/output token counts) and calculates what I actually paid under the new peak/off-peak rules, split by weekday vs weekend.

import csv
from datetime import datetime, timezone

# Pricing per 1M tokens (cache miss), off-peak / peak
PRICING = {
    "deepseek-v4-flash": {
        "input_offpeak": 0.22, "input_peak": 0.44,
        "output_offpeak": 0.66, "output_peak": 1.32,
    },
    "deepseek-v4-pro": {
        "input_offpeak": 0.66, "input_peak": 1.32,
        "output_offpeak": 1.98, "output_peak": 3.96,
    },
}

def is_peak(dt_utc):
    # Peak: Mon-Fri, 01:00-04:00 and 06:00-10:00 UTC
    if dt_utc.weekday() >= 5:  # Sat=5, Sun=6
        return False
    hour = dt_utc.hour
    return (1 <= hour < 4) or (6 <= hour < 10)

def calc_cost(model, input_tokens, output_tokens, dt_utc):
    rates = PRICING[model]
    peak = is_peak(dt_utc)
    in_rate = rates["input_peak"] if peak else rates["input_offpeak"]
    out_rate = rates["output_peak"] if peak else rates["output_offpeak"]
    cost = (input_tokens / 1_000_000) * in_rate + (output_tokens / 1_000_000) * out_rate
    return cost, peak

def analyze_log(csv_path):
    weekday_cost, weekend_cost = 0.0, 0.0
    weekday_calls, weekend_calls = 0, 0

    with open(csv_path) as f:
        reader = csv.DictReader(f)
        for row in reader:
            dt = datetime.fromisoformat(row["timestamp"]).astimezone(timezone.utc)
            cost, peak = calc_cost(
                row["model"],
                int(row["input_tokens"]),
                int(row["output_tokens"]),
                dt,
            )
            if dt.weekday() >= 5:
                weekend_cost += cost
                weekend_calls += 1
            else:
                weekday_cost += cost
                weekday_calls += 1

    print(f"Weekday: {weekday_calls} calls, ${weekday_cost:.4f}")
    print(f"Weekend: {weekend_calls} calls, ${weekend_cost:.4f}")

if __name__ == "__main__":
    analyze_log("api_usage_log.csv")

Expected CSV format:

timestamp,model,input_tokens,output_tokens
2026-08-22T14:32:00+00:00,deepseek-v4-flash,1200,340
2026-08-23T09:15:00+00:00,deepseek-v4-flash,980,410

You'll need to export your own usage log — DeepSeek's dashboard lets you download call history, or you can log it yourself at request time if you're not already.

What I Actually Found

Running this against about six weeks of my own logs: my weekend calls were consistently cheaper per-token than my weekday calls even before this change (because some of my weekend hours already fell outside the old off-peak window by luck). After the update, the gap widened — my weekend cost-per-call dropped further since Saturday and Sunday are now unconditionally off-peak, no matter the hour.

For someone running a handful of batch jobs on weekends, that's a real, if modest, saving. Your mileage depends entirely on when you actually run your workload — if most of your usage happens on weekday afternoons, this change does nothing for you.

The Question I Couldn't Answer With This Script

Once I had this data, the obvious next question was: how would the same workload cost on a different model? The script above only works because I already know DeepSeek's pricing structure. Answering the same question for Qwen or GLM would mean writing a whole new pricing table and, more annoyingly, a whole new API integration to actually generate comparable token logs.

That's the part I didn't script around — I switched my project to call models through RouteAI, an OpenAI-compatible API gateway, mainly so I could point the same request format at different models without rebuilding my client code each time. It didn't change the cost-tracking logic above, but it meant I could actually go collect that data for other models instead of just wondering about it. Worth noting this only saved me integration time — the actual per-token pricing is still whatever each model provider sets.

If You Want to Run This Yourself
Export or log your own usage with timestamps in UTC (not local time — this bit me on my first run)
Add pricing tiers for whichever models you're using; I only included flash and pro here
If you're on a different provider, check whether they have time-based pricing at all before assuming this script applies

TL;DR: DeepSeek's new pricing makes weekends fully off-peak (previously only certain hours were). I wrote a Python script to calculate actual cost split between weekday/weekend usage from a call log — script included above, MIT-license it however you like.

Worth exploring if this is relevant to your stack: www.fastrouteai.com

'Cheap AI' Is a Moving Target — Here's the Framework for Evaluating It Correctly

Felix — Thu, 20 Aug 2026 08:13:11 +0000

$0.14 per million tokens looks cheap. $0.14 per million tokens, multiplied by a feature that goes viral next Tuesday, might not be.

Most "is this AI cheap" evaluations stop at the first number. That's the mistake I want to walk through, because the actual question was never "what does it cost right now" — it's "what shape does the cost take as usage grows," and those are very different questions that happen to share a price tag.

The Number Everyone Compares, and Why It's Incomplete

When people evaluate whether an AI API is "cheap," they almost always mean: what's the per-token or per-request price, compared to alternatives. That number is real and it matters — but it's a snapshot, not a trajectory. It tells you what one unit costs. It tells you nothing about what a thousand units cost, or a million, or what happens at the specific volume your actual product will hit if it works.

Cost only behaves the way that snapshot implies if it scales linearly forever — the 1,000th call costs exactly what the 1st call cost, with nothing changing in between. That's the assumption baked into "cheap AI" as most people evaluate it. It's also, in practice, rarely how these systems actually behave once you're operating at real scale.

Three Shapes Cost Actually Takes, Not Two

Cost as a function of usage tends to fall into a few recognizable shapes, and knowing which one you're dealing with matters more than the sticker price:

Linear. Cost per unit stays flat regardless of volume. The 1,000th call costs what the 1st call cost. This is what a simple per-token price implies, and it's genuinely how some providers price things — but it's worth verifying rather than assuming, because it's the exception more often than the default once you look closely.

Step function. Cost stays flat within a tier, then jumps at a threshold — a rate limit you exceed and get bumped to a more expensive plan, a concurrency cap that forces you onto premium infrastructure, a free-tier boundary you cross without noticing until the bill arrives. This is common with anything that has both a free and paid tier, and with time-based pricing structures where crossing from off-peak into peak hours effectively doubles your rate mid-workload.

Marginal-decreasing. Cost per unit actually drops as volume increases — committed-use discounts, volume pricing tiers, negotiated enterprise rates. Less common at small scale, but real once you're a big enough customer for a provider to want to keep you.

Almost nobody evaluating "is this cheap AI" is thinking in these terms. They're comparing a single number, implicitly assuming shape one, when the actual answer might be shape two — which means the "cheap" evaluation was accurate at low volume and actively misleading at the volume that matters.

Where This Actually Goes Wrong

Here's the concrete failure mode: a team evaluates a model at prototype-scale usage, sees a low per-token cost, concludes "this is cheap," and builds a cost model for the product around that number. Then the product works. Usage climbs. And somewhere along the way, they cross a threshold they didn't know existed — a concurrency limit that forces an upgrade, a rate structure where their peak-hours traffic pattern (because of course their users are active during business hours) now costs meaningfully more than the number they budgeted around.

None of this is deceptive pricing. It's usually clearly documented — but documented in a pricing page that gets read once, at evaluation time, at a volume that doesn't reflect where the product ends up. The gap between "cheap at the volume I tested" and "cheap at the volume I'll actually run" is exactly the gap this framework is trying to make visible before it becomes a surprise on an invoice.

This is also worth connecting to a specific, current example: several major model providers have recently introduced peak/off-peak pricing structures, where identical usage costs meaningfully more during business hours than at night. That's a step function hiding inside what looks, on the pricing page, like a single number — and it's a good illustration of why "what's the price" and "what's the cost curve" are genuinely different questions right now, not just a hypothetical concern.

How to Actually Evaluate "Cheap" Correctly

A few concrete things worth doing differently, given this framework:

Ask about the shape, not just the number, when you evaluate a provider. Is this price linear indefinitely? Is there a rate limit or concurrency threshold where the effective cost jumps? Does pricing vary by time of day or usage tier? These are usually documented, just rarely asked about upfront.

Model your cost at the volume you expect to hit if things go well, not the volume you're at during evaluation. A price that's cheap at prototype scale and a price that's cheap at the scale you're actually building for are different claims, and testing at the wrong scale is how teams get surprised.

Treat unpredictability as its own cost, separate from the price itself. A cost structure that varies by time of day or usage threshold requires monitoring and budgeting work that a flat, predictable rate doesn't — and that overhead is real, even when the headline price is competitive.

Standardized, flat-rate access reduces the step-function risk specifically. This is part of why unified access layers — services that sit in front of multiple model providers with flat, consistent pricing rather than tiered or time-based structures — have become a relevant category to know about, separate from any single provider's rate card. RouteAI is one example of this, offering flat per-token pricing without a peak-hour surcharge structure across several model providers — worth mentioning as a case study in what a linear-shaped alternative looks like, not as a claim that it's the cheapest option available, which would require a comparison this piece isn't making.

The Actual Point

"Cheap AI" isn't a property of a number on a pricing page. It's a property of how that number behaves as your usage grows into whatever your product actually becomes. Two providers can have an identical headline price and radically different actual cost outcomes at scale, depending entirely on the shape underneath that number — and the shape is almost never what gets compared when people search "cheapest AI API" or "cheap AI" and pick based on the first number they see.

The question worth asking isn't "what does this cost me today." It's "what does this cost me at the volume where it actually matters" — and that's a question the pricing page alone usually won't answer for you.

TL;DR: Comparing "cheap AI" options by per-token price alone assumes cost scales linearly, but real pricing often follows a step function (rate limits, peak-hour surcharges, tier jumps) or occasionally marginal-decreasing pricing (volume discounts) instead. Evaluate providers by asking about the shape of cost as usage grows, not just the sticker price, and model your cost at the volume you'll actually hit — not the volume you tested at. Standardized, flat-rate access layers (like RouteAI, among other options) can reduce step-function risk specifically, though that's a different claim from being the cheapest option outright.

Here's the tool I referenced in this post: www.fastrouteai.com

'Free AI API' Isn't a Pricing Tier — It's a Funnel. Here's How It Actually Works

Felix — Tue, 18 Aug 2026 09:23:23 +0000

Zero dollars, for a while, under conditions you didn't read.

That's the actual terms of almost every "free AI API" you'll find right now. Not a lie, exactly — just incomplete, in a way that shapes a lot of decisions before anyone notices it's happening.

I want to walk through why "free" in this market behaves less like a pricing tier and more like a funnel, and what that means for how you should actually evaluate these offers.

The Pattern, Once You Notice It

Search "free AI API" and you'll land on some combination of: a signup-gated trial credit, a rate-limited forever-free tier, or an open-source model you self-host. All three get marketed as "free," and all three have a different actual cost structure hiding underneath the word.

Signup trial credits are the most common. You get a fixed dollar amount — enough to build a prototype, rarely enough to run anything in production for long. The moment it runs out, you're a paying customer with an existing integration, which is exactly the point. This isn't a criticism — it's a completely standard acquisition mechanic, the API equivalent of a free trial on any SaaS product. But it's worth naming it accurately: it's not a pricing tier, it's a funnel stage.

Rate-limited free tiers are the second version — genuinely free indefinitely, but capped hard enough (low requests per minute, low concurrency) that they work for testing and personal projects, not for anything with real traffic. These are sustainable for the provider precisely because they're too small to represent meaningful cost or meaningful competition to the paid tier.

Self-hosted open-source models are the third, and the only one that's "free" in a literal, ongoing sense — you're just trading API cost for infrastructure cost, DevOps time, and the work of keeping up with model updates yourself. Free in one column, expensive in another.

None of these are bad options. They're just not the same thing, and treating them as interchangeable answers to "is there a free AI API" is where people get tripped up.

Why "Free" Behaves Like a Funnel, Not a Tier

A pricing tier is a stable offering — you can build a long-term plan around it. A funnel stage is designed to convert you into the next stage. The tell is in how these free options are structured: they're calibrated to be just enough to prove the product works, and just limited enough that any real usage pushes you past the edge.

This isn't unique to AI APIs — it's the same shape as freemium software, free cloud tiers, free CDN bandwidth. What's different about AI APIs specifically is how fast the terms of the funnel can move underneath you, because the underlying compute cost is still genuinely high and providers are actively experimenting with pricing structure.

A concrete, recent example: several major model providers have started introducing peak/off-peak pricing — cheaper rates at night, higher rates during business hours — as a way to manage daytime infrastructure load. One provider's recent repricing moved certain rates by several hundred percent between off-peak and peak windows on the same day. That kind of structural change doesn't just affect paid usage — it reshapes what the free tier's limits are worth, and how predictable "free enough for now" actually is going forward.

If your evaluation of "is this free API good enough" was done once, at signup, and never revisited, you're evaluating a snapshot of a funnel stage that's actively designed to change.

What This Actually Means for How You Should Evaluate "Free"

Reframing free-tier evaluation around the funnel model changes what questions are worth asking:

Not: "Is this free?" Instead: "What stage of the funnel is this free tier designed to move me through, and what does the next stage cost?"

Not: "How much free credit do I get?" Instead: "What's the actual rate limit and concurrency cap, and does that match my real traffic pattern — not my testing pattern?"

Not: "Which provider has the best free tier?" Instead: "If I build against this provider's free tier and it changes, how much of my integration breaks?"

That last question is the one people skip most often, and it's the one that actually costs money later. A free tier that requires rewriting your error handling, retry logic, and request format the moment you outgrow it is a more expensive "free" than one that lets you scale into paid usage without touching your code.

The Honest Nuance Here

None of this means free tiers are a trap to avoid — they're genuinely useful, and almost every serious project should start on one. The point isn't "don't use free APIs." It's that treating "free" as a stable, comparable pricing category leads to worse decisions than treating it as what it actually is: a funnel stage with a shelf life.

This is also, worth being direct about, part of why unified API gateways exist as a category — services like RouteAI, which sit in front of multiple model providers behind one OpenAI-compatible interface, are a response to exactly this problem. Not because they eliminate the funnel dynamic (they don't — someone's still paying for compute), but because standardizing the interface means that whichever free tier you start on, and whatever it turns into later, the actual code you've built doesn't have to be the thing that breaks.

I'm not making a "just use this instead" argument here — that's a separate decision with its own tradeoffs. The point of this piece is narrower: the next time you're comparing "free AI API" options, the question that actually predicts how this goes for you isn't "how much free stuff do I get." It's "what happens to my integration when this stops being free."

That's the question the marketing page never answers, and it's the one worth answering yourself before you build on top of any of them.

TL;DR: "Free AI API" almost always means one of three things — trial credits (a funnel stage), a rate-limited forever-free tier (too small to matter), or self-hosted open source (free API cost, real infrastructure cost). None of these are stable pricing tiers — they're calibrated to convert or contain you. The question worth asking isn't "is this free," it's "what breaks in my integration when this stops being free," and building against a standardized interface (whether that's a single provider's stable API or a gateway like RouteAI in front of several) is what keeps that answer cheap instead of expensive.

Here's the tool I referenced in this post: www.fastrouteai.com

DeepSeek API Price Just Changed (Aug 17): Peak/Off-Peak Rates, With Real Numbers

Felix — Mon, 17 Aug 2026 10:46:25 +0000

If you're calling the DeepSeek API and your bill looked different this morning, you're not misreading it. As of 00:00 Beijing time today (August 17), DeepSeek's new peak/off-peak pricing is live.

Here's the short version, then the actual numbers, then what I changed in my own setup.

What changed

DeepSeek announced the update on August 13. The mechanism is simple: peak hours are 9:00–12:00 and 14:00–18:00 Beijing time, off-peak is everything else, and off-peak pricing is set at half the peak rate.

That sounds like a discount structure. It mostly is — relative to peak. But peak pricing itself is meaningfully higher than the price DeepSeek was charging just five days earlier, when the official V4 Pro model launched with aggressive introductory rates. So the honest way to describe this is: DeepSeek didn't get cheaper at off-peak hours, it got more expensive at peak hours, and off-peak is the smaller price to pay.

Here's the official pricing, straight from DeepSeek's own pricing page, for both V4 Flash and V4 Pro:

Two things jump out once you see the real numbers side by side. First, peak is exactly 2x off-peak across every single tier for both models — the "half price off-peak" framing is literally accurate, not marketing rounding. Second, and more relevant if you're building anything serious: v4-pro's concurrency limit is 500, five times lower than v4-flash's 2500. If you're getting throttled during peak hours, that limit — not just the price — might be the bigger constraint on your architecture.

Compared to V4 Pro's introductory pricing when it launched on August 12 (cache-hit input was priced at a small fraction of a cent), even off-peak cache-hit input today is roughly 6x higher, and peak is over 10x higher. Cache-miss and output pricing moved up too, just less dramatically. If your app leans on cache hits (repeated system prompts, shared context, RAG pipelines reusing chunks), that's the number to watch — not the headline "price change" itself.

Why "just move workloads off-peak" isn't a full fix

The advice you'll see everywhere — including from DeepSeek's own docs — is to schedule non-real-time jobs during off-peak hours. That's genuinely good advice for batch jobs, evals, and anything async.

It's less useful if your product serves live traffic. Peak hours are 9–12 and 14–18 Beijing time specifically because that's when Chinese business-hour demand spikes — but if your users are in other timezones, or your app is interactive, "wait until after 6 PM Beijing time" isn't an option you can hand to a user waiting on a response.

So for anything latency-sensitive, off-peak scheduling helps your batch/background work, but it doesn't solve the problem for your live traffic.

What I actually changed

Two things, neither of which is "leave DeepSeek."

I split workloads by urgency. Anything that can tolerate a delay (nightly summarization, embedding refreshes, eval runs) now explicitly targets off-peak windows. That's a scheduling change, not a code change — just a cron adjustment.
I stopped hardcoding a single provider's base URL. This is the part that actually mattered. My integration used to call DeepSeek directly:

client = OpenAI(
    base_url="https://api.deepseek.com/v1",
    api_key=DEEPSEEK_KEY,
)

Since DeepSeek's API is OpenAI-compatible, swapping providers is mostly a base_url and api_key change — if you've abstracted that config out instead of hardcoding it, which I hadn't. I fixed that first, independent of any specific provider:

client = OpenAI(
    base_url=os.environ["AI_GATEWAY_BASE_URL"],
    api_key=os.environ["AI_GATEWAY_API_KEY"],
)

Once that was config-driven, I could route different request types to whichever model/provider combination made sense, without a redeploy every time a pricing page changes.

For the gateway piece specifically, I've been testing RouteAI — it's an OpenAI-compatible proxy in front of DeepSeek, Qwen, Kimi, GLM, and a few others. I'm not going to claim it's the cheapest option out there — I haven't benchmarked every gateway or reseller, and pricing across this space changes often enough that any "cheapest" claim would probably be outdated by the time you read this. What it did solve for me: I'm no longer locked into rewriting integration code every time a single provider adjusts pricing with five days' notice. Worth evaluating yourself against your own traffic pattern rather than taking my word for it.

If you only do one thing today

Pull your last week of DeepSeek usage logs and check what percentage of your token volume falls inside the 9–12 / 14–18 Beijing-time windows. That number tells you whether this pricing change is a rounding error or a real line-item for you — before you make any architecture decisions based on it.

TL;DR: DeepSeek's peak/off-peak API pricing went live Aug 17 (Beijing time) — peak is 9–12 and 14–18, exactly 2x off-peak across every tier, but even off-peak is higher than the Aug 12 launch price, especially for cached input tokens (~6x off-peak, ~10x+ at peak). V4 Pro's concurrency limit (500) is also 5x lower than V4 Flash's (2500) — worth checking if you're hitting throttling. Batch/async work can shift to off-peak; live traffic mostly can't. Abstracting your base_url/api_key behind config (optionally through an OpenAI-compatible gateway like RouteAI) makes future pricing changes a config edit instead of a rewrite.

More on how I set this up: www.fastrouteai.com