DEV Community: you.bot

Cut AI API Costs by Up to 80% with a Safe Provider Fallback

you.bot — Wed, 29 Jul 2026 09:20:45 +0000

Lower AI API prices are useful only when the routing design preserves predictable behavior.

The practical pattern is:

use the lower-cost route as primary;
keep the existing provider integration as fallback;
fall back only after a confirmed terminal outcome; and
reconcile unknown task states instead of duplicating them.

With you.bot, failed or resultless media tasks are refunded. For model configurations where its listed price is below the standard comparison price, a successful primary call costs less, while a refunded failure followed by fallback has the same model execution cost as calling the standard provider directly.

Price snapshot: up to 80% less

The largest differences are easiest to understand side by side. This selection includes both the highest-saving configurations and widely used text, image, and video models:

Model and configuration	Standard comparison	you.bot listed price	Max savings with bonus
Grok Imagine Video 1.5, image-to-video, 480p (per second)	$0.0800	$0.0178	80%
GPT Image 2, text-to-image, 1K (per generation)	$0.2190	$0.0564	77%
GPT-5.6 Luna, input (per 1M tokens)	$1.0000	$0.5097	54%
Claude Sonnet 5, input (per 1M tokens)	$3.0000	$1.5367	53%
Gemini 3.6 Flash, input (per 1M tokens)	$1.5000	$0.7871	52%
Gemini Omni, 4s 4K video without video input (per generation)	$1.8667	$1.0565	49%

Max savings with bonus includes the qualifying 10% bonus on the $1,250 top-up. It is calculated as 1 - ((you.bot listed price / 1.10) / standard comparison price) and rounded to the nearest whole percent. Without that bonus, the corresponding base-price differences are 77.8%, 74.2%, 49.0%, 48.8%, 47.5%, and 43.4%.

Prices are a point-in-time snapshot and vary by exact model, operation, resolution, duration, and billing unit. Check the current you.bot price table before making a routing decision.

The cost model

Let:

Y = you.bot price
S = standard provider price
p = primary-route success rate

Then:

route_first_cost = pY + (1 - p)S
direct_only_cost = S
savings = p(S - Y)

If Y < S, the route-first architecture reduces blended spend whenever some requests succeed through the primary route.

For example, the current GPT Image 2 snapshot lists 1K text-to-image generation at $0.0564 through you.bot and $0.219 as the standard comparison price, a 74.2% difference for that configuration.

Do not generalize one row to the entire catalog. Compare the same model, operation, resolution, duration, and billing unit.

Why a normal timeout is not a fallback signal

Assume an application creates an asynchronous video task and waits 60 seconds. The client times out, but the task continues running. If the application immediately creates the same video with another provider, it can receive and pay for two outputs.

The correct response to an unknown state is reconciliation:

type FinalResult =
  | { state: "completed"; outputUrl: string; route: "you.bot" | "fallback" }
  | { state: "failed"; reason: string };

type PrimaryState =
  | { state: "completed"; outputUrl: string }
  | { state: "terminal_failure"; reason: string }
  | { state: "running"; taskId: string }
  | { state: "unknown"; taskId?: string };

async function generateWithSafeFallback(input: GenerationInput): Promise<FinalResult> {
  const primary = await createWithYouBot(input);

  if (primary.state === "completed") {
    return { ...primary, route: "you.bot" };
  }

  const reconciled =
    primary.state === "running" || primary.state === "unknown"
      ? await reconcileYouBotTask(primary.taskId)
      : primary;

  if (reconciled.state === "completed") {
    return { ...reconciled, route: "you.bot" };
  }

  if (reconciled.state !== "terminal_failure") {
    throw new Error("Primary execution state is unknown; fallback is not safe yet.");
  }

  const fallback = await createWithExistingProvider(input);
  return { ...fallback, route: "fallback" };
}

The adapter functions above are intentionally separated. Each one should normalize provider-specific responses into a small internal state machine.

Persist state before polling

Store at least:

interface RoutedGeneration {
  localRequestId: string;
  primaryTaskId?: string;
  modelId: string;
  primaryState: "creating" | "running" | "completed" | "terminal_failure" | "unknown";
  fallbackState: "not_started" | "running" | "completed" | "failed";
  fallbackReason?: string;
  primaryPriceUsd?: number;
  fallbackPriceUsd?: number;
  createdAt: string;
  updatedAt: string;
}

Persist the primary task ID before the worker begins polling. That lets a replacement worker reconcile the original task instead of creating a new one after a crash.

Separate user latency from execution state

A product can stop making the user wait without declaring the generation failed.

For example:

after 45 seconds, return a pending response to the frontend;
continue reconciliation in a background worker;
notify the user through a webhook, WebSocket, or status page;
use a longer execution deadline before considering manual intervention.

This keeps user experience responsive without producing duplicate billable work.

Use documented failure classes

Create an allowlist of outcomes that can trigger fallback. Examples might include:

task reached a documented terminal failed state;
completed response contains no usable result and is refund-eligible;
create request was definitively rejected before acceptance; or
an operator has reconciled an otherwise unknown task.

Do not use a broad catch block that sends every exception to the fallback provider.

Authentication errors, invalid inputs, insufficient balance, and policy failures may also fail through the fallback route. Retrying those problems can add latency without improving completion.

Measure blended cost

The metric that matters is not the advertised price in isolation:

blended cost per successful output =
  (primary charges + fallback charges) / successful outputs

Track it by exact model configuration together with:

primary completion rate;
fallback rate;
unknown-state rate;
end-to-end latency;
refunded-task reconciliation; and
duplicate-output rate.

The goal is both economic and operational: capture the lower primary price while preserving an independent route for continuity.

The current you.bot price table is available at https://you.bot/pricing, and its task lifecycle is documented at https://you.bot/docs.

What I learned building one API in front of 70+ AI models (LLM, image, video, music)

you.bot — Tue, 21 Jul 2026 14:14:47 +0000

Disclosure up front: I work on you.bot, the gateway described here. This is a build-log about the engineering problems, not a sales pitch — the patterns apply whether you build your own gateway or use one.

The problem that started it

Every AI feature I shipped came with its own tax. The LLM had one SDK and one dashboard. The image model had another. Video was a third provider with a totally different async flow. Music was a fourth. Four SDKs, four invoices, four sets of retry logic, four ways to handle failures.

The moment I wanted to A/B two image models, or swap an LLM for a cheaper one, I was rewriting integration code and reconciling another bill. So we built a single endpoint that fronts 70+ models across text, image, video and music, where switching models is a one-line change. Here's what turned out to be hard.

1. Every provider has a different idea of "a request"

LLMs are mostly request/response. Image is sometimes sync, sometimes not. Video is almost always a long-running job — you submit, you poll or wait for a webhook, and it can take minutes. Music (like Suno) is the same: you don't get a song back on the open socket.

If you expose these differences to the caller, you've moved the complexity onto them. The decision that mattered most: normalize everything to one create-then-poll task shape. You create a task, then poll it until it succeeds (text models return their result inline in the create response, so you can skip the poll):

POST https://you.bot/api/v1/generate
Authorization: Bearer YOUR_API_KEY

{ "modelId": "suno-v4-5",
  "input": { "prompt": "lofi beat, rainy night", "title": "Rainy Lofi", "style": "lofi, chill" } }

→ { "taskId": "t_abc", "creditsCharged": 13 }

GET https://you.bot/api/v1/task/t_abc?model=suno-v4-5

→ { "state": "success", "resultUrls": ["https://…"] }

Same two-call contract for a fast LLM call and a three-minute video render. Callers can poll, or pass a callbackUrl and get a signed webhook on completion. Switching from music to video (Kling) or image (GPT Image 2) is just a different modelId and a different input.

2. The modelId is the only thing that should change

The whole point is that trying a new model shouldn't be a project. The surface a caller touches is deliberately tiny: one endpoint, one Authorization header, and a modelId.

import requests

def generate(model_id, inp):
    r = requests.post(
        "https://you.bot/api/v1/generate",
        headers={"Authorization": f"Bearer {API_KEY}"},
        json={"modelId": model_id, "input": inp},
    )
    return r.json()   # -> {"taskId": "...", "creditsCharged": ...}

# swap models by changing one field
# (input fields vary per model — each model page documents its own)
generate("gemini-3-1-pro",  {"prompt": "..."})
generate("nano-banana-pro", {"prompt": "..."})
generate("kling-3-0",       {"prompt": "..."})

Every model has a page with its own inputs, price and an in-browser playground (e.g. Gemini 3.1 Pro, Nano Banana Pro) so you can test with your own prompt before writing a line of integration code. That "try before you wire it up" step removed most of our support questions.

3. Only charging for successful output changes your whole billing model

If a generation fails, errors, or comes back empty, the user should not pay for it. Sounds obvious; it's annoying to implement because "success" is defined differently per provider, and refunds have to be idempotent so a retried webhook doesn't double-credit.

We ended up with a per-model success predicate and a ledger where every debit can be reversed by task id. Credits come from one prepaid wallet shared across all models (the create response tells you creditsCharged per call), so there's no per-provider balance to juggle — and because you only pay on success, the effective price drops below the sticker price.

4. Reliability: keep the caller's existing key as a fallback

The honest objection to any gateway is "now you're a single point of failure between me and the model." The pattern that made people comfortable: let the app keep its existing provider key as a fallback route. Primary traffic goes through the gateway; if a call fails, it falls back to the provider directly. Multi-provider routing targets high uptime, but the caller never bets reliability on us alone.

5. Make the docs machine-readable

A lot of "how do I call X model" questions now start in ChatGPT or Perplexity, not Google. So every model page also ships a plain-markdown version (append /md to the URL, e.g. /models/suno-v4-5/md) that an LLM can read cleanly. If assistants are going to answer developer questions anyway, the least you can do is give them accurate, structured source material.

What I'd tell my past self

Normalize to one create-then-poll task shape early. Retrofitting it later is painful.
Define "success" per provider before you promise charge-on-success billing.
Make refunds idempotent from day one.
A playground per model is worth more than another paragraph of docs.
Assume an LLM will read your docs before a human does.

The model catalog with per-model pricing and playgrounds is at you.bot/market. Happy to answer architecture questions in the comments — especially on fallback-routing and charge-on-success, since those drove the most debate on our side.