<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Long AI</title>
    <description>The latest articles on DEV Community by Long AI (@longai).</description>
    <link>https://dev.to/longai</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4256034%2F663325e8-5c54-4582-b9c5-d5808b82512f.png</url>
      <title>DEV Community: Long AI</title>
      <link>https://dev.to/longai</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/longai"/>
    <language>en</language>
    <item>
      <title>Why your LLM API invoice doesn't match the price page: a unit-by-unit guide</title>
      <dc:creator>Long AI</dc:creator>
      <pubDate>Sat, 21 Nov 2026 10:57:37 +0000</pubDate>
      <link>https://dev.to/longai/why-your-llm-api-invoice-doesnt-match-the-price-page-a-unit-by-unit-guide-4adn</link>
      <guid>https://dev.to/longai/why-your-llm-api-invoice-doesnt-match-the-price-page-a-unit-by-unit-guide-4adn</guid>
      <description>&lt;p&gt;Every "your price page is wrong" ticket we open ends up being the same thing: two different units compared as if they were one. Text models are billed per million tokens. Image models per picture. Video models per second of output. Audio per minute of input. Four different rulers — and a single line that says "¥0.8" without the unit attached is a bug waiting to happen.&lt;/p&gt;

&lt;p&gt;Here is the unit system behind the &lt;a href="https://ai.dijinlong.com/pricing" rel="noopener noreferrer"&gt;LONG AI price table&lt;/a&gt;, and the five places where a price page and an invoice can differ while both being correct.&lt;/p&gt;

&lt;h2&gt;
  
  
  1. Text is two prices, not one
&lt;/h2&gt;

&lt;p&gt;Input tokens (system prompt, conversation history, tool definitions, retrieved context) and output tokens are metered separately, and output usually costs several times more. A page that shows one number is almost always quoting the input price.&lt;/p&gt;

&lt;p&gt;Two consequences that surprise people:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;An agent loop is not one call. Ten steps resend the history ten times, so input billing roughly multiplies by the step count.&lt;/li&gt;
&lt;li&gt;Reasoning models bill the thinking tokens you never see. A short answer can carry a long invisible invoice.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;When comparing providers, compare output price. That is where the spread between channels is widest, and it is the number that decides your monthly bill.&lt;/p&gt;

&lt;h2&gt;
  
  
  2. Images are priced per picture — and the tier is part of the price
&lt;/h2&gt;

&lt;p&gt;The same image model often has several resolution or quality tiers at different unit prices. A quote taken at the cheapest tier sits next to a request made at 2K, and the invoice is arithmetically right. Pin the tier in the request, then compare prices at that tier.&lt;/p&gt;

&lt;h2&gt;
  
  
  3. Video is per second, audio is split in two directions
&lt;/h2&gt;

&lt;p&gt;Video generation is billed per second of output, sometimes with extra charges for reference frames or upscaling. Five seconds at ¥0.3/s is ¥1.5 before extras. Speech synthesis goes per character or per second of audio; transcription goes per minute of input, so a long recording costs more regardless of how short the transcript is.&lt;/p&gt;

&lt;h2&gt;
  
  
  4. Currency is a settlement detail, not a hidden fee
&lt;/h2&gt;

&lt;p&gt;A price quoted in CNY but settled in another currency moves with the exchange rate. That is why the &lt;a href="https://ai.dijinlong.com/en" rel="noopener noreferrer"&gt;English version of the site&lt;/a&gt; states the display currency and the settlement currency separately: displayed in USD, settled in CNY, with the rate visible.&lt;/p&gt;

&lt;h2&gt;
  
  
  5. Discounts are per model, not per site
&lt;/h2&gt;

&lt;p&gt;"Up to 90% off" describes the deepest quote on the cheapest model in the catalogue. Your model has its own seller quotes, and those are the ones that apply. A site-wide average is useless for budgeting, which is why &lt;a href="https://ai.dijinlong.com/model-plaza" rel="noopener noreferrer"&gt;the model plaza&lt;/a&gt; shows per-model quotes with the channel behind each one.&lt;/p&gt;

&lt;p&gt;Two smaller cases in the same family: retries (a failed upstream request should be billed once — if your client retries on its own, you made two billable requests) and tiers inside one model (long-context variants and peak multipliers change the unit price under the same model name).&lt;/p&gt;

&lt;h2&gt;
  
  
  The one-minute check
&lt;/h2&gt;

&lt;p&gt;Before opening a ticket, write down four things: the unit (token / image / second / minute), the tier you requested, whether the counter is input-only or input plus output, and the settlement currency. If all four match between the price page and the invoice, the difference is almost always your own retry loop. If they do not match, it is a documentation bug worth reporting.&lt;/p&gt;

&lt;p&gt;Every archived model on &lt;a href="https://ai.dijinlong.com/model-docs" rel="noopener noreferrer"&gt;LONG AI&lt;/a&gt; lists its billing unit, its tier and both sides of the price for exactly this reason. Units are boring until they are expensive.&lt;/p&gt;

</description>
      <category>api</category>
      <category>llm</category>
      <category>openai</category>
      <category>billing</category>
    </item>
    <item>
      <title>Serving 100+ LLMs behind one OpenAI-compatible endpoint: the engineering notes</title>
      <dc:creator>Long AI</dc:creator>
      <pubDate>Fri, 20 Nov 2026 16:11:08 +0000</pubDate>
      <link>https://dev.to/longai/serving-100-llms-behind-one-openai-compatible-endpoint-the-engineering-notes-p3h</link>
      <guid>https://dev.to/longai/serving-100-llms-behind-one-openai-compatible-endpoint-the-engineering-notes-p3h</guid>
      <description>&lt;p&gt;Most "one API for every model" services still route you to a single upstream vendor per model. The harder problem is what happens when that vendor throttles you at 3 a.m.: retries stack up, latency explodes, and your bill keeps growing for requests that never returned a token.&lt;/p&gt;

&lt;p&gt;I have been building an OpenAI-compatible gateway that answers a different question: what if the supply side were a marketplace instead of a fixed vendor list? Here are the engineering notes from that build, the parts that were genuinely hard, and the trade-offs I would not make again.&lt;/p&gt;

&lt;h2&gt;
  
  
  1. One endpoint, many sellers
&lt;/h2&gt;

&lt;p&gt;Sellers list spare model capacity with their own prices. A request arrives at &lt;code&gt;POST /v1/chat/completions&lt;/code&gt;, the gateway picks a seller whose quote matches the model and the request shape, forwards it, and settles the difference afterwards. Clients keep the OpenAI SDK - only &lt;code&gt;base_url&lt;/code&gt; and &lt;code&gt;api_key&lt;/code&gt; change.&lt;/p&gt;

&lt;p&gt;Keeping it protocol-compatible sounds trivial until you hit the small divergences: &lt;code&gt;stream_options&lt;/code&gt;, tool-call deltas, &lt;code&gt;logprobs&lt;/code&gt;, &lt;code&gt;response_format&lt;/code&gt;, and providers that quietly drop &lt;code&gt;max_completion_tokens&lt;/code&gt;. Every adapter needs a conformance test against the OpenAI schema, otherwise clients break in ways users cannot debug.&lt;/p&gt;

&lt;h2&gt;
  
  
  2. Failover is a scheduling problem, not a retry loop
&lt;/h2&gt;

&lt;p&gt;The naive fix for 429s is exponential backoff. That makes p99 worse, because every client retries into the same saturated seller. What worked better:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Track per-seller health separately for each model, not per seller overall.&lt;/li&gt;
&lt;li&gt;Treat 429, 5xx and mid-stream disconnects as routing signals, not errors to hide. Re-route to a healthy seller on the first token; if tokens have already streamed, finish the stream and mark the seller degraded.&lt;/li&gt;
&lt;li&gt;Cap concurrent in-flight requests per seller so you saturate but do not stampede.&lt;/li&gt;
&lt;li&gt;Never bill a request that produced no tokens. Billing on delivery, not on attempt, removes the incentive to silently retry.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Across roughly 51 upstream providers, this moved rate-limit failures from a few percent of calls to a rounding error in daily dashboards.&lt;/p&gt;

&lt;h2&gt;
  
  
  3. Transparent pricing beats cheap pricing
&lt;/h2&gt;

&lt;p&gt;Discounts are easy to advertise and hard to operate. What developers actually ask for is a per-request price they can see before they send it, and a line item afterwards that matches. That means the gateway has to price the request (tokens, model, region, cache) before routing, then reconcile with the delivered usage - including the annoying case where a seller reports different usage than the tokenizer predicted.&lt;/p&gt;

&lt;h2&gt;
  
  
  4. Agents need a different surface than chat
&lt;/h2&gt;

&lt;p&gt;Half of the traffic now comes from agents and IDEs rather than chat UIs. They want model discovery and tool-calling with schemas, not a wall of text. Shipping an MCP server next to the HTTP API turned out to be the cheapest integration path: the agent lists models, calls chat completions, and never needs a bespoke SDK.&lt;/p&gt;

&lt;h2&gt;
  
  
  What I would do differently
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;Build the conformance suite before the second adapter. Retrofitting nine adapters is unpleasant.&lt;/li&gt;
&lt;li&gt;Store every upstream response body for 24 hours. Debugging "the model answered differently through your gateway" is impossible without the raw payload.&lt;/li&gt;
&lt;li&gt;Decide early whether you optimize for latency or for price. Trying to win both at once produces a router nobody can reason about.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;If you want to poke at the result, it is running at &lt;a href="https://ai.dijinlong.com" rel="noopener noreferrer"&gt;LONG AI&lt;/a&gt; - 100+ live LLM and video models, up to 78% below list price, OpenAI SDK compatible. The MCP server is open source, and I am happy to answer questions about the routing or settlement design in the comments.&lt;/p&gt;

</description>
      <category>ai</category>
      <category>webdev</category>
    </item>
  </channel>
</rss>
