DEV Community: Oskar

Tokenminning (for people who write openai.chat.completions.create)

Oskar — Fri, 19 Jun 2026 17:49:24 +0000

Your team shipped the agent. The demo was magic. Then finance asked why one engineer's coding assistant burned $40k in a month while the feature backlog looked the same.

Welcome to the hangover after tokenmaxxing — the habit of maximizing raw LLM usage, often celebrated on internal leaderboards, without tying token spend to shipped outcomes.

The counter-move has a name: tokenminning. Journalists call it token minimizing. Engineering teams call it tokenminning when they formalize it — metering, model routing, context hygiene, hard budgets, and CI enforcement.

This is the developer's cut. Philosophy lives in the Manifesto. Law lives in the Constitution. This post is what you ship before Friday.

TL;DR

A token is a billable unit (~word fragment). Providers charge for input and output separately.
In agent loops, input usually dominates — full history and tool dumps are re-sent every turn.
Tokenminning = same output quality, minimum viable token spend.
Order matters: meter → prompt hygiene → model routing → caching → context hygiene → caps.
Soft guidance produces soft budgets. Hard ceilings belong in code.

Tokenmaxxing vs tokenminning (for people who write `openai.chat.completions.create`)

	Tokenmaxxing	Tokenminning
Goal	Maximize token volume	Maximize value per token
Model choice	Frontier default	Cheapest model that passes your quality bar
Prompts	Grow by accretion	Versioned, schema-enforced, audited
Agents	Run until done	Session budget + graceful degradation
Measurement	"Who used the most AI?"	USD per feature, user, session
Enforcement	Slack reminders	CI gates, runtime caps, immutable ledgers

Meta, Uber, Walmart, and Amazon all reversed course in 2026 — caps, limits, leaderboard removals. Uber burned through its projected annual AI budget in four months. The NYT documented the shift. Your invoice is not a personal failure. It is a missing constraint layer.

Why your bill exploded (three bugs in production, not one)

1. Hardcoded frontier models

// tokenmaxxing
const response = await openai.chat.completions.create({
  model: 'gpt-4', // every request, regardless of task
  messages,
});

Classification, extraction, and JSON transforms rarely need frontier tiers. Teams report 60–90% savings from model routing alone — after they benchmark on real traffic.

// tokenminning
const response = await router.complete({
  capability: 'classify-intent',
  messages,
});
// router maps capability → cheapest model passing SLA

2. Agent loops with unbounded context

A chat turn might cost hundreds of tokens. An agent that retains full tool payloads, never summarizes state, and re-sends everything each step can cost tens of thousands per session. Per-token prices look flat; context inflation makes total tokens per request grow 10×. See what is context inflation.

3. Prompts treated as Google Docs

Every token in a system prompt is billed on every request. A prompt that grows 200 tokens sounds free. At 10M requests/month, it is a line item.

# scaffolding waste (delete these)
"You are a helpful assistant."
"Take a deep breath and think step by step."
// TODO: update this when we switch models

Replace prose formatting rules with schema enforcement:

{
  "response_format": {
    "type": "json_schema",
    "json_schema": {
      "name": "summary_response",
      "schema": {
        "type": "object",
        "properties": {
          "title": { "type": "string" },
          "summary": { "type": "string", "maxLength": 500 }
        },
        "required": ["title", "summary"]
      }
    }
  }
}

Schema outputs validate in CI. Vibes do not. Full guide: prompt hygiene.

The optimization sequence (do not skip steps)

From Where to start on tokenminning.ai:

Step	Technique	Typical impact
0	Metering + attribution	Prerequisite
1	Prompt hygiene	~20–25% on high-volume templates
2	Model routing	60–95% depending on task mix
3	Prompt caching	41–80% on stable prefixes
4	Context hygiene	40–60% in agentic workloads
5	Output control + RAG discipline	15–40%
6	Semantic caching	Eliminates inference on cache hits

Anti-pattern: optimizing prompts before metering. You cannot prove savings or find the real bottleneck.

Anti-pattern: trimming max_tokens on output when input is 80% of spend.

Start this week: five PRs that actually move the needle

PR 1 — Log input and output tokens separately

Before changing anything, answer: "What did this user action cost, in USD, when it completed?"

Minimum viable instrumentation:

const result = await model.complete({ messages, max_tokens: 1024 });

await ledger.record({
  feature: 'document-analysis',
  userId: session.userId,
  sessionId: session.id,
  inputTokens: result.usage.prompt_tokens,
  outputTokens: result.usage.completion_tokens,
  model: result.model,
  usd: priceResolver.toUsd(result.usage, result.model),
});

Implement Article I: immutable metering before you optimize prompts.

PR 2 — Audit top 10 templates for scaffolding

Pull highest-volume or highest-cost prompts. Remove signal-free tokens. Diff token counts before/after. Target 20–25% on bloated templates.

PR 3 — Add `max_tokens` to every bounded task

If the answer shape is known, cap it in the API call:

await model.complete({
  messages,
  max_tokens: 256, // not a paragraph essay for a boolean classification
});

PR 4 — Stable prefix for caching

Static system prompts and tool definitions go first. Dynamic user content goes after.

const messages = [
  { role: 'system', content: STABLE_SYSTEM_PROMPT }, // cacheable prefix
  { role: 'user', content: dynamicUserInput },
];

Enable provider prefix caching. Guide: prompt caching.

PR 5 — Session budgets with graceful degradation

Hard ceilings are architecture, not suggestions. From Article IV:

const session = createSession({
  userId: 'usr_123',
  feature: 'document-analysis',
  tokenBudget: 50_000,
  usdCeiling: 0.5,
});

// at 90% budget: inject wrap-up directive
// at 100%: return best partial with status `completed_degraded`
// never: abort mid-sentence with a raw 429

No budget, no session. No "internal testing" exceptions in prod.

Model routing: cascade, don't default upward

Request → small model
            ↓ quality check passes → return
            ↓ fails → mid-tier model
                        ↓ passes → return (log escalation)
                        ↓ fails → frontier (log justification)

Every frontier call needs an auditable reason:

Complexity classifier score above threshold
Cheaper model failed quality checks (logged)
User explicitly chose a billed "high quality" tier

Not acceptable: developer preference, "the prompt is long," no routing layer at all.

Benchmark on your data — 1,000 real inputs, parallel runs, compare quality × cost × latency. Blog post savings are not your savings.

What tokenminning is not

Stripping required instructions to shave counts (noise removal ≠ signal removal)
Using the smallest model for everything without benchmarks
Fragile micro-edits that save 12 tokens and break edge cases
Asking engineers to "write shorter prompts" without infrastructure

Tokenminning is building systems where token bloat physically cannot deploy and every inference call maps to a ledger.

What is the most expensive inference call pattern in your stack right now — and do you actually know its USD cost per session?

Launching @ai-billing package for Vercel AI SDK

Oskar — Tue, 16 Jun 2026 17:44:48 +0000

Problem worth solving

If you are building an AI-powered product, you’ve likely run into the "billing wall." Unlike traditional SaaS, AI products are often billed per inference, token, or request. Building a robust metering and billing system that handles real-time usage data, balances, and provider integrations is a major distraction from your core product.

`pnpm add @ai-billing/openai`

@ai-billing is an npm package designed to handle the heavy lifting of usage-based billing in Vercel AI SDK. It does two things:

1. Adds cost in dollars to your provider

2. (optionally) Sends usage events to billing providers

How it works?

The package acts as a middleware for your usage telemetry. Instead of writing custom logic for every provider, you use a unified interface to track consumption and sync it with your chosen billing platform.

When working on it, we prioritized:

being provider agnostic: Easily switch or add billing providers as your needs scale.
having ready-to-use templates: We’ve provided full-stack examples for popular combinations like OpenRouter + Polar or OpenAI + Stripe.
thinking of the developer experience: Less time configuring APIs, more time building features.

Basic example - adding cost to OpenAI calls

import {
  UIMessage,
  convertToModelMessages,
  generateText,
  wrapLanguageModel,
} from 'ai';
import { createOpenAI } from '@ai-sdk/openai';
import { createOpenAIMiddleware } from '@ai-billing/openai';
import {
  consoleDestination,
  createObjectPriceResolver,
  ModelPricing,
} from '@ai-billing/core';

const openai = createOpenAI({
  // eslint-disable-next-line turbo/no-undeclared-env-vars
  apiKey: process.env.OPENAI_API_KEY,
});

const customPricingMap: Record<string, ModelPricing> = {
  'gpt-5': {
    promptTokens: 1.25 / 1_000_000,
    completionTokens: 10.0 / 1_000_000,
    inputCacheReadTokens: 0.125 / 1_000_000,
  },
  'gpt-4o': {
    promptTokens: 5.0 / 1_000_000,
    completionTokens: 15.0 / 1_000_000,
  },
};

const priceResolver = createObjectPriceResolver(customPricingMap);

const billingMiddleware = createOpenAIMiddleware({
  destinations: [consoleDestination()],
  priceResolver: priceResolver,
});

export async function POST() {
  try {
    const messages: UIMessage[] = [
      {
        id: 'test-gen-1',
        role: 'user',
        parts: [{ type: 'text', text: 'What is the capital of Sweden?' }],
      },
    ];

    const model = 'gpt-5';

    const wrappedModel = wrapLanguageModel({
      model: openai(model),
      middleware: billingMiddleware,
    });

    const result = await generateText({
      model: wrappedModel,
      messages: await convertToModelMessages(messages),
    });

    return Response.json(result);
  } catch (error) {
    console.error('Generate Error:', error);
    return Response.json({ error: (error as Error).message }, { status: 500 });
  }
}

Full-stack projects built with @ai-billing

Name	Demo Link	Repo	Deploy
Chatbot (OpenRouter + Polar)	View Demo	GitHub	Deploy with Vercel
Chatbot (OpenAI + Polar)	View Demo	GitHub	Deploy with Vercel
Chatbot (Stripe)	View Demo	GitHub	Deploy with Vercel

One more time, repo link

https://github.com/narevai/ai-billing

FinOps X 2026 recap: the great token panic

Oskar — Mon, 15 Jun 2026 07:25:27 +0000

If you were following the FinOps X 2026 conference that just wrapped up in San Diego (June 8–11, 2026), you probably noticed a massive shift. The discipline has officially outgrown its cloud-cost-management origins. While the expo floor had the usual vendor announcements, the real story for those of us building AI products was what the keynotes dubbed: "The Great Token Panic."

With Gartner forecasting $2.59 trillion in AI spending in 2026, the era of treating AI costs as an unattributed R&D expense is over. Here are the core takeaways from the event and why they matter for engineering teams.

1. The Arrival of Token Economics

For the last decade, FinOps was about CPU hours, memory, and reserved instances. Now, the atomic unit of technology spend is the token.
Because reasoning and agentic workloads consume 5 to 30 times more tokens per task than simple chat interactions, enterprise consumption is swamping the gradual decline in per-token list prices. AI token costs are variable, hard to forecast, and prone to rapid volatility. Recognizing this, the Linux Foundation and FinOps Foundation used the event to launch the Tokenomics Foundation. It's a massive industry effort uniting hyperscalers and enterprises to establish open standards for AI cost management and billing data.

2. Agentic FinOps and Moving Beyond Dashboards

We are officially moving past static dashboards and natural-language wrappers on billing exports. The conference floor was dominated by the concept of "Agentic FinOps" - autonomous systems that don't just report on costs, but actively investigate and orchestrate remediation.
Instead of analysts manually digging through blast radiuses, the next generation of tooling uses AI to continuously scan Kubernetes clusters, cloud environments, and AI service consumption to automatically route root-cause analysis straight into engineering workflows.

3. FinOps Scopes: Expanding the Definition

The State of FinOps 2026 report made it clear: 90% of teams now manage SaaS costs or plan to within the year. The FinOps umbrella is expanding to cover private clouds, data platforms, software licensing, and, crucially, multi-model AI endpoints.
Optimization efforts fail without accurate allocation. If you can't attribute the cost of an Anthropic or OpenAI API call to a specific product feature or user, you can't optimize it. Teams are shifting to virtual tagging and strict governance frameworks to isolate these costs before they scale.

4. Widespread Adoption of FOCUS

The FinOps Open Cost and Usage Specification (FOCUS) continues to gain traction as the universal billing format. As organizations juggle multiple AI providers, standardizing billing data into a universal schema is the only way to maintain portability. If your infrastructure emits FOCUS-compliant data, you can seamlessly integrate new ecosystem tools without having to rebuild your underlying data pipeline.

How we cut our Dagster Cloud bill by 60% by collapsing our dbt assets

Oskar — Mon, 15 Jun 2026 05:17:03 +0000

If your team is using Dagster Cloud's Solo or Starter tiers, the May 2026 pricing update probably gave you a bit of a shock. By removing the monthly credit allowance and charging for each step-executed asset materialization, workspaces that used to cost $10–$100 a month suddenly spiked into the hundreds or even over a thousand dollars.

At Narev, we love the developer experience of Dagster, but our 24/7 schedule was driving up our bill to the point where self-hosting Airflow was starting to look attractive.

Before jumping ship, we audited our setup and found a massive quick win: we were paying Dagster to coordinate a graph that dbt already understood. Here is how we fixed it and cut our orchestration bill by 60%.

The culprit: `@dbt_assets`

The standard dagster-dbt component is beautiful. You use @dbt_assets, and Dagster maps every single dbt model and seed into an individual node in your Dagster UI.

But under the new pricing model, every asset materialization has a cost (around $0.035 to $0.040 per credit). If you have a couple hundred dbt models running on an hourly or daily partition, you are burning credits just to have Dagster say, "Yep, dbt built this view."

The fix: One asset, One command

We decided to collapse our entire dbt project into a single plain @asset that just shells out to dbt. Same dbt logic, same data, but far fewer orchestration steps.
Here is what our refactored asset looks like:

import json
from dagster import AssetExecutionContext, asset
from dagster_dbt import DbtCliResource
from etl.defs.partitions import daily_partition
from etl.defs.utils import DeploymentType, get_deployment_type

@asset(
    partitions_def=daily_partition,
    group_name="transformers",
    deps=["dw_load"],
)
def dw_transform(context: AssetExecutionContext, dbt_resource: DbtCliResource):
    time_window = context.partition_time_window
    deployment = get_deployment_type()
    dbt_target = "prod" if deployment == DeploymentType.PROD else "dev"

    dbt_vars = {
        "start_date": time_window.start.strftime("%Y-%m-%d"),
        "end_date": time_window.end.strftime("%Y-%m-%d"),
    }

    args = ["build", "--target", dbt_target, "--vars", json.dumps(dbt_vars)]

    # The crucial part:
    result = dbt_resource.cli(args).wait()

    if not result.is_successful():
        raise Exception("dbt build failed")

    return "All models updated"

The gotcha: Drop the context

If you try this, there is one massive trap you need to avoid. Do not pass context=context to dbt_resource.cli().

If you do result = dbt_resource.cli(args, context=context).wait(), the dagster-dbt package will still emit individual materialization events behind the scenes for every single model. You will end up paying for per-model orchestration without getting the per-model UI benefits.

Using .wait() instead of .stream() and dropping the context ensures you only pay for one materialization per partition per run.

The trade-offs

This optimization isn't entirely free. By hiding dbt's complexity from Dagster, you are giving up a few things:

No per-model UI lineage: You won't see individual staging models in the Dagster graph.
Coarser alerting: If a single model fails, the entire dw_transform asset goes red. You'll have to check the dbt logs to see exactly which model broke.

For us, this was a no-brainer. We rely on dbt docs and our warehouse for deep lineage anyway, and we only need Dagster to know: "Extract done → Load done → Transform done."

If you want to see exactly how this changes downstream asset dependencies and how to wire up your jobs with this new pattern, I wrote a full breakdown on our blog (along with a drop-in Cursor prompt to automate the refactor for your workspace).

Read the full guide on the Narev blog

It's time to get familiar with what FinOps for AI is

Oskar — Sun, 31 May 2026 17:05:10 +0000

No Generative AI was used in writing this article. If you enjoyed reading it, please let me know. It helps me dedicate more time to things that resonate with my readers.

It took Uber 4 months to burn the entire AI budget for 2026. The cost if AI, the ROI and the question of AI investment have entered the picture.

Uber’s COO, Andrew MacDonald, said in his recent interview that the costs became “harder to justify” because the link between spending on AI tokens and creating more useful features “was not there”.

It started with all you can eat subscriptions.

Since the early days of tech, we’ve been conditioned to expect all you can eat subscriptions. For a month, you could watch Netflix non-stop, for 24 hours, 7 days a week and you’d still pay the same price as if you watched only one.

That’s a great deal!

In fact, such a good deal, that we see it everywhere. On Netflix, on Amazon Prime, pretty much all the software.

The magic of zero marginal cost (in digital goods)

In traditional businesses, selling one more unit of physical product costs the company an extra unit of resources.

Each flower bouquet of 5 roses sold requires 10 roses. Selling 100 bouquets will require 100 bouquets x 5 roses = 500 roses. This is called marginal cost.

But that’s not the case in the digital world!

Once Netflix pays the massive upfront cost to produce Stranger Things ($30 million an episode), it costs them exactly the same amount of money whether 10 people watch it or 100 million people watch it. As long as they get enough subscriber to cover the initial investment, every new subscriber after that is 100% pure profit. This is a zero marginal cost.

Large Language Models are more like a flower shop

The cost of generating tokens requires significant amount of energy. This makes token generation expensive.

This week news cycle brought us a story about an anonymous company accidentally burning through $500M of tokens.

This very nature of LLMs requires that all-you-can-eat subscriptions have to be subsidized.

Why is it a good thing?

Ultimately, it’s about the unit economics.

We went through a very similar pain with a shift to cloud computing. The solution was to not abandon AWS completely due to spiraling bills.

Instead we built dashboards, tracked usage, and an entire discipline was born: FinOps.

When the costs are real, the value has to be real too.

Where would you start?

I'll leave you with the FinOps for AI framework from Narev.

It was never easier to make software. It was never harder to make good software

Oskar — Sun, 31 May 2026 11:23:21 +0000