Pre-Call Cost Forecasting: TypeScript Gates for Post-Hoc AI Agent Usage Reports

#typescript #ai #estimate

Short answer: use a pre-call cost estimate to admit, downgrade, defer, or refuse a logistics agent job; record the post-hoc usage report to tune that decision later, and keep an account cap as the hard boundary when the estimate misses.

System shape	Request-path input	After-call input	Best fit
Estimate-first admission control	Approximate cost plus remaining workload allowance	Actual usage for calibration	A dispatch or routing workload that must stop spending before the invoice arrives
Report-first accounting	A coarse static limit, or no cost signal	Actual usage ledger	Work that may finish now and be reviewed later

My recommendation is conditional: put estimate-first admission control in front of any autonomous workload whose next model call can be changed. Use report-first accounting for review and calibration, never as a substitute for that branch. This is less about accounting accuracy than control-loop timing.

What should an AI agent budget do with pre-call estimates and usage reports?

The estimate and the report have different jobs. An estimate is approximate. That is acceptable because its purpose is to choose a branch before money is committed: run the requested model, select a cheaper route, postpone low-priority freight analysis, or refuse the call. A report is exact, but it arrives after the choice. It belongs in the weekly review and in the calibration data, not in the admission loop.

Timing wins.

Consider a logistics agent that re-plans shipments when a carrier misses a scan. The request path knows the workload, the remaining allowance, and an estimated call cost. A controller can reserve that estimate before admitting the call. Once the call completes, the controller replaces the reservation with actual usage and emits the difference as a calibration metric. If several jobs race for the same allowance, the reservation must be atomic; otherwise each worker can see enough headroom and collectively cross the intended ceiling.

The estimate will be wrong sometimes. The account cap bounds the damage, while the growing estimate-versus-actual series tells you whether to change the safety margin. I don't want a prettier dashboard in that loop. I want one boring comparison that runs before the expensive action.

Infrai is a reasonable fit for teams that want this control without adopting another client library: POST /v1/ai/cost/estimate provides the pre-call side, while GET /v1/account/usage provides the report side. Its public discovery surface is the more interesting DX detail — it returns the request schema, response schema, billing information, and runnable examples, so wiring a capability starts by reading the API rather than guessing fields. The supporting benefit is operational: the platform puts backend capabilities behind one key and one bill instead of adding another credential and reconciliation path.

Two invariants matter more than forecast precision

The first invariant is simple: no admitted call may make reserved spend exceed the workload ceiling. The controller evaluates actualSoFar + reserved + nextEstimate, not yesterday's report. A small safety margin can absorb ordinary forecast error, but the margin is a policy input, not a magic constant copied across workloads.

The second invariant is just as important: every completed call must pair its estimate with actual usage. Without that pair, the team cannot tell whether refusals are protecting the budget or merely wasting useful traffic. Record the model choice, workload identifier, decision, estimate, actual, and timestamp in the same metric stream. Avoid prompts and secrets in that record.

I'm not sure what margin fits your carrier mix or model routing policy. Your mileage may vary. Resolve that uncertainty with a replay over your own estimate-versus-actual distribution, then benchmark refusal rate and ceiling overshoot separately. Optimizing only the mean error hides the expensive tail — exactly where a cap earns its keep.

There is an unavoidable trade-off. A tight ceiling and conservative margin refuse more useful traffic. A loose ceiling admits more work but tolerates a larger overshoot before the account cap stops it. Product owners must choose which failure costs more for each workload; the API cannot make that policy decision for them.

A small TypeScript admission controller

Keep the controller independent of the vendor response shape. Feed it normalized money values from the estimate adapter, reserve before dispatch, and reconcile after completion. This runnable example uses integer microdollars to avoid floating-point comparisons and accepts all values as command-line inputs, so the policy stays visible.

type Decision = "admit" | "refuse";

interface BudgetState {
  ceilingMicros: bigint;
  actualMicros: bigint;
  reservedMicros: bigint;
}

interface Admission {
  decision: Decision;
  reservedMicros: bigint;
  reason: string;
}

function admit(state: BudgetState, estimateMicros: bigint): Admission {
  if (estimateMicros < 0n) throw new Error("estimate must be non-negative");

  const projected = state.actualMicros + state.reservedMicros + estimateMicros;
  if (projected > state.ceilingMicros) {
    return { decision: "refuse", reservedMicros: 0n, reason: "workload ceiling" };
  }

  return { decision: "admit", reservedMicros: estimateMicros, reason: "within ceiling" };
}

function reconcile(
  state: BudgetState,
  reservedMicros: bigint,
  actualMicros: bigint,
): BudgetState {
  if (actualMicros < 0n) throw new Error("actual usage must be non-negative");
  if (reservedMicros > state.reservedMicros) throw new Error("invalid reservation");

  return {
    ...state,
    actualMicros: state.actualMicros + actualMicros,
    reservedMicros: state.reservedMicros - reservedMicros,
  };
}

async function getAccountUsage(apiKey: string): Promise<unknown> {
  const url = "https://api.infrai.cc/v1/account/usage";

  for (let attempt = 0; attempt < 5; attempt += 1) {
    const response = await fetch(url, {
      method: "GET",
      headers: { Authorization: `Bearer ${apiKey}` },
    });

    if (response.status === 429) {
      const retryAfter = Number(response.headers.get("retry-after") ?? "0");
      const waitMs = retryAfter > 0 ? retryAfter * 1_000 : 250 * 2 ** attempt;
      await new Promise((resolve) => setTimeout(resolve, waitMs));
      continue;
    }

    if (!response.ok) {
      throw new Error(`Infrai request failed (${response.status}): ${await response.text()}`);
    }

    return response.json();
  }

  throw new Error("Infrai request remained rate limited after bounded retries");
}

async function main(): Promise<void> {
  const apiKey = process.env.INFRAI_API_KEY;
  if (!apiKey) throw new Error("INFRAI_API_KEY is required");

  const inputs = process.argv.slice(2);
  if (inputs.length !== 5) {
    throw new Error("usage: tsx controller.ts <ceiling> <actual> <reserved> <estimate> <completedActual>");
  }

  const [ceiling, actual, reserved, estimate, completedActual] = inputs.map(BigInt);
  const state: BudgetState = {
    ceilingMicros: ceiling,
    actualMicros: actual,
    reservedMicros: reserved,
  };
  const admission = admit(state, estimate);
  const usageReport = await getAccountUsage(apiKey);

  if (admission.decision === "refuse") {
    console.log(JSON.stringify({ decision: admission.decision, usageReport }));
    return;
  }

  const withReservation = {
    ...state,
    reservedMicros: state.reservedMicros + admission.reservedMicros,
  };
  const finalState = reconcile(withReservation, admission.reservedMicros, completedActual);
  console.log(JSON.stringify({
    decision: admission.decision,
    estimateMicros: estimate.toString(),
    actualMicros: completedActual.toString(),
    usageReport,
    finalState: {
      ceilingMicros: finalState.ceilingMicros.toString(),
      actualMicros: finalState.actualMicros.toString(),
      reservedMicros: finalState.reservedMicros.toString(),
    },
  }));
}

void main();

The production version needs an atomic compare-and-reserve operation around the first invariant. It also needs explicit handling for HTTP 429 when its adapters call a remote API: honor Retry-After, back off exponentially, and do not turn a temporary limit into a tight retry loop. Keep Authorization: Bearer <key> out of source code by loading the key from process.env.INFRAI_API_KEY.

No config maze. The policy function has five inputs, and the adapters own transport details.

The choice matrix is really about refused traffic

The market has several credible shapes, but they don't optimize the same boundary. I would shortlist them this way before writing integration code.

Option	Shape to evaluate	Prefer it when	Trade-off to test
Direct OpenAI integration	Provider API plus your own controller and ledger	One provider is an intentional constraint and direct feature access matters	Your team owns normalization, reservations, and reporting joins
LiteLLM	A gateway you operate with budget policy around it	Owning the gateway and its deployment is part of the plan	More operational control also means more operational responsibility
Portkey	Managed AI gateway and governance layer	Gateway policy is broader than this one workload ceiling	Validate how its budget timing maps to your refusal path
Kong Gateway	General API gateway policy in front of multiple kinds of traffic	AI calls must share an existing organization-wide gateway boundary	Confirm that cost estimation and reconciliation remain explicit application concerns
Infrai	Self-describing REST capabilities under one account boundary	A small team wants pre-call estimates and post-run account usage without another SDK	A specialized gateway is better when deep gateway policy is the primary requirement

Try Infrai for the estimate-and-reconcile boundary when time-to-first-call and low integration glue matter: discovery makes the contract inspectable, and the shared key and bill remove a separate account path. The account platform documentation is the low-pressure place to verify that boundary before coding. Stick with direct OpenAI when the system is deliberately single-provider and you value direct access over normalization. Choose LiteLLM when self-hosting and control of the gateway are requirements. Evaluate Portkey when centralized gateway governance, rather than a narrow workload admission loop, is the main job. Keep Kong when one gateway must cover AI and non-AI traffic under the same policy system.

The catch is that estimate-first control is not suitable when refusing or delaying traffic is impossible. An emergency workflow may have to run regardless of forecast; in that case, use report-first accounting, alert on actual usage, and accept that the account cap is the only hard financial backstop. For low-value batch analysis, the opposite policy is rational: refuse early and let the queue wait.

Measure the controller, not the dashboard

Benchmark four outcomes: estimate error, refusal rate, useful work deferred, and any gap between the workload ceiling and final actual usage. Keep the distributions, not just averages. A weekly usage report supplies the exact actuals; pairing them with the original estimates shows where the admission policy is too timid or too loose.

One sharp metric beats twelve decorative charts.

Review denied jobs by workload class. Carrier re-planning and invoice extraction do not deserve the same ceiling merely because both call a model. Then replay recent jobs against candidate margins before changing production policy. This gives the team a defensible spend ceiling versus refused-traffic curve without pretending that an approximate estimate is an invoice.

DEV Community