Beyond the Wrapper: Architecting AI-Native SaaS (and Billing For It)

If you shipped a "ChatGPT wrapper" in 2023, you already know how this story ends. A thin UI in front of a single chat.completions.create() call was enough to get a Product Hunt launch. It is no longer enough to get a paying customer to stay past month two.

The apps winning right now don't call a model once and format the response. They run multi-step, multi-agent, asynchronous workflows — retrying failed tool calls, waiting on human approval, checking a budget before spending another dollar, and billing the customer for exactly what was consumed. That's a different animal architecturally, and it's a different animal commercially.

This post covers both halves of that problem:

1. How to structure the execution layer so agentic workflows are reliable instead of a pile of try/catch blocks.
2. How to structure the billing layer underneath it, because "$20/seat/month" makes no sense when your "user" is an autonomous agent running 400 tool calls a night.

---

Part 1: Why a Single API Call Isn't an Architecture

A wrapper app looks like this:

User request → LLM call → Response → Done

An AI-native app looks like this:

flowchart LR
    A[User / Trigger] --> B[Orchestrator]
    B --> C{Plan Step}
    C --> D[Tool Call 1: DB Query]
    C --> E[Tool Call 2: External API]
    D --> F[LLM Reasoning]
    E --> F
    F --> G{Needs Human Approval?}
    G -- Yes --> H[Human-in-the-loop Queue]
    G -- No --> I[Execute Action]
    H --> I
    I --> J{Success?}
    J -- No --> K[Retry / Fallback Model]
    K --> F
    J -- Yes --> L[Persist State + Emit Event]
    L --> M[Billing Meter]

The moment you introduce multiple steps, tool calls, and waiting periods (for a human, an API, or a rate limit), you inherit every problem distributed systems engineers have been solving for a decade: state persistence, retries, idempotency, and failure isolation. The difference is your "task" now includes a non-deterministic LLM call in the middle of it.

The execution layer: don't build this yourself

The naive approach is an in-memory loop with a while statement and a prayer that your server doesn't restart mid-workflow. Don't do this past a prototype. Two solid patterns:

Option A — Durable execution (Temporal, Restate, Inngest)

Good when workflows are long-running (minutes to days), need human-in-the-loop pauses, and must survive process crashes without losing state.

// Temporal workflow — survives crashes, retries automatically
export async function agentWorkflow(taskInput: TaskInput) {
  const plan = await activities.generatePlan(taskInput);

  for (const step of plan.steps) {
    const result = await activities.executeToolCall(step, {
      retry: { maximumAttempts: 3, backoffCoefficient: 2 },
    });

    if (step.requiresApproval) {
      // Workflow literally pauses here — no polling, no cron job
      await condition(() => approvalSignal.received);
    }

    await activities.persistStepResult(result);
    await activities.emitBillingEvent(result.tokensUsed, result.toolCallsUsed);
  }
}

Option B — Job queues (BullMQ, SQS + Lambda)

Good when workflows are shorter, more parallel, and you don't need the workflow to "remember" it's mid-conversation for days at a time.

// BullMQ worker for a single agent step
worker.process('agent-step', async (job) => {
  const { conversationId, stepIndex } = job.data;
  const context = await loadConversationState(conversationId);

  const result = await callLLMWithTools(context);

  if (result.needsRetry) {
    throw new Error('Retryable failure'); // BullMQ handles backoff
  }

  await saveState(conversationId, result);
  await billingQueue.add('meter-usage', {
    conversationId,
    tokens: result.usage.total_tokens,
    toolCalls: result.toolCallCount,
  });
});

Neither is "better" in the abstract — Temporal buys you visibility and long-lived pauses at the cost of operational complexity; BullMQ is lighter but you own more of the retry/idempotency logic yourself.

Handling latency, fallbacks, and flaky models

Three things WILL happen in production that never happen in your demo:

• A model provider has a slow day (P99 latency triples).
• A tool call times out against a third-party API.
• The model returns a malformed tool call your parser chokes on.

The fix isn't "add a try/catch." It's designing for it up front:

Failure mode	Pattern
Provider latency spike	Timeout + fallback to a secondary model/provider
Tool call timeout	Idempotent retries with exponential backoff
Malformed output	Schema validation (e.g. Zod) + one automatic "repair" reprompt before failing the step
Runaway loop (agent stuck)	Hard step-count ceiling per workflow run

A minimal fallback wrapper:

async function callWithFallback(prompt: string) {
  try {
    return await withTimeout(callModel('primary-model', prompt), 8000);
  } catch (err) {
    logFallbackTriggered(err);
    return await callModel('fallback-model', prompt);
  }
}

This single function is the difference between "our AI feature went down for two hours" and nobody noticing the primary provider had an incident.

---

Part 2: The Billing Layer Underneath It

Here's the part most architecture posts skip: once your app runs asynchronous, variable-length agent workflows, "$20/user/month" stops mapping to anything real.

Two customers on the same seat-based plan can have wildly different cost profiles:

flowchart TB
    subgraph SeatModel["Old Model: Per-Seat"]
    U1[User A: 3 logins/week, 12 AI actions] -->|pays| P1[$20/mo]
    U2[User B: agent running 24/7, 8,000 AI actions] -->|pays| P2[$20/mo]
    end

User B is costing you 600x more in inference spend than User A, and charging them the identical price. That's not a pricing quirk—it's a margin problem that gets worse as adoption grows, because your best customers (heaviest usage) become your worst unit economics.

Moving to outcome/credit-based metering

The fix is metering the thing that actually costs you money: tokens, tool calls, or completed outcomes (whichever maps most cleanly to value for your customer).

flowchart LR
    A[Agent Action Completes] --> B[Emit Usage Event]
    B --> C[Metering Service]
    C --> D{Balance Check}
    D -- Sufficient credit --> E[Deduct + Continue]
    D -- Insufficient --> F[Block Action + Notify]
    C --> G[(Usage Ledger)]
    G --> H[Nightly Aggregation]
    H --> I[Stripe Usage Record]
    I --> J[Invoice / Webhook]

The key architectural shift: billing is no longer a monthly cron job that reads a users table. It's an event stream, fed by the same execution layer that runs your agents.

// Emitted from inside the workflow/worker, not from a nightly batch job
async function emitBillingEvent(event: UsageEvent) {
  await ledger.record({
    customerId: event.customerId,
    metric: 'tool_call', // or 'token', 'completed_task'
    quantity: event.quantity,
    metadata: { workflowId: event.workflowId, model: event.model },
    timestamp: new Date(),
  });

  await stripe.billing.meterEvents.create({
    event_name: 'agent_tool_calls',
    payload: {
      value: String(event.quantity),
      stripe_customer_id: event.stripeCustomerId,
    },
  });
}

Stripe's usage-based billing (meter events + billing meters) will aggregate this into the invoice for you — you don't need to compute totals yourself, just emit events reliably and idempotently (attach a unique event ID so a retried job doesn't double-bill).

FinOps controls: don't let an agent bankrupt you or your customer

Usage-based billing without spend controls is how you end up with a customer support ticket titled "why is my invoice $40,000." Three controls to build before you ship metering, not after:

1. Pre-flight spend caps

Check the budget before the expensive call, not after.

async function checkSpendCap(customerId: string, estimatedCost: number) {
  const { spent, cap } = await getBudget(customerId);

  if (spent + estimatedCost > cap) {
    await notifyCustomer(customerId, 'approaching_spend_cap');
    throw new SpendCapExceededError(customerId);
  }
}

2. Token/request rate-limiting per tenant

Prevents one runaway agent loop (or one compromised API key) from consuming shared model capacity meant for every other customer.

const limiter = new RateLimiter({
  keyPrefix: 'tenant-tokens',
  points: 100_000,   // tokens
  duration: 3600,    // per hour
});

await limiter.consume(tenantId, estimatedTokens);

3. Webhook-driven billing triggers, not polling

React to usage in near-real-time via webhooks (from Stripe, or your own internal event bus) rather than a nightly job that discovers the overage twelve hours late.

app.post('/webhooks/stripe', async (req, res) => {
  const event = stripe.webhooks.constructEvent(req.body, req.headers['stripe-signature'], secret);

  if (event.type === 'billing.meter.error_report_triggered') {
    await pauseCustomerAgents(event.data.object.customer);
  }

  res.sendStatus(200);
});

---

Putting It Together

The two halves aren't actually separate systems — they're the same event stream viewed from two angles. Every step your orchestration layer executes is simultaneously a unit of work (which needs retries, state, and human gates) and a unit of cost (which needs metering, caps, and invoicing).

flowchart TD
    A[Orchestrator: Temporal/BullMQ] -->|step completes| B[Event]
    B --> C[State Persistence]
    B --> D[Billing Meter]
    D --> E[Spend Cap Check]
    E -->|over cap| F[Pause Workflow]
    E -->|under cap| A
    D --> G[Stripe Meter Event]
    G --> H[Customer Invoice]

If you design the execution layer and the billing layer as one pipeline from day one, you avoid the painful retrofit most teams go through: building the agent orchestration first, shipping it on a flat monthly price, and then bolting metering on eighteen months later once a handful of power users have quietly destroyed the gross margin.

Takeaways

• A single LLM call is a feature, not a product. Reliability comes from treating agent workflows as durable, retryable, resumable jobs — use Temporal/Restate for long-lived or human-in-the-loop flows, job queues for shorter parallel work.
• Design fallback and retry paths before you need them: timeouts, secondary models, schema validation, and step ceilings.
• Per-seat pricing breaks down the moment usage per "seat" becomes agent-driven and variable. Meter the thing that costs you money (tokens, tool calls, or completed outcomes).
• Treat billing as an event stream emitted from your execution layer, not a separate nightly job.
• Ship spend caps and rate limits before usage-based billing, not after your first runaway-agent invoice.

---

Have you shipped usage-based billing for an AI product yet? Curious what metering unit you landed on — tokens, tool calls, or something more outcome-based like tasks completed. Drop it in the comments.