Charging credits for async AI jobs without double-spending

Image and video jobs do not finish in the same HTTP request. They take long enough that a dropped response, a double-click, or a refresh will retry. If you charge on "done", you eat unpaid work. If you charge on "start" and the job dies, you owe people money back. Both bugs showed up in our first design, so we moved the money and the task into one write.

This is how Epochal does it today: a 202 on submit, a durable task, a client request id, and a refund flag that can only flip once. No customer webhooks. The browser polls.

I am writing this as the person who has to explain a missing refund.

The failure that forced the design

A user hits Generate. The server creates a job and charges the quoted cost. The tab dies before the JSON comes back. The client retries with the same payload.

Without a request id you get two tasks and twice the spend. With a request id but a late charge, the retry can create a second task that never got paid. The only version that stayed sane for us:

1. The client sends clientRequestId.
2. One database transaction locks the user, checks the in-flight job cap, spends credits, inserts a pending task.
3. The HTTP response is 202 with the task id. The file is not ready.

Same user + same clientRequestId returns the existing task. No second spend.

// Same user + same clientRequestId → existing row, created: false
const existing = await findTask(userId, clientRequestId);
if (existing) {
  return { task: existing, created: false };
}

await consumeCredits(tx, { userId, amount });
await insertTask(tx, { status: "pending", creditsReserved: true });

The column is unique per user.

Reserve credits and create the task atomically

Spend happens inside the same transaction as the task insert, not after the model returns.

The user row is locked (FOR UPDATE) so two tabs cannot both pass the balance check. Credits live in expiring lots. We spend the lots that expire first, and a cron expires any unused balance and records the change in the ledger.

If the balance is short, we throw before insert. The generate routes map that to 402. No empty task, no "reserved 0 then fail."

Local safety or text-moderation blocks write a failed task with nothing reserved. Those never touch the ledger.

202 means accepted, not completed

Synchronous image generation is deprecated on our API. Submit returns immediately:

HTTP/1.1 202 Accepted
{
  "creation": { "id": "…", "status": "pending" },
  "remainingCredits": 142
}

Statuses are only pending, running, completed, failed. A completed or failed row cannot move back. That sounds obvious until two workers both try to "fix" a stuck job.

The workbench polls the task endpoint while it is active. Each GET asks the server to sync that task before it reads. A slower task-center poll lets users navigate away without losing visibility.

We do not offer "POST this URL when the video is done." Billing uses Stripe webhooks. Generation completion is our problem: client poll, a server-side follow-up after the 202, and a cron that rescans stale pending / running rows.

Refund exactly once

Failure does two things in order: mark the task failed (only if it is not already terminal), then refund if the row is eligible.

The refund is its own transaction. The refund flag is not a record that a refund was attempted. It is a claim that the refund transaction completed.

const [claimed] = await tx
  .update(generationTask)
  .set({ creditsRefunded: true })
  .where(
    and(
      eq(generationTask.id, taskId),
      eq(generationTask.status, "failed"),
      eq(generationTask.creditsRefunded, false)
    )
  )
  .returning({ id: generationTask.id });

if (!claimed) return; // another worker already refunded

await addCredits(tx, userId, task.creditsCost);
await insertLedgerRow(tx, { type: "REFUND", amount: task.creditsCost });

If the add-credits step throws, the transaction rolls back and creditsRefunded stays false. The next retry can try again. That comment is in the orchestrator because we shipped the opposite bug: flag flipped, money never added.

A second cron looks for failed + reserved + not refunded and runs the same claim. I do not trust a single in-process path after a deploy mid-job.

Not every terminal failure is refundable, so refund eligibility is stored explicitly rather than inferred from status = failed.

Completed means we own the output

running means we handed the job off. completed means we downloaded the result, stored it on our object storage, and linked it to the task. That last write is also claimed so two finishers cannot create two library rows.

Upstream success is not a durable result. If persist fails after the file exists, a later sync or the recovery cron can finish the row. Users see the same object in the workbench, the task center, and the library because both tables map to one list item.

Concurrency belongs inside the same lock

In-flight means pending or running. We enforce the user's concurrency limit under the same user lock as the credit spend. That prevents two simultaneous requests from both passing the limit check before either inserts its task.

Over the cap is 429. The pricing UI reads the same plan-cap resolver as the server, so the displayed limit and the enforced limit cannot drift independently.

What I would not do again

Charge after success. You will eat unpaid GPU time on every abandoned tab, and you will invent a "pending charge" state that nobody can explain in support.

Refund with UPDATE … SET refunded = true then a second query for the money. One of those queries will fail. Claim the slot and move the balance in one transaction.

Customer webhooks as the only completion signal. We would have to sign payloads, retry 4xx, and still keep polling for the workbench. Poll + server sync + cron is enough for a product that is a website, not an API company.

Treat "the model finished" as "the user has a file." Until the object is on our storage and linked, the task is not done. People refresh. The row has to survive that.

Try the failure path

Open a model or tool on epochal.app, submit once, then refresh mid-job. The same clientRequestId should not take credits twice.

If you run a similar queue: put the request id, the spend, and the pending row in one lock. Everything after that is catching up.