The Most Expensive AI Bug Is a Job That Runs Twice

A failed AI request does not always mean the work failed.

Sometimes the client loses the response after the model already completed. Sometimes a worker restarts while a tool call is still running. Sometimes a timeout triggers a retry while the original request is still alive.

Then one user action becomes:

• two model calls
• two extraction jobs
• two support tickets
• two emails
• two database writes
• two bills

That is not a model-quality problem.

It is an idempotency problem.

A retry should not create a new job

Retries are normal in production AI systems.

Networks fail. Providers slow down. Tool calls time out. Queues redeliver jobs. A fallback route may begin after the primary route becomes uncertain.

The mistake is treating every retry as a new piece of work.

Instead, define one durable job for each business action:

• one uploaded document
• one invoice extraction
• one support conversation summary
• one scheduled report
• one moderation event
• one agent task

Every attempt should belong to that same job.

Use an idempotency key

Give each business action a stable idempotency key.

For example:

json
{
  "idempotency_key": "extract_invoice_9821",
  "workflow": "invoice_extraction",
  "status": "running",
  "attempt_count": 2,
  "model": "model-a",
  "route": "fallback",
  "result_reference": null
}
When the same request arrives again, do not immediately run the workflow again.
First check the job state.
The correct response may be:
the job already completed, return the saved result
the job is still running, return its status
the previous attempt is uncertain, investigate before replaying
the job failed safely, create a controlled retry
The key represents the business outcome, not one HTTP request.
Persist state before calling the model
A risky pattern looks like this:
call the model
receive a result
save the job record
If the application crashes between steps two and three, the model may have finished but the system has no evidence.
The next retry can call the model again.
Create the job record before invoking a model, tool, or external API.
Record enough information to reconstruct what happened:
idempotency key
workflow name
input reference
model and configuration version
selected route
attempt count
timestamps
provider request ID
tool execution ID
result reference
error class
This turns an unknown retry into something that can be inspected.
“Unknown” is more dangerous than “failed”
A clear failure is easy to retry.
An unknown outcome is harder.
Imagine a request times out after the application sends a model call.
Did the provider receive it?
Did the model finish?
Did the tool execute?
Was the output stored?
Blindly retrying may duplicate work. Before replaying, inspect the evidence:
request logs
provider request IDs
tool execution records
callback events
database writes
output storage
usage records
A production system should reconcile uncertain work before creating more of it.
A fallback route is still the same job
Switching from a primary model to a fallback model does not create a new business task.
Keep both attempts under the same idempotency key:
Job: extract_invoice_9821

Attempt 1: primary route timed out
Attempt 2: fallback route completed
Final result: stored once
This matters in multi-model applications.
Without a shared job record, teams cannot tell whether higher cost came from an intentional fallback, a retry, or an accidental duplicate.
Separate the request from the work
For longer workflows, avoid doing all work inside one user request.
Create a durable job, return a job ID, and process the job asynchronously.
Then the client can safely ask for status without submitting the same task again.
This is useful for:
long document processing
RAG indexing
batch summarization
agent workflows
media generation
tool-heavy automation
A timeout should not force a user to start over.
Test the duplicate paths
Do not test only the happy path.
Test what happens when:
a client retries after a slow response
two workers receive the same job
a worker restarts during execution
a model response arrives after the timeout
a tool succeeds but its callback is lost
a fallback begins while the primary route is uncertain
a user clicks submit twice
Success is not “the workflow eventually completed.”
Success is one intentional business result.
Final thought
Retries are necessary.
Duplicate AI work is not.
A stable idempotency key, durable job state, and request-level evidence make retries safer, cheaper, and easier to debug.
VectorNode helps teams access, monitor, and manage global and Chinese frontier models through one multi-model AI infrastructure layer.
Learn more: https://www.vectronode.com/