Retries are useful.
But some AI jobs still fail.
A document extraction task exhausts its retries. An agent stops after a tool timeout. A RAG indexing job cannot access a source file. A batch workflow hits a context limit.
What happens next?
If the answer is “write an error log and move on,” the application is losing more than a request.
It is losing the evidence needed to understand, repair, and safely replay the work.
This is where dead letter queues matter.
A dead letter queue is a recovery boundary
A dead letter queue, or DLQ, holds jobs that could not be completed safely after their normal retry policy was exhausted.
It is not a place to hide errors.
It is a place to preserve failure context.
For AI workflows, that context can include:
- the workflow name
- a reference to the input data
- the selected model and route
- prompt or configuration version
- retry count
- fallback history
- error classification
- whether the job is safe to replay
That is much more useful than a line that says request failed.
AI failures are rarely just provider failures
A failed model request can be caused by many things:
- a temporary provider outage
- a rate limit
- an oversized context
- invalid structured output
- a missing source document
- broken retrieval
- a tool-call timeout
- an unsupported parameter
- an unapproved model route
Some of these problems may recover with a retry.
Others need a prompt change, a schema fix, a route change, or manual review.
A DLQ stops the system from pretending that every failure has the same solution.
What should an AI DLQ record?
A useful record might look like this:
json
{
"job_id": "job_8421",
"workflow": "document_extraction",
"payload_reference": "file_2388",
"model": "model-a",
"model_config_version": "v12",
"route": "primary",
"attempt_count": 3,
"fallback_used": true,
"error_class": "structured_output_validation",
"last_error": "Required field missing",
"safe_to_replay": true
}
Notice what is missing: a requirement to store every raw prompt forever.
Sensitive inputs may require masking, encryption, access controls, or a reference to the original source instead of a full payload copy.
The important part is keeping enough context to investigate the failure.
A DLQ should not become an invisible retry loop
A common mistake is to automatically replay every dead-lettered job every few hours.
That is just a retry loop with a longer delay.
Before replaying a failed job, ask:
Is the provider healthy again?
Did the model configuration change?
Was the input too large?
Is the original source still available?
Did the job already trigger an external action?
Is replaying it safe?
Should it use the same model route or a reviewed replacement?
A job that failed because of a temporary timeout may be safe to replay.
A job that failed because the JSON schema was wrong needs repair first.
A job that sent an external email may need manual approval.
Separate capture, diagnosis, and replay
A clean workflow has three stages:
Capture the failed job in the DLQ.
Diagnose the actual failure cause.
Replay or repair the job deliberately.
For example:
Primary model returns invalid JSON
→ retry with a constrained prompt
→ fallback route also fails validation
→ send the job to the DLQ
→ inspect source file and output schema
→ update the configuration
→ replay selected failed jobs
This is much safer than repeatedly changing models until a request happens to succeed.
A DLQ is also product feedback
Failed jobs reveal where the product needs work.
A DLQ may show that:
one document type breaks extraction
a prompt fails for long inputs
a model route struggles with multilingual content
a provider limit affects batch traffic
a tool integration frequently times out
a model update changed structured-output behavior
Track metrics such as:
failed jobs by workflow
failure rate by model and route
retry exhaustion rate
time spent in the DLQ
replay success rate
repeated error classes
cost of failed and replayed jobs
The goal is not merely to replay failed work.
It is to reduce the reasons work reaches the queue.
Final thought
Retries help with temporary failures.
Dead letter queues help with failures that are not temporary.
They prevent silent data loss, preserve the context needed for debugging, and make replay an operational decision rather than an automatic gamble.
VectorNode helps teams access, manage, monitor, and optimize global and Chinese frontier models through one multi-model AI infrastructure layer.
https://www.vectronode.com/
Top comments (0)