When LLM agents hit production traffic, manual error triage stops scaling. The same five failure types appear under different traces, each re-diagnosed from scratch. Duplicate alerts flood Slack. Engineers mute the channel.
This pipeline automates the entire flow: webhook ingestion, deduplication fingerprinting, Gemini-based classification into five real failure categories, playbook attachment, and severity-scored Slack reports. The workflow refuses to force a classification when the model lacks confidence.
The Five Failure Categories That Matter
Agent errors cluster into a small set of repeatable patterns:
| Category | Description | Common Trigger |
|---|---|---|
| Tool Call Failure | External API, database, or function call failed outright | Network timeout, auth token expiry, malformed parameters |
| Context Window Exhaustion | Conversation or tool history exceeded model context limit | Long-running sessions, verbose tool outputs, recursive calls |
| State Corruption | Agent memory or task state in unanticipated condition | Race condition, partial write, schema mismatch |
| Retry Loop | Agent stuck repeating the same failed step without progress | Missing exit condition, identical tool call pattern |
| Uncertain | Error doesn't cleanly fit any named category | Novel failure mode, insufficient context |
Without memory, every instance of "API timeout on customer lookup" gets diagnosed fresh. One bug generates dozens of identical alerts. No severity signal means everything looks urgent, so nothing is.
Architecture Flow
┌──────────────────────────────────────────────────────────┐
│ n8n AI Agent Triage Workflow │
│ │
│ [Webhook: POST /agent-error-triage] │
│ ↓ │
│ Accepts: Sentry native payload OR { message: "..." } │
│ ↓ │
│ [Extract & Dedup Code Node] │
│ → Normalize payload (title, culprit, stack_trace) │
│ → Build fingerprint (hash of identifying fields) │
│ → Check 30-min in-memory suppression window │
│ ↓ ↓ │
│ [isDuplicate=true] [isDuplicate=false] │
│ ↓ ↓ │
│ [Skip No-Op] [Classify with Gemini] │
│ ↓ │
│ Structured Output Parser │
│ → { category, severity, │
│ confidence, reasoning } │
│ ↓ │
│ [Attach Fix Playbook] │
│ ↓ │
│ [Post to Slack] │
│ → Thread with severity badge │
│ → Playbook steps as bullets │
└──────────────────────────────────────────────────────────┘
The workflow exposes a single webhook endpoint. It accepts either Sentry's native JSON payload or a minimal { message: "..." } object for custom error reporters.
Deduplication Fingerprinting
The Extract & Dedup node normalizes the incoming payload into three fields: title, culprit, and stack_trace. It builds a SHA-256 fingerprint from these fields and checks an in-memory Map with a 30-minute TTL.
// Extract & Dedup Code Node (simplified)
const crypto = require('crypto');
// Normalize payload
const title = $input.item.json.title ||
$input.item.json.exception?.values?.[0]?.type ||
"Unknown Error";
const culprit = $input.item.json.culprit ||
$input.item.json.exception?.values?.[0]?.stacktrace?.frames?.[0]?.function ||
"unknown";
const stackTrace = $input.item.json.exception?.values?.[0]?.stacktrace?.frames || [];
// Build fingerprint
const fingerprintInput = JSON.stringify({ title, culprit, stackTrace });
const fingerprint = crypto.createHash('sha256').update(fingerprintInput).digest('hex');
// Check suppression window
const now = Date.now();
const suppressionWindow = 30 * 60 * 1000; // 30 minutes
const lastSeen = $node["Dedup Map"].getItem(fingerprint);
if (lastSeen && (now - lastSeen) < suppressionWindow) {
return { isDuplicate: true, fingerprint };
}
// Store new fingerprint
$node["Dedup Map"].setItem(fingerprint, now);
return { isDuplicate: false, fingerprint, title, culprit, stackTrace };
The Map lives in n8n's execution context. It resets on workflow restart, which is acceptable for a 30-minute window. For longer suppression or multi-instance deployments, swap the Map for Redis with TTL keys.
Gemini Structured Output for Classification
The Classify node sends the normalized error to Gemini 1.5 Flash with a structured output schema. The schema enforces five fields: category, severity (1-5), confidence (0-1), reasoning, and suggestedPlaybook.
{
"type": "object",
"properties": {
"category": {
"type": "string",
"enum": [
"Tool Call Failure",
"Context Window Exhaustion",
"State Corruption",
"Retry Loop",
"Uncertain"
]
},
"severity": {
"type": "integer",
"minimum": 1,
"maximum": 5
},
"confidence": {
"type": "number",
"minimum": 0,
"maximum": 1
},
"reasoning": {
"type": "string"
},
"suggestedPlaybook": {
"type": "string"
}
},
"required": ["category", "severity", "confidence", "reasoning"]
}
The prompt includes all five category definitions and instructs the model to return "Uncertain" with low confidence when the error doesn't fit cleanly. This prevents forced classifications that degrade routing accuracy.
Gemini's structured output mode guarantees valid JSON. No regex parsing, no retry loops for malformed responses. The output parser node extracts the fields directly.
Playbook Routing Logic
The Attach Fix Playbook node maps each category to a predefined remediation checklist. These live as static strings in the workflow but could be fetched from a CMS or Git repo.
// Playbook mapping
const playbooks = {
"Tool Call Failure": `
1. Check external service status page
2. Verify API credentials in environment config
3. Inspect rate limit headers in last successful call
4. Test tool function in isolation with same parameters
5. Add exponential backoff if not present`,
"Context Window Exhaustion": `
1. Review conversation length in agent state
2. Implement sliding window summarization
3. Prune old tool outputs from context
4. Switch to model with larger context window
5. Add context length monitoring to agent loop`,
"State Corruption": `
1. Dump agent state JSON to file
2. Compare against expected schema
3. Check for concurrent writes in logs
4. Add state validation at loop entry
5. Implement state rollback on validation failure`,
"Retry Loop": `
1. Identify repeated tool call pattern in trace
2. Add max retry counter to agent loop
3. Implement circuit breaker for failing tool
4. Add exponential backoff between retries
5. Log loop detection event for analysis`,
"Uncertain": `
1. Review full error context and stack trace
2. Search internal incident database for similar patterns
3. Escalate to on-call engineer for manual classification
4. Document new failure mode if novel
5. Update classification prompt with new category`
};
const category = $input.item.json.category;
const playbook = playbooks[category] || playbooks["Uncertain"];
return { playbook };
The playbook becomes part of the Slack message payload. Engineers see the error, the classification reasoning, and the next steps in one view.
Slack Reporting with Severity Badges
The Post to Slack node formats the message with severity-based color coding and thread structure. High-severity errors (4-5) use red, medium (3) uses orange, low (1-2) uses yellow.
// Slack message formatting
const severityColors = {
5: "#d32f2f", // red
4: "#f57c00", // orange
3: "#fbc02d", // yellow
2: "#7cb342", // light green
1: "#388e3c" // green
};
const color = severityColors[$input.item.json.severity] || "#9e9e9e";
const blocks = [
{
"type": "header",
"text": {
"type": "plain_text",
"text": `🚨 ${$input.item.json.category} (Severity ${$input.item.json.severity})`
}
},
{
"type": "section",
"text": {
"type": "mrkdwn",
"text": `*Error:* ${$input.item.json.title}\n*Confidence:* ${($input.item.json.confidence * 100).toFixed(0)}%\n*Reasoning:* ${$input.item.json.reasoning}`
}
},
{
"type": "section",
"text": {
"type": "mrkdwn",
"text": `*Fix Playbook:*\n${$input.item.json.playbook}`
}
}
];
return { color, blocks };
The workflow posts to a dedicated #agent-errors channel. Each error becomes a thread root. Follow-up messages (resolution status, related errors) can be posted as replies using the thread_ts from the initial post.
State Management for Resolution Tracking
The current workflow is stateless after Slack posting. To track resolution status across threads, add a Redis node or n8n's built-in database node to store:
-
fingerprint(primary key) slack_thread_ts-
status(open, investigating, resolved) assigned_toresolved_at
A second webhook endpoint (POST /agent-error-resolve) accepts fingerprint and status, updates the database, and posts a resolution message to the Slack thread. This closes the loop from detection to fix confirmation.
For multi-instance n8n deployments, the dedup Map must move to Redis. Use SET fingerprint timestamp EX 1800 to maintain the 30-minute TTL atomically.
Backpressure and Rate Limiting
The webhook node has no built-in rate limiting. If 50 agents fail simultaneously, 50 Gemini API calls fire in parallel. Gemini's default quota is 60 requests per minute for Flash.
Add a Queue node before the Classify step:
// Queue node configuration
{
"mode": "limiter",
"maxConcurrent": 10,
"minTime": 1000 // 1 second between batches
}
This caps concurrent Gemini calls at 10 and spaces batches by 1 second. Errors beyond the queue depth (default 100) return HTTP 429 to the caller, which should implement exponential backoff.
For persistent queuing, replace the Queue node with a Redis-backed job queue (Bull, BullMQ). The webhook writes to Redis, a separate n8n workflow polls the queue, and failed classifications retry with backoff.
Failure Modes
Gemini API outage: The Classify node times out after 30 seconds. The workflow posts to Slack with category "Uncertain" and severity 3. The playbook instructs manual classification.
Slack API outage: The Post to Slack node retries three times with exponential backoff (n8n default). After three failures, the workflow logs the error to n8n's execution history but does not block the webhook response. The caller receives HTTP 200. Unposted errors are lost unless you add a dead-letter queue.
Dedup Map memory overflow: The Map has no size limit. If fingerprints accumulate faster than the 30-minute TTL clears them, n8n's memory usage grows unbounded. Monitor heap size and restart the workflow daily, or move to Redis with EXPIRE.
Structured output schema mismatch: Gemini occasionally returns valid JSON that violates the schema (e.g., severity: 6). The output parser node throws an error. Add a validation step that clamps severity to 1-5 and defaults confidence to 0.5 on parse failure.
Technical Verdict
Use this pipeline when:
- You run multiple LLM agents in production and see repeated failure patterns
- You have 5-20 known error categories and want deterministic routing to playbooks
- You need deduplication to prevent alert fatigue
- You want human-in-the-loop confirmation before auto-remediation
Avoid this pipeline when:
- You have fewer than 10 agent errors per day (manual triage is faster)
- Your errors are highly novel and don't cluster into categories (classification accuracy drops below 70%)
- You need sub-second triage latency (Gemini adds 2-5 seconds per classification)
- You require strict ordering guarantees (n8n's queue is in-memory and non-durable)
The workflow JSON is production-ready for single-instance n8n deployments with moderate error volume (10-100 per hour). For higher scale, add Redis for dedup state, a persistent job queue, and multi-region Slack posting.
Top comments (0)