Building a Multi-Signal Renewal-Risk Workflow with n8n, Deterministic Scoring, and a Dual-LLM Audit

Churn signals never live in one place. Contract data sits in the CRM. Health scores sit in the customer success platform. Ticket sentiment sits in the support desk. Engagement dates sit somewhere else entirely. None of these systems alone tells you whether an account is actually at risk.

I built a workflow in n8n that pulls all four sources together, scores risk with a deterministic model, and only calls an LLM when the score is high enough to justify the cost and the hallucination risk. This post covers the architecture, the specific bugs that taught me the most, and how I'd scale it past a single webhook.

Why not just ask an LLM to score risk

An LLM is good at writing a coherent risk narrative. It is not a good scorer. Ask the same model to rate the same account twice and the number moves. That is fine for a summary, not fine for something that triggers a Slack alert to an account owner.

So the scoring itself is a separate deterministic service. The LLM only gets involved once an account has already crossed a threshold, and even then its only job is to explain and recommend, never to set the number.

Architecture at a glance

The flow, start to finish:

1. Webhook fires on a renewal-risk event.
2. Request gets validated.
3. Account gets resolved across the four source systems (or logged as unresolved and stopped).
4. Signals get fetched from all four in parallel.
5. A deterministic scoring service computes a risk score.
6. Below threshold: logged as low risk, done.
7. At or above threshold: one LLM generates a churn analysis, a second LLM audits it.
8. Auditor disagrees: routed to a human in Slack.
9. Auditor approves: guardrails check runs.
10. Guardrails fail: logged and stopped.
11. Guardrails pass: duplicate check runs against the last alert for that account.
12. Duplicate: suppressed and logged.
13. Not a duplicate: alert sent, and the outcome recorded back in the CRM and CS platform.

Step 1: resolving one account across four systems

Every downstream system has its own ID for the same account. The resolution step accepts either an internal ID or a company name, and joins across a mapping table with a fallback ILIKE match on name. If nothing resolves, the workflow logs it and stops rather than guessing.

Step 2: a deterministic scoring service, not a prompt

Scoring runs in a small FastAPI service with fixed weights:

• Health score drop of 30+ points: +0.35
• Health score drop of 15-29 points: +0.20
• Support sentiment below -0.5: +0.25
• Support sentiment below 0: +0.10
• No engagement data on record: +0.20
• No engagement in 30+ days: +0.25
• No engagement in 14-29 days: +0.10
• Two or more open tickets: +0.15

The total is capped at 1.0. Anything at 0.65 or above is treated as high risk and moves on to LLM analysis. Everything else is logged and closed out.

One bug taught me more than the rest of the service combined. Missing engagement data was defaulted to zero through a COALESCE. Zero read as "engaged today," which suppressed exactly the accounts that most needed flagging: the ones with no engagement history at all. The fix was a -1 sentinel treated as its own explicit risk factor instead of a silent default. Missing data is a signal. Don't let a default value hide it.

Step 3: one model writes, a different model checks

For accounts above the threshold, one LLM generates a churn analysis and a recommended action. A second, different model then audits that output: does the evidence actually support the conclusion, is anything overstated, is the recommended action proportionate to the risk.

Using a different model for the audit step matters. Two calls to the same model share the same blind spots. A genuinely different model catches more. In testing, the auditor caught a real disagreement: a set of support tickets that on the surface looked negative but actually indicated an engaged customer working through a migration, not a disengaged one about to churn. That case got routed to a human instead of auto-alerting on a wrong conclusion.

Step 4: guardrails before anything leaves the workflow

Even after the auditor approves, a guardrails step checks the alert content before it goes anywhere: correct account owner attached, no unsupported claims, structure matches what downstream systems expect. If it fails, it gets logged and stopped, not sent.

Step 5: idempotency that holds under real concurrency

Renewal-risk events can arrive more than once for the same account. The naive fix is "check if an alert already exists, then insert if not." That has a race condition: two concurrent requests can both pass the check before either has inserted.

The actual fix was a UNIQUE constraint at the database level on account and risk score, so the database itself rejects the duplicate insert instead of relying on application logic to catch it. I stress-tested this with concurrent requests hitting the same account and confirmed only one alert made it through.

Step 6: eight terminal states, not just try/catch

Every path through the workflow ends in one of eight explicit logged outcomes: alert sent, low risk, duplicate suppressed, guardrails violation, auditor disagreement, unresolved ID, invalid request, or scoring service unavailable. When something goes wrong, the question is never "why did the workflow fail," it's "which of these eight states did this execution land in, and why."

Scaling past a single webhook

The version above handles one event at a time. Scaling it out changes a few things:

• An API gateway acknowledges the webhook immediately and hands off to a queue, so the source system never waits on the full pipeline.
• The queue has a dead-letter topic for anything that fails processing repeatedly.
• n8n runs in queue mode: a main instance plus separate workers, coordinated through Redis, so heavy executions don't block new incoming events.
• The database sits behind a connection pooler, since a fleet of workers hitting Postgres directly will exhaust connections fast.
• Every execution also writes to a warehouse table as a "golden record," so risk patterns can be analyzed over time instead of only reacted to in the moment.
• Outbound calls go through a static egress IP, since most CRM and support-desk APIs allowlist by IP rather than accepting traffic from anywhere.

What I'd tell someone building this kind of thing

• Score deterministically first. Let the LLM explain, not decide.
• Treat missing data as its own risk factor, never a silent default.
• Put a second, different model in an adversarial role instead of just re-running the same one.
• Push idempotency into the database. Application-level checks lose races.
• Log outcomes, not just errors. "It failed" is not a diagnosis. Which of your terminal states it hit, is.

None of this is complicated in isolation. What made it work was refusing to let the LLM touch anything that needed to be consistent, and refusing to let the "happy path" architecture skip the concurrency and observability work until it broke in production.

---

More on automation architecture at automiq.fi, more n8n templates on the n8n Creator Hub, or connect on LinkedIn.