What it actually takes to keep an LLM pipeline running for a year, unattended

Farzam — Thu, 06 Aug 2026 10:11:34 +0000

Most AI document-processing demos fall over on the third weird PDF. Here's a contract-intelligence pipeline I built that's been running in production for over a year the architecture decisions that made that possible.

The problem
A legal team was reading thousands of sponsorship contracts by hand. The obvious fix; throw an LLM at each document breaks in production for a specific reason: a wrong extraction early in the chain propagates into every downstream report, and a job that dies late in a long chain can't afford to re-burn every prior stage of tokens.

The architecture
I modeled each document as a work unit with an explicit status enum, moving through named stages; extraction, citation, deal typing, vendor identification, overlap detection, amendment detection, redaction checks — as separate, idempotent background jobs. A stage only enqueues the next on success. If a job dies mid-chain, it resumes exactly where it broke instead of restarting from zero.

Every model call routes through a provider gateway with bounded retries, a hard timeout, and typed handling for read/connect timeouts so one provider's bad afternoon doesn't take the whole pipeline down.

Before anything reaches reporting, the pipeline writes a QA record of the model's disagreements and open issues, then halts at a human review gate. Only the flagged item blocks; the rest of the queue keeps draining.

Stack: Ruby on Rails 7.2, Hotwire, Sidekiq, Portkey, OpenAI, Google Drive API, MySQL

Result: live and unattended for over a year, processing documents in batches with a 98%+ classification confidence rate before anything hits human review.

The pattern that generalizes: write the failure path before the happy path. If you can't answer "what happens when this specific stage fails at 2am," you don't have a production system yet, you have a demo.

More on how I build these → https://farzamazhar.com/#work

DEV Community: Farzam

What it actually takes to keep an LLM pipeline running for a year, unattended