Hardening Document Extraction in a NestJS CDSS

Latency and Failure Modes in Clinical AI Pipelines

In a clinical decision support system (CDSS), document ingestion forms the base of the reasoning pipeline. When a clinician uploads an unstructured record, scanned note, or lab report, the system must parse the text, invoke medical AI agents via an LLM gateway (such as OpenRouter), extract clinical entities, and update the patient's longitudinal record in PostgreSQL.

In early iterations of this pipeline, document extraction suffered from three distinct architectural bottlenecks:

1. Network Overhead on LLM Invocations: Every extraction prompt initiated a new connection, paying the latency cost of a fresh TLS handshake per agent call.
2. Ambiguous Parsing States: Network timeouts and schema mismatches returned empty payloads that the backend could not differentiate from genuinely empty documents.
3. Fragile Orchestration: An unhandled exception in an individual extraction agent halted execution across the entire reasoning case, while interrupted ingestion jobs left documents in a permanent "reading" state.

Recent updates to Documedic resolve these issues by introducing persistent connection pools, explicit ingestion state transitions, and isolated agent execution.

Persistent LLM Connection Pooling in NestJS

To minimize round-trip latency when dispatching prompts to OpenRouter, the NestJS extraction service now maintains a persistent agent pool with keep-alive enabled, avoiding repeated TLS negotiations.

import { Injectable, OnModuleDestroy } from '@nestjs/common';
import { Agent } from 'undici';

@Injectable()
export class OpenRouterClientService implements OnModuleDestroy {
  private readonly dispatcher = new Agent({
    keepAliveTimeout: 30_000,
    keepAliveMaxTimeout: 60_000,
    connections: 50,
  });

  async queryAgent(prompt: string, model: string) {
    return fetch('https://openrouter.ai/api/v1/chat/completions', {
      method: 'POST',
      dispatcher: this.dispatcher,
      headers: {
        'Authorization': `Bearer ${process.env.OPENROUTER_API_KEY}`,
        'Content-Type': 'application/json',
      },
      body: JSON.stringify({ model, messages: [{ role: 'user', content: prompt }] }),
    });
  }

  async onModuleDestroy() {
    await this.dispatcher.destroy();
  }
}

Graceful shutdown hooks ensure that connection pools drain cleanly on service restarts rather than leaking open sockets or leaving pending requests unresolved.

Differentiating Empty Documents from Extraction Failures

A critical failure mode in clinical document processing is the silent dropping of data. If an OCR scanner yields unparseable text or an LLM call fails downstream, treating the result as an empty document leads to missing diagnoses in the patient chart.

Documedic now enforces strict status checks across the PostgreSQL ingestion schema. The document pipeline models state transitions through distinct terminal states:

CREATE TYPE document_status AS ENUM (
  'pending',
  'extracting',
  'completed',
  'empty_content',
  'extraction_failed'
);

When processing incoming scans:

• Documents with zero detected text tokens are classified as empty_content.
• Agent exceptions or API transport errors trigger retries before transitioning the record to extraction_failed.
• Interrupted jobs are recovered on startup using heartbeat timestamps, preventing jobs from stalling indefinitely in the extracting state.

Fault Isolation in Multi-Agent Clinical Reasoning

When synthesizing insights from clinical records, Documedic runs specialized sub-agents to extract discrete medical entities, such as diagnoses, active medications, and visit summaries.

Previously, a failure in one sub-agent caused the entire case execution to abort. The current pipeline executes agent tasks using an isolated settlement strategy:

const results = await Promise.allSettled([
  this.diagnosisAgent.extract(documentText),
  this.medicationAgent.extract(documentText),
  this.encounterAgent.extract(documentText),
]);

for (const result of results) {
  if (result.status === 'fulfilled') {
    await this.persistEntities(result.value);
  } else {
    this.logger.error(`Sub-agent failure: ${result.reason}`);
    // Log partial failure to audit trail without invalidating valid extractions
  }
}

By decoupling individual extraction routines, healthy agents continue populating valid chart data while failed components flag explicit audit entries for review, ensuring reliable clinical decision support under real-world runtime conditions.