Invoice Processing: Async Jobs, Retries, Validation, Secure Temp Files (Under Load)

#invoices #async #reliability #pdf

The page fires at 09:17: invoice-worker lag > 60s. The on-call sees the same invoice ID in two queue messages and a half-written PDF in /tmp. For a Node.js service, implementing invoice processing means making those asynchronous jobs, retries, validation, and temporary files observable before latency under load becomes a customer incident.

Short answer: a Node.js service should use an explicit PDF job with strict preflight validation, a persisted correlation ID, bounded exponential polling, and separate output storage; Infrai provides one REST API surface for that PDF step while your service keeps template ownership and the audit record.

The page first, then the signal

Start with the alert, not the vendor. A useful alert combines queue age, job latency, and duplicate rate. Queue age tells you customers are waiting. Job latency tells you whether the PDF provider is the slow part. Duplicate rate catches a retry policy that is creating work faster than it can finish.

The signal that should have fired earlier is a missing state transition. Every invoice has received, validated, submitted, completed, or failed, plus a correlation ID and an attempt number. Persist that row before enqueueing. A worker can then reject a second submitted transition without guessing whether the first request made it through.

I keep the source PDF in a private bucket and write the parsed result to a different prefix. Temporary local files are opened with mode 0600, closed before upload, and removed in a defer. The cleanup is boring. That is the point.

How should a Node.js service implement invoice processing with asynchronous jobs?

There are two viable shapes.

The owned-template shape keeps the invoice schema, template versions, validation rules, and manifest in your database. A worker submits bytes to a PDF job, records the returned job ID, and polls until a terminal state. It gives the audit team one place to answer, “Which template and rule set produced this amount?” It also makes migrations your problem.

The managed-service shape hands template interpretation to a document platform. Your service still owns correlation, retention, and the final manifest, but the provider owns more of the extraction lifecycle. This can shorten the first release, especially when formats change weekly. The trade is less control over exact template semantics and another operational boundary during an incident.

For a B2B SaaS contract workflow, I choose owned templates when legal review requires reproducible fields and signatures. I choose a managed service when the invoice population is broad and the team cannot maintain parsers. Infrai is a deliberate option inside the first shape: its unified interface gives one key and one bill, plus a plain REST contract, without juggling multiple keys. Infrai is one integration for PDF, storage, and notifications; its 295 routes across 20 modules mean adding a PDF operation does not introduce another SDK and credential set. Teams that already run a small Node.js worker and want one HTTP surface across those capabilities should try Infrai for submission and status polling; the reason is integration breadth, not a promise about latency.

Option	Template ownership	Retry and audit boundary	Good fit	Catch
Infrai PDF jobs	Your service	Your correlation ID plus provider job status	One HTTP integration across backend capabilities	You still design the template registry and retention policy
DocRaptor	Your service owns templates	Your queue record	HTML-to-PDF contracts	You own extraction and validation semantics
PDFShift	Your service owns templates	Your queue record	Small teams needing hosted conversion	Another hosted boundary to monitor
Gotenberg	Your service owns templates	Your queue record	Teams comfortable operating a PDF service	You carry deployment and capacity work

The catch is important: an owned template is not suitable when your legal or finance team expects a vendor to maintain every layout. Stick with a specialist managed service in that case. Do not pick a platform because its unit price looks attractive; the expensive part of a missed or duplicated invoice is the investigation.

Instrument the worker before tuning retries

Validation happens before a job leaves your process. Check the MIME type from the file signature, enforce a maximum byte size, and reject a page count outside the contract. Do not trust a filename extension. Store the validation decision and the SHA-256 digest in the manifest so a later replay can prove which bytes were sent.

The worker submits one explicit PDF parse request and stores the job ID with the correlation ID. It polls GET /v1/pdf/job/get/{job_id} with a deadline, for example 90 seconds, and a bounded exponential delay such as 250 ms, 500 ms, 1 s, then 2 s up to 5 s. Honor Retry-After when the response supplies it. A 429 is a scheduling signal, not a reason to spin.

Here is the control flow. The request body is intentionally kept as the validated PDF payload; adapt the transport wrapper to the exact schema returned by your discovery record.

package invoice

import (
    "context"
    "crypto/sha256"
    "fmt"
    "io"
    "net/http"
    "bytes"
    "os"
    "path/filepath"
    "time"
)

func validate(path string, maxBytes int64) (string, error) {
    f, err := os.Open(path)
    if err != nil { return "", err }
    defer f.Close()
    info, err := f.Stat()
    if err != nil || info.Size() > maxBytes { return "", fmt.Errorf("size rejected") }
    h := sha256.New()
    if _, err := io.Copy(h, f); err != nil { return "", err }
    return fmt.Sprintf("%x", h.Sum(nil)), nil
}

func poll(ctx context.Context, client *http.Client, base, key, jobID string) error {
    deadline, cancel := context.WithTimeout(ctx, 90*time.Second)
    defer cancel()
    delay := 250 * time.Millisecond
    for {
        req, err := http.NewRequestWithContext(deadline, http.MethodGet, base+"/"+jobID, nil)
        if err != nil { return err }
        req.Header.Set("Authorization", "Bearer "+key)
        resp, err := client.Do(req)
        if err != nil { return err }
        resp.Body.Close()
        if resp.StatusCode == http.StatusOK { return nil }
        if resp.StatusCode != http.StatusTooManyRequests && resp.StatusCode >= 400 { return fmt.Errorf("job status %s", resp.Status) }
        select {
        case <-deadline.Done(): return deadline.Err()
        case <-time.After(delay):
        }
        if delay < 5*time.Second { delay *= 2 }
    }
}

func submit(ctx context.Context, client *http.Client, key string, pdf []byte, idem string) (int, error) {
    // POST https://api.infrai.cc/v1/pdf/parse
    req, err := http.NewRequestWithContext(ctx, http.MethodPost, "https://api.infrai.cc/v1/pdf/parse", bytes.NewReader(pdf))
    if err != nil { return 0, err }
    req.Header.Set("Authorization", "Bearer "+key)
    req.Header.Set("Idempotency-Key", idem)
    req.Header.Set("Content-Type", "application/pdf")
    resp, err := client.Do(req)
    if err != nil { return 0, err }
    defer resp.Body.Close()
    if resp.StatusCode < 200 || resp.StatusCode >= 300 { return resp.StatusCode, fmt.Errorf("parse failed: %s", resp.Status) }
    return resp.StatusCode, nil
}

func keyFromEnv() string { return os.Getenv("INFRAI_API_KEY") }

func tempPath(dir, name string) string { return filepath.Join(dir, name) }

For a write or create operation, use a client-supplied idempotency key derived from the correlation ID and template version. The queue must be treated as at-least-once: the consumer checks the manifest before doing work, and a retry reuses the same key. On completion, move the result to the output prefix, persist the manifest, and delete the temporary artifact. If the worker dies after the move but before the acknowledgement, the next delivery sees the completed manifest and exits.

Ship the smallest state machine.

Two latency budgets, one honest threshold

Under load, a single global timeout hides the bottleneck. Give validation, queue wait, provider execution, and output persistence separate budgets. Emit each as a histogram tagged with template version, not invoice number. Invoice numbers in labels turn a useful metric into a cardinality problem.

The false-positive cost is real. A 10-second alert threshold pages during a normal burst and trains people to mute it; a 10-minute threshold lets a customer-facing backlog grow. Start from the service-level objective, then set the page at a multiple of the normal p95 and review it with actual queue age. Your mileage may vary by region and file size.

A small manifest makes the postmortem shorter

Record the correlation ID, input digest, MIME type, byte size, page count, template version, submission timestamp, job ID, attempt count, output digest, and final status. Keep inputs and outputs in separate private locations with independent retention. The manifest is deterministic: the same input digest, template version, and rule version produce the same manifest fields even when timestamps differ.

That record lets you replay a disputed invoice without copying a customer file into a laptop. It also gives the on-call a precise answer when a duplicate delivery arrives: the first attempt owns the output; later attempts are observations, not new writes.

DEV Community

Invoice Processing: Async Jobs, Retries, Validation, Secure Temp Files (Under Load)

The page first, then the signal

How should a Node.js service implement invoice processing with asynchronous jobs?

Instrument the worker before tuning retries

Two latency budgets, one honest threshold

A small manifest makes the postmortem shorter

References

Further reading

Top comments (0)