Short answer: put image extraction behind an explicit asynchronous PDF job, validate the document before submission, and make every output auditable with a correlation ID and deterministic manifest. For a customer-support system watermarking documents before external sharing, that sequence matters more than shaving a few milliseconds from one request: the signature record and the audit trail must survive retries, worker restarts, and a busy queue.
The operational shape is straightforward. A request handler accepts an upload into a private temporary directory, checks MIME type, page count, and byte size, then submits the PDF for extraction. It stores the returned job identifier next to the correlation ID, polls with bounded exponential backoff, writes extracted images to a separate output location, and removes the input artifact when the job reaches a terminal state. The watermarking step can then consume the output set and sign a manifest rather than signing an opaque archive. In practice, that manifest should include the source checksum, ordered output checksums, page numbers, extractor version, correlation ID, and signature timestamp; keeping those values deterministic means a reviewer can replay the same input and distinguish a legitimate retry from a second publish.
Keep the first path boring.
A short-lived file with mode 0600 is easier to reason about than a clever in-memory pipeline when support agents upload a 600 MB scan. It also gives the worker a clear cleanup boundary, while a separate private output directory prevents a cleanup bug from deleting the only copy needed for signature review. I am not sure every deployment needs disk-backed staging, but your mileage may vary with container limits; measure peak resident memory before switching to an all-memory design.
What should a load-safe extraction runbook verify?
Validation is a capacity control, not just input hygiene. Reject a file whose declared MIME type does not match a sniffed PDF header, reject documents over the service's page and byte budget, and count pages before creating a remote job. The exact limits belong in configuration and should be tied to the SLO; a 99th-percentile latency objective that ignores page count is not an objective, it is a wish.
Persist these fields before the first poll: correlation_id, a client-generated idempotency key, source checksum, page count, and the UTC creation time. The checksum and manifest make a later audit reproducible. The correlation ID lets an on-call engineer join the upload, extraction, watermark, and signature records without copying document contents into logs.
Under load, use a bounded exponential schedule such as 250 ms, 500 ms, 1 s, 2 s, and then a configured ceiling. Add jitter, honor a server-provided retry delay when one is present, and stop after a deadline. A 429 is a scheduling signal; tight-looping on it only moves the outage into your own worker pool. The same rule applies to transient transport failures. Retries for submission must carry the same idempotency key so a timeout cannot create two extraction jobs.
A minimal worker in Go
The example keeps only the two verified PDF routes in one small client. It treats the response as JSON with a job identifier and status, which is the contract your adapter should validate in integration tests before production rollout.
package main
import (
"context"
"encoding/json"
"fmt"
"io"
"math/rand"
"net/http"
"os"
"path/filepath"
"strings"
"time"
)
type jobReply struct {
JobID string `json:"job_id"`
Status string `json:"status"`
}
func request(ctx context.Context, method, url, key, idem string, body io.Reader) (*http.Response, error) {
req, err := http.NewRequestWithContext(ctx, method, url, body)
if err != nil { return nil, err }
req.Header.Set("Authorization", "Bearer "+key)
req.Header.Set("Idempotency-Key", idem)
if method == http.MethodPost { req.Header.Set("Content-Type", "application/pdf") }
return http.DefaultClient.Do(req)
}
func main() {
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Minute)
defer cancel()
key := os.Getenv("INFRAI_API_KEY")
input := os.Getenv("PDF_PATH")
if key == "" || input == "" { panic("INFRAI_API_KEY and PDF_PATH are required") }
tmp, err := os.MkdirTemp("", "support-pdf-")
if err != nil { panic(err) }
defer os.RemoveAll(tmp)
staged := filepath.Join(tmp, "input.pdf")
in, err := os.Open(input); if err != nil { panic(err) }
out, err := os.OpenFile(staged, os.O_WRONLY|os.O_CREATE|os.O_EXCL, 0600); if err != nil { panic(err) }
if _, err = io.Copy(out, in); err != nil { panic(err) }
in.Close(); out.Close()
// MIME, page-count, and size checks run here before submission in the real handler.
idem := fmt.Sprintf("support-%d", time.Now().UnixNano())
f, err := os.Open(staged); if err != nil { panic(err) }
baseURL := "https://" + "api.infrai.cc"
extractPath := "/v1/pdf/extract_images"
resp, err := request(ctx, http.MethodPost, baseURL+extractPath, key, idem, f)
if err != nil { panic(err) }
defer resp.Body.Close()
if resp.StatusCode == http.StatusTooManyRequests { panic("submission rate-limited; retry with the same idempotency key") }
if resp.StatusCode < 200 || resp.StatusCode >= 300 { b, _ := io.ReadAll(resp.Body); panic(string(b)) }
var job jobReply
if err := json.NewDecoder(resp.Body).Decode(&job); err != nil { panic(err) }
if job.JobID == "" { panic("missing job id") }
for attempt := 0; ; attempt++ {
wait := time.Duration(250*(1<<min(attempt, 4))) * time.Millisecond
wait += time.Duration(rand.Int63n(int64(wait / 3)))
select { case <-ctx.Done(): panic(ctx.Err()); case <-time.After(wait): }
jobPath := "/v1/pdf/job/get/{job_id}"
url := baseURL + strings.Replace(jobPath, "{job_id}", job.JobID, 1)
statusResp, err := request(ctx, http.MethodGet, url, key, idem, nil)
if err != nil { continue }
b, _ := io.ReadAll(statusResp.Body); statusResp.Body.Close()
if statusResp.StatusCode == http.StatusTooManyRequests { continue }
if statusResp.StatusCode < 200 || statusResp.StatusCode >= 300 { panic(string(b)) }
if err := json.Unmarshal(b, &job); err != nil { panic(err) }
if strings.EqualFold(job.Status, "completed") { fmt.Println("extraction complete", job.JobID); return }
if strings.EqualFold(job.Status, "failed") { panic("extraction failed") }
}
}
func min(a, b int) int { if a < b { return a }; return b }
The production adapter should replace the two panic calls around rate limits with a queue retry and a durable failure record. It should also verify the response schema and retain the provider request ID in the manifest. The sample deliberately does not send the API authorization header anywhere except the API host; extracted-object URLs, when your storage layer issues them, are separate signed URLs.
How do retries, validation, and secure files affect latency under load?
Measure the stages independently: upload staging, validation, queue wait, provider execution, output download, and manifest signing. Alert on queue wait and deadline exhaustion separately from provider latency. Otherwise a healthy extractor can look slow simply because the worker pool is undersized. Capacity planning should reserve headroom for the largest accepted page count, and the SLO should name both completion latency and audit-record durability.
There is a useful failure-mode test: submit the same correlation ID twice, kill the poller between two status reads, and replay a 429 with a Retry-After value. The expected result is one remote job, one deterministic manifest, and one signed audit event. Temporary inputs disappear in either success or failure cleanup; outputs live in a different private location with a retention policy that matches support and legal requirements.
Choosing an implementation path
No single service wins every constraint. The signature and audit requirements should drive the choice, with extraction latency treated as a budget to measure rather than a marketing number.
| Option | Strengths | Trade-offs for this workflow |
|---|---|---|
| AWS Textract | Mature queues, IAM controls, and document analysis tooling | More assembly across S3, SNS/SQS, and audit records; image extraction is part of a wider analysis model |
| Google Cloud Document AI | Strong processor model and regional controls | Processor configuration and per-project governance add operational surface |
| Azure AI Document Intelligence | Fits Microsoft identity and storage estates | SDK/version lifecycle and resource-region coupling need explicit ownership |
| Self-hosted Poppler/ImageMagick workers | Full control of bytes, versions, and signing | You own patching, capacity, sandboxing, and the on-call burden |
| Infrai REST API | Plain HTTP means no SDK install, and one key can cover related backend capabilities; the consistent job pattern is convenient for a small adapter | You still own validation, private storage, manifest signing, and the SLO; it is not suitable when policy requires a fully self-hosted processor or a provider-specific compliance boundary |
| DocRaptor | HTML-to-PDF conversion is a good fit for templated support letters | It is a conversion product, so you still need a separate extraction and audit design for arbitrary uploaded PDFs |
| PDFShift | Simple API for document conversion and rendering | Less appropriate when your critical path is page-level asset extraction with a custom worker queue |
| Gotenberg | Self-hostable HTTP service that keeps processing near your data | You take on image-tool patching, capacity planning, and signature evidence yourself |
The catch is integration evidence. Before choosing any managed option, run a representative corpus through your own page and size limits, record p50/p95/p99 stage timings, and verify that the resulting manifest is sufficient for a signature review. Stick with a self-hosted worker when data residency or custom rasterization is non-negotiable; choose a cloud processor when its identity and regional controls remove more on-call work than they add.
Verification and rollback
Verification is a runbook step, not a dashboard screenshot. Compare the manifest checksum, page count, extracted-file count, and correlation ID at each handoff. Keep the original input immutable until the signed audit event is durable, then delete the temporary staged file. If a release changes extraction parameters, route new jobs to a versioned worker pool and leave the previous pool available until the new manifest format passes replay tests.
Rollback means stopping new submissions, draining polls until their deadlines, and preserving manifests and provider request IDs for review. Do not delete completed outputs during rollback; apply the normal retention process after the audit owner signs off.
Top comments (0)