A PDF endpoint becomes an operational liability when schema discovery mutates the document, retains the upload without a deadline, or hides signature evidence behind a render response. Short answer: a US/EU SaaS should own four narrow application contracts: discover, render, evidence, and erase. Keep discovery read-only, make render idempotent, and bind every signed output to immutable evidence and an explicit retention decision.
I've been paged for missed jobs and duplicate deliveries. The relevant lesson is dull but durable: retries are normal, and a document pipeline must make a repeated request boring. A duplicate render must return the same logical result; a discovery retry must never create a signature or flatten a form.
These are contracts at your service boundary, not routes copied from a PDF vendor. An adapter can call an in-process parser, a self-hosted service, or an external processor behind them. That separation is the main decision: select the execution location per document class without changing the product-facing API.
Read-only means read-only.
How should a US/EU SaaS choose PDF endpoints for form schema discovery?
Use four capabilities, with authorization and tenant isolation applied to every call. Keep the nouns and response envelopes stable even if the underlying processor changes.
| Capability | Suggested application contract | Side effect | Operational decision |
|---|---|---|---|
| Discover | POST /pdf/forms/discover |
None beyond bounded processing | Run locally for sensitive or latency-critical files; use a remote adapter only under an approved data-flow policy |
| Render | POST /pdf/forms/render |
Produces a filled, optionally flattened artifact | Require an idempotency key and the discovered schema version |
| Evidence | GET /pdf/forms/evidence/{operation_id} |
Read only | Return hashes, timestamps, actor references, policy version, and signature-verification state |
| Erase | DELETE /pdf/forms/artifacts/{operation_id} |
Deletes eligible retained artifacts | Preserve only records that the applicable retention policy requires |
Why POST for discovery? The request can contain document bytes and processing options; it is read-only in domain terms even though it is not an HTTP safe method. Don't pretend a large PDF is a query string. DELETE expresses an erase request, but the service still has to evaluate legal-hold and retention policy before removing evidence.
The discovery response should expose stable field identifiers, field type, page, rectangle, required state, and a schema digest. It should not return a flattened PDF. A practical response also distinguishes warnings from rejection: an unknown field appearance may be a warning, while an encrypted file that cannot be opened is a terminal client-visible result. I'm not sure one latency budget fits both born-digital forms and scanned documents; measure those classes separately, because OCR changes the work rather than merely making the same parser slower.
Treat schema discovery as an untrusted read path
PDF bytes are untrusted input. Put discovery behind byte, page, execution-time, and concurrency limits chosen from your own workload. Parse in an isolated worker with no ambient credentials, and allow outbound network access only when the selected processor requires it. Those controls are architectural recommendations; the actual limits must come from load tests and the largest legitimate forms you accept.
Do less here.
Discovery should calculate a digest over the original bytes, normalize field metadata into your internal schema, and discard temporary files when the request's policy says no retention. The browser Blob abstraction represents immutable raw data and can be converted to an ArrayBuffer; it does not define server-side privacy or lifecycle policy. Don't confuse a convenient client object with consent, regional processing, or deletion.
This Go handler shows the boundary. The paths are illustrative application routes, and the parser interface is deliberately generic. Production code still needs authentication, tenant authorization, a hardened multipart reader, and worker isolation.
package main
import (
"context"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"io"
"net/http"
)
const maxPDFBytes int64 = 25 << 20 // Example policy: tune from accepted document classes.
type Field struct {
ID string `json:"id"`
Type string `json:"type"`
Page int `json:"page"`
Rect [4]float64 `json:"rect"`
Required bool `json:"required"`
}
type Parser interface {
Discover(context.Context, []byte) ([]Field, error)
}
type Server struct{ parser Parser }
func (s Server) discover(w http.ResponseWriter, r *http.Request) {
if r.Method != http.MethodPost {
http.Error(w, "method not allowed", http.StatusMethodNotAllowed)
return
}
body := http.MaxBytesReader(w, r.Body, maxPDFBytes)
pdf, err := io.ReadAll(body)
if err != nil {
http.Error(w, "document exceeds policy or cannot be read", http.StatusRequestEntityTooLarge)
return
}
if len(pdf) < 5 || string(pdf[:5]) != "%PDF-" {
http.Error(w, "unsupported document", http.StatusUnsupportedMediaType)
return
}
fields, err := s.parser.Discover(r.Context(), pdf)
if err != nil {
http.Error(w, "document cannot be processed", http.StatusUnprocessableEntity)
return
}
sum := sha256.Sum256(pdf)
response := struct {
DocumentDigest string `json:"document_digest"`
SchemaVersion string `json:"schema_version"`
Fields []Field `json:"fields"`
}{
DocumentDigest: hex.EncodeToString(sum[:]),
SchemaVersion: "1",
Fields: fields,
}
w.Header().Set("Content-Type", "application/json")
_ = json.NewEncoder(w).Encode(response)
}
A 422 here means the bytes passed transport checks but not document processing. Keep that distinct from 413, because the runbook action differs: clients can reduce a document that violates size policy, while an unreadable form needs inspection or a different accepted input. Don't log the bytes or field values in either case.
Make fill and flatten one idempotent operation
Filling and flattening belong in one render request when the desired artifact is fixed at request time. Splitting them into two externally retried mutations creates an ambiguous middle state: the form may be filled but not flattened, and a worker replay may produce another artifact. The render request should carry the original document digest, discovered schema version, field values, flatten mode, and an idempotency key. Store the request digest beside the key. If the same key arrives with different content, return 409 Conflict; if it matches, return the existing operation.
Consider the ordinary failure sequence. A client submits key media-release-184, the service commits operation op-73, and the response is lost before the client receives it. The retry carries the same key and request digest, so the API returns op-73 rather than enqueuing another render. If an operator edits a field and retries with that old key, the changed digest produces 409 Conflict; the client must use a new key for the new intent. This is why the key belongs to the intended render, not the network attempt, and why a unique database constraint must arbitrate concurrent requests instead of a check-then-insert in application memory.
Retries happen.
type RenderRequest struct {
DocumentDigest string `json:"document_digest"`
SchemaVersion string `json:"schema_version"`
Values map[string]string `json:"values"`
Flatten bool `json:"flatten"`
}
type Operation struct {
ID string `json:"id"`
RequestDigest string `json:"request_digest"`
Status string `json:"status"`
}
type OperationStore interface {
FindByKey(context.Context, string) (Operation, bool, error)
Create(context.Context, string, string, RenderRequest) (Operation, error)
}
var ErrKeyConflict = errors.New("idempotency key reused with different content")
func render(ctx context.Context, store OperationStore, key string, req RenderRequest) (Operation, error) {
encoded, err := json.Marshal(req)
if err != nil {
return Operation{}, err
}
sum := sha256.Sum256(encoded)
digest := hex.EncodeToString(sum[:])
existing, found, err := store.FindByKey(ctx, key)
if err != nil {
return Operation{}, err
}
if found {
if existing.RequestDigest != digest {
return Operation{}, ErrKeyConflict
}
return existing, nil
}
return store.Create(ctx, key, digest, req)
}
The second snippet shares imports with the first and adds errors. The JSON hashing assumes one service implementation controls serialization. If different languages generate the digest, define and test a canonical representation before relying on it. Otherwise equivalent maps can produce different byte sequences across implementations. More important, don't reuse the PDF digest as the render-request digest: one identifies input bytes, while the other identifies an intended operation.
Flatten only after validation confirms that every required field has a value and the signature policy permits it. A flattened file is useful for distribution because fields no longer invite casual editing, but it is not, by itself, evidence of who approved anything. Keep the unflattened source only when the retention policy calls for it.
Bind signatures to evidence, not to a download URL
A signed-media workflow needs an audit object that outlives a short-lived artifact URL and can be verified without trusting logs scattered across workers. Record the input and output digests, operation ID, schema version, policy version, actor reference, event times, signature method, verification state, and the reason for any manual override. Use append-only semantics for events; corrections become new events rather than edits to history.
This does not make every signature legally sufficient. In the United States, the ESIGN framework and its implementing context matter; in the European Union, eIDAS defines categories and effects for electronic signatures. Product and counsel must decide the required signature level and evidence for the transaction. Engineering's job is to preserve the decision, bind it to exact bytes, and make later verification possible.
The invariant is byte identity. Approval of digest A cannot silently migrate to rendered digest B. Verify the output digest before release, then write the verification result to the evidence stream. Emit metrics for discovery rejection by reason, render age, duplicate-key hits, evidence write latency, erase decisions, and queue redelivery. Alert on age and invariant violations, not raw request volume.
Privacy changes the storage topology. Classify uploads and field values, document each processing region and subprocessor boundary, encrypt transport and storage, restrict operator access, and put a deletion deadline or legal basis on every retained object. GDPR data minimization and storage limitation make "keep it in case support asks" a poor default for EU personal data. For US deployments, retention still needs a declared business and legal policy rather than an infinite object-store lifecycle. Keep audit evidence and document content as separate classes so policy can erase one without accidentally destroying the other.
Keep the bytes out of logs.
Where does this four-endpoint design stop fitting?
The catch is that an application-owned adapter adds code, on-call ownership, contract tests, and migration work. It is not suitable when a one-off internal workflow has no retry path, no signature consequence, and documents can be handled under an existing approved tool; stick with that controlled process instead of building a service. It also does not remove processor-specific limits. Your adapter must surface capability differences honestly, not reduce every failure to "invalid PDF."
Run parsing in-process when the library's security posture, format fidelity, licensing, and resource profile meet your review, and when keeping bytes inside your boundary is the overriding constraint. Choose an isolated self-hosted worker when crash containment or independent scaling matters. Choose remote processing only when its fidelity or managed operations justify the added network latency, regional transfer analysis, retention configuration, and supplier oversight. Your mileage may vary by form corpus; a representative acceptance suite resolves more uncertainty than a feature matrix.
Before release, build a corpus containing blank and prefilled forms, repeated field names, checkboxes, radio groups, multiline text, rotated pages, embedded fonts, encrypted samples you are authorized to test, and signed documents. Assert schema stability, rectangle placement, required-field validation, flatten behavior, digest binding, retry identity, and erasure decisions. Deploy the adapter behind a tenant allowlist, compare results without retaining extra content, and keep rollback at the adapter level.
The final choice is a control-plane decision, not a parser popularity contest. Own four small contracts, test fidelity against the documents that matter, measure latency by document class, and make privacy, retention, retry, and evidence behavior explicit before the first production upload.
Top comments (0)