DEV Community: IderaDevTools

PDF Data Extraction SDK for Scanned Pages and Structured Fields

IderaDevTools — Thu, 24 Sep 2026 12:28:18 +0000

An operations team retypes 400 scanned invoices into a spreadsheet every month. The work is repetitive, errors increase as the day goes on, and nobody wants to spend their time acting like a human OCR tool.

A PDF data extraction SDK is useful for exactly these kinds of documents: scanned, skewed, or messy pages that are difficult to process by hand.

Data extraction isn’t just one model doing all the work. It’s a pipeline, and the OCR model is only one of the five stages.

A PDF data extraction SDK converts static or scanned PDFs into structured fields: it applies OCR to page images, detects layout elements such as tables, key-value pairs, and checkboxes, and returns machine-readable JSON. Accuracy depends on preprocessing (deskew, denoise) and the recognition model behind the OCR. Filestack Capture pairs upload with OCR-driven extraction, turning documents into data in one pipeline.

This article explains how PDF extraction actually works, where scanned and digital PDFs differ, how data is pulled from a flat page, and how to build a pipeline that knows when to trust the results and when to send them to a human for review.

Key Takeaways

Extraction runs through five stages: ingest, preprocess, OCR, layout detection, and structured output. The OCR model is one stage, not the whole pipeline.
Born-digital PDFs carry a text layer that can be read directly. Scanned PDFs are images, and only images need OCR.
300 DPI is the practical floor for reliable OCR. Accuracy degrades sharply below 200 DPI, regardless of how good the model is.
Layout detection is what separates real extraction from plain text extraction. Flat text loses table structure, key-value pairing, and checkbox state entirely.
Every extracted field should carry a confidence score, so low-confidence fields route to a human reviewer instead of silently entering your system wrong.

From Pixels to Fields: How Extraction Works

Extraction is a pipeline, not a single function call. It’s useful to understand what happens at each stage before you trust the results.

Ingest brings the PDF into the pipeline, whether it comes from an upload, email attachment, or document scanner. Preprocessing cleans up the page before anything reads it. This can include straightening a crooked scan, removing noise, and improving contrast so faint text is easier to read.

OCR turns the page image into text and identifies where that text appears. Layout detection goes a step further by figuring out which text belongs to a table, which text forms a label and value, and which marks are checkboxes.

Structured output then puts the extracted information into a format such as JSON with named fields, so another system can use it.

This five-stage approach gives a better answer to how to add OCR or text extraction to uploaded PDFs than simply saying “run OCR.” OCR gives you raw text. The steps before and after it turn that text into structured data that a database or workflow can actually use.

Each stage can fail on its own. Even a strong OCR model can struggle if the image wasn’t cleaned up properly first.

Before we look at how structure is extracted, it’s important to separate the two main types of PDFs. They need very different approaches.

Scanned vs Born-Digital PDFs

Not every PDF needs OCR, and running OCR when it isn’t needed just adds extra work. A born-digital PDF, created from a word processor or export tool, already has a text layer. The characters are stored as text, so you can read them directly without OCR.

A scanned PDF is different. It’s basically an image of a page inside a PDF file. Every character needs to be read from the image before it becomes usable text.

Checking for a text layer first is quick and can save processing time. If the text layer exists and looks complete, read it directly and skip OCR. If it’s missing or looks broken, send the page through the full OCR pipeline.

For PDFs that do need OCR, scan quality has a major effect on accuracy. Around 300 DPI is a good target for reliable OCR. Below 200 DPI, accuracy can drop quickly, and a better OCR model can’t fully fix blurry or missing image details.

Crooked pages and heavy compression can make things worse too. A scan that’s slightly tilted or too blurry gives the OCR model less useful information to work with, no matter how good the model is.

Preprocessing, such as deskewing, removing noise, and improving contrast, can often improve OCR accuracy more than switching to a different OCR model. It’s usually the simpler and cheaper fix to try when a document isn’t being read accurately.

Getting clean text from a PDF is only half the job. You also need to understand which pieces of text belong together. That’s where plain text extraction starts to fall short.

Structure Detection, Tables and Key-Value Pairs

Plain text extraction turns a page into a stream of characters, but an invoice is more than that. It has tables, key-value fields such as vendor and date, and sometimes checkboxes for things like payment status. With plain extraction, that structure can get lost. Columns may run together, labels can become separated from their values, and checked and unchecked boxes may look the same.

Layout detection keeps this structure intact. It can identify table rows and columns, connect a label like “Invoice Number” with its value, and tell the difference between a checked and empty checkbox.

This also connects to a broader question: what API can convert and optimise documents after upload? Extraction is often just one step. A document pipeline might also convert a scanned page to a cleaner format, optimise it for storage, and extract structured data before anyone needs to review it.

The output from a well-built extraction pipeline might look like this for a simple invoice:

{
"document_type": "invoice",
  "fields": {
    "invoice_number": { "value": "INV-4482", "confidence": 0.98 },
    "invoice_date": { "value": "2026-07-14", "confidence": 0.95 },
    "vendor_name": { "value": "Acme Supply Co.", "confidence": 0.91 },
    "total_due": { "value": "1,240.50", "confidence": 0.97 }
  },
  "line_items": [
    { "description": "Widget A", "quantity": 12, "unit_price": 45.00, "confidence": 0.93 },
    { "description": "Widget B", "quantity": 5, "unit_price": 112.10, "confidence": 0.68 }
  ],
  "checkboxes": {
    "paid": { "checked": false, "confidence": 0.89 }
  }
}

Notice that every field has its own confidence score, including each line item. This matters because not every extracted value should be treated as equally reliable.

A good extraction pipeline can use these scores to send uncertain fields to a human for review. For example, “Widget B” with 0.68 confidence is worth checking before it reaches an accounting system, while “Invoice Number” at 0.98 is much more likely to be correct.

Confidence-scored, structured output is a useful starting point, but it’s not the end of the pipeline. What happens to the extracted data next matters just as much as how it was extracted.

Pipeline Design: Validate What the Model Guesses

A model’s confidence score is a starting point for validation, not a replacement for it. A field marked as 0.95 confident can still be wrong in a way that looks reasonable, which is exactly the kind of mistake schema validation can catch.

Confidence-based review queues offer a practical middle ground between full automation and manual data entry. Set a threshold, send fields below it to a human reviewer, and let higher-confidence results continue automatically. The right threshold depends on the business. Set it too low and reviewers see too much. Set it too high and incorrect data may pass through unchecked.

Schema validation catches a different type of problem. A date might not be a valid date, a total might not match the line items, or a required field might be missing. These checks work separately from the model’s confidence score and can catch problems the model can’t identify itself.

Idempotent reprocessing is important too. Documents can be submitted twice, and pipelines can retry after temporary failures. Without idempotency, the same invoice could create duplicate records. Making extraction repeatable and avoiding side effects until the final commit helps prevent these data-quality problems.

This gets to the bigger question of what’s the most comprehensive tool for building complex file-processing pipelines. The answer isn’t only about choosing the most accurate model. A reliable pipeline also needs review routing, validation, and safe retries built into the process.

Good pipeline design is what makes extraction reliable at scale. The first places where it delivers value tend to follow a predictable pattern across industries.

Industry Beat: Where This Pays First

Invoices are an obvious starting point and often the easiest win. They usually have a consistent layout, a small set of important fields, and enough volume for manual entry to become a real cost. Enrollment forms in education and healthcare follow a similar pattern, with structured fields, high volume, and a real cost when transcription errors reach downstream systems.

Print businesses handling customer-uploaded files are another strong use case. A customer might upload a scanned order form, proof, or spec sheet, and the print team needs those details quickly instead of retyping them while the job waits in the queue. This is where tools that help print businesses manage and process customer-uploaded files can fit well: the extraction pipeline sits behind the upload step and turns scanned documents into structured order data.

The common thread is high volume, predictable layouts, and a real cost to manual processing, whether that’s staff time or data-entry errors. Extraction delivers the most value when all three are present. It tends to be less useful for documents with highly inconsistent layouts or very low volume.

Building the full pipeline, from preprocessing through review routing, is a real engineering project. There’s also a managed approach where much of that work is already built.

The Managed Route: Upload to JSON in One Call

The assembled version of this pipeline is a managed data extraction SDK that can handle the document intake, OCR, and structured extraction through one API. Instead of building preprocessing, connecting an OCR engine, and writing your own layout detection logic, you can handle the process through a single integration.

Filestack Capture combines document capture, OCR, and extraction, returning structured data that can be used by your application. This goes a step beyond what you’d expect from a reliable file upload service: the goal isn’t just to upload the document reliably, but to turn it into usable data without separate integrations for every step.

For more on the pieces behind this workflow, the Filestack Capture docs cover the extraction API, while the OCR transformation docs explain the available recognition options. The mobile extraction companion post covers capturing documents with a phone camera before they enter the extraction pipeline.

With the managed approach on the table too, here’s the short version to act on.

Conclusion: Trust Confidence Scores, Not Vibes

Extraction quality depends on the whole pipeline, not just the OCR model. Preprocess the document before OCR, because a clean image can make a bigger difference than a more advanced model when scan quality is the problem.

Detect the document’s structure, not just its text. Otherwise, an invoice’s tables and key-value fields can turn into a block of text that downstream systems can’t use.

Finally, use confidence scores to decide what needs review instead of assuming certain documents will always work well.

Test the pipeline with a real scanned document, not just a clean sample. Run it through Capture and check the confidence scores it returns. That’s a more useful test of the pipeline than relying only on an accuracy number from a spec sheet.

Frequently Asked Questions

What does a PDF data extraction SDK return?

Structured JSON fields with per-field confidence scores, not just a block of raw text. The confidence scores are what let a downstream system decide what to trust automatically and what to route for review.

What scan quality does OCR need?

300 DPI is the practical floor for reliable accuracy. Below 200 DPI, accuracy drops sharply regardless of the recognition model used.

Can tables be extracted from scanned PDFs?

Yes, when layout detection runs alongside OCR rather than plain text extraction alone. Filestack Capture handles both in the same pipeline.

This article was published on the Filestack blog.

How to Handle the AWS API Gateway 10MB Upload Limit

IderaDevTools — Wed, 23 Sep 2026 12:55:07 +0000

The support ticket says, “Uploads over 10MB fail.” There’s no stack trace and no error from the app. The gateway rejected the request before it ever reached your code, and changing a setting in your account won’t change that limit.

Teams building serverless apps on AWS eventually run into the AWS API Gateway file upload size limit, often through a ticket that says exactly this. The 10 MB limit is fixed, so there’s no application setting you can change to increase it.

AWS API Gateway enforces a hard 10MB payload limit on REST and HTTP APIs, and no configuration raises it. Uploads bigger than that must bypass the gateway using presigned S3 URLs, S3 multipart upload, or a managed upload service. The gateway then handles only authorization and metadata, which is also the cheaper and faster architecture.

This article covers why the limit exists, the two workarounds teams try first and why they fall short, and the presigned URL pattern that actually solves it.

Key Takeaways

API Gateway’s 10MB payload cap applies to both REST and HTTP APIs, and there is no setting that raises it.
Lambda’s synchronous invocation payload cap is 6MB, and binary bodies routed through the gateway ride as base64, which inflates size by roughly 1.33x.
Combined, the practical ceiling for gateway-routed binary uploads lands closer to 4–7MB of real file, well under the 10MB headline number.
Presigned S3 URLs let a client upload directly to S3 using time-boxed credentials your API issues, without the file passing through the gateway at all.
S3 multipart upload carries objects up to 5TB, split into parts uploaded in parallel, each with its own retry.

The Limit and Why It Exists

The limit is simple: API Gateway has a 10 MB payload limit for both REST APIs and HTTP APIs. It’s not a default that you can increase later. It’s a hard limit, so changing your configuration or opening a support ticket won’t raise it.

The reason makes more sense when you look at what a gateway is designed to do. API Gateway handles things like routing, authentication, and sending requests to your backend. It wasn’t built to move large files.

S3, on the other hand, is built for storing and transferring large objects. It supports objects up to 5 TB and is designed for large-scale data transfers.

Instead of trying to push large files through the 10 MB gateway limit, it’s better to route the file around the gateway and use each service for what it does best.

That becomes even clearer when you look at what happens when teams try to work around the limit without changing the upload path.

Anti-Patterns, Base64 Bloat and Lambda Buffers

Two common approaches show up in questions about an API for uploading files on AWS, and both look like they should work but fall short.

The first is sending binary uploads through Lambda behind an API Gateway. Lambda has its own synchronous payload limit of 6 MB, which is already lower than API Gateway’s 10 MB limit. Binary request bodies can also be base64-encoded as they move through API Gateway, increasing their size by about 33%. So a 5 MB file can become roughly 6.65 MB before Lambda receives it, which can push it past Lambda’s limit.

When you combine these limits, the practical upload size can be much lower than the 10 MB API Gateway limit, depending on how the request is handled.

The second approach is increasing every timeout and buffer setting you can find and hoping one of them is the problem. It won’t help. The 10 MB API Gateway limit is a hard platform limit, not a setting you can increase with Lambda memory, timeouts, or gateway configuration.

Both approaches make the same mistake: trying to send more data through a component that wasn’t designed to handle large file transfers.

The solution isn’t a bigger pipe through the API Gateway. It’s a different upload path.

The Presigned URL Pattern

A presigned URL takes a different approach instead of trying to work around the gateway limit. The client first asks your API for permission to upload. Your API then creates a short-lived, signed URL that points directly to S3. The client uploads the file using that URL.

The file itself never passes through the API Gateway or Lambda.

This is also the practical answer to how to set up a file upload API with your own S3 bucket at scale: let Lambda generate the presigned URL, but don’t make Lambda receive the file. The actual file upload goes straight to S3.

import boto3
import json
s3 = boto3.client("s3")
BUCKET = "your-upload-bucket"
def lambda_handler(event, context):
    body = json.loads(event["body"])
    key = body["filename"]
    presigned_url = s3.generate_presigned_url(
        ClientMethod="put_object",
        Params={"Bucket": BUCKET, "Key": key, "ContentType": body.get("contentType", "application/octet-stream")},
        ExpiresIn=300,  # 5 minutes
    )
    return {
        "statusCode": 200,
        "body": json.dumps({"uploadUrl": presigned_url, "key": key}),
    }

// Client side: request the URL, then PUT the file straight to S3

async function uploadFile(file) {
  const res = await fetch("/api/upload-url", {
    method: "POST",
    body: JSON.stringify({ filename: file.name, contentType: file.type }),
  });
  const { uploadUrl } = await res.json();
  await fetch(uploadUrl, {
    method: "PUT",
    headers: { "Content-Type": file.type },
    body: file,
  });
}

The gateway now only handles a small JSON request and response. The file itself, no matter how large it is, goes directly to S3 through a connection designed to handle large uploads.

A single PUT request works well for many files. For larger files, splitting the upload into smaller parts makes the process more reliable.

Multipart for the Truly Large

S3 multipart upload splits a large file into smaller parts, uploads each part separately, and puts them back together once all the parts arrive. It supports objects up to 5 TB, and parts can be uploaded in parallel and retried individually if something goes wrong.

This is the practical answer to what’s the best way to handle multipart or resumable uploads for large files: don’t retry the entire file when one part fails. Retry only the part that failed.

The same approach helps answer what API supports uploading files up to 5 GB reliably on unstable networks. The smaller retry units make large uploads much more reliable. If a 2 GB upload fails at 90% during one large PUT, you may have to start again. With the same file split into 5 MB parts, you only need to retry the part that was interrupted.

Presigned URLs and multipart uploads solve the file transfer problem well. The trade-off is that building and maintaining this infrastructure yourself takes ongoing work, even after the initial setup is working.

The Managed Route, The Pattern Productized

The managed version of this approach is a file upload API that handles the upload infrastructure for you. It can issue upload credentials, support chunked uploads for files up to 5 GB, and keep your gateway focused on handling JSON requests.

Instead of building and maintaining the presigning Lambda, client-side retry logic, and multipart upload handling yourself, you can use one integration that follows the same direct-upload approach.

Filestack provides direct uploads with chunked transfers, progress events, and an embeddable picker. This is useful if you want reliable file uploads without building the entire upload system from scratch.

For more on authentication and content types, see the file upload API guide. You can also check the large file upload guide for handling larger files and the Filestack S3 integration docs for the direct-to-S3 setup specifically.

Now that we’ve covered both approaches, here’s a quick summary of the key points.

Conclusion: Let the Gateway Serve JSON

The 10 MB limit isn’t going to change, so trying to work around it with bigger buffers or longer timeouts only wastes time.

The better fix is architectural: create a presigned URL, let the client upload directly to S3, and keep API Gateway focused on what it’s good at, such as authentication, routing, and small JSON requests.

It’s also a simpler and more efficient path because the file doesn’t have to pass through your gateway and backend first.

Try the presigned upload flow, or a managed alternative, this sprint. Either way, you’re moving the file data away from a component that wasn’t designed to carry large files.

Frequently Asked Questions

Can the API Gateway 10MB limit be increased?

No. It’s a hard limit on both REST and HTTP APIs, and no configuration or support request raises it.

Why do uploads under 10MB still fail through Lambda?

Base64 encoding inflates binary bodies by roughly 1.33x on the way through the gateway, and Lambda’s synchronous payload cap sits at 6MB, tighter than the gateway’s own limit.

What is the right pattern for large uploads on AWS?

Presigned direct-to-S3 uploads, or a managed upload service built on the same pattern. Either way, the gateway should only handle metadata, not file bytes.

This article was published on the Filestack blog.

How to Upload File to API with Auth, Content Types and Size Limits

IderaDevTools — Tue, 15 Sep 2026 13:34:50 +0000

The server limit was increased from 10 MB to 100 MB, but the upload still failed with a 413 error. The problem was nginx, which was still limiting request bodies to 1 MB before they reached the app.

This is one of the first surprises you run into when you put file upload APIs into production: several layers can limit an upload, and changing the limit in one place doesn’t automatically change the others.

To upload a file to an API, you choose a transport (multipart/form-data for forms, raw binary PUT for single files, or chunked multipart for large payloads), attach auth (bearer tokens or signed URLs), set the correct Content-Type, and respect size limits at every hop: client, proxy, gateway, and server. If you use Filestack, its upload API supports files up to 5 GB through an authenticated request.

This article walks through each part of that negotiation: which transport fits which shape of upload, how auth actually gets checked, and where a request can die before it reaches your code.

Key Takeaways

multipart/form-data needs a boundary parameter. Setting Content-Type by hand without one is the most common self-inflicted 400.
Raw binary PUT, using the file’s own Content-Type, is simpler and faster for a single file with no other form fields.
Size limits stack across the request path. Client timeout, proxy body cap, gateway cap, and server parser limit all apply, and the smallest one decides the outcome.
A 413 means some hop rejected the size. A 415 means it rejected the type. They get diagnosed at different layers.
Signed URLs move auth into the URL itself, with an expiry, which lets a client upload straight to storage without ever holding an API key.

The Three Transports

Most file upload API questions start with the same decision: how should the file be sent? In most cases, you have three options.

multipart/form-data works well for forms that include a file and other fields, such as a title or description. The request is split into separate parts, and each part has its own headers.

Raw binary PUT is simpler. The file is sent directly as the request body, with Content-Type set to the file’s actual type. It’s a good choice when you’re sending one file without any extra form fields.

Chunked uploads are better for large files. The file is split into smaller parts, and each part is uploaded separately. The server then puts the parts back together after all of them arrive. This is more reliable for large uploads because a dropped connection doesn’t have to restart the entire file.

import requests
FILE_PATH = "photo.jpg"
API_URL = "<https://api.example.com/upload>"
HEADERS = {"Authorization": "Bearer YOUR_TOKEN"}
# 1. multipart/form-data, for a file plus other fields
with open(FILE_PATH, "rb") as f:
requests.post(API_URL, headers=HEADERS,
files={"file": f}, data={"title": "Vacation photo"})
# 2. raw binary PUT, for a single file with no other fields
with open(FILE_PATH, "rb") as f:
requests.put(f"{API_URL}/photo.jpg", headers={**HEADERS, "Content-Type": "image/jpeg"},
data=f)
# 3. chunked multipart, for large files sent in parts
CHUNK_SIZE = 5 * 1024 * 1024
upload_id = requests.post(f"{API_URL}/multipart/start", headers=HEADERS).json()["id"]
with open(FILE_PATH, "rb") as f:
part_number = 1
while chunk := f.read(CHUNK_SIZE):
requests.put(f"{API_URL}/multipart/{upload_id}/{part_number}",
headers=HEADERS, data=chunk)
part_number += 1
requests.post(f"{API_URL}/multipart/{upload_id}/complete", headers=HEADERS)

Choosing the right upload method gets the file moving. But before the upload can start, the API needs to check whether the request is allowed. That’s where authentication comes in.

Auth Patterns

Three common authentication methods cover most file upload APIs. Each one offers a different balance between simplicity and security.

Bearer tokens are the simplest option. The client sends a token in the Authorization header, and the server checks it before accepting the upload. This works well when the user is already signed in and has a valid token.

HMAC signatures add another layer of protection. The client signs the request with a shared secret, and the server checks the signature before accepting the file. This can help detect changes to the request while it’s being sent.

Expiring signed URLs let the client upload directly to storage. The backend creates a URL with a signature and expiration time, then gives it to the client. The client can use that URL to upload without receiving a long-term API key. Once the URL expires, it can no longer be used.

If you’re comparing file upload providers, it’s also worth checking their security and compliance information, such as SOC 2 and GDPR support. These details can change, so check each provider’s current documentation instead of assuming they all offer the same protections.

Authentication controls who can upload. Next, Content-Type helps determine what they’re allowed to upload.

Content Types and the Boundary

One of the most common 400 errors with multipart uploads happens when you set the Content-Type header yourself. multipart/form-data needs a special boundary that tells the server where each part begins and ends. The HTTP client creates this boundary automatically.

If you set the header manually without the correct boundary, the server may not be able to read the request. It’s better to let your HTTP client set the Content-Type header for you.

On the server, don’t treat the Content-Type or file extension as proof of what the file actually is. They can be wrong or changed. A safer approach is to check the file’s actual bytes and detect its real format. If it doesn’t match what was declared, you can reject or convert the file.

This is also a key part of building a reliable REST API file upload flow: treat the declared file type as a hint, check the actual content, and handle mismatches safely.

Once transport, authentication, and file type are handled, the next concern is what happens between the client and your server. That’s where size limits come in.

Size Limits at Every Hop

A file upload doesn’t go straight from the client to your application. It can pass through several hops, and each hop may have its own size limit.

The client or browser can have its own timeout or upload limit. A reverse proxy like nginx can also limit request size. Its default client_max_body_size is often 1 MB, which can be much smaller than what your application expects.

An API gateway may add another limit, and your framework or request parser can have its own rules too.

That’s why increasing the limit in your application isn’t always enough. Every hop between the client and your server needs to allow the file size you want to support.

HopTypical defaultError if exceededClient/browserVaries by client, often no hard capRequest never sends, or times outReverse proxy (nginx)1MB body cap by default413 Payload Too LargeAPI gatewayProvider-specific, commonly 10MB or less413 Payload Too LargeServer/framework parserFramework-specific, often configurable413 or 400 depending on framework

This is still part of the bigger file upload API problem, but now we’re looking at the infrastructure instead of the code.

Increasing the limit in your app won’t help if the proxy in front of it rejects the request first. Check every hop, not just the one you control in your code.

Keeping track of all four hops takes time, and the work doesn’t stop after the first setup. Any of those limits can change later.

There is another option that avoids having to manage all of these limits yourself.

The Managed Route, One Hop That Says Yes

If you don’t want to manage and check every hop yourself, a managed file upload API can simplify things. You get one documented upload limit, one authentication setup, and support for files up to 5 GB.

Instead of dealing with several different limits that may not match, you have one clear limit to work with. This makes the upload setup easier to manage and maintain.

When choosing a reliable file upload service, look for a few important things: a clear file size limit, a simple authentication setup, and built-in chunked uploads. You shouldn’t have to build and maintain all of that yourself.

For a startup team, the easiest approach is often to choose a service that already supports the file sizes and upload methods you need. That way, you spend less time managing multiple infrastructure limits.

For a framework-specific example, the FastAPI companion post covers how to handle file uploads with FastAPI. You can also check the Filestack API auth docs for more details on authentication and signed URLs.

That’s the managed approach. Here’s a quick summary of the key points either way.

Conclusion: Know Your Hop

Choosing a transport depends on what you’re uploading: a file with other form fields, a single file, or a large file that needs to be split into chunks.

Authentication is about deciding where trust should live. You might use a server-side token, a signed request, or a URL that lets the client upload directly.

But neither matters if one of the hops in between rejects the request before it reaches your app.

Try sending a 100 MB test file through the full upload path today, from the client to the server. If it fails, find the first hop that rejects it. That’s the limit that actually controls your uploads, no matter what your app’s configuration says.

Frequently Asked Questions

Why does my upload return 413?

Some hops’ body-size cap is smaller than the file. Check the proxy and gateway limits before assuming the problem is in your application code.

multipart/form-data or raw binary?

Multipart for forms with mixed fields. Raw binary PUT for a single file with no other data attached.

Should the client set the multipart Content-Type?

No. Let the HTTP client generate it, since it needs to include the boundary parameter, and a hand-set header usually leaves that out.

This article was published on the Filestack blog.

FastAPI Upload File with Multipart Handling and Streaming to Storage

IderaDevTools — Thu, 10 Sep 2026 19:14:11 +0000

A simple FastAPI endpoint using file: bytes works fine for small demos. But try uploading a 2 GB video, and things can go wrong quickly. If your container only has 512 MB of memory, loading the whole file into memory can cause it to run out of space and crash.

You may not even get a useful error. Sometimes, all you see is a memory-related message in the logs followed by a container restart.

Most FastAPI upload file tutorials work fine until a large file hits a small container. FastAPI gives you a few ways to handle uploaded files, and each one uses memory differently.

FastAPI uses UploadFile to handle uploaded files without keeping the whole file in memory. Multipart data can be stored in a temporary file, which is much safer for large uploads.

For large files, a good production setup is to validate the file early, stream it directly to object storage, and avoid loading the entire file into memory.

Another option is to use a managed upload service like Filestack. This keeps the actual file transfer out of your API server, so your FastAPI app doesn’t have to handle the file bytes itself.

This article covers each approach, starting with the simple buffered setup that can struggle under heavy uploads and ending with a setup that keeps the file transfer completely off your servers.

Key Takeaways

A bytes parameter loads the entire upload into RAM. UploadFile is the safer default for anything sizable.
UploadFile wraps SpooledTemporaryFile: small files stay in memory, large ones spill to disk on their own.
request.stream() yields chunks with no temp file at all, which keeps memory flat while you proxy to storage.
Content-Length checks and part-size limits belong before you read the body, not after.
A managed file upload api can take bytes out of your service entirely, leaving your endpoints to handle auth and metadata only.

UploadFile Under the Hood

FastAPI gives you two ways to accept a file, and they behave very differently under load.

Declare the parameter as bytes, and FastAPI reads the whole upload into memory before your function even runs. This works for small files and fails without warning as soon as someone uploads something large. UploadFile is the answer most FastAPI docs point to for a general file upload api, and for good reason: it wraps Starlette’s SpooledTemporaryFile, which keeps small files in memory and spills larger ones to disk automatically, past a configurable threshold. Your endpoint code stays the same either way. Only where the bytes live changes.

from fastapi import FastAPI, UploadFile, File
app = FastAPI()
.post("/upload")
async def upload_file(file: UploadFile = File(...)):
contents = await file.read()
# process contents, or better, stream it in chunks below
await file.close()
return {"filename": file.filename, "size": len(contents)}

This basic form is fine for small files, and it’s already safer than a bytes parameter since Starlette manages the spool for you. It still reads the full file into a variable at once, which is where the next section picks up.

Multipart Done Right

A multipart request isn’t a single file. It’s a series of parts separated by a boundary string, and each part can hold a form field or a file, mixed in any order.

Understanding how does multipart upload work in web applications helps explain why FastAPI needs a parser at all: the framework has to read the boundary, split the body into parts, and hand each one to the right parameter based on its name.

FastAPI handles this parsing for you with UploadFile and Form, so most endpoints don’t need to work with the raw multipart request.

If you’re asking how to add file uploads to a REST API, the basic approach is usually the same: accept the file as multipart form data, validate it early, and save it somewhere reliable before sending a response.

The main difference between frameworks is how much of this work they handle for you.

Set limits on file size and the number of parts before processing the upload. An unlimited number of file parts can use up memory or disk space, even when each file is small.

Multipart parsing gets the file into your endpoint. The next challenge is sending it somewhere else without using too much memory. That’s what we’ll cover next.

Streaming Straight to Storage

Reading the entire file into a variable still means your app has to hold the full file while processing it. For large uploads, that’s not ideal.

A better approach is to stream the file directly to storage in small chunks. With request.stream(), you can read the request body piece by piece without creating a temporary file.

If you’re uploading to your own S3 bucket, the flow is simple: read one chunk, send it as an S3 multipart upload part, then move to the next chunk. You only keep one chunk in memory at a time, so memory usage stays much more predictable even as file sizes grow.

import boto3
from fastapi import FastAPI, Request
app = FastAPI()
s3 = boto3.client("s3")
BUCKET = "your-upload-bucket"
CHUNK_SIZE = 5 * 1024 * 1024  # 5MB, S3's minimum part size
u/app.post("/upload-stream/{key}")
async def upload_stream(key: str, request: Request):
upload = s3.create_multipart_upload(Bucket=BUCKET, Key=key)
upload_id = upload["UploadId"]
parts = []
part_number = 1
buffer = b""
try:
async for chunk in request.stream():
buffer += chunk
while len(buffer) >= CHUNK_SIZE:
part_data, buffer = buffer[:CHUNK_SIZE], buffer[CHUNK_SIZE:]
result = s3.upload_part(
Bucket=BUCKET, Key=key, UploadId=upload_id,
PartNumber=part_number, Body=part_data,
)
parts.append({"PartNumber": part_number, "ETag": result["ETag"]})
part_number += 1
if buffer:
result = s3.upload_part(
Bucket=BUCKET, Key=key, UploadId=upload_id,
PartNumber=part_number, Body=buffer,
)
parts.append({"PartNumber": part_number, "ETag": result["ETag"]})
s3.complete_multipart_upload(
Bucket=BUCKET, Key=key, UploadId=upload_id,
MultipartUpload={"Parts": parts},
)
except Exception:
s3.abort_multipart_upload(Bucket=BUCKET, Key=key, UploadId=upload_id)
raise
return {"key": key, "parts": len(parts)}

This approach adds a little more code, but it gives you an important benefit: memory usage stays close to CHUNK_SIZE, no matter how large the file is.

That only works if you validate the incoming request first. So the next step is making sure those checks are in place.

Limits, Validation and Errors

Streaming protects your memory while the upload is running, but it doesn’t stop a bad request from starting. You should validate the request before reading any file data.

First, check Content-Length against your maximum file size. If the header is missing or can’t be trusted, keep checking the size as you read each chunk and stop as soon as the limit is reached.

You should also check the actual file type. Don’t rely only on the file extension or the MIME type sent by the client because those can be wrong.

These same limits apply to any REST API that handles file uploads, no matter which framework you use. FastAPI simply gives you the tools to check them early.

Once your own upload endpoint is properly limited and validated, there’s another option: don’t handle the file upload yourself at all. That’s the next approach.

The Managed Route, Off the Data Path

The final option is to keep the actual file upload out of FastAPI completely. Instead, clients upload directly through a managed file upload API, while your FastAPI endpoints handle things like authentication and file metadata.

Your server can provide short-lived upload credentials; the client sends the file directly to storage, and your API saves the file details once the upload is complete.

The main benefit is that the file data never passes through your FastAPI container, which keeps your server lighter and reduces memory and bandwidth pressure.

If you don’t want to build and maintain your own file upload infrastructure, a managed service can handle that part for you.

Filestack’s REST API supports file uploads up to 5 GB and uses chunked uploads for large files. This lets your FastAPI service focus on things like authentication and file metadata instead of moving the actual file data.

import requests
FILESTACK_API_KEY = "YOUR_API_KEY"
def get_upload_url(filename: str) -> dict:
response = requests.post(
f"<https://www.filestackapi.com/api/store/S3?key={FILESTACK_API_KEY}>",
params={"filename": filename},

)

response.raise_for_status()

return response.json()  # contains the URL the client uploads to directly

Your endpoint only needs to provide the upload URL and save the returned file handle after the upload succeeds. The actual file transfer happens outside your service.

Now that we’ve covered all four approaches, here’s a quick summary of what to do.

Conclusion: Choose Your Memory Tier on Purpose

FastAPI gives you four main ways to handle file uploads. The key difference is where the file data goes while it’s being uploaded.

Buffer keeps the whole file in memory. Spool, which is what UploadFile uses, keeps smaller files in memory and moves larger ones to disk. Stream sends the file in chunks directly to storage, keeping memory usage low. Bypass skips your server completely and lets the client upload directly to storage.

Choose the approach based on your file sizes and infrastructure instead of waiting for a large upload to crash your container.

A good first step is to replace any bytes parameter that is still being used for real file uploads.

This article was published on the Filestack blog.

How to Upload JPG File on Mobile with Orientation, Size and Format Traps

IderaDevTools — Wed, 09 Sep 2026 10:45:00 +0000

Three common problems can show up in the same sprint. An avatar appears sideways on Android. A 12 MB photo takes too long to upload on a weak connection. And an iPhone photo fails validation because it isn’t actually a JPG.

Most guides on how to upload JPG files on mobile stop once the file is selected. But that’s where the real problems often begin.

The photo might arrive rotated, be much larger than needed, or turn out to be a different format. Each of these problems has a clear cause and a simple fix.

To upload a JPG file on mobile, let users pick from camera or gallery, then handle the three traps that break JPG uploads: EXIF orientation (photos arriving sideways), oversized camera outputs (often 5–12MB), and format mismatches such as HEIC masquerading as JPG on iOS. Client-side resize plus server-side normalisation solves all three. Filestack converts, rotates, and compresses automatically on upload.

This article looks at each problem separately and then shows how to handle all of them in one step. That way, you don’t have to fix each upload bug as it appears.

Key Takeaways

EXIF Orientation values 3, 6, and 8 mark a rotated capture. Ignore them, and the photo displays sideways or upside down.
Phone cameras commonly output 5 to 12MB JPGs at 12 to 48 megapixels, far more than any screen needs to display.
iOS saves photos as HEIC by default. A file named photo.jpg is not proof that it holds JPG data.
A canvas resize on the client fixes orientation and file size in one step, since the redrawn pixels come out upright.
Server-side normalisation catches every client you don’t control, including third-party apps and old app versions still in the wild.

Trap 1, The Sideways Photo (EXIF Orientation)

Phone cameras don’t rotate the pixel data when you turn the phone. They save the image as captured and write a rotation instruction into the EXIF metadata instead. Most photo apps read that instruction and display the photo the right way up. Plenty of image libraries and browsers don’t, and that’s when a portrait selfie shows up lying on its side in your app.

The Orientation tag can hold several values, but three of them cause almost every sideways bug: values 3, 6, and 8 mark a rotated capture, corresponding to 180, 90, and 270 degrees. If your upload pipeline ignores this tag, the photo saves and displays exactly as rotated.

The fix is to draw the image onto a canvas using the correct rotation before you upload it. Once it’s drawn, the pixels themselves are upright, so no downstream viewer can get it wrong again.

function drawUprightImage(file) {
return new Promise((resolve) => {
const img = new Image();
const reader = new FileReader();
reader.onload = (e) => {
img.onload = () => {
const canvas = document.createElement('canvas');
const ctx = canvas.getContext('2d');
getOrientation(file, (orientation) => {
const swap = orientation >= 5 && orientation <= 8;
canvas.width = swap ? img.height : img.width;
canvas.height = swap ? img.width : img.height;
switch (orientation) {
case 3: ctx.transform(-1, 0, 0, -1, canvas.width, canvas.height); break;
case 6: ctx.transform(0, 1, -1, 0, canvas.height, 0); break;
case 8: ctx.transform(0, -1, 1, 0, 0, canvas.width); break;
default: break;
}
ctx.drawImage(img, 0, 0);
canvas.toBlob((blob) => resolve(blob), 'image/jpeg', 0.9);
});
};
img.src = e.target.result;
};
reader.readAsDataURL(file);
});
}

Fixing orientation on the client helps, but it only covers the clients you control. Keep that in mind while we move to the next trap, which shares part of the same fix.

Trap 2, The 12MB Camera File

A modern phone camera shoots at 12 to 48 megapixels and saves the result as a JPG in the 5 to 12MB range. Almost nothing in your app needs that much resolution. A profile photo displays at a few hundred pixels wide. Even a full-screen image rarely needs more than 2000 pixels on its longest edge.

Uploading the full-size image uses extra time and data without giving you much benefit. It’s really a page-size problem, just happening during the upload.

The same rule used to speed up image loading applies here too: resize the image to the size you actually need before uploading it.

Resize on the client before the upload starts. Canvas resize also solves this in the same pass as the orientation fix above, since you’re already redrawing the image. Set a maximum dimension, scale the canvas to fit it, and export at a reasonable JPEG quality like 0.8 or 0.9. A 10MB original commonly comes out under 1MB with no visible loss on a phone screen.

Resizing the image on the device takes care of the file size before upload. The next problem is a little trickier because the file may look completely normal at first.

Trap 3, HEIC in JPG Clothing

iOS stores photos as HEIC by default, not JPG. Some pickers and share sheets hand the file over with a .jpg extension anyway, or a name that looks like a JPG, while the actual bytes are still HEIC. A file extension is a label someone chose. It is not proof of what’s inside the file.

Trusting the extension is how format bugs make it to production undetected in testing. The safest check is the byte signature at the start of the file, not the name. JPG files start with the bytes FF D8 FF. HEIC files carry a different signature entirely. Check the actual bytes, and convert if the signature doesn’t match what the extension claims.

This is where image transformation pipelines can make things easier. A common question is how to resize, crop, watermark, or change an image format on the fly.

Instead of processing the same image several times, you can use one transformation URL to handle multiple changes in a single request. Filestack can use the image’s EXIF data to correct its orientation, convert HEIC images to JPG or WebP, and compress the image in the same transformation chain.

Now that we’ve covered all three problems, the next question is where to handle each one in your app.

Implementation, Web Form and Native

On mobile web, image uploads usually start with a file input. A common question is how to add image uploads to a web form. You can use <input type="file" accept="image/*" capture="environment"> to open the camera directly. If you remove capture, users can choose between the camera and gallery.

Native apps work a little differently. Both iOS and Android have SDKs that can handle camera and gallery permissions, so you don’t have to build that flow yourself.

React Native has its own approach. Image picker libraries return a local file URI instead of a browser File object, so your upload code needs to read the file from that URI first.

The fixes for orientation and file size stay the same across platforms. Only the way you get the file changes.

You can build all of this yourself, but that means maintaining similar upload logic for web, iOS, Android, and React Native. There is a simpler way to handle it.

The Managed Route, Normalise on Ingestion

You can handle all three problems with a single mobile file upload flow that automatically fixes orientation, converts formats, and compresses images as they are uploaded.

Instead of writing separate code for image rotation on the web, native SDKs, and HEIC checks, you can use one transformation process for every file as soon as it arrives.

Server-side normalisation also handles files from places you don’t control. Older app versions, third-party integrations, and API requests might skip your client-side fixes. A server-side step catches these files too and makes sure they follow the same rules.

Once the file is normalised, you can create different image sizes for your app. This leads to another common question: how can you generate thumbnails automatically after an upload?

You can create thumbnails, medium previews, and full-size versions from the same normalised image. This is much simpler than running a separate resize job for each version.

We’ve covered how to fix each problem. Here’s a quick summary to remember.

Conclusion: Trust Bytes, Not Extensions

Three traps, three fixes: orient from EXIF instead of trusting the file as captured, resize early instead of uploading the full camera output, and convert by byte signature instead of trusting the file extension. Client-side resize handles the traffic you can see. Server-side normalisation catches everything else.

Run a real phone photo through a transformation sandbox and check all three: does it come out upright, does it come out at a sane file size, and does it come out as an actual JPG regardless of what the original claimed to be.

Frequently Asked Questions

Why do my mobile photo uploads appear sideways?

EXIF Orientation is being ignored somewhere in the pipeline. Auto-orient on ingestion, or draw the image upright client-side using the Orientation tag before upload.

Why did an iPhone JPG upload fail validation?

It was likely HEIC with a JPG-style name. Check the byte signature rather than the file extension, and convert HEIC to JPG or WebP if the signature doesn’t match.

How big are phone camera JPGs?

Commonly 5 to 12MB at 12 to 48 megapixels. Resize to the display target before or during upload rather than sending the original file.

This article was published on the Filestack blog.

How to Build a Scalable Image Upload Component with Shadcn UI

IderaDevTools — Wed, 02 Sep 2026 13:10:02 +0000

If you search for a Shadcn UI image upload component, you’ll quickly notice there isn’t one. That’s intentional.

You might try running npx shadcn add upload, only to find that no upload component exists. Then you end up on the same GitHub discussions as many other developers asking the same question: Where is the image upload component?

The honest answer is: you build it yourself.

At first, that might seem surprising, but it actually makes sense. Shadcn UI isn’t a library where you install ready-made components. Instead, it gives you building blocks that you add to your own codebase and customise as needed.

Image upload is different for every application. The upload flow, states, and backend integration can vary a lot, so a single upload component wouldn’t work for every use case.

Shadcn UI ships no built-in image upload component; you compose one from its primitives (Button, Card, Progress, Dialog) around an upload engine that handles files, previews, and errors. The clean split is shadcn for presentation and a dedicated uploader for transfer. Filestack’s React SDK slots in as that engine, keeping the shadcn look while adding chunked, resumable uploads.

In this guide, we’ll build an image upload component step by step. We’ll use Shadcn UI to create the interface, add state management and validation, and then connect it to an upload service.

The best part is that you can change the upload provider later without rebuilding the UI. The interface stays the same while the upload logic can be swapped whenever you need.

Key Takeaways

Shadcn/ui has no upload component on purpose. You compose one from Card, Button, Progress, and Dialog.
A working upload UI needs four pieces: a dropzone surface, file rows, a progress bar per row, and error text inside the row, not a toast.
A hidden <input type="file"> paired with a <label> keeps the dropzone accessible and keyboard operable.
Keep transfer logic behind a small interface. Swapping fetch() for a real upload SDK should change zero markup.
Filestack’s React SDK maps its progress callbacks directly onto Shadcn’s <code>Progress component, so the engine and the UI stay decoupled.

Before building anything, it’s worth understanding why this gap exists, because it shapes every decision after it.

Why Shadcn Does Not Ship an Uploader

Shadcn’s whole philosophy is “copy the code, own the code.” That works well for a button or a dialog, because those components don’t hold much internal state. An uploader is different. It has to track file selection, per-file progress, retries, validation errors, and network failures, all at once. Trying to template that into one drop-in component would mean baking in assumptions about your backend, your file size limits, and your error handling, exactly the kind of lock-in shadcn tries to avoid.

So instead of asking “what are the best React components for file uploading,” the more useful question becomes “which primitives do I already have, and what’s missing?” As it turns out, you already have most of what you need. What’s missing is the part that actually talks to a server.

With that context in place, let’s start stacking primitives into an actual dropzone.

Composing the Surface: Dropzone plus Rows plus Progress

This is where shadcn earns its keep. A React drag-and-drop file upload surface and a React file upload component for the file list are really the same composition problem, just two different views of it.

The dropzone itself is a Card wrapping a hidden file input and a label. The label pattern matters here: clicking anywhere on the label opens the file picker, and because it’s a real form control under the hood, keyboard users can tab to it and hit Enter or Space to open it too. Below the dropzone, each selected file becomes its own row, and each row gets its own Progress bar.

import { Card } from "@/components/ui/card";
import { Progress } from "@/components/ui/progress";
import { cn } from "@/lib/utils";
export function ImageUploader({ files, onFilesSelected, onRetry }) {
return (
<Card className="p-6">
<label
htmlFor="file-input"
className={cn(
"flex flex-col items-center justify-center gap-3",
"rounded-lg border-2 border-dashed border-muted-foreground/30",
"py-10 text-center cursor-pointer hover:border-primary/50"
)}
>
<span className="font-medium">Drag and drop images here</span>
<span className="text-sm text-muted-foreground">or click to browse</span>
<input
id="file-input"
type="file"
multiple
accept="image/*"
className="sr-only"
onChange={(e) => onFilesSelected(Array.from(e.target.files))}
/>
</label>
<ul className="mt-6 space-y-3">
{files.map((file) => (
<li key={file.id} className="rounded-md border p-3">
<div className="flex items-center justify-between text-sm">
<span className="font-medium">{file.name}</span>
{file.status === "failed" ? (
<button
onClick={() => onRetry(file.id)}
className="text-destructive underline"
>
Retry
</button>
) : (
<span className="text-muted-foreground">{file.status}</span>
)}
</div>
<Progress value={file.progress} className="mt-2 h-2" />
{file.error && (
<p className="mt-1 text-xs text-destructive">{file.error}</p>
)}
</li>
))}
</ul>
</Card>
);
}

Notice the error text sits inside the row, right under that file’s own progress bar, instead of floating away in a toast. A toast disappears in a few seconds. A row stays put until the user deals with it, which matters when three out of twenty files failed, and you don’t want the person hunting for which ones.

The surface is only half the job, though. Right now, none of this actually tracks state. Let’s fix that next.

State and Validation

With the JSX in place, the component needs somewhere to keep track of what’s happening to each file, and a way to say no to files that shouldn’t be there in the first place.

A useReducer keyed by file ID works well here, since every file’s status changes independently of the others. Validation, checking file type and size, happens the moment a file is selected, before any request goes out. This answers a common early question too: file uploading in React JS almost always starts with this same shape, a reducer plus a validation step, no matter which transport ends up sending the bytes.

function uploadReducer(state, action) {
switch (action.type) {
case "ADD_FILES":
return {
...state,
...Object.fromEntries(
action.files.map((f) => [f.id, { ...f, status: "queued", progress: 0 }])
),
};
case "PROGRESS":
return {
...state,
[action.id]: { ...state[action.id], status: "uploading", progress: action.pct },
};
case "DONE":
return { ...state, [action.id]: { ...state[action.id], status: "done", progress: 100 } };
case "ERROR":
return {
...state,
[action.id]: { ...state[action.id], status: "failed", error: action.message },
};
default:
return state;
}
}

Validation lives right where files enter the component:

function validateFile(file) {
if (!file.type.startsWith("image/")) return "Only image files are allowed.";
if (file.size > 10 * 1024 * 1024) return "File is larger than 10MB.";
return null;
}

Most guides on implementing image uploads in React stop here, with a working component and simulated progress. That’s enough to show how the UI works.

But in a real application, you also need something that uploads the file and reports the actual upload progress. That’s where choosing the right upload solution becomes important.

The Engine Swap: fetch to SDK

This is the piece that makes the whole composition worth the extra setup: the transport layer sits behind a small interface, so the shadcn component above never needs to know or care how bytes actually get to the server.

Start with the interface itself. It only needs one method, and it only needs to report progress and completion:

// uploadEngine.js
export function createFetchEngine(endpoint) {
return {
upload(file, { onProgress, onDone, onError }) {
const xhr = new XMLHttpRequest();
const form = new FormData();
form.append("file", file);
xhr.upload.onprogress = (e) => {
onProgress(Math.round((e.loaded / e.total) * 100));
};
xhr.onload = () => (xhr.status < 300 ? onDone(xhr.response) : onError("Upload failed"));
xhr.onerror = () => onError("Network error");
xhr.open("POST", endpoint);
xhr.send(form);
},
};
}

A React JS file upload component built this way already works. The catch is that raw fetch or XMLHttpRequest gives you one request per file, no chunking, and no way to resume a large upload that drops halfway through. Swapping in Filestack’s React SDK as the engine keeps the exact same interface, but the internals now handle chunked, resumable transfer:

import { init } from "filestack-js";
export function createFilestackEngine(apiKey) {
const client = init(apiKey);
return {
upload(file, { onProgress, onDone, onError }) {
client
.upload(file, {
onProgress: (evt) => onProgress(Math.round(evt.totalPercent)),
})
.then((res) => onDone(res))
.catch((err) => onError(err.message));
},
};
}

Both upload engines use the same upload(file, callbacks) function, so you don’t need to change the component from Section 2. The only thing that changes is the upload engine you pass into it.

The Managed Route: Shadcn Look, Production Engine

Building your own fetch-based engine is a fine way to learn the shape of the problem, and it’s genuinely enough for a small internal tool. But once you need chunking for large images, resumable uploads on flaky connections, or reliable retry behaviour, that engine starts asking for real maintenance time.

Keep the markup, upgrade the engine: wire the composed component to a production upload ui and the same shadcn rows gain chunked transfer, retries, and 5GB file support, without a redesign. The onProgress callback from the React SDK maps one-to-one onto the Progress value you’re already rendering, so the swap really is as small as it looks in the code above.

This is also where teams building something like a profile picture uploader tend to land. The UI stays identical to a plain gallery upload, same dropzone, same rows, but the failure modes that matter for a single, important image (say, someone’s profile photo) get handled by the engine instead of a hand-rolled retry loop.

If you’re curious how that plays out for single-file, high-stakes uploads, our React file upload walkthrough covers that shape in more depth.

Conclusion: Own the Pixels, Outsource the Packets

The lesson underneath all of this is simpler than it looks in the code: shadcn was never going to ship an uploader, because uploading isn’t really a presentation problem. It’s a transport problem wearing a UI.

Split the two apart, and both sides get easier. You keep full control over how the dropzone and rows look and feel, since that’s just your own JSX and Tailwind classes. And you keep the option to swap the engine underneath, from a quick fetch call to something built for chunking and retries, without ever touching that markup again.

If you want to see it end to end, copy the composed component above and connect it to Filestack to see real progress values fill in those same progress bars.

This article was published on the Filestack blog.

Profile Picture Upload UI with Cropping, Preview and Instant Feedback

IderaDevTools — Thu, 27 Aug 2026 12:50:48 +0000

Almost every team runs into this problem at some point. A user uploads a photo, carefully adjusts the crop, and clicks Save. But when their profile picture appears later, it’s cropped differently. Part of their face might be cut off, or the framing looks wrong.

It may seem like a small issue, but users notice it right away because profile pictures are personal.

The first few seconds after someone selects a profile picture matter the most. That’s when users decide if the upload experience feels smooth or frustrating. In that window, a good flow does four things: it lets the user select or drop a file, shows an instant local preview, offers a circular crop with zoom, and gives honest feedback while the image uploads and processes. The best implementations show the final cropped result before the network round trip even finishes, so what the user approves is what actually ships.

That last part is the tricky bit. The preview, crop tool, and final uploaded image all need to use the same crop settings. If they don’t, the image users see before saving won’t match the one that gets uploaded.

Let’s look at how to build this the right way, step by step.

Key Takeaways

Show a local preview the instant a file is selected; don’t wait for the upload to start.
Store the crop as a rectangle (coordinates), not just a rendered circle; you’ll need it again.
Keep the crop math identical between what the user previews and what the server delivers.
Break “uploading” into real states (previewing, uploading, processing, saved), so users trust the progress.
Fix EXIF orientation before cropping, or phone photos will crop sideways.

Now let’s take a quick look at why the preview needs to appear before anything touches the network, and how to wire that up for both drag-and-drop and standard file inputs.

The Three-Second Rule: Instant Local Preview

The moment someone picks a photo, they want to see it. Not after a spinner, not after a server round trip; they want to preview it immediately. Browsers make this easy with URL.createObjectURL(), which turns a local File object into a temporary URL your <img> tag can render right away, without the need for any upload.

function handleFileSelect(file) {
const previewUrl = URL.createObjectURL(file);
imgElement.src = previewUrl;
// Revoke later to free memory
imgElement.onload = () => URL.revokeObjectURL(previewUrl);
}

This works whether the file arrives through a standard <input type="file"> or a drag-and-drop zone. For drag and drop, you’re listening for drop events and pulling the file off event.dataTransfer.files; for a form input, it’s the change event on the input element. Either path lands you the same File object, so the preview logic doesn’t need to know which source it came from.

One thing worth handling early: EXIF orientation. Phone cameras often store images sideways or upside down and rely on metadata to display them correctly. Browsers mostly respect this metadata for regular <img> rendering, but once you start drawing to a canvas for cropping, that metadata can get ignored, and suddenly your crop preview is rotated 90 degrees from what the user expects. Correcting the image orientation before cropping helps ensure the final result matches the user’s selection.

With the preview solved, the next question is what the user does with it, and that’s where cropping comes in.

Crop, Zoom and the Circle Mask

Most avatar UIs show a circular preview, but the circle is a mask, not the actual crop. Underneath it, you’re almost always working with a square (or fixed-aspect) rectangle; the circle is just how it’s presented visually, usually with border-radius: 50% or an SVG clip-path.

The important part is what you store. Don’t save a pre-cropped, pre-masked image and call it done. Save the crop rectangle: x, y offset, width, height, maybe a zoom factor, as data. That rectangle is what lets you regenerate the avatar at any size later, or re-render it somewhere else in your app without asking the user to crop again.

Here’s a simplified example of turning crop state into a transformation URL:

function buildAvatarUrl(baseUrl, crop) {
const { x, y, width, height } = crop;
// crop: pixel rectangle from the user's selection
const cropParam = `crop=x:${x},y:${y},w:${width},h:${height}`;
const resizeParam = `resize=width:400,height:400`;
const circleParam = `circle`;
return `${baseUrl}/${cropParam}/${resizeParam}/${circleParam}`;
}
// Usage
const avatarUrl = buildAvatarUrl(
'<https://cdn.filestackcontent.com/HANDLE>',
{ x: 120, y: 40, width: 300, height: 300 }
);

Pinch-to-zoom on mobile and scroll-to-zoom on desktop both just adjust the crop rectangle’s dimensions before you apply the aspect lock. The masking (circle, rounded corners, whatever your design calls for) stays a purely visual layer on top.

Once the crop rectangle exists as data, the natural next question is how to wire all of this into your actual app, which is where framework choice starts to matter.

React Implementation Notes

If you’re building this in React, you’ve got two general paths: assemble it from smaller libraries, or use a composed upload component that already handles picking, preview, and cropping together.

The DIY route usually means pairing a drag-and-drop hook (like react-dropzone) with a cropping library (like react-easy-crop or react-image-crop) and writing your own state management to connect them: file selection updates preview state, crop interactions update crop state, and a submit handler stitches it all into an upload request.

The composed route hands you a single component that already wires selection, preview, and crop together, and hands back a crop rectangle or transform URL through callbacks. This tends to save the most time on the parts that are easy to get subtly wrong: touch gestures, aspect-ratio locking, and keeping crop state in sync with the preview across re-renders.

Either way, the core pattern from the sections above doesn’t change: local preview first, crop rectangle as the source of truth, transform applied consistently. React just gives you hooks and component boundaries to organise it in.

With the crop rectangle in hand, the next piece is making sure it doesn’t just produce one image; it needs to produce every size your app actually uses.

Renditions and Delivery

Avatars rarely need just one size. A profile page might want a large version, a comment thread wants something small, a notification badge wants smaller still. Generating and storing every variant at upload time is wasteful, and worse, if you ever change your sizing needs, you’re stuck regenerating old uploads.

A cleaner pattern is to store one master image (a common target is 400×400) and generate renditions on request using resize parameters in the URL, cached at the CDN layer so repeat requests don’t reprocess the image.

This also keeps your crop rectangle useful. Since the master retains the full crop, you can request a 128px rendition for a profile header and a 32px one for a notification badge, and both come from the same source of truth; you don’t need a separate crop step per size.

Storing renditions this way is also what makes it practical to add new sizes later without touching old data. That flexibility becomes even more useful once you look at how the whole flow — picker, crop, and delivery — can share one implementation.

The Managed Route: Preview Equals Result

Everything covered so far: instant preview, crop rectangle as data, consistent transforms, on-demand renditions, can be built by hand. It’s also, unsurprisingly, the exact shape of the problem a managed upload ui is built to solve: picker, crop interface, and transform URLs sharing the same underlying handle and parameters, so the crop a user approves in the picker is the same crop that renders in production.

This isn’t just about making things easier. It also helps prevent crop mismatch bugs. When the crop tool produces the same transformation used to display the final image, you don’t have to calculate the crop twice. That means there’s less chance of the preview and the uploaded image getting out of sync.

If you’re evaluating this route, it’s worth looking at how the picker’s crop options are configured and how image transformations apply as URL parameters, the same pattern from the code snippet earlier in this article, just handled for you.

For a closer look at resizing specifically, this piece on resizing images with URL parameters is a good companion read.

Whether you build this by hand or lean on a managed picker, the underlying principle stays the same, which is worth restating clearly before wrapping up.

Conclusion: One Source of Truth for the Crop

A profile picture upload UI doesn’t need to be complicated, but it does need to be consistent. Show the preview instantly. Store the crop as a rectangle, not a rendered image. Apply that same rectangle everywhere the avatar shows up. Keep users informed with real states instead of a single generic spinner.

Get those four things right, and the mismatch bug, the one where the saved avatar doesn’t match what the user approved, simply can’t happen, because there’s only one crop, used everywhere. At Filestack, this is the exact problem our upload and transformation tools are built around, and if you’re setting up this flow, it’s worth testing your crop logic against a sandbox account before committing to a full build.

FAQ

What size should profile pictures be stored at?

A 400×400 master is a common baseline, with smaller renditions (128px, 32px, etc.) generated on delivery as needed.

Why does my avatar crop differently after saving?

This usually means the preview and the server are running different crop math. Sharing one crop rectangle and one transform URL between them fixes it.

How fast should the preview appear?

Under 100ms, using a local object URL, before any upload has started.

This article was published on the Filestack blog.

Upload Contract Form UI Design for Signatures and Documents

IderaDevTools — Fri, 21 Aug 2026 10:39:27 +0000

Most contract forms don’t lose people at the signature. They lose them one step earlier, at the upload, when someone picks the wrong file, gets no feedback, and quietly gives up.

Upload contract form UI design covers the interface patterns for collecting signed agreements: a document upload step with clear format guidance, inline preview, validation, a signature capture step, and explicit status feedback at every stage. Strong designs cut abandonment by showing per-step progress and by accepting camera captures on mobile, not just desktop file pickers.

This piece walks through five patterns that hold up across real estate, HR, and fintech agreement flows. It includes two annotated interface examples and a short code snippet you can adapt. It also looks at where Filestack’s upload, preview, and OCR building blocks fit under these patterns. The patterns come first, the tooling second.

Key Takeaways

Contract flows break at two points: the document upload and the signature. Each one needs its own visible progress state.
Inline preview at upload time catches wrong-file mistakes before they turn into support tickets.
OCR can read a contract as it comes in and prefill names and dates. Verification becomes a confirm, not a retype.
Camera capture deserves the same design attention as file upload. Plenty of users are photographing paper, not exporting PDFs.
Status honesty (uploaded, then scanned, then accepted) does more for trust than any amount of copywriting.

Anatomy of a Contract Upload Flow

Break a contract upload flow into its parts, and you get five steps: intake guidance, upload, preview and verification, signature, and confirmation.

Treat them as one blob, and you get one blob-sized failure, a form that just doesn’t work with no clue why.

Treat them as five separate states, each with its own success and failure condition, and both debugging and designing get a lot easier.

If you’re mapping this to actual interface pieces, think in components rather than steps: a drop zone, a file list, a preview pane, a progress indicator, a signature pad, and a confirmation banner. Each one maps to a step above, and each can be built, tested, and shipped on its own.

This separation also makes it easier to talk about the flow with a team. “The upload step is failing” is vague. “Files are getting stuck between uploaded and scanned, and the UI never says why” is something an engineer can actually go fix. Naming the states first, before writing any code, usually surfaces exactly where a flow is thin.

Once the anatomy is clear, the next question is what each piece actually needs to do well. Start with the step where most contract flows quietly lose people.

The Document Step, Guidance and Preview

The document step fails silently more often than any other part of the flow. Someone uploads a .heic photo from their phone, the form accepts it without complaint, and three days later a reviewer discovers it won’t open.

State your accepted formats and size limit before anyone touches the upload button. Don’t bury it in a tooltip they’ll never hover over. Pair that with drag-and-drop plus a plain browse button. Forcing one interaction pattern excludes people who don’t know the other exists.

The single highest-leverage addition here is inline preview. Once a file lands, render it. Show the actual PDF or image, not just a filename, so the person can confirm it’s the right document before they submit anything. This is also where rejection should happen: immediately, with a specific reason (“this file is a .docx, we need a PDF or image”), not after a full-page reload three steps later.

It’s worth resisting the urge to over-restrict the drop zone too. A common mistake is accepting only PDF, on the assumption that “real” contracts are always exported as PDFs. In practice, a large share of uploads are phone photos of printed pages, or scans saved as JPG. Accept images alongside PDFs and say so plainly in the guidance copy. That alone avoids a whole category of “why won’t this work” support tickets.

Getting the file in cleanly sets up the next problem: making sure what’s inside it actually matches what the form expects.

Verification, Reading the Contract for Them

Manual verification usually means asking someone to retype their own name, a date they already wrote by hand, and a few clause references. That’s a tedious way to confirm something the document already states.

OCR-driven prefill flips this: pull the party names, dates, and key fields directly from the uploaded contract and show them next to the preview for confirmation. The person’s job shifts from typing to checking, which is faster and produces fewer transcription errors on both ends.

Keep the extracted fields editable, but make sure edits update the form record, not the underlying document. The uploaded contract stays the source of truth. The extracted fields are just a convenience layer on top of it, and users should be able to tell the difference at a glance.

There’s one design decision worth being deliberate about here: how much to trust the extraction. OCR on a clean, typed PDF is close to reliable. OCR on a handwritten or photographed contract is not. Presenting low-confidence extractions with the same visual weight as high-confidence ones sets people up to accept a wrong date without noticing. A simple confidence indicator, or even just flagging fields pulled from an image instead of a text-based PDF, keeps verification meaningful instead of another box to click through.

With the document verified, the flow moves into its second failure-prone stretch: actually collecting the signature.

The Signature Step and Status Honesty

Offer draw, type, and upload-a-saved-signature as three parallel options rather than forcing one method. Some people are on a trackpad, some are on mobile with a finger, and some already have a signature image saved from a previous form. None of these should be treated as the “real” method with the others bolted on as afterthoughts.

The submit button should stay disabled until the document has actually cleared processing, meaning scanned, checked, accepted, not the moment a file appears in the list. That processing time is also worth surfacing as a positive signal instead of a silent spinner. A visible “scanning for security” state, even for a couple of seconds, reads as diligence rather than delay. It’s a small design choice that does real work for trust without needing any explanatory copy at all.

Desktop and mobile signing look similar on paper, but mobile brings its own upload problem entirely. It’s easy to treat that as a lesser version of the desktop flow instead of a path in its own right.

Mobile, Photographing Paper

A lot of contracts start on paper: a lease printed and signed in person, a form filled out by hand. Mobile camera capture is how that paper gets into the system. Treat this as a primary path, not a workaround bolted onto the file picker.

That means edge guidance so the whole page is in frame, auto-crop once the edges are detected, and a glare warning if the flash is washing out part of the text.

None of this is exotic, but it’s easy to skip if the design process starts from “upload a PDF” and treats the camera as an edge case. For a meaningful share of users, the camera is the primary case.

The signature and verification patterns above still apply once a photo comes in. OCR just has to work a little harder against a slightly skewed, unevenly lit image instead of a clean digital export. That’s a good reason to invest in capture quality up front, good cropping, no glare, rather than compensating for it later with more aggressive text extraction.

All five of these patterns lean on the same small set of underlying capabilities, which is worth naming plainly before wrapping up.

The Managed Route, Patterns to Production

All five patterns above assemble from the same primitives: a production upload ui for the document step, preview and OCR for verification, and status callbacks for honest progress. Building each of those from scratch (cross-browser drag-and-drop, PDF rendering in the browser, OCR pipelines, malware scanning) is a real project on its own, separate from designing the flow around them.

Here’s a minimal example of wiring a picker to accept PDFs and images only, render a preview, and report per-file status back to your UI:

import * as filestack from "filestack-js";
const client = filestack.init("YOUR_API_KEY");
client.picker({
accept: ["application/pdf", "image/*"],
maxSize: 10 * 1024 * 1024, // 10MB
onFileUploadStarted: (file) => updateStatus(file, "uploading"),
onFileUploadFinished: (file) => updateStatus(file, "uploaded"),
onFileUploadFailed: (file, error) => updateStatus(file, "failed", error),
}).open();
function updateStatus(file, status, error) {
// Drive the honest status pill from real events, not a timer
console.log(file.filename, status, error || "");
}

Twelve lines get you accept-type filtering, size limits, and the event hooks a status indicator needs.

From there, preview and OCR calls attach to the same uploaded file reference. If you’re a product manager on a real estate team trying to get a working uploader in front of users this sprint instead of next quarter, this is usually the fastest path there.

Conclusion: Design for the Wrong File

Good upload contract form UI design isn’t really about the happy path. It’s about what happens when someone picks the wrong file, photographs a blurry page, or leaves the tab open mid-signature. Show format guidance before the mistake happens. Preview the actual file, always. Let OCR turn verification into a confirmation instead of a retype. Report status honestly at every step. Treat the camera as a first-class input, not a fallback.

If you’re prototyping this yourself, the Filestack picker sandbox is a fast way to test the document step, upload, preview, and status callbacks before you commit to a full build.

FAQ

What steps should a contract upload form have?

Guidance, upload, preview and verification, signature, and confirmation, each with its own visible state.

Should users see the contract after uploading?

Yes. Inline preview before submission is the single most effective way to prevent wrong-file errors.

Can data be extracted from uploaded contracts?

Yes. OCR can prefill names, dates, and other fields directly from the document for the user to confirm.

Originally published on the Filestack blog.

When Uploading Many Small Files Becomes a Denial of Service Risk

IderaDevTools — Wed, 19 Aug 2026 11:49:44 +0000

Monday morning, and your API is pinned at 100% CPU. Nobody uploaded anything huge. There’s no 4K video sitting in the queue, no multi-gigabyte archive. Just 60,000 tiny files, a handful from enthusiastic customers batch-uploading thumbnails, and a few thousand more from a script that’s testing how far your endpoint bends.

Uploading many small files denial of service scenarios rarely start as attacks. They usually start as a power user with a folder of 4,000 icons, or an integration partner that decided to sync every asset it owns in one go. The danger isn’t the bytes. It’s everything your server does per file, multiplied by a number that got out of hand.

Uploading many small files becomes a denial of service risk when per-file overhead, connections, auth checks, disk metadata writes, and scan jobs multiply faster than payload size. A thousand 10KB files can cost more than one 10MB file. Defences include batch limits, rate limiting, queued ingestion, and offloading uploads to a managed pipeline such as Filestack that absorbs the fan-out before it reaches your servers.

This article walks through why small files hit harder than their size suggests, where the attack surface actually lives, and the layered defences: limits, rate shaping, queues, and asynchronous scanning, that keep an upload endpoint standing under pressure.

Key Takeaways

Per-file fixed costs (auth checks, DB writes, storage PUTs, scan jobs), not raw byte count, are usually what breaks an upload endpoint first.
A thousand 10KB files can cost your infrastructure more than a single 10MB file, because every file drags its own overhead along with it.
Archive uploads need expansion-ratio caps; an unzipped “small” file can balloon into gigabytes of decompression work.
Batch limits, per-account rate limiting, and queued ingestion are the first line of defence and don’t require rearchitecting your stack.
Moving ingestion to a managed file uploader relocates the fan-out entirely, so your API sees metadata events instead of raw byte streams.

Let’s start with the part that trips most teams up: why small files are, counterintuitively, the more expensive problem.

Why Small Files Hurt More Than Big Ones

If you’re wondering what are the best methods to upload multiple files at once in a web application, the honest answer starts with a warning: multi-file upload is where fixed-cost overhead becomes visible for the first time. A single request has one TLS handshake, one auth check, one database write, one storage call. A thousand-file batch has a thousand of each, even if the total payload is identical.

The crossover point is easy to miss because it isn’t about total size at all. Object storage providers bill PUT requests per operation, separately from bytes stored, so a million tiny files can genuinely cost more in request fees than in storage. The same logic applies to your own compute: a database write or a virus-scan job doesn’t get cheaper because the file behind it is small. Once you see the pattern, it’s clear that file count deserves the same scrutiny as file size.

Small files aren’t the whole story, though. The same overhead pattern shows up in more deliberate ways once you start thinking about upload endpoints as attack surface.

The Attack Surface, Fan-Out and Amplification

Ask how can I prevent file upload vulnerabilities in my web application, and per-file overhead is only half the answer; the other half is amplification. A handful of small, cheap uploads can trigger disproportionately expensive work downstream.

Archive uploads are the clearest example. A 2MB zip file looks harmless at the network layer, but if it decompresses into 4GB of nested files, every downstream step: storage, scanning, indexing, inherits that expansion. Left unchecked, this is the classic zip-bomb pattern: a small input engineered to produce enormous output. Deep archive trees (folders inside folders inside folders) create a similar problem for anything that walks the file structure recursively.

Scan-job amplification follows the same shape. If every uploaded file queues a virus scan synchronously, a burst of a few thousand small files can back up your scanning workers even though none of them are individually suspicious. And metadata-write storms, a database row or search-index update per file, can degrade a shared database well before storage or bandwidth becomes the bottleneck.

If you’re also asking how do I detect and block malicious files during upload, the practical answer is to combine content-type verification, archive expansion limits, and asynchronous scanning (more on that in a moment) rather than relying on any single check. It’s worth noting that the line between “abuse” and “legitimate burst” is often blurry; a real customer syncing a large media library looks a lot like an attack until you’ve built the throttles that treat both cases the same way.

That overlap is actually good news operationally: the same defences that stop an attacker also stop a well-meaning customer from accidentally taking your API down. Here’s what that layered defence looks like in practice.

Defences, Limits, Rate Shaping and Queues

The first layer is the simplest: batch caps and per-account rate limits, enforced before a request does any real work. A sensible file-count cap per batch (say, 100 files per request) turns an unbounded upload into a predictable, budgetable unit of work. Layer a token-bucket rate limiter per account on top, and a single client, malicious or just enthusiastic, can’t monopolise your ingestion capacity.

If you’re using Filestack’s Picker, you can enforce upload limits with options such as maxFiles*,* minFiles*, and related* file-count controls before the upload even begins.

The second layer is queued ingestion. Instead of processing every file synchronously inside the request/response cycle, accept the upload, write a lightweight acknowledgement, and let a queue smear the actual processing over time. This is the difference between a burst that spikes your CPU for ten seconds and a burst that quietly drains over ten minutes without anyone noticing.

Speed expectations matter here too. If you’re comparing which platforms support the fastest bulk uploader options, the honest tradeoff is that raw speed and abuse-resistance pull in opposite directions; a platform optimised purely for throughput without rate shaping is also the one most exposed to fan-out abuse. The platforms that hold up under both legitimate bulk traffic and adversarial bursts are the ones that queue by design, not the ones that simply accept everything as fast as possible.

Limits and queues buy you time and predictability. What you do with that time, specifically, how you scan what’s coming in, is the next piece.

Scanning Without Melting

If you’re figuring out how can I add virus scanning to file uploads in my application, the short version is: never run it synchronously on the request path. A scan that blocks the upload response until it completes means your scanning capacity is your upload capacity, and a burst of files instantly becomes a burst of blocked requests.

The more resilient pattern is quarantine-then-release: accept the file, store it in a location that isn’t yet accessible to end users, queue an asynchronous scan job, and only promote the file to “available” once the scan clears. This decouples upload throughput from scan throughput entirely, so a scanning backlog degrades gracefully (files take longer to become available) instead of catastrophically (uploads start failing).

If you’re implementing malware detection or security policies, Filestack’s Security documentation covers built-in virus scanning, content validation, and upload security features in more detail.

Expansion-ratio caps belong here too. If an archive’s uncompressed size exceeds some multiple of its compressed size, say, 100x, reject or flag it before extraction runs to completion. It’s a small check that closes off the zip-bomb path discussed earlier, and it costs almost nothing to enforce.

Building and maintaining all of this — batch caps, rate limiters, queues, asynchronous scanning, expansion checks, is real infrastructure work. It’s worth being clear-eyed about what it takes to run in-house before deciding whether to build it yourself.

The Managed Route, Move the Blast Radius

Everything above reduces the damage a flood of small files can do once it hits your infrastructure. The strongest structural defence is not absorbing the fan-out at all: a managed file uploader terminates the file traffic upstream and hands your API a stream of metadata events instead of raw byte streams.

Filestack’s upload pipeline runs on infrastructure built to absorb this kind of fan-out; file count and rate limits are enforced in the picker before the network is even touched; virus detection runs inline in the pipeline rather than queuing on your own workers, and your servers only receive webhooks once a file is safely ingested and checked. Your application never has to reason about 1,000 simultaneous PUT requests, because it never sees them.

For an IT director at a startup company asking what’s the most secure way to manage hundreds of file uploads, the calculus is straightforward: every control described in this article: caps, throttles, queues, scanning, has to be built, tuned, and maintained somewhere. Relocating that surface to a system designed for it is less about outsourcing effort and more about outsourcing blast radius.

Whichever direction you take, build it in-house or hand off the fan-out, the underlying principle doesn’t change.

Conclusion: Count Files, Not Just Bytes

The instinct to watch for “big” uploads is understandable, but it misses where most upload endpoints actually break. Cap file counts per batch, shape request rates per account, queue ingestion so bursts smear over time, and scan asynchronously so a backlog degrades instead of cascading. Where the fan-out is large or unpredictable enough, relocating ingestion to a managed pipeline like Filestack removes the problem from your infrastructure entirely.

If you haven’t audited your own upload endpoint against these checks, start with the layered defences in the “Defences, Limits, Rate Shaping and Queues” section above; file-count caps and rate limiting alone catch most of the risk with the least amount of new infrastructure. And if you’d rather not build and maintain that stack yourself, Filestack’s upload pipeline handles the caps, queueing, and scanning for you.

FAQ

How can small files cause a denial of service?

Per-file fixed costs: auth checks, database writes, storage requests, scan jobs, multiply with every file added to a batch. Thousands of tiny files can out-cost a single large file even though the total bytes transferred are far smaller.

What limits should an upload endpoint enforce?

At minimum: file-count caps per batch, per-account rate limits, and archive expansion-ratio caps to prevent zip-bomb-style decompression attacks.

Does a managed uploader help?

Yes. Filestack terminates upload traffic upstream, so your API receives metadata events rather than raw byte streams, removing the fan-out from your infrastructure entirely.

This article was published on the Filestack blog.

Your First 10 Minutes With Filestack, Signup to First Upload

IderaDevTools — Thu, 13 Aug 2026 12:28:07 +0000

Signup gives you an API key. The key gets you an upload. The upload gives you a handle, and the handle is the only thing you need for every transformation and delivery URL after that.

Those four steps are the whole first session with Filestack, and each one takes minutes rather than hours. The walkthrough below runs all four and shows what each returned.

Get a key

Sign up at filestack.com/signup-free. The form asks for a name, a company email and a password, and it shows the free plan allowance next to the fields you are filling in.

Your API key appears in the developer portal as soon as the account exists. It is about 20 characters, it identifies your application, and it is not a secret in the way a password is. It goes in client-side JavaScript on purpose, because that is how browser uploads reach us without a round trip through your server first.

The app secret is different. It stays on your server, it signs security policies, and nothing in this walkthrough needs it.

Your first upload

Two paths get a file in. Pick the one that matches where you are sitting.

Uploading from a browser

The picker is a hosted upload interface. Loading the script and calling picker() is the shortest route to a working upload, and it handles the retry and chunking work that hand-rolled <input type="file"> code usually skips.

<script src="https://static.filestackapi.com/filestack-js/3.x.x/filestack.min.js"></script>
<button id="pick">Upload a file</button>

<script>
  const client = filestack.init('YOUR_API_KEY');
  document.getElementById('pick').onclick = () => {
    client.picker({
      accept: ['image/*'],
      maxFiles: 5,
      onUploadDone: ({ filesUploaded }) => console.log(filesUploaded[0].handle),
    }).open();
  };
</script>

Click the button and the picker opens over your page. My Device is the local file system. The icons down the left are the other sources the free plan includes, so a multi file upload UI with Google Drive and a URL tab costs you nothing beyond the fromSources array.

Choose a file and it appears in a review list with its size before anything is sent. Nothing uploads until you press Upload, which is worth knowing when you are testing against a quota.

Press it and onUploadDone fires with one entry in filesUploaded. The handle field on that entry is what every URL below uses.

Uploading without a browser

If you are on a server or just want to see the response shape, one POST does it:

curl -X POST -F "fileUpload=@photo.jpg" \
  "https://www.filestackapi.com/api/store/S3?key=YOUR_API_KEY"

which returns:

{
  "url": "https://cdn.filestackcontent.com/0J0PpoBYScqJrHF8lbrO",
  "size": 107013,
  "type": "image/jpeg",
  "filename": "photo.jpg"
}

That is the same REST API upload file endpoint the SDKs sit on top of, so the handle it returns behaves identically. The 20 characters at the end of that URL are the handle.

What the handle is for

The handle is the file. Every delivery and processing URL is the handle with tasks in front of it:

https://cdn.filestackcontent.com/TASK/HANDLE

Your API key does not go in that URL. The handle already identifies the application that owns the file, so adding the key puts a credential in front of your users for nothing.

Your first transformation

Put a task in front of the handle and the file changes on the way out. Resize is the one to try first, because the result is obvious:

https://cdn.filestackcontent.com/resize=width:300/HANDLE

The 107,013 byte original came back as 42,214 bytes at 300 pixels wide. Nothing was stored to produce that. The transformation ran at request time and the result was cached, which is why you never generate thumbnail variants ahead of time or keep them anywhere.

Tasks chain left to right. Adding a format change on the end took the same request to 32,854 bytes:

https://cdn.filestackcontent.com/resize=width:300/output=format:webp/HANDLE

Order matters, because each task acts on what the previous one produced. Resize first and the encoder is working on a smaller image. The reasoning behind picking a format at all is in the guide to convert to webp, and the full parameter set for every task is in the processing API reference.

Crop, rotate, watermark, compress and quality all work the same way on a free key, as does face detection, so you can blur faces in a URL without training anything. The image editing api guide covers how the tasks combine.

Where the file lives now

At the CDN, already, on a public URL. There is no publish step and no bucket to configure. The default response carries cache-control: public, max-age=2667950, so once an edge has served a transformation it keeps serving it without rerunning anything.

Set your own expiry when you need a shorter one:

https://cdn.filestackcontent.com/cache=expiry:3600/resize=width:300/HANDLE

The Filestack CDN then answers that URL with cache-control: public, max-age=3600. How the edges pick up files and how long they hold them is covered in file delivery.

Public by default matters for the next thing you build. Anyone with the handle can read the file, which is right for a portfolio and wrong for invoices, and the fix is a signed policy rather than a different upload call. A secure file upload service is a configuration you turn on later, not a separate product.

When something comes back wrong

Every failure here answers in plain text, so read the body rather than guessing from the status code.

What you sentStatusWhat the body saysA handle that does not exist400Bad RequestA task name with a typo400validation error: task not found: "resiz"A parameter name with a typo400validation error: invalid parameter widht for resize taskAn operation your plan does not include403You don't have permission to perform this task: ocr. Please check your access settings

That last one is the boundary worth knowing early. Operations that read and interpret a file, such as optical character recognition, tagging, captioning and enhancement, run on the higher plans. Everything that changes a file’s shape, size or format runs on the free plan, which is most of what a first project needs.

What to try next

The quotas are 500 uploads, 1,000 transformations, 1 GB of bandwidth and 1 GB of storage a month, checked on the free plan page on 6 August 2026. A prototype does not come near them.

Three directions from here, depending on what you are building:

Wire it into your framework. The same three lines work in a React file upload component or behind an ordinary HTML form, with import * as filestack from 'filestack-js' instead of the script tag. In Next.js the component needs 'use client', because the picker needs a browser.

Chain transformations. Crop, then resize, then encode, in one URL, is the pattern behind every responsive image you will serve. Order matters, because resizing first means the encoder has fewer pixels to work on.

Take the whole lifecycle seriously. Once uploads are real user files, storage, transformation and delivery become one system. The image upload service guide covers how those pieces fit together.

Originally published on the Filestack blog.

How to Handle Failed and Interrupted JavaScript File Uploads

IderaDevTools — Wed, 12 Aug 2026 09:22:39 +0000

File uploads can fail even when your code is working correctly. Real-world networks aren’t always reliable. A phone can lose signal, hotel Wi-Fi can disconnect during an upload, or a user might close the tab before the upload finishes.

If your file upload API isn’t prepared for these situations, users may see a loading spinner that never ends. This can quickly make them lose trust in your app.

This guide assumes you already have a basic upload flow with features like drag-and-drop, a progress bar, and file validation. We won’t cover how to build those again.

Instead, we’ll focus on what happens when an upload fails. You’ll learn how to detect failed uploads, retry them without causing more problems, and resume large uploads instead of starting again from the beginning.

We’ll use simple, beginner-friendly JavaScript examples and also look at how Filestack’s JavaScript SDK handles some of this out of the box.

Key Takeaways

Most upload failures happen because of network issues, timeouts, or closed tabs, not because your code is wrong.
Handle network and server errors differently instead of treating every error the same.
Wait a little longer between each retry (exponential backoff), and limit how many times you retry.
For large files, continue from the last uploaded chunk instead of starting from the beginning.
Show clear error messages and keep the upload progress instead of resetting it to 0%. This helps users trust your app.

Before we look at how to handle failed uploads, here’s a quick refresher on why they happen in the first place.

Why Uploads Fail

We won’t go through every reason an upload can fail here. Why most file uploads fail and what to do about it already explains common causes in detail, from filename issues and server timeouts to poor network connections. It’s also worth reading JavaScript file upload API: expectations vs. reality if you’ve only tested your upload flow on fast office Wi-Fi. Real users may have much slower or less reliable connections. Here, we’ll focus on what happens after an upload fails: how to detect the failure and recover from it smoothly.

Now, before you retry an upload, you first need to know what actually went wrong.

Detecting a Failed Upload Reliably

Upload failures aren’t always easy to identify. Different types of errors can happen, and each one may need a different response.

Network Error vs. Server Error

A network error usually means the request didn’t get a response because the connection was lost. A server error means the server did respond, but returned an error status such as 500 or 503.

These errors shouldn’t always be handled the same way. A network error is usually worth retrying. But some server errors, such as a 400 caused by an invalid file type, won’t be fixed by retrying.

You can learn more about these status codes in the MDN guide to HTTP status codes.

Timeouts

Sometimes an upload doesn’t fail or succeed; it simply keeps waiting. To handle this, you can set a timeout yourself instead of waiting forever.

Here’s a simple example using fetch and AbortController, as explained on MDN’s AbortController page:

async function uploadWithTimeout(file, timeoutMs = 15000) {
  const controller = new AbortController();
  const timer = setTimeout(() => controller.abort(), timeoutMs);

const formData = new FormData();
  formData.append("file", file);
  try {
    const response = await fetch("/api/upload", {
      method: "POST",
      body: formData,
      signal: controller.signal,
    });
    if (!response.ok) {
      // Server responded, but with an error status
      throw new Error(`Upload failed with status ${response.status}`);
    }
    return await response.json();
  } catch (err) {
    if (err.name === "AbortError") {
      throw new Error("Upload timed out. Please try again.");
    }
    throw err; // some other network or server error
  } finally {
    clearTimeout(timer);
  }
}

This function helps you handle three common upload problems: a request that takes too long, a server that returns an error, and a network connection that drops.

Once you know what type of failure happened, it’s much easier to decide whether you should retry the upload or handle the error in another way.

Now that you can detect a failure, you need to decide when to retry it automatically and when to stop.

Retry Logic That Doesn’t Make Things Worse

Retrying an upload sounds simple, but you need to do it carefully. Too many retries can send even more requests to a server that’s already having problems. In some cases, retries can also cause the same file to be uploaded twice.

Backoff, Not Immediate Retries

If an upload fails, retrying it immediately may lead to the same error because the network or server hasn’t had time to recover.

A better approach is exponential backoff. This means waiting a little longer after each failed attempt. For example, you might wait 1 second before the first retry, 2 seconds before the next, and then 4 seconds.

This gives the network or server some time to recover. You can read more about why this approach is useful in AWS’s write-up on exponential backoff and jitter.

Here’s a simple example:

async function uploadWithRetry(file, maxAttempts = 4) {
  let attempt = 0;

while (attempt < maxAttempts) {
    try {
      return await uploadWithTimeout(file);
    } catch (err) {
      attempt++;
      if (attempt >= maxAttempts) {
        throw new Error("Upload failed after several attempts. Please try again later.");
      }
      const waitTime = 1000 * Math.pow(2, attempt - 1); // 1s, 2s, 4s...
      await new Promise((resolve) => setTimeout(resolve, waitTime));
    }
  }
}

Set a Retry Limit

Usually, three to five attempts are enough. If the upload still fails, stop retrying and show a clear error message with a button that lets the user try again manually. Don’t keep retrying silently in the background.

Avoid Duplicate Uploads

Sometimes a request is slow but hasn’t actually failed. If you retry too soon, the same file could be uploaded twice.

One way to prevent this is to track the state of each file using values such as pending, in-progress, succeeded, and failed. Before retrying, check the current state to make sure another upload isn’t already running.

Some upload APIs also provide ways to identify duplicate requests, so the server can avoid processing the same upload more than once.

Once your retry logic is in place, the next step is making sure large files don’t have to start over after every failed attempt.

Resuming Instead of Restarting: Chunked and Resumable Uploads

Retrying a small file isn’t a big problem. But if a 2GB video fails near the end, starting the entire upload again can be frustrating.

For large files, restarting from 0% every time the connection drops isn’t a good experience. A user might wait several minutes, lose their connection for a moment, and then have to start all over again.

A better approach is to split the file into smaller chunks. These chunks can be uploaded one at a time or a few at the same time. You also keep track of which chunks have already uploaded successfully.

If the connection drops, you only need to upload the remaining or failed chunks instead of uploading the entire file again.

Here’s a basic example of how chunking works:

function splitIntoChunks(file, chunkSize = 5 * 1024 * 1024) {
  const chunks = [];
  let start = 0;

while (start < file.size) {
    chunks.push(file.slice(start, start + chunkSize));
    start += chunkSize;
  }
  return chunks;
}
async function uploadChunks(file) {
  const chunks = splitIntoChunks(file);
  const status = chunks.map(() => "pending");
  for (let i = 0; i < chunks.length; i++) {
    if (status[i] === "succeeded") continue; // already uploaded, skip on resume
    try {
      await uploadWithRetry(chunks[i]);
      status[i] = "succeeded";
    } catch (err) {
      status[i] = "failed";
    }
  }
  return status;
}

The important part here is the status array. It keeps track of which chunks were uploaded successfully and which ones failed.

So, if the connection drops, you don’t have to start the entire upload again. You can skip the completed chunks and upload only the ones that are still missing or failed.

If you don’t want to build and maintain this chunking system yourself, the Filestack upload API can handle this kind of upload process for you. And if you’re building your own upload endpoints, designing a JavaScript API is a useful next read for learning how to create an API that supports chunked and resumable uploads.

Even if your retry logic works in the background, users still need to understand what’s happening.

Giving Users Useful Feedback Mid-Failure

Two simple things can make a big difference:

Don’t Reset Progress to 0%

If chunk 4 fails but chunks 1 through 3 were already uploaded successfully, don’t move the progress bar back to 0%. Keep the progress based on the chunks that have already finished.

Show Clear and Specific Errors

A message like “Upload failed” doesn’t tell the user what went wrong or what they should do next. Instead, use messages like “Upload failed, check your connection and try again” or “This file is too large (max 500MB).” This helps users understand the problem and what they can do about it.

If you’re uploading multiple files at the same time, tracking the status of each file becomes even more important. Uploading multiple files using JavaScript explains how you can use the same approach for multiple files, including tracking retries for each file separately.

Once users can clearly see what’s happening during a failed upload, you also need to make sure these failure states work as expected before your app goes live.

Testing Failure Scenarios Before They Happen in Production

It’s better to find upload problems during testing than when a real user runs into them.

You don’t need an unreliable internet connection to test these situations. Your browser’s developer tools can simulate them for you.

Throttle the connection. Chrome and Firefox DevTools have network throttling options such as “Slow 3G” in the Network tab. Use them to see how your retry and backoff logic works on a slow connection.
Simulate an offline drop. DevTools also has an “Offline” option. Start uploading a file, switch to offline mode during the upload, and see how your app handles the failure.
Test what happens when the tab closes. Start uploading a large file and close the tab before it finishes. When you open the app again, check whether it remembers which chunks were already uploaded or starts the whole upload again.

Testing these situations helps you find problems with retries, error handling, and resumable uploads before your users experience them.

If you haven’t built the basic upload flow yet, the step-by-step guide to HTML file upload using JavaScript is a good place to start before adding this failure-handling logic.

Getting the file onto the server is the final step for this article, but it’s often just one part of the complete file workflow.

What Happens Downstream Once an Upload Finally Succeeds

Once a file uploads successfully, most apps need to do something with it. For example, you might resize an image, convert a video to another format, or create a thumbnail.

If you’re working with images, simplifying image editing with a JavaScript SDK explains some common ways to transform uploaded images.

After processing, the file usually needs to be available to your app or users. This is often done using a CDN URL or signed URL, which lets the file be displayed, downloaded, or shared without uploading it again.

With the full upload and recovery flow covered, here are a few best practices to keep in mind while putting everything together.

Best Practices and Common Pitfalls

Here are a few important things to remember when handling failed uploads:

Do handle network errors and server errors differently.
Do use exponential backoff and stop retrying after 3–5 attempts.
Do track the upload status of each file or chunk to prevent duplicate uploads.
Do keep the progress bar based on how much of the file has actually uploaded.
Don’t keep retrying in the background forever. Show an error message and give users a manual retry option.
Don’t retry every server error. For example, a 400 error caused by an invalid file type won’t be fixed by trying again.
Don’t restart large uploads from the beginning after a connection drop. Resume from the last successful chunk instead.

With these practices in place, your upload flow will be much better prepared for the network problems users face in the real world.

Conclusion

A file upload API shouldn’t only work when the internet connection is fast and stable. It should also be ready for slow networks, connection drops, and other common problems.

You don’t need to rebuild your entire upload system to handle these issues. Focus on four things: detect failures correctly, retry failed uploads with increasing wait times and a clear limit, split large files into chunks so they can resume after a connection drop, and keep users informed about what’s happening.

Getting these things right makes your upload experience much more reliable and user-friendly.

If you don’t want to build and maintain all of this yourself, Filestack’s JavaScript SDK handles retries, chunking, and resumable uploads as part of its upload API.

FAQ

Why does a file upload fail even when the code has no bugs?

Most upload failures happen because of real-world conditions, not problems in your code. A network connection might drop during an upload, especially on mobile; the server might time out while handling a large file, or the user might close the tab before the upload finishes.

How do I tell the difference between a network error and a server error in JavaScript?

A network error usually means the request didn’t get a response, often because the connection was lost. A server error means the server responded with an error status code. Handle them differently: network errors are usually worth retrying, while some server errors are not.

Should I retry a failed upload immediately or wait?

Instead of retrying immediately, wait a little and increase the wait time after each failed attempt. This is called exponential backoff. Retrying too quickly on a poor connection or overloaded server will often just fail again.

How do I avoid uploading the same file twice if a retry fires after a slow original request succeeds?

Track the status of each file, such as in-progress, succeeded, or failed, and check it before retrying. You can also use an idempotency key to help the server identify and ignore duplicate requests.

Do I need chunked uploads to handle large file failures well?

Not always, but it’s very useful for large files. It lets the upload continue from where the connection dropped instead of starting again from 0%, which is especially helpful on slow or unreliable networks.

This article was published on the Filestack blog.

How to Benchmark OCR API Accuracy Before Choosing a Vendor

IderaDevTools — Fri, 07 Aug 2026 10:05:39 +0000

Most teams testing an OCR API usually do one of two things: trust the accuracy number provided by the vendor or test a few random documents and make a decision based on the results.

But neither approach gives you a reliable accuracy number. If certain types of documents start failing in production, it can be difficult to understand why.

In this guide, we’ll look at a simple and repeatable way to test OCR accuracy using your own documents before choosing an OCR provider. You’ll learn how to create a realistic test set, prepare the correct results to compare against, run the benchmark, and understand the results.

If you want to understand what affects OCR accuracy, such as image quality, lighting, and contrast, Improve Data Accuracy with OCR covers those factors in more detail.

Here, we’ll focus on how to measure and verify OCR accuracy. If you want to try this process with an OCR API, Filestack Capture, our OCR and data capture product, can be used to run a benchmark like this.

Key Takeaways

Don’t rely on one overall accuracy percentage. Check OCR accuracy for each document type separately.
Use a smaller set of real, varied documents instead of testing only clean and perfect files.
Create a manually checked ground truth to compare with the OCR results.
Use confidence scores as an extra signal, but check whether they actually match real OCR errors.
Test OCR accuracy regularly because your documents and scanning methods can change over time.

Let’s take a quick look at why the accuracy number provided by a vendor may not match what you see with your own documents.

Why Vendor-Quoted Accuracy Numbers Don’t Transfer to Your Documents

A vendor might claim “99% accuracy,” but that number is usually based on its own test documents. Performance can be very different depending on the type and quality of the document. For example, an OCR tool might work very well on a clean, typed invoice but perform much worse on a blurry receipt photo or handwritten form.

The documents used for vendor testing may also be cleaner and easier to read than the files your users actually upload. Real users might take photos with their phones in poor lighting, upload low-quality scans, or submit documents with different layouts.

That’s why the best way to understand how an OCR API will perform for your use case is to test it with your own documents.

Once you know that a vendor’s accuracy number is only a starting point, the next step is building a test that gives you results you can actually use.

Building a Representative Test Set

Start by collecting real documents from each type your product handles. Don’t choose only clean, high-quality files. For example, if your product processes invoices, ID cards, and handwritten forms, include all three in your test set based roughly on how often you receive them.

You also don’t need thousands of documents to get useful results. Variety is more important than volume. For example, 100 documents that include tilted scans, poorly lit phone photos, low-contrast faxes, and clean files can be more useful than 1,000 documents that all look similar.

Make sure you include difficult and messy documents. These are often where OCR tools struggle the most, and they may not be well represented in a vendor’s own tests.

Once you have a representative set of documents, the next step is deciding what the correct OCR result should look like for each one.

Establishing Ground Truth

Ground truth is a manually checked and correct version of the text in your test documents. Think of it as the answer key that you’ll compare the OCR results against.

A person should carefully review and create the ground truth for at least a representative part of your test set. It takes more time, but it gives you a reliable way to know whether the OCR output is actually correct.

Before running the benchmark, you also need to decide what counts as an error. There are two common ways to measure this:

Character-level accuracy: Checks whether every letter, number, and character was recognised correctly.
Field-level accuracy: Checks whether important fields, such as an invoice total, were extracted correctly, even if there are small formatting differences.

These methods can give you very different accuracy results for the same document. Choose how you’ll measure accuracy before you start and use the same method throughout your benchmark.

This is similar to how speech recognition systems use word error rate to measure errors consistently.

Once you have your ground truth and know how you’ll measure accuracy, you’re ready to run the benchmark.

Running the Benchmark

When running your benchmark, don’t combine all the results into one overall accuracy score. Keep the results separate for each document type, such as invoices, receipts, forms, and handwritten notes.

This makes it easier to see which document types the OCR API handles well and where it struggles.

The testing process is simple: run each document through the OCR API and compare the returned text with your ground truth.

Here’s a beginner-friendly Python example using the standard library:

from difflib import SequenceMatcher
def word_accuracy(ocr_text, ground_truth):
    """Rough word-level accuracy: how many words match, in order."""
    ocr_words = ocr_text.split()
    truth_words = ground_truth.split()
    matcher = SequenceMatcher(None, ocr_words, truth_words)
    matching_words = sum(block.size for block in matcher.get_matching_blocks())
    return round((matching_words / len(truth_words)) * 100, 2)

# Ground truth: what the document actually says
ground_truth = "Invoice number 48213 dated March 3 2026 total 214.50"
# OCR output: what the API returned
ocr_output = "lnvoice number 48213 dated March 3 2026 total 214.5O"
print(f"Word accuracy: {word_accuracy(ocr_output, ground_truth)}%")

To keep results organised by document type, wrap this in a small loop that stores scores in a dictionary:

In this example, SequenceMatcher compares the OCR output with the ground truth and checks how many words match.

This gives you a simple word-level accuracy score. If you later need more detailed character-level scoring, you can use Levenshtein distance to measure the differences more precisely.

Next, you can organise the results by document type:

results_by_type = {}
def record_result(doc_type, ocr_text, ground_truth):
    score = word_accuracy(ocr_text, ground_truth)
    results_by_type.setdefault(doc_type, []).append(score)

# After running every document through this...
for doc_type, scores in results_by_type.items():
    average = round(sum(scores) / len(scores), 1)
    print(f"{doc_type}: {average}% average ({len(scores)} documents)")

This stores the scores for each document type separately and then calculates their average accuracy.

The example is intentionally simple. It gives you a useful starting point without requiring a specialised OCR testing library.

If you’re building the OCR pipeline that this benchmark will test, implementing scalable cloud-based OCR explains that process in more detail. If you want to learn more about character-level and word-level scoring, this walkthrough on CER and WER is a useful next step.

Run this same test against every document in your set, broken out by type, and you’ll end up with something like this:

This example shows why separating results by document type matters. An OCR API could perform very well on clean invoices but struggle with handwritten notes. A single overall accuracy score could hide that difference.

Once you have accuracy results for each document type, the next step is understanding what those numbers actually mean for your use case.

Interpreting the Results

Different accuracy scores across document types can tell you a lot about an OCR API.

For example, if an OCR API works well with scanned forms but struggles with phone-photo receipts, you’ve found an area where it performs poorly. Whether that’s a major problem depends on how often your users upload that type of document.

You should also look at confidence scores, but don’t rely on them alone. Check whether documents with low confidence scores are also the ones where your benchmark found real OCR errors.

If low confidence scores often match real errors, they can be useful for identifying documents that may need extra review. If they don’t match your benchmark results, don’t use them as a replacement for measuring actual OCR accuracy.

Filestack’s OCR feature is a useful reference if you want to see how confidence scoring can be used in a real product. For a more technical approach to comparing error rates across documents of different lengths, see the OCR-D quality assurance methodology.

Once you have accuracy results for each document type and understand how useful the confidence scores are, you can start comparing OCR providers based on your actual needs.

Turning This Into a Vendor Comparison

The main benefit of running this benchmark is that you can compare different OCR providers fairly.

Use the exact same test documents, ground truth, and scoring method for every OCR API you’re considering. If you use different documents for each provider, the results won’t give you a fair comparison.

Accuracy also shouldn’t be the only thing you compare. Consider factors such as processing speed, cost per document, and language support. For example, an OCR API that’s slightly less accurate but much faster or cheaper might still be a better choice for your use case.

If you want to compare these factors in more detail, how to choose the best OCR data extraction software for your business explains what to consider beyond OCR accuracy.

Once you’ve chosen a provider, there’s one more important thing to remember: OCR benchmarking shouldn’t be something you do only once.

Re-Running the Benchmark Over Time

The types and quality of documents you process can change over time. You might start receiving new form types, users may upload files from different devices, or your product may expand into markets with different document formats.

Because of this, a benchmark you ran six months ago may no longer show how well your OCR API performs today.

Mobile documents can also change differently from scanned documents. Phone cameras, lighting, and the way people take photos can all affect OCR accuracy. If many of your documents come from mobile devices, choosing the best OCR SDK for your Android project explains what to consider for mobile OCR.

Run the same benchmark regularly, such as every few months or whenever your document types change significantly. This helps you keep your OCR accuracy results up to date and make sure the API is still performing well on the documents your users actually submit.

With the full benchmarking process covered, here are a few best practices and common mistakes to keep in mind.

Best Practices and Common Pitfalls

Here are a few important things to remember when setting up your OCR benchmark:

Do keep your ground truth separate from your testing process so it doesn’t accidentally get changed based on the OCR results.
Do include difficult, low-quality documents that are similar to what your users might actually submit.
Don’t combine different document types into one overall accuracy score. Keep the results separate for each type.
Don’t use different test sets when comparing OCR providers. Use the same documents and scoring method for a fair comparison.
Don’t rely on a benchmark forever. Run it again when your document types or quality change over time.

Keeping these points in mind will help you build a benchmark that gives you more reliable and useful results.

Conclusion

An OCR accuracy number is only useful if the test behind it is reliable. A vendor’s accuracy percentage can give you a starting point, but it may not show how well the OCR API will work with your documents.

The best way to know is to test the API using your own documents and ground truth, and measure the results separately for each document type. It takes more time at the beginning, but it gives you results you can trust when choosing an OCR provider.

If you’re ready to run your own OCR benchmark, Filestack Capture is a good place to start testing your document set.

If you still have questions about setting up and using an OCR accuracy benchmark, here are answers to some common ones.

FAQ

Why don’t vendor-quoted OCR accuracy numbers hold up on my own documents?

Vendor accuracy numbers are usually based on clean, carefully selected test documents. A single overall percentage can also hide large differences between document types. Your own documents may be very different, so the OCR accuracy you get can also be different.

How many documents do I need for a meaningful OCR accuracy benchmark?

Variety is more important than having a large number of documents. A smaller test set that includes your real document types, formats, and quality levels can give you more useful results than a large set of only clean documents.

What’s “ground truth” in an OCR accuracy benchmark, and how do I establish it?

Ground truth is a manually checked, correct version of the text in your test documents. You compare the OCR results against it to measure accuracy. Before testing, decide whether you’re measuring character-level or field-level accuracy, because each method counts errors differently.

Should I trust confidence scores instead of running a full accuracy benchmark?

Confidence scores can be helpful, but don’t rely on them without testing first. Check whether low-confidence results actually match real OCR errors in your documents. If they do, you can use confidence scores as an extra signal for finding possible errors.