The first version of a PDF-merging Lambda I shipped timed out on roughly one batch in twenty, always the larger ones, always at the same point in the logs. The function was doing everything correctly. It just wasn't finishing in time, and the reason had almost nothing to do with the PDF logic itself.
Where the budget actually goes
A 10-second Lambda timeout sounds generous for "merge a few files," and it is, if the entire budget went to merging files. It doesn't. Cold start alone can eat close to two seconds on a function that isn't kept warm. Downloading the input files from S3 into the function's temp storage takes real time, especially for anything above a few megabytes. The actual PDF operation, whether it's a merge, a compress, or both, is often the smallest single line item in the budget, not the biggest.
Once I actually measured it instead of assuming, the shape of the problem became obvious: the timeout failures weren't PDF processing taking too long, they were cold start and file transfer eating into a budget that had never been sized with those costs in mind.
The fix that mattered most: get out of the download business
The biggest single change was not downloading input files into the Lambda's local storage at all. Instead of pulling files from S3 into /tmp and then uploading them to the PDF API, the function passes signed S3 URLs directly to the API and lets it fetch the files itself:
import boto3
s3 = boto3.client("s3")
def handler(event, context):
keys = event["file_keys"]
urls = [
s3.generate_presigned_url("get_object", Params={"Bucket": BUCKET, "Key": k}, ExpiresIn=300)
for k in keys
]
result = pdf_api.run({"action": "merge", "files": urls})
if result.status != "success":
return {"statusCode": 500, "body": result.status}
s3.put_object(Bucket=BUCKET, Key=f"output/{context.aws_request_id}.pdf", Body=result.content)
return {"statusCode": 200, "body": f"output/{context.aws_request_id}.pdf"}
That one change removed an entire download phase from the Lambda's own execution time. The download still happens, it's just happening on the API's side of the request instead of inside the function, which doesn't count against the Lambda's timeout the same way.
Cold starts, and why keeping the function warm matters more here than usual
Cold start cost is a familiar serverless problem, but it bites harder on a function with a tight timeout than on one with generous headroom, because a couple of seconds lost to initialization is a much bigger fraction of a 10-second budget than of a 30-second one. Provisioned concurrency, keeping a small number of instances warm, turned a meaningful chunk of timeout failures into a non-issue, at the cost of paying for idle capacity. Whether that tradeoff is worth it depends entirely on traffic pattern: bursty, unpredictable PDF jobs benefit from it a lot more than a function that's already getting called constantly and staying warm on its own.
Package size turned out to matter here too, in a way I hadn't expected going in. A Lambda's cold start time scales with how much code and how many dependencies it has to load before your handler runs, and an early version of this function pulled in a full PDF-manipulation library as a dependency, purely as a fallback path that never actually got used once the API call was doing the real work. Trimming the deployment package down to just the S3 client and a thin HTTP wrapper for the API shaved a few hundred milliseconds off cold start on its own, which doesn't sound like much until you remember it's competing against a 10-second ceiling.
Batching within the function's own limits
There's a second failure mode worth planning for explicitly: a batch large enough that even the fixed costs, cold start plus one API round trip, don't leave enough room for the operation itself. Rather than letting a Lambda invocation time out on an oversized batch, it's worth chunking large batches into multiple invocations upstream, each sized comfortably under the timeout, and combining results afterward if needed. This is a case where fighting the platform's constraint is more expensive than designing around it.
def chunk_batch(files, max_per_chunk=10):
for i in range(0, len(files), max_per_chunk):
yield files[i:i + max_per_chunk]
Timeouts as a signal, not just a failure
Once the function stopped timing out under normal load, the remaining timeouts became genuinely useful signals rather than noise, they almost always meant something unusual about that specific batch, an oversized file, a slow upstream S3 region, worth investigating individually instead of dismissed as "serverless is just like that sometimes."
That distinction turned out to be the most useful mental shift in the whole project. Before the fix, a timeout meant almost nothing, it could have been cold start, download time, batch size, or the API call itself, and there was no cheap way to tell which. After the fix, with the fixed costs trimmed down and accounted for, a timeout almost always pointed at one specific, investigable cause. Instrumenting each phase separately, logging cold start duration, download duration, and API call duration as distinct fields rather than one combined execution time, made that diagnosis fast instead of guesswork.
The API side of the equation
None of this required the Lambda function to get faster at PDF manipulation, because it was never doing PDF manipulation directly. It calls a serverless-friendly PDF API that handles merge, split, compress, rotate, watermark, and convert against URLs it's given, priced per successful result, so the only latency the function actually owns is cold start, a lightweight request, and writing the result back to S3. If you're running PDF logic inside a tight serverless timeout and seeing occasional failures under load, check where the time is actually going before assuming the fix is more compute.
Top comments (0)