There's a particular kind of technical mistake that's easy to make and expensive to fix. It doesn't come from ignorance, exactly. It comes from taking something that works well in one context and applying it everywhere because it's available, familiar, and fits the current architecture on paper.
Lambdas are a good example of this.
I've seen this mistake made in production, at cost, by people who should have known better. But before I get into that, let's establish the foundation.
What is AWS Lambda?
Lambda is Amazon's serverless compute offering. The concept is elegant: you write a function, you define it's triggers, and AWS handles everything else. No server provisioning, no infrastructure management and no paying for idle time. The function runs, does what it has to and then disappears.
That last part is vital: it disappears. Lambdas are ephemeral by design. They spin up to handle a task and shut down when they are done. This isn't a limitation, it's the point. It's what makes them powerful in the right context.
Where Lambda Really Shines
The sweet spot for Lambda is tasks that are:
Short-lived: the task needs to complete within a bounded time
Event-driven: something happens(trigger) and a function responds
Intermittent: the task does not need to be running constantly
Stateless: the task does not need to remember anything between calls
Classic examples are: image processing after an upload to S3, sending notifications triggered by a user action, responding to an API Gateway request. These are tasks that happen, finish and move on. Lambda handles them cleanly and cheaply.
Where It Breaks Down
Lambda has hard constraints that don't bend regardless of how you configure it. This is where teams get into trouble:
- 15-minute maximum execution time. If your task runs longer, Lambda kills it.
- Cold starts. When a function hasn't been invoked recently, there's a startup delay. For latency-sensitive workflows, this matters.
- Pricing model. You're billed per invocation and per GB-second of compute time. For short, infrequent tasks this is excellent. For long-running or high-frequency workloads, costs compound fast.
- Statelessness. Every invocation starts fresh. If your process needs to maintain state across a long operation, you're working against the architecture.
Let's consider a long running background job. One that needs to run daily, process a large dataset, maintain context between executions, and potentially run for much longer than 15 minutes.
Obviously, Lambda is the wrong tool for this. Not because it can't be coerced into the scenario described above with workarounds, but because you'd be fighting the design of the service at every step. Increasing your cost and increasing the fragility of your application.
There are tools that have been purpose-built for this kind of workload, such as EC2 Instances, ECS tasks, containerised services, or even a simple scheduled job on a properly configured server. These options exist precisely because sustained, stateful, long-running compute is a fundamentally different issue than ephemeral event-driven execution.
The Mistake Is in the Reasoning, Not the Tool
Lambda is a well-designed service. The issue is never the tool itself. It's the reasoning (or lack thereof) behind the decision to use it.
"We're using Lambda because we're serverless first" is a strategy.
"We're using Lambda because we're using Lambda" is not.
When the justification for a technical decision is the decision itself, that's a signal worth paying attention to. Good system design is not just about asking if we can user this tool, but if we should, here, for this specific requirement.
I learned this the expensive way, watching a system architect itself into a corner because no one was allowed to ask that second question.
More on that in Part 2.
Top comments (0)