Before Serverless: The EC2 Era
EC2 (Elastic Compute Cloud) is one of the foundational services in AWS. It gives you a virtual server where you run applications on AWS infrastructure. You control the computing resources. You scale up and down with a button. Unlike traditional on-premises servers that require physical management, EC2 abstracts the hardware. But you still manage a lot.
For years, companies used EC2 for everything. It worked. But certain use cases made EC2 feel like overkill: processing an event, reacting to an image upload, responding to a database change. With EC2, you are not just writing code. You provision resources, deploy applications, patch operating systems, and handle infrastructure. For a simple function that runs for two seconds, that overhead adds up.
Lambda: Serverless Compute Arrives
In November 2014, AWS introduced Lambda, the first core serverless compute service.
Lambda lets you run code in response to events. You do not manage the underlying servers. AWS handles provisioning, scaling, and patching. You write your function, deploy it, and Lambda runs it when triggered.
How Lambda Works
Lambda runs on Firecracker, a micro virtual machine technology that AWS developed and open-sourced. Dedicated hosts run these micro VMs. When you invoke a Lambda function, AWS creates an execution environment, allocates resources, and runs your code. After execution, the environment can be reused for subsequent invocations or cleaned up.
You focus on your code. AWS handles everything else. This was the shift that made serverless real.
Three Invocation Types
Lambda supports three invocation patterns. Each serves different use cases.
Asynchronous Invocation
The caller sends an event to Lambda and moves on. It does not wait for a response. Lambda processes the event in the background. This is used for background tasks: sending emails, processing images, and updating analytics. The event source fires and forgets.
For example, an S3 bucket with S3 events enabled - when you upload a new image to this S3 bucket, this causes an event to be generated and sent through to Lambda.
Synchronous Invocation
The caller sends a request, waits for Lambda to execute, and receives a response. The caller is blocked until the function completes. This is used for user-facing applications. A user submits a form, API Gateway triggers Lambda, Lambda processes the request, and the response goes back to the user.
Poll-based Invocation
Lambda polls a source for new records and processes them in batches. This works with SQS queues, Kinesis streams, and DynamoDB Streams. Lambda pulls messages or records, processes them, and deletes them from the source. Use this for stream processing and queue-based workloads.
Why Serverless Matters
Serverless is not just about avoiding server management. It changes how you build.
Automatic scaling: Functions scale in response to demand without configuration. You do not provision capacity for peak load.
Cost efficiency: You pay for execution time, not idle infrastructure. If your function runs for 100 milliseconds, you pay for 100 milliseconds.
Built-in resilience: Without much configuration, you get high availability. AWS handles infrastructure-level reliability.
Serverless fits well for:
- Startups wanting minimal ops overhead
- Event-driven workloads (IoT, APIs, webhooks)
- Burst or unpredictable traffic
- Applications decomposed into microservices
Lambda Limitations
Lambda is not the right tool for every job.
Cold starts introduce latency: When a new execution environment spins up, your function takes longer to respond. The impact depends on runtime and package size. For latency-sensitive applications, consider Provisioned Concurrency, which keeps environments warm.
Timeout limits: Lambda functions can run for a maximum of 15 minutes. Long-running or CPU-intensive tasks may need containers or EC2.
Vendor lock-in: Using Lambda-specific features ties you to AWS. Moving to another cloud requires rewriting.
Debugging complexity: Serverless architectures need new patterns and tooling for testing and debugging.
What Changed at re:Invent 2025
AWS addressed several Lambda limitations at re:Invent 2025.
Lambda Managed Instance
One argument against Lambda was it is expensive at scale. With Lambda managed instance, the argument got weaker. By default, your Lambda functions run on shared EC2 instances across AWS accounts. You can now use dedicated instances from your own account instead. AWS still manages everything: OS patching, load balancing, and auto-scaling. You get isolation without the infrastructure burden.
Durable Functions
AWS introduced Lambda Durable Functions, similar to Step Functions but without Amazon States Language. You write workflows in familiar code with your existing dependencies. Unlike standard Lambda's 15-minute timeout, durable function execution can span multiple invocations with a maximum timeout of one year.
Official AWS documentation for Durable Functions
The tradeoff with these new options: you now manage instance capacity alongside function code. Not harder, just different.
Your existing Lambda integrations still work. EventBridge, CloudWatch, X-Ray, AWS Config. Nothing changes in how you trigger or observe functions.
API Gateway: The HTTP Trigger
In 2015, AWS introduced API Gateway, making Lambda practical for web and mobile applications.
Lambda needs a trigger. For web applications, that trigger is usually an HTTP request. API Gateway provides the HTTP endpoint that receives requests and invokes your Lambda function.
API Gateway is fully managed. AWS handles provisioning, scaling, and patching. You define your routes and connect them to backend logic.
Beyond Routing
API Gateway does more than route requests to Lambda.
- Authentication and authorization: Control access using IAM roles, Cognito user pools, or Lambda authorizers.
- Request validation: Define a schema and reject invalid requests before they reach your function.
- Rate limiting: Protect your backend by limiting requests per second.
- Usage quotas: Limit requests per day or month. Useful for SaaS applications with tiered plans.
- Caching: Store responses to reduce Lambda invocations and improve latency.
- Monitoring: Integrates with CloudWatch for metrics and logging.
Three API Types
REST API is the original API Gateway product. It includes the most features: API keys, per-client throttling, request and response transformation, caching, AWS WAF integration, and usage plans. Use REST API when you need advanced features or when building APIs for third-party consumption.
HTTP API is newer, simpler, and cheaper. AWS designed it for performance with minimal features. HTTP APIs cost up to 70% less than REST APIs. They support Lambda and HTTP backends, JWT authorization (OAuth 2.0 and OpenID Connect), CORS configuration, and automatic deployments. For most serverless applications, HTTP API is sufficient.
WebSocket API enables real-time, two-way communication. Unlike REST and HTTP APIs where the client initiates every request, WebSocket keeps a persistent connection open. The server pushes messages without waiting for a request. Use WebSocket API for chat applications, live notifications, real-time dashboards, and multiplayer games.
DynamoDB: The Serverless Database
DynamoDB is a NoSQL database fully managed by AWS. You do not provision servers or manage resources. AWS handles all of that. You focus on your data.
AWS introduced DynamoDB in 2012, before Lambda existed. It was foundational for serverless architectures from the start.
Why Traditional Databases Fail with Lambda
When you build serverless applications, traditional databases like PostgreSQL or MySQL create problems.
Serverless is ephemeral. Lambda execution environments spin up, handle a request, and get destroyed. Traditional databases need persistent connections. Your application opens a connection, keeps it alive, and reuses it. Lambda does not stay alive. Environments get killed. New environments spin up. Each one tries to open a new connection. With high traffic, you exhaust your connection pool. Your database chokes.
DynamoDB works differently. It operates like an API. You make an HTTP request, and you get a response. No persistent connections. No connection pools to manage. Each request is independent. This is why DynamoDB works well with Lambda.
Capacity Modes
DynamoDB offers two capacity modes.
Provisioned capacity: You specify reads and writes per second. You pay for that capacity whether you use it or not. This is used when traffic is predictable, and you want to optimize costs.
On-demand capacity: Pay per request. This came out later in 2018, perfect for serverless. No capacity planning. DynamoDB scales automatically. This is used when traffic is unpredictable or when you are starting.
For serverless applications, on-demand mode usually makes sense. It matches the pay-per-use model of Lambda.
DynamoDB Resources
If you want to go deeper on DynamoDB:
- The DynamoDB Book by Alex DeBrie: If you are serious about DynamoDB, this is the book. Covers data modelling, access patterns, and advanced techniques. Essential reading for serverless developers.
- Awesome DynamoDB: A curated list of resources for modelling, operating, and using DynamoDB.
Communication Patterns: Request-Response vs Event-Driven
Serverless applications use two main communication patterns.
Request-response is synchronous. The client sends a request and waits. This works for real-time scenarios: fetching data, submitting a form, and logging in. The user expects an immediate answer. API Gateway with Lambda is request-response in action.
Event-driven is asynchronous. A publisher announces something happened and does not wait. Consumers react when ready. This works for background processing: sending emails, updating analytics, processing images, syncing data. Choreography and orchestration are both patterns within event-driven architecture.
Most serverless applications use both. API Gateway handles user-facing requests where you need immediate responses. EventBridge or SNS handles background processing where work happens asynchronously.
Step Functions: Orchestration
AWS introduced Step Functions in 2016 to coordinate multiple Lambda functions into workflows.
Orchestration means a central controller manages the workflow. Think of a conductor leading an orchestra. The conductor tells each musician when to play. The orchestrator tells each component what to do and when.
Step Functions is the orchestrator. You define your workflow as a state machine. Step Functions executes it, calling Lambda functions in sequence, handling retries, managing state, and dealing with errors.
When to Use Orchestration
Orchestration works well when:
- You need visibility into the entire workflow
- Order of execution matters
- You want centralized error handling
- The workflow has many conditional paths
- Processes run for hours or days
Consider an order processing workflow: validate payment, check inventory, reserve items, send confirmation, notify shipping. You could write one Lambda that calls other Lambdas, handling retries and errors yourself. AWS calls this the "Lambda as orchestrator" anti-pattern. It works, but it gets messy.
Step Functions handles this for you.
What Step Functions Provides
Built-in error handling: Define retry strategies and catch blocks. If a step fails, Step Functions retries automatically or routes to a fallback.
State management: Step Functions tracks where you are in the workflow. If something fails, you know which step failed and can resume from there.
Visual monitoring: You see your workflow as a diagram. Each execution shows which steps succeeded, failed, or are in progress.
Long-running workflows: Lambda has a 15-minute timeout. Step Functions Standard workflows can run for up to one year.
EventBridge: Choreography
AWS introduced EventBridge in 2019, evolving from CloudWatch Events, to enable event-driven architectures at scale.
Choreography has no central controller. Each service listens for events and decides how to respond. Think of dancers who know their moves and react to the music without someone directing each step.
EventBridge is the choreography tool in AWS serverless. Services publish events to an event bus. Other services subscribe to events they care about and react independently.
When to Use Choreography
Choreography works well when:
- Services should be truly independent
- Different teams own different services
- You want to avoid a single point of failure
- The workflow is simple (event happens, services react)
EventBridge and SNS are not competing. You often use both together. EventBridge excels at content-based routing with complex rules. SNS excels at fan-out to multiple subscribers. Choose based on the scenario.
When Lambda Is Not Enough: Fargate
Lambda handles most serverless workloads. But sometimes you hit limits.
Lambda has a 15-minute timeout. Memory caps at 10 GB. Cold starts affect latency-sensitive workloads. The package size is limited. Some workloads need persistent processes or specific runtimes that Lambda does not support.
When you hit these limits, Fargate is an option.
Fargate is a serverless container. You run containers without managing servers. Traditional container hosting (ECS on EC2, Kubernetes on EC2) requires you to provision and manage instances. You decide server count, instance types, and scaling. You patch operating systems.
Fargate removes that. You define your container, specify CPU and memory, and AWS runs it. No servers to manage. No clusters to configure. You pay for the computing resources your containers use.
Fargate works with ECS (Elastic Container Service) and EKS (Elastic Kubernetes Service). If you already use Kubernetes, Fargate can run your pods without managing nodes.
The Serverless Mindset
Serverless is not just "no servers." The servers exist. You just do not manage them.
The real shift is in how you design systems. You think about events and reactions, not servers and capacity. You think about how signals flow, not where code runs. You design around system boundaries, coupling, and error propagation.
Lambda, API Gateway, DynamoDB, Step Functions, EventBridge. These are not just services. They are building blocks for a different way of thinking about applications.
- Start with Lambda for compute
- Add API Gateway for HTTP triggers
- Use DynamoDB for data
- Coordinate with Step Functions when you need orchestration
- Decouple with EventBridge when you need choreography
- Reach for Fargate when you outgrow Lambda
That is the serverless toolkit. The rest is learning when to use each piece.
What Comes Next
This article covered the building blocks. Production serverless requires two more skills: observability and cost management.
Observability means understanding what your system is doing. CloudWatch collects logs and metrics from Lambda, API Gateway, and DynamoDB. X-Ray traces requests across services, showing you where time is spent and where errors occur. Without observability, debugging distributed systems becomes guesswork.
Serverless promises you pay only for what you use. No traffic, no cost. That is mostly true. But serverless costs can surprise you. Lambda invocations are cheap, but they add up. DynamoDB writes cost more than you expect. CloudWatch logs accumulate. Without attention, you can spend more than planned.
Understanding observability and cost patterns helps you build production-ready serverless applications. The services covered here give you the foundation. The next step is putting them together in a real project.
Further Resources
Books
- Serverless Essentials: A guide for leaders and beginners by Paul Obayuwana. This book covers the essentials: Lambda, API Gateway, DynamoDB, EventBridge, Step Functions, Fargate, security, cost management, and monitoring. Broad enough to give you the full picture.
Blogs and Newsletters
- The Burning Monk by Yan Cui: Deep dives on serverless architecture, patterns, and best practices. Yan consults for companies running serverless in production. His insights come from real experience.
- Off By None by Jeremy Daly: Weekly serverless newsletter. Curates the best serverless content each week.
- AWS Compute Blog: Official AWS blog covering Lambda, Step Functions, and other compute services. Announcements, tutorials, and best practices from AWS.
Open Source Projects
- Serverless Framework Examples: A collection of serverless application examples built with the Serverless Framework. Covers different languages, use cases, and AWS services.
- AWS Lambda Handler Cookbook: A production-ready serverless template by Ran Isenberg. Shows best practices for Lambda functions with CDK, including testing, CI/CD, and operational patterns.
- Powertools for AWS Lambda: Open source toolkit for implementing serverless best practices.
Courses
- Production-Ready Serverless: The most comprehensive serverless course available. Covers testing, monitoring, security, CI/CD, and operational best practices. Hands-on, practical, and based on real production experience.
- AppSync Masterclass: If you are building GraphQL APIs on AWS, this is the course. Covers AppSync from basics to production patterns, including real-time subscriptions, authentication, and integrating with Lambda and DynamoDB.
Top comments (0)