Bry

Posted on Oct 20 Originally published at Medium

Serverless vs Containers: A Technical Deep Dive

#serverless #docker #aws #devops

Key Points

Lambda = pay per invocation, fully managed, no idle cost. Containers = pay per compute hour, you manage the runtime.
Cold starts are real: Node.js Lambda hits 100–500ms; Java Lambda can hit 3000ms. This matters for latency SLOs.
At 1M requests/month with 500ms avg duration and 256MB memory, Lambda costs ~$2.79. An always-on Fargate task costs ~$14.93/month — but scales to handle millions with no extra config.
Lambda can't hold persistent connections or in-memory state across invocations. WebSocket servers belong on containers.
Neither is universally better. The right answer depends on your traffic shape, latency budget, and team's ops maturity.

Introduction

Every new backend project triggers the same debate: serverless or containers? Both camps have strong advocates, and both camps are partially right. I've built production systems on both sides of this line — Lambda APIs that scaled effortlessly through a product launch and ECS services that saved significant cost once traffic stabilized past the Lambda crossover point — so the answer I give now is less "it depends" and more a structured set of tests you can run against your own workload.

The honest answer is that Lambda and ECS solve different problems. Lambda is a function executor — it runs your code, handles scaling automatically, and charges you only when something actually fires. ECS (or any container runtime) is a process host — it keeps your application running continuously, gives you full control of the runtime, and charges you for the uptime regardless of traffic.

Picking the wrong one doesn't just cost money. It creates operational debt. A CRUD API on a full Kubernetes cluster is over-engineered noise. A WebSocket server on Lambda is architecturally impossible. A batch job that runs for 20 minutes on Lambda will never finish — Lambda's hard limit is 15 minutes.

This article gives you the technical facts: cold start mechanics, real pricing math, scaling behavior, state management constraints, deployment complexity, and a decision framework you can apply to your next project.

The Mental Model

Lambda is a function-as-a-service. You ship code, not a server. AWS provisions compute, handles OS patching, manages concurrency, and tears down the execution environment when your function finishes. You pay per invocation and per GB-second of compute time.

Containers are a process model. You define a runtime environment in a Dockerfile, run it as a long-lived process, and that process handles requests as they arrive. AWS ECS (with Fargate) manages cluster placement and removes the EC2 management overhead, but your container is still a running process that consumes resources between requests.

Lambda: event → cold start (if needed) → handler → response → teardown
Container: deploy → always running → request → handler → response → next request

The difference is not "new vs old" or "simple vs complex." It is about the fundamental execution model and what that model implies for cost, scaling, and architecture.

Cold Starts: The Hidden Cost of Serverless

A cold start happens when Lambda needs to spin up a new execution environment for your function. This occurs when there is no warm instance available — on first invocation, after a period of inactivity, or when scaling beyond the number of existing warm instances.

Cold start duration breaks into three phases:

Container init — AWS provisions a microVM (Firecracker), loads the runtime, and downloads your code package or container image.
Runtime init — The language runtime starts (Node.js loads V8, JVM bootstraps, etc.).
Handler init — Your initialization code outside the handler function runs (DB connections, SDK clients, config loading).

Cold Start Timing by Runtime

Runtime	Typical Cold Start	With VPC	With Container Image
Node.js 20.x	100–300ms	+200–500ms	+500–1500ms
Python 3.12	100–250ms	+200–500ms	+500–1500ms
Java 21 (JVM)	500–3000ms	+200–500ms	+1000–3000ms
Java 21 (SnapStart)	100–200ms	+200–500ms	+200–500ms
Go	50–100ms	+200–500ms	+400–1000ms

VPC attachment is a significant multiplier. If your Lambda function must attach to a VPC (to reach RDS, ElastiCache, or internal resources), cold starts increase substantially because Lambda must provision an elastic network interface. AWS has improved this with hyperplane ENIs — pre-allocating them per function rather than per instance — but VPC cold starts still add latency. I've seen this catch teams when they migrate a Node.js Lambda into a VPC to access a new internal service and suddenly their p95 response time jumps from 180ms to over 700ms — the VPC attachment overhead is invisible in local testing and doesn't show up until the first cold start hits in production.

Mitigation Strategies

Provisioned Concurrency keeps a specified number of Lambda instances permanently warm. You pay for those instances whether they receive traffic or not — effectively converting Lambda's invocation model to something closer to containers for that baseline capacity.

Keep-warm pings use EventBridge to invoke your function on a schedule (every 5 minutes). This prevents the execution environment from being recycled due to inactivity. It works, but it is a workaround, not a solution — it keeps one instance warm, not enough for burst traffic.

Minimize init code. Every SDK client, DB connection pool, and config file you initialize outside your handler adds to cold start time. Import only what you need. Lazy-initialize where possible. Keep your deployment package small.

For latency-critical APIs with a p99 SLO under 500ms, cold starts are a real risk. Either use provisioned concurrency, choose a fast runtime (Node.js, Go), or move to containers.

Pricing Model Comparison

Lambda Pricing (us-east-1, x86)

Requests: $0.20 per 1 million invocations
Compute: $0.0000166667 per GB-second
Free tier: 1M requests/month + 400,000 GB-seconds/month (always free)

Fargate Pricing (us-east-1, Linux/x86)

vCPU: $0.04048 per vCPU-hour
Memory: $0.004445 per GB-hour
Storage: First 20GB free; $0.000111/GB-hour beyond that

Real Cost Calculation: 1M Requests/Month

Scenario: 1 million requests/month, 500ms average duration, 256MB (0.25GB) memory allocation.

Lambda cost:

Request cost:    1,000,000 / 1,000,000 × $0.20   = $0.20
Compute cost:    1,000,000 × 0.5s × 0.25GB
                 × $0.0000166667/GB-s             = $2.08
                                             ──────────────
Subtotal:                                         $2.28
Free tier offset (400,000 GB-s → $6.67 credit):  -$0.00 (absorbed)
Monthly Lambda cost:                              ~$2.28

(Free tier covers the first 400,000 GB-seconds. 1M req × 0.5s × 0.25GB = 125,000 GB-s, so you stay well within free tier. After the free tier, the raw compute adds $2.08.)

Fargate cost (always-on, 0.25 vCPU / 0.5GB task):

vCPU cost:    0.25 vCPU × 720 hrs × $0.04048   = $7.29
Memory cost:  0.5 GB   × 720 hrs × $0.004445   = $1.60
                                           ──────────
Monthly Fargate cost:                          ~$8.89

Fargate cost (right-sized for 1M req/month at low concurrency):

At 1M requests/month and 500ms average, peak concurrency is about 6–8 requests/second. A single 0.5 vCPU / 1GB task handles this easily:

vCPU cost:    0.5 vCPU × 720 hrs × $0.04048   = $14.57
Memory cost:  1.0 GB   × 720 hrs × $0.004445  =  $3.20
                                          ──────────
Monthly Fargate cost:                          ~$17.77

Cost Comparison Table

Scenario	Lambda	Fargate (0.25 vCPU/0.5GB)	Fargate (0.5 vCPU/1GB)
1M req/month, 500ms, 256MB	~$2.28	~$8.89	~$17.77
10M req/month, 500ms, 256MB	~$22.80	~$8.89	~$17.77
100M req/month, 500ms, 256MB	~$228.00	~$8.89 (saturated)	~$17.77 (1 task)
100M req/month, needs 5 tasks	~$228.00	~$44.45	~$88.85

The crossover point: For this workload (500ms, 256MB), Lambda becomes more expensive than a single Fargate task somewhere around 5–8M requests/month. Above that volume with sustained traffic, containers win on cost. Below that volume, Lambda wins — especially at zero or near-zero traffic where Fargate still charges for idle time. In practice, I use this math as the first filter: if a service is projected to stay under 5M requests/month for the next 12 months, I start with Lambda and migrate if I'm wrong — migration is far cheaper than over-engineering the initial deployment.

Scaling Behavior

Lambda scaling is automatic and fast. When traffic spikes, Lambda creates new execution environments in parallel — up to 1,000 concurrent invocations by default (soft limit, increases on request). There is no autoscaling policy to write, no target tracking rule, no warmup period.

Containers require deliberate autoscaling configuration:

# ECS Service Auto Scaling — the config Lambda does for you automatically
ScalingTarget:
  MinCapacity: 1
  MaxCapacity: 50
  ResourceId: service/my-cluster/my-service

ScalingPolicy:
  PolicyType: TargetTrackingScaling
  TargetTrackingScalingPolicyConfiguration:
    TargetValue: 70.0  # CPU utilization %
    ScaleInCooldown: 300
    ScaleOutCooldown: 60
    PredefinedMetricSpecification:
      PredefinedMetricType: ECSServiceAverageCPUUtilization

For burst workloads — a campaign launch, a batch trigger, a viral event — Lambda handles the spike without pre-warming or capacity planning. Containers need headroom (over-provisioned min capacity) or they will queue and drop requests during the scaling warmup period. I've watched ECS services get overwhelmed during a flash sale because the autoscaling cooldown period was 60 seconds and requests were piling up in the ALB target group — Lambda would have absorbed that spike silently.

For steady-state, predictable traffic, containers are more efficient. A constant 500 requests/second fits cleanly into a fixed number of container tasks running at healthy utilization. Lambda at that volume means 500 concurrent executions burning compute time with per-invocation overhead.

State Management

Lambda execution environments are ephemeral. Each invocation may land on a different environment. Files written to /tmp persist within the same environment but disappear when the environment is recycled. In-memory data (connection pools, local caches) is environment-local, not global — and environments are recycled without notice.

This means Lambda cannot hold:

WebSocket connections (long-lived, bidirectional)
In-memory application state shared across requests
Database connection pools (every cold start opens a new connection — this exhausts RDS connection limits fast)

Containers hold state naturally. A running container process accumulates in-memory cache, holds open database connections across requests, and maintains WebSocket sessions indefinitely.

WebSocket: Impossible on Lambda, Trivial on Containers

// This CANNOT run on Lambda — Lambda terminates after response
// WebSocket server requires a persistent process

import { WebSocketServer } from 'ws';

const wss = new WebSocketServer({ port: 8080 });
const rooms = new Map<string, Set<WebSocket>>();

wss.on('connection', (ws, req) => {
  const room = new URL(req.url!, `http://localhost`).searchParams.get('room') ?? 'default';

  if (!rooms.has(room)) rooms.set(room, new Set());
  rooms.get(room)!.add(ws);

  ws.on('message', (data) => {
    // Broadcast to all clients in room — in-memory state
    for (const client of rooms.get(room)!) {
      if (client.readyState === WebSocket.OPEN) {
        client.send(data.toString());
      }
    }
  });

  ws.on('close', () => {
    rooms.get(room)?.delete(ws);
  });
});

For Lambda, external services carry the state:

S3 for files and large objects
DynamoDB for persistent key-value state
ElastiCache (Redis) for in-memory caching and session state
RDS Proxy to pool database connections at the infrastructure layer (bypasses the Lambda connection exhaustion problem)

Code Examples

TypeScript Lambda Handler

import { APIGatewayProxyEvent, APIGatewayProxyResult, Context } from 'aws-lambda';
import { DynamoDBClient, GetItemCommand } from '@aws-sdk/client-dynamodb';
import { marshall, unmarshall } from '@aws-sdk/util-dynamodb';

// Initialize outside handler — reused across warm invocations
const ddb = new DynamoDBClient({ region: process.env.AWS_REGION });
const TABLE_NAME = process.env.TABLE_NAME!;

export async function handler(
  event: APIGatewayProxyEvent,
  context: Context
): Promise<APIGatewayProxyResult> {
  const userId = event.pathParameters?.userId;

  if (!userId) {
    return {
      statusCode: 400,
      headers: { 'Content-Type': 'application/json' },
      body: JSON.stringify({ error: 'userId path parameter required' }),
    };
  }

  try {
    const result = await ddb.send(new GetItemCommand({
      TableName: TABLE_NAME,
      Key: marshall({ userId }),
    }));

    if (!result.Item) {
      return {
        statusCode: 404,
        headers: { 'Content-Type': 'application/json' },
        body: JSON.stringify({ error: 'User not found' }),
      };
    }

    return {
      statusCode: 200,
      headers: { 'Content-Type': 'application/json' },
      body: JSON.stringify(unmarshall(result.Item)),
    };
  } catch (err) {
    console.error('DynamoDB error:', err);
    return {
      statusCode: 500,
      headers: { 'Content-Type': 'application/json' },
      body: JSON.stringify({ error: 'Internal server error' }),
    };
  }
}

Multi-Stage Dockerfile (TypeScript/Node.js)

# Stage 1: Build
FROM node:20-alpine AS builder

WORKDIR /app

COPY package*.json tsconfig.json ./
RUN npm ci --only=production=false

COPY src/ ./src/
RUN npm run build

# Stage 2: Prune dev dependencies
FROM node:20-alpine AS deps

WORKDIR /app
COPY package*.json ./
RUN npm ci --only=production

# Stage 3: Production image
FROM node:20-alpine AS runtime

# Non-root user for security
RUN addgroup -S appgroup && adduser -S appuser -G appgroup

WORKDIR /app

COPY --from=deps /app/node_modules ./node_modules
COPY --from=builder /app/dist ./dist
COPY package.json ./

USER appuser

EXPOSE 3000

HEALTHCHECK --interval=30s --timeout=5s --start-period=10s --retries=3 \
  CMD wget -qO- http://localhost:3000/health || exit 1

CMD ["node", "dist/server.js"]

SAM Template (Lambda Configuration)

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31

Globals:
  Function:
    Runtime: nodejs20.x
    Architectures: [arm64]
    Timeout: 30
    MemorySize: 256
    Environment:
      Variables:
        TABLE_NAME: !Ref UsersTable

Resources:
  GetUserFunction:
    Type: AWS::Serverless::Function
    Properties:
      Handler: dist/handler.handler
      CodeUri: ./
      Description: Get user by ID
      AutoPublishAlias: live
      DeploymentPreference:
        Type: Canary10Percent5Minutes
      ProvisionedConcurrencyConfig:
        ProvisionedConcurrentExecutions: 5  # Keep 5 warm — removes cold starts for baseline
      Policies:
        - DynamoDBReadPolicy:
            TableName: !Ref UsersTable
      Events:
        GetUser:
          Type: Api
          Properties:
            Path: /users/{userId}
            Method: GET

  UsersTable:
    Type: AWS::DynamoDB::Table
    Properties:
      BillingMode: PAY_PER_REQUEST
      AttributeDefinitions:
        - AttributeName: userId
          AttributeType: S
      KeySchema:
        - AttributeName: userId
          KeyType: HASH

Outputs:
  ApiEndpoint:
    Value: !Sub "https://${ServerlessRestApi}.execute-api.${AWS::Region}.amazonaws.com/Prod"

docker-compose.yml (Local Container Dev)

version: '3.9'

services:
  api:
    build:
      context: .
      target: runtime
    ports:
      - "3000:3000"
    environment:
      NODE_ENV: development
      DB_HOST: postgres
      DB_PORT: 5432
      DB_NAME: appdb
      DB_USER: appuser
      DB_PASS: devpassword
    depends_on:
      postgres:
        condition: service_healthy
    volumes:
      - ./src:/app/src:ro  # hot-reload with ts-node-dev in dev target
    restart: unless-stopped

  postgres:
    image: postgres:16-alpine
    environment:
      POSTGRES_DB: appdb
      POSTGRES_USER: appuser
      POSTGRES_PASSWORD: devpassword
    ports:
      - "5432:5432"
    volumes:
      - pg_data:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U appuser -d appdb"]
      interval: 10s
      timeout: 5s
      retries: 5

  redis:
    image: redis:7-alpine
    ports:
      - "6379:6379"
    command: redis-server --maxmemory 256mb --maxmemory-policy allkeys-lru

volumes:
  pg_data:

Deployment and Ops Complexity

Lambda deployment is intentionally minimal:

# Deploy a Lambda function with SAM
sam build && sam deploy --guided

# Update function code only (no infra change)
sam build && sam deploy

# Rollback a bad deploy
aws lambda update-alias \
  --function-name GetUserFunction \
  --name live \
  --function-version 7

Container deployment on ECS requires more moving parts:

# 1. Build and tag the image
docker build -t my-api:latest .

# 2. Authenticate to ECR
aws ecr get-login-password --region us-east-1 | \
  docker login --username AWS --password-stdin \
  123456789.dkr.ecr.us-east-1.amazonaws.com

# 3. Tag and push to ECR
docker tag my-api:latest \
  123456789.dkr.ecr.us-east-1.amazonaws.com/my-api:latest
docker push 123456789.dkr.ecr.us-east-1.amazonaws.com/my-api:latest

# 4. Update ECS service (triggers rolling deployment)
aws ecs update-service \
  --cluster my-cluster \
  --service my-api-service \
  --force-new-deployment

Beyond the CLI commands, ECS requires you to define and manage: Dockerfile, ECR repository, ECS cluster, task definition (CPU/memory, IAM role, port mappings, env vars, log configuration), ECS service, Application Load Balancer, target group, listener rules, security groups, and IAM roles for task execution. That is real infrastructure — IaC (CDK or Terraform) is non-negotiable at any serious scale.

Lambda abstracts all of that. SAM or CDK defines your function, IAM policy, and event source in under 50 lines of YAML. AWS manages everything else.

Common Mistakes

Putting long-running tasks in Lambda. Lambda's maximum execution time is 15 minutes. Anything that may run longer — large file processing, ML inference on big inputs, multi-step ETL — will time out. Use ECS Fargate tasks, AWS Batch, or Step Functions with activity workers instead.
Ignoring cold starts in latency SLOs. Teams set a p99 SLO of 500ms, deploy a Java Lambda, and then wonder why SLOs breach. Cold starts are not sampled out of p99 — they show up directly in your percentiles, especially at low request volumes where cold starts are frequent. Benchmark cold start times for your runtime before committing to a latency SLO.
Using containers for simple CRUD APIs. A three-endpoint user management API does not need an ECS cluster, an ALB, a task definition, and an autoscaling policy. Lambda + API Gateway deploys in minutes and costs under $5/month at typical startup traffic levels. Container-first thinking applied to simple APIs is over-engineering that adds operational burden with no meaningful benefit.
Not setting Lambda concurrency limits. By default, Lambda scales to the regional account limit (1,000+ concurrent executions). Each execution opens a database connection. If your RDS instance supports 100 connections and a traffic spike triggers 300 concurrent Lambda invocations, you will exhaust the connection pool instantly. Always set ReservedConcurrentExecutions on functions that hit relational databases, or use RDS Proxy. I've seen this cause a 45-minute outage: an unthrottled Lambda function connected to a db.t3.medium RDS instance, a marketing email went out, connection count spiked to 300 in under 30 seconds, and every subsequent invocation got too many connections errors from PostgreSQL — all while Lambda reported zero errors in CloudWatch because the failures happened inside the handler, not at the invocation layer.
Mixing serverless and containers in the same data flow without budgeting latency. A common pattern: API Gateway → Lambda → internal ECS service → Lambda → SQS. Each hop adds network latency. Lambda cold starts in the middle of a synchronous chain compound. If your end-to-end latency budget is 200ms and you have three Lambda hops that each might cold-start at 300ms, your architecture is broken before a single line of business logic runs. Map your latency budget before your deployment topology. The fix I reach for is placing Lambda only at the edges of a data flow — ingestion and fan-out — and keeping synchronous internal calls inside the container tier where round-trip latency is consistent and predictable.

Decision Framework

When to Pick Lambda

Event-driven workloads — S3 triggers, SQS consumers, EventBridge rules, scheduled jobs (cron via EventBridge).
Low-to-moderate, spiky API traffic — less than 10M requests/month with unpredictable burst patterns.
Small teams or early-stage products — Lambda's operational simplicity lets a two-person team ship a production backend without a dedicated DevOps engineer.
Cost-sensitive low-traffic services — internal tooling, webhooks, notification processors that receive hundreds of requests per day. Lambda's free tier covers most of it.
Microservices with clear boundaries — stateless functions with one responsibility each fit Lambda's model perfectly.

When to Pick Containers (ECS Fargate)

WebSockets, gRPC streaming, long-poll APIs — anything requiring persistent connections.
Long-running compute — batch processing, ML inference pipelines, video encoding, multi-step ETL.
High-volume, steady-state APIs — above ~10M requests/month at typical compute sizes, containers undercut Lambda on price.
In-memory state or connection pooling — applications that benefit from holding cached data, open DB connections, or shared application state between requests.
Strict cold-start SLOs without the provisioned concurrency budget — if you need sub-100ms p99 without paying for always-warm Lambda instances.
Compliance or custom runtime requirements — need a specific OS, kernel version, custom binary, or GPU access that Lambda doesn't support.

When to Pick Kubernetes (EKS)

Multi-cloud or hybrid cloud requirements — Kubernetes is portable across AWS, GCP, Azure, and on-premises.
Large engineering organizations with a dedicated platform team — k8s overhead (cluster management, node upgrades, networking plugins) makes sense only when a dedicated team owns it.
Complex workload colocation — GPU nodes, spot instance pools, specialized node types all in one cluster.
Advanced deployment patterns — canary, blue/green, A/B across multiple services with Istio or Linkerd service mesh.

Decision Matrix

Workload	Lambda	ECS Fargate	Kubernetes
Burst / spiky traffic	✅ Best	⚠️ Needs autoscaling config	⚠️ Needs headroom
WebSockets / long connections	❌ Impossible	✅ Native	✅ Native
Scheduled jobs (cron)	✅ EventBridge	⚠️ Scheduled tasks	⚠️ CronJob resource
ML inference (<15 min)	✅ With container image	✅	✅
APIs with p99 <100ms	⚠️ Provisioned concurrency needed	✅ Warm always	✅ Warm always
Cost under low traffic	✅ Best (pay per use)	⚠️ Idle cost	❌ High idle cost
Cost under high traffic	⚠️ Linear cost growth	✅ Predictable	✅ With spot/reserved
Team with ops experience	✅	✅	⚠️ High complexity
Compliance / VPC isolation	✅ VPC Lambda (+ latency)	✅ Native VPC	✅ Native VPC

Conclusion

Lambda and containers are not competing standards — they are different answers to different questions. The mistake I see most often is architects treating this as an identity choice ("we're a serverless team") rather than a workload-by-workload engineering decision. Run the five questions in the decision framework above against each component independently. Most systems end up with both — Lambda at the edges for event-driven ingestion and async processing, containers in the core for stateful services that need persistent connections and predictable latency. That hybrid approach is not indecision; it is the correct answer for most production architectures.

DEV Community