DEV Community: Muhammad Umair Virk

Why Serverless Engineers Already Understand Containers

Muhammad Umair Virk — Wed, 02 Sep 2026 15:48:36 +0000

The outage that teaches you deployment

A service passes every test locally. It fails in staging because the API calls localhost:5432 for Postgres — but Postgres is in another container, reachable only as db:5432.

This is not a Docker problem. It is a boundary problem: your code assumed an environment it does not own.

Engineers who have shipped on AWS Lambda already avoid a class of these mistakes. They never SSH into a function to hot-fix. They inject config at deploy time. They treat each invocation as disposable. Containers reward the same discipline with different vocabulary.

This article maps what transfers, what breaks, and what I require before any Python backend goes to production in a container.

What serverless already taught you

Immutable deployments

Lambda versions are replaced, not patched. Container images work the same way: build a new image, roll out, roll back by tag. If your incident runbook includes "edit files inside the running box," you have a design problem.

Configuration at runtime

Secrets belong in Secrets Manager or injected env vars — not in source control, not in the image layer cache. Docker does not change the rule; it changes where you mount the values.

Single responsibility per unit

One Lambda, one job. One container, one main process. Compose and Kubernetes add orchestration; they do not remove the rule.

Cold start awareness

Slim packages on Lambda map to slim base images (python:3.12-slim, multi-stage builds). Startup time affects autoscaling and health-check windows the same way cold starts affect user-facing latency.

If you understand why a Lambda deployment package should stay small, you understand why a 2 GB container image is a liability.

Where the mental model breaks

1. Network identity

Inside Compose or Kubernetes, localhost is the container itself. Services discover each other by DNS name (api, db, redis). This is the most common first-production failure I see in teams moving from bare metal or single-host deploys.

2. Port publishing

EXPOSE in a Dockerfile documents intent. It does not bind a host port. You still need explicit port mapping or an ingress controller. Confusing the two produces "the container is healthy but nothing reaches it from outside."

3. Image vs. container vs. process

The image is immutable. The container is a running instance. You can scale ten replicas from one image. Debugging "which instance handled this request?" requires logs with instance identity — hostname, pod name, task id.

4. Health checks are part of the contract

A process that listens but cannot serve traffic should fail the health check. Load balancers and orchestrators depend on this. HEALTHCHECK in Docker and /health endpoints in your app are not optional polish.

A production-shaped Dockerfile (Python / FastAPI)

FROM python:3.12-slim

WORKDIR /app

ENV PYTHONDONTWRITEBYTECODE=1 \
    PYTHONUNBUFFERED=1

RUN useradd --create-home appuser
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY app ./app

USER appuser
EXPOSE 8000

HEALTHCHECK --interval=30s --timeout=5s --start-period=10s --retries=3 \
  CMD python -c "import urllib.request; urllib.request.urlopen('http://127.0.0.1:8000/health')"

CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000"]

Non-root user, slim base, explicit health check, no secrets in layers. This is the minimum bar I review for in PRs.

Serverless vs. containers — a decision lens

Constraint	Lean serverless	Lean containers
Traffic shape	Spiky, unpredictable	Steady or long-running workers
Unit of work	Short, event-driven	Persistent connections, WebSockets, background workers
Ops surface	Minimize	Team can own runtime and patching
Dependency weight	Small handler bundles	Custom system libs, ML models, legacy binaries

Most mature platforms use both. The engineer who can reason about trade-offs — not just "we use Kubernetes" — is the one I want reviewing architecture.

What I look for in a backend engineer's deploy story

Can they explain why the last outage happened at the infrastructure boundary, not in a line of Python? Can they roll back without data loss? Do they know where config lives and who can read it?

Containers are not a separate career. They are the next layer of the same responsibility: ship code that survives real environments.

REST vs SOAP vs GraphQL vs gRPC — Interview Notes for Backend Engineers

Muhammad Umair Virk — Sat, 29 Aug 2026 17:31:49 +0000

I have shipped REST APIs for four years (Flask, AWS Lambda + API Gateway, FastAPI). This is the reference I use in design reviews and phone screens — focused on when to pick each style, not memorising definitions.

Companion on Medium: REST vs SOAP vs GraphQL vs gRPC: How I Actually Choose — decision framework with trade-offs.

1. REST in 60 seconds

REST models resources (nouns) and uses HTTP methods (verbs).

GET    /users/7      read          safe + idempotent
POST   /users        create        not idempotent
PUT    /users/7      replace all   idempotent
PATCH  /users/7      change some   usually not idempotent
DELETE /users/7      remove        idempotent

Safe = does not change server data (GET).

Idempotent = 1 call or 10 identical calls leave the same state (GET, PUT, DELETE).

Status codes I treat as required:

Code	Meaning I say in interviews
200	Success
201	New resource created (typical POST)
400	Client sent a bad request (invalid JSON / missing field)
401	Unauthenticated — I do not know who you are
404	That URL or id does not exist
500	Server failed — do not "fix the JSON"

401 vs 403: 401 = who are you? 403 = I know you; you are not allowed.

What makes an API RESTful (5 bullets):

Resources in the URL, not action names
Uniform HTTP methods and status codes
Stateless requests (auth + ids on every call)
Representations (usually JSON)
Correct verb + code pairing (GET does not write; create → 201)

2. The comparison interviewers want

	REST	SOAP	GraphQL	gRPC
Style	Resources + HTTP	XML envelope + WSDL	Query language, one endpoint	RPC + Protobuf
Payload	JSON (usually)	XML	JSON	Binary protobuf
Contract	Informal / OpenAPI	Strict WSDL	Schema (SDL)	`.proto`
Typical transport	HTTP/1.1	HTTP POST	HTTP POST	HTTP/2
Browser-friendly	Yes	Painful	Yes	Not natively
Caching	Natural on GET	Weak	Harder	Weak
Best at	Public / mobile APIs	Enterprise partner contracts	Many clients, custom shapes	Internal service-to-service
Weak at	Chatty mobile graphs	New greenfield APIs	Simple CRUD, HTTP cache	Public `curl` + JSON

SOAP

SOAP is a protocol. You send XML. The operations live in a WSDL. Almost every call is POST. HTTP methods are not the API.

Use it when the other organisation already requires the contract (finance, government, older healthcare integrations). Do not start a new B2C API in SOAP in 2026 if you control both sides.

GraphQL

One endpoint. The client asks for exact fields. Solves over-fetching.

Watch for: cache keys, expensive queries, auth on a single URL that does many jobs.

gRPC

HTTP/2 + Protobuf. Typed generated clients. Streaming. Fast inside your own network.

Do not expose this as your public mobile API unless you have a gateway. Partners and browsers want JSON.

Also mentioned, not replacements

JSON-RPC — call a method name. Fine internally. Easy to invent RPC soup.
WebSockets — persistent two-way. Live updates, not CRUD.

3. Decision rules I actually use

Need a public or mobile API?                     → REST
Partner handed you a WSDL?                       → SOAP
Clients need different slices of the same graph? → GraphQL
Service-to-service, low latency, typed?          → gRPC
Need live updates after first load?              → WebSockets (+ REST for the first load)
Unsure?                                          → REST, then add a BFF later

Starting with the heaviest option is how designs stall. REST is reversible.

4. Tiny examples (same "create user" idea)

REST

POST /users HTTP/1.1
Content-Type: application/json

{"name":"Aisha","email":"aisha@example.com"}

201 Created + Location: /users/7

SOAP (shape only — do not memorise the XML)

<soap:Envelope>
  <soap:Body>
    <CreateUser>
      <name>Aisha</name>
      <email>aisha@example.com</email>
    </CreateUser>
  </soap:Body>
</soap:Envelope>

Posted to one /service URL. The operation is inside the body.

GraphQL

mutation {
  createUser(name: "Aisha", email: "aisha@example.com") {
    id
    email
  }
}

Usually POST /graphql. You choose the fields in the response.

gRPC (.proto)

service UserService {
  rpc CreateUser (CreateUserRequest) returns (User);
}

Binary on the wire. Generated stubs in Python, Go, Java.

5. What I say in 45 seconds

I default to REST: nouns in the URL, verbs in the method, JSON on the wire. GET is safe. PUT and DELETE are idempotent. POST is not — a double-tap can create two rows.

SOAP when the partner already has a WSDL.

GraphQL when over-fetching is a real product problem.

gRPC for internal service-to-service, not browsers.

The decision is the problem, not the acronym.

Idempotency Keys for Python Payment APIs

Muhammad Umair Virk — Sat, 29 Aug 2026 17:31:43 +0000

A user double-taps Pay. The network retries. Your API creates two orders for one charge.

POST is not idempotent. If you handle payments without an idempotency strategy, you will eventually ship a bug that finance notices before engineering does.

I've debugged checkout flows in production — post-payment 500s, guest checkout edge cases, gateway parity. Here are the patterns I use for idempotent payment endpoints in Python.

Companion on Medium: What Production Checkout Taught Me About Idempotency — reconciliation, webhooks, and state machines.

1. The problem in one request

POST /orders HTTP/1.1
Content-Type: application/json

{"amount": 9900, "currency": "AED", "payment_method_id": "pm_abc"}

The client sends this. The server charges the card and creates an order. The response never reaches the app. The client retries the same POST.

Without protection: two orders, or one order and one orphan charge.

2. Idempotency keys — the standard fix

The client generates a unique key once per user action and sends it on every retry:

POST /orders HTTP/1.1
Content-Type: application/json
Idempotency-Key: 7c9e6679-7425-40de-944b-e07fc1f90ae7

{"amount": 9900, "currency": "AED", "payment_method_id": "pm_abc"}

Server rules:

First request with key K → process normally, store (K → response)
Duplicate request with key K → return stored response, do not re-charge

Stripe popularized this header. You can implement the same pattern on any checkout API.

3. Database schema

CREATE TABLE idempotency_keys (
    key         VARCHAR(64) PRIMARY KEY,
    request_hash TEXT NOT NULL,
    response_body JSONB NOT NULL,
    status_code  INT NOT NULL,
    created_at   TIMESTAMPTZ DEFAULT NOW()
);

-- Optional: tie to business entity
CREATE UNIQUE INDEX orders_idempotency_key_idx ON orders (idempotency_key);

Store the key before calling the payment gateway when possible (with status processing) to win race conditions between concurrent retries.

4. FastAPI implementation

from fastapi import FastAPI, Header, HTTPException, Request
from sqlalchemy.orm import Session
import hashlib
import json

app = FastAPI()

def request_hash(body: dict) -> str:
    return hashlib.sha256(json.dumps(body, sort_keys=True).encode()).hexdigest()


@app.post("/orders")
async def create_order(
    body: dict,
    db: Session = Depends(get_db),
    idempotency_key: str | None = Header(default=None, alias="Idempotency-Key"),
):
    if not idempotency_key:
        raise HTTPException(400, "Idempotency-Key header required")

    existing = db.query(IdempotencyRecord).filter_by(key=idempotency_key).first()
    if existing:
        if existing.request_hash != request_hash(body):
            raise HTTPException(422, "Idempotency key reused with different body")
        return JSONResponse(existing.response_body, status_code=existing.status_code)

    # Reserve the key (status: processing) — prevents duplicate charges under concurrency
    record = IdempotencyRecord(key=idempotency_key, request_hash=request_hash(body), status="processing")
    db.add(record)
    db.commit()

    try:
        charge = payment_gateway.charge(body)
        order = Order(amount=body["amount"], charge_id=charge.id, idempotency_key=idempotency_key)
        db.add(order)
        response = {"order_id": order.id, "status": "paid"}
        record.response_body = response
        record.status_code = 201
        record.status = "completed"
        db.commit()
        return JSONResponse(response, status_code=201)
    except Exception:
        db.rollback()
        raise

Key details:

Reject keys reused with a different body (422)
Store response before returning to client
Use a DB transaction around order + idempotency record

5. Flask — lighter version

from flask import Flask, request, jsonify
from functools import wraps

app = Flask(__name__)

# Production: Redis or Postgres, not in-memory
_idempotency_store = {}


@app.post("/orders")
def create_order():
    key = request.headers.get("Idempotency-Key")
    if not key:
        return jsonify({"error": "Idempotency-Key required"}), 400

    body = request.get_json()
    if key in _idempotency_store:
        stored = _idempotency_store[key]
        if stored["body_hash"] != hash(body):
            return jsonify({"error": "Key reused with different payload"}), 422
        return jsonify(stored["response"]), stored["status"]

    charge = charge_customer(body)
    order_id = save_order(body, charge.id)
    response = {"order_id": order_id, "status": "paid"}
    _idempotency_store[key] = {"body_hash": hash(body), "response": response, "status": 201}
    return jsonify(response), 201

Use Redis with TTL (24–72 hours) or Postgres in production — not a module-level dict.

6. Gateway-level idempotency

Most gateways accept their own idempotency key:

# Stripe example
stripe.PaymentIntent.create(
    amount=9900,
    currency="aed",
    idempotency_key=idempotency_key,  # same key as your API
)

Defense in depth: idempotency at your API layer and at the gateway. Your order service should not depend on the gateway alone — webhooks and retries can still duplicate rows.

7. Status reconciliation

Sometimes the charge succeeds but your API crashes before saving the order. Run a reconciliation job:

def reconcile_orphan_charges():
    pending = db.query(Order).filter_by(status="payment_pending").all()
    for order in pending:
        gateway_status = payment_gateway.get_charge(order.charge_id)
        if gateway_status == "succeeded":
            order.status = "paid"
        elif gateway_status == "failed":
            order.status = "failed"
    db.commit()

Webhooks (payment_intent.succeeded) are the other half — never rely only on the synchronous response.

8. Guest checkout edge case

Guest orders often have user_id: null in JSON. Some serializers treat missing field and null differently. Downstream services that expect "no user" as absent field break when they receive explicit null.

Normalize at the API boundary:

def normalize_guest(body: dict) -> dict:
    if body.get("user_id") is None:
        body.pop("user_id", None)  # or store as NULL consistently — pick one
    return body

Parity bugs here cause post-payment 500s that look like payment failures but are mapping issues.

Checklist

[ ] Idempotency-Key required on POST /orders and POST /payments
[ ] Store key → response in DB or Redis with TTL
[ ] Reject same key + different body (422)
[ ] Pass idempotency key through to payment gateway
[ ] Webhook handler updates order status
[ ] Reconciliation job for stuck payment_pending orders
[ ] Unique constraint on orders.idempotency_key

Docker Compose for Python Backends — A Minimal Production Stack

Muhammad Umair Virk — Sat, 29 Aug 2026 17:29:34 +0000

Most Python backends eventually leave "run on my laptop." The first container deploy is where teams discover that localhost is not a universal address and that a Dockerfile without a health check is a black box to the load balancer.

This is the minimal stack I use and review in PRs: FastAPI, a hardened Dockerfile, Compose for local parity with production topology.

Companion article on Medium: Why Serverless Engineers Already Understand Containers — same patterns, more on mental models.

1. Dockerfile essentials

FROM python:3.12-slim

WORKDIR /app

ENV PYTHONDONTWRITEBYTECODE=1 \
    PYTHONUNBUFFERED=1

RUN useradd --create-home appuser
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY app ./app

USER appuser
EXPOSE 8000

HEALTHCHECK --interval=30s --timeout=5s --start-period=10s --retries=3 \
  CMD python -c "import urllib.request; urllib.request.urlopen('http://127.0.0.1:8000/health')"

CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000"]

Review checklist:

[ ] Non-root user — container escape should not mean root on the host
[ ] Slim base — smaller attack surface and faster pulls
[ ] HEALTHCHECK hits a real endpoint that verifies dependencies if possible
[ ] No secrets in layers — use env injection at runtime

2. docker-compose.yml

services:
  api:
    build: .
    ports:
      - "8000:8000"
    environment:
      ENVIRONMENT: local-compose

docker compose up --build
curl http://localhost:8000/health

If /health returns a hostname that changes between restarts, you are talking to the container — not a stale local process.

3. The rule that prevents the most outages

Inside Compose, localhost is the container itself.

When you add Postgres:

services:
  api:
    build: .
    environment:
      DATABASE_URL: postgresql://app:secret@db:5432/appdb
    depends_on:
      - db
  db:
    image: postgres:16-alpine
    environment:
      POSTGRES_USER: app
      POSTGRES_PASSWORD: secret
      POSTGRES_DB: appdb

The API must use host db, not 127.0.0.1. This single mistake causes more first-week container incidents than any Dockerfile syntax error.

4. Health endpoint

import os
import socket
from fastapi import APIRouter

router = APIRouter()

@router.get("/health")
def health():
    return {
        "status": "healthy",
        "hostname": socket.gethostname(),
        "environment": os.getenv("ENVIRONMENT", "development"),
    }

Orchestrators, load balancers, and Compose use this to decide whether to route traffic. A listening socket that cannot serve requests should fail the check.

5. Local vs. production gaps

Concern	Local Compose	Production
Secrets	`.env` (gitignored)	Secrets Manager / platform env
Database	Named volume	Managed RDS + migrations
Logs	stdout	Structured JSON → aggregation
Deploy	Manual	CI pipeline → registry → roll out

Closing these gaps deliberately — not on the day of launch — is what separates a demo from a service.

AWS Secrets Manager Patterns for Python Microservices

Muhammad Umair Virk — Sat, 29 Aug 2026 17:29:33 +0000

Hardcoding secrets is the fastest way to get paged at 3 AM — or worse, flagged in a security audit.

Centralized secret management is not optional once you run more than one microservice. I have implemented shared retrieval layers in Python (Flask, FastAPI, Lambda) using AWS Secrets Manager. These are the patterns that survive production — caching, least privilege, rotation, and what breaks when you skip any of them.

Pattern 1: Never bake secrets into images

# ❌ NEVER
DATABASE_URL = "postgresql://user:password123@host/db"

# ✅ Runtime injection
import os
DATABASE_URL = os.environ["DATABASE_URL"]

In Docker, use .env files locally and AWS-injected env vars in production. Add .env to .dockerignore and .gitignore.

Pattern 2: Centralized retrieval helper

import boto3
import json
from functools import lru_cache

client = boto3.client("secretsmanager", region_name="me-south-1")


@lru_cache(maxsize=32)
def get_secret(secret_name: str) -> dict:
    response = client.get_secret_value(SecretId=secret_name)
    return json.loads(response["SecretString"])


def get_database_url() -> str:
    secrets = get_secret("prod/taskflow/database")
    return (
        f"postgresql://{secrets['username']}:{secrets['password']}"
        f"@{secrets['host']}:{secrets['port']}/{secrets['dbname']}"
    )

Why @lru_cache? Secrets Manager charges per API call. Cache in-process for Lambda warm instances and long-running containers. Invalidate on rotation events if needed.

Pattern 3: IAM least privilege

Each service gets its own IAM role with access to only its secrets:

{
  "Effect": "Allow",
  "Action": "secretsmanager:GetSecretValue",
  "Resource": "arn:aws:secretsmanager:me-south-1:123456789:secret:prod/taskflow/*"
}

Never use one mega-role for all secrets across all services.

Pattern 4: Lambda vs. long-running services

Runtime	Strategy
Lambda	Fetch on cold start, cache in module scope
ECS / Docker	Fetch on startup, cache in app singleton
Local dev	`.env` file or LocalStack

# Lambda — module-level cache survives warm invocations
_db_url = None

def get_db_url():
    global _db_url
    if _db_url is None:
        _db_url = get_database_url()
    return _db_url

Pattern 5: Secret rotation

Enable automatic rotation in Secrets Manager for database credentials. Your app must handle connection drops gracefully:

from sqlalchemy import create_engine
from sqlalchemy.pool import NullPool

engine = create_engine(get_database_url(), pool_pre_ping=True)

pool_pre_ping=True verifies connections before use — critical after rotation.

Pattern 6: Structured logging — never log secrets

import logging
logger = logging.getLogger(__name__)

# ❌ logger.info(f"Connecting to {DATABASE_URL}")
# ✅
logger.info("database_connection_initialized", extra={"host": secrets["host"]})

Redact secrets in error handlers and APM tools.

Pattern 7: Compose locally, Secrets Manager in prod

# docker-compose.yml (local)
services:
  api:
    env_file:
      - .env

# ECS task definition (prod)
secrets:
  - name: DATABASE_URL
    valueFrom: arn:aws:secretsmanager:...:secret:prod/taskflow/database-url

Same app code. Different secret source per environment.

What I built

At work, I implemented a shared secret retrieval layer used across Python and Go services — standardizing how services access database credentials, API keys, and third-party tokens. Result: zero hardcoded secrets in repos, audit-friendly access logs, and rotation without redeploying application code.

Checklist for your next project

[ ] All secrets in Secrets Manager (or Parameter Store for non-sensitive config)
[ ] .env in .gitignore and .dockerignore
[ ] IAM role per service, least privilege
[ ] In-process caching with rotation awareness
[ ] pool_pre_ping on database connections
[ ] No secrets in logs, CI output, or error messages

DEV Community: Muhammad Umair Virk

Why Serverless Engineers Already Understand Containers

The outage that teaches you deployment

What serverless already taught you

Where the mental model breaks

A production-shaped Dockerfile (Python / FastAPI)

Serverless vs. containers — a decision lens

What I look for in a backend engineer's deploy story

Further reading

REST vs SOAP vs GraphQL vs gRPC — Interview Notes for Backend Engineers

1. REST in 60 seconds

2. The comparison interviewers want

SOAP

GraphQL

gRPC

Also mentioned, not replacements

3. Decision rules I actually use

4. Tiny examples (same "create user" idea)

5. What I say in 45 seconds

Further reading

Idempotency Keys for Python Payment APIs

1. The problem in one request

2. Idempotency keys — the standard fix

3. Database schema

4. FastAPI implementation

5. Flask — lighter version

6. Gateway-level idempotency

7. Status reconciliation

8. Guest checkout edge case

Checklist

Further reading

Docker Compose for Python Backends — A Minimal Production Stack

1. Dockerfile essentials

2. docker-compose.yml

3. The rule that prevents the most outages

4. Health endpoint

5. Local vs. production gaps

Further reading

AWS Secrets Manager Patterns for Python Microservices

Pattern 1: Never bake secrets into images

Pattern 2: Centralized retrieval helper

Pattern 3: IAM least privilege

Pattern 4: Lambda vs. long-running services

Pattern 5: Secret rotation

Pattern 6: Structured logging — never log secrets

Pattern 7: Compose locally, Secrets Manager in prod

What I built

Checklist for your next project

Further reading