DEV Community

Python-T Point
Python-T Point

Posted on • Originally published at pythontpoint.in

⚙️ Setting up Kubernetes HPA for a FastAPI application made easy

#devops #kubernetes #cloud #tutorial

💡 Overview — Why Scaling Matters

kubernetes hpa for fastapi application

To configure kubernetes hpa for fastapi application , define a HorizontalPodAutoscaler resource that watches CPU utilization and automatically adjusts the replica count of the associated Deployment.

📑 Table of Contents

  • 💡 Overview — Why Scaling Matters
  • 🐳 Containerizing FastAPI — How Images Work
  • 📦 Deploying FastAPI — Using a Deployment
  • 📈 Setting up HPA — Configuring Autoscaling
  • 🔧 Choose Metrics — Which Signal to Use
  • ⚙️ Tuning Parameters — Thresholds and Limits
  • 🛠 Monitoring & Testing — Verifying Behavior
  • 🟩 Final Thoughts
  • ❓ Frequently Asked Questions
  • Can I use memory instead of CPU for the HPA target?
  • What happens if the HPA reaches the maxReplicas limit?
  • Do I need to install any additional components to use HPA?
  • 📚 References & Further Reading

🐳 Containerizing FastAPI — How Images Work

A Docker image bundles the FastAPI code, its dependencies, and the runtime environment into a single artifact that Kubernetes can schedule on any node.

# Dockerfile
FROM python:3.11-slim # Install system dependencies needed for uvicorn and fastapi
RUN apt-get update && apt-get install -y -no-install-recommends \ build-essential && rm -rf /var/lib/apt/lists/* # Create a non‑root user
RUN useradd -create-home appuser
WORKDIR /app
COPY requirements.txt .
RUN pip install -no-cache-dir -r requirements.txt # Copy application source
COPY . . # Switch to non‑root user
USER appuser # Expose the default port
EXPOSE 8000 # Run the FastAPI app with uvicorn
CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000"]
Enter fullscreen mode Exit fullscreen mode

What this does:

  • FROM: selects a minimal Python base image.
  • RUN apt-get: installs build tools required for compiled dependencies.
  • COPY requirements.txt & pip install: creates a reproducible layer for Python packages.
  • COPY .: adds the FastAPI source code.
  • USER appuser: runs the container without root privileges, improving security.
  • CMD: starts uvicorn, the ASGI server that serves the FastAPI app.

Why this, not a plain python app.py command? Using uvicorn provides native async handling and HTTP/2 support, which a simple script lacks; the container also isolates dependencies, making deployments repeatable.

Key point: A well‑crafted image ensures that autoscaling affects only the application code, not the build environment.

📦 Deploying FastAPI — Using a Deployment

A Deployment object manages a set of identical pods and provides declarative updates, which is essential for reliable scaling.

# deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata: name: fastapi labels: app: fastapi
spec: replicas: 2 selector: matchLabels: app: fastapi template: metadata: labels: app: fastapi spec: containers: - name: fastapi image: yourregistry/fastapi:latest ports: - containerPort: 8000 resources: requests: cpu: "250m" memory: "128Mi" limits: cpu: "500m" memory: "256Mi" readinessProbe: httpGet: path: /health port: 8000 initialDelaySeconds: 5 periodSeconds: 10
Enter fullscreen mode Exit fullscreen mode

What this does:

  • replicas: initial pod count; HPA will modify this field.
  • selector & template.labels: tie the Deployment to its pods.
  • resources.requests: baseline CPU the scheduler uses for placement.
  • resources.limits: maximum CPU a pod may consume; the HPA observes usage against this ceiling.
  • readinessProbe: ensures traffic is only sent to pods that have passed the health check.

Why this, not a bare Pod? A Deployment adds a control loop that automatically recreates failed pods and permits rolling updates; a plain Pod would terminate on failure and require manual recreation.

“HorizontalPodAutoscaler reacts to real‑time metrics, turning a static replica count into a responsive control system.”

Key point: The Deployment provides the stable target that the HPA will scale up or down. (Also read: ⚙️ Docker Swarm vs Kubernetes for small teams — which one should you use?)

📈 Setting up HPA — Configuring Autoscaling

This section creates a HorizontalPodAutoscaler that watches CPU utilization and adjusts the FastAPI Deployment accordingly.

# hpa.yaml
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata: name: fastapi-hpa
spec: scaleTargetRef: apiVersion: apps/v1 kind: Deployment name: fastapi minReplicas: 2 maxReplicas: 10 metrics: - type: Resource resource: name: cpu target: type: Utilization averageUtilization: 60
Enter fullscreen mode Exit fullscreen mode

What this does:

  • scaleTargetRef: points the HPA at the Deployment defined earlier.
  • minReplicas / maxReplicas: bounds the scaling range.
  • metrics.type: Resource: tells the controller to use CPU usage.
  • averageUtilization: 60: the target CPU percentage; the HPA adds or removes pods to keep average usage near 60 %.

Why this, not a custom metric? CPU is a built‑in metric collected by the kube‑metrics‑server, requiring no extra instrumentation; custom metrics introduce additional components and operational overhead.

🔧 Choose Metrics — Which Signal to Use

CPU utilization is the default metric because the metrics server is always present in a standard cluster. Memory utilization or custom Prometheus metrics can be added later if the workload is memory‑intensive. (Also read: 🐍 Mastering Kubernetes interview questions on Python async)

⚙️ Tuning Parameters — Thresholds and Limits

Setting averageUtilization to 60 % balances latency and cost. A lower threshold triggers more pods, reducing response time but increasing resource consumption. A higher threshold risks saturation and higher error rates.

According to the Kubernetes documentation, the HPA controller evaluates metrics every 15 seconds and applies scaling decisions after a stabilization window, preventing rapid flapping. (Also read: 🐍 Configure Kubernetes liveness and readiness probes in Python made easy) (More onPythonTPoint tutorials)

Aspect Manual Scaling HPA Scaling
Responsiveness Hours (operator‑initiated) Seconds to minutes (controller‑driven)
Resource Utilization Often over‑provisioned Matches load dynamically
Complexity Simple CLI commands Requires metrics server and YAML

Key point: The HPA turns a static replica count into a feedback‑controlled system that reacts to real‑time load.

🛠 Monitoring & Testing — Verifying Behavior

After applying the resources, confirm that the HPA reacts as expected.

$ kubectl apply -f deployment.yaml
deployment.apps/fastapi created



$ kubectl apply -f hpa.yaml
horizontalpodautoscaler.autoscaling/fastapi-hpa created



$ kubectl get hpa fastapi-hpa
NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE
fastapi-hpa Deployment/fastapi 0%/60% 2 10 2 30s
Enter fullscreen mode Exit fullscreen mode

Initially the CPU usage is 0 % because the service is idle. Generate load with a simple HTTP benchmark tool.

$ hey -n 5000 -c 50 http://localhost:8000/items/1
Summary: Total: 12.3456 secs Slowest: 0.789 secs Fastest: 0.001 secs Average: 0.025 secs Requests/sec: 405.00



# Expected output from kubectl after load
NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE
fastapi-hpa Deployment/fastapi 78%/60% 2 10 4 1m
Enter fullscreen mode Exit fullscreen mode

The HPA observed an average CPU utilization of 78 % and increased the replica count from 2 to 4 to bring usage back toward the 60 % target. Verify the new pods are running:

$ kubectl get pods -l app=fastapi
NAME READY STATUS RESTARTS AGE
fastapi-7d9c5b8f5b-2k9f9 1/1 Running 0 2m
fastapi-7d9c5b8f5b-2k9f9 1/1 Running 0 2m
fastapi-7d9c5b8f5b-9l1c2 1/1 Running 0 30s
fastapi-7d9c5b8f5b-9l1c2 1/1 Running 0 30s
Enter fullscreen mode Exit fullscreen mode

Observe the metrics continuously to ensure the HPA stabilizes:

$ kubectl top pods -l app=fastapi
NAME CPU(cores) MEMORY(bytes)
fastapi-7d9c5b8f5b-2k9f9 120m 80Mi
fastapi-7d9c5b8f5b-9l1c2 115m 78Mi
Enter fullscreen mode Exit fullscreen mode

When CPU drops below the target, the HPA will scale down after the stabilization window, confirming the feedback loop works.

🟩 Final Thoughts

Implementing kubernetes hpa for fastapi application turns a static deployment into a self‑adjusting service that matches traffic patterns without manual intervention. The key steps—containerizing the app, defining a Deployment, creating an HPA, and validating the controller—are reusable across any stateless microservice.

For developers, the practical benefit is reduced operational overhead: you no longer need to predict peak load or manually resize pods. The cluster continuously optimizes resource usage, keeping latency low while preventing wasteful over‑provisioning.

❓ Frequently Asked Questions

Can I use memory instead of CPU for the HPA target?

Yes. Replace the metric definition with name: memory and set an appropriate averageUtilization percentage. Memory metrics are also provided by the metrics server.

What happens if the HPA reaches the maxReplicas limit?

The controller stops scaling out and maintains the highest allowed replica count. At that point consider increasing maxReplicas or optimizing the application code.

Do I need to install any additional components to use HPA?

The default Kubernetes installation includes the metrics server, which supplies CPU and memory usage. If you want custom metrics (e.g., request latency), you must deploy a Prometheus Adapter or a comparable exporter.

💡 Want to practise this hands-on? DigitalOcean gives new accounts $200 free credit for 60 days — enough to spin up a full Linux/Docker/Kubernetes environment at no cost.

📚 Recommended reading: Best DevOps & cloud books on Amazon — from Linux fundamentals to Kubernetes in production, curated for working engineers.

📚 References & Further Reading

  • Official FastAPI docs — comprehensive guide to building async APIs: fastapi.tiangolo.com
  • Kubernetes Horizontal Pod Autoscaler documentation — authoritative source on HPA behavior and configuration: kubernetes.io
  • Docker best practices for Python applications — recommended image construction patterns: docs.docker.com

Top comments (0)