How modern cloud native platforms dynamically adjust to demand — and how serverless takes that abstraction even further.
Based on cloud native curriculum from the @Linux Foundation | CNCF
What Is Autoscaling?
Autoscaling is the dynamic adjustment of compute resources in response to current demand. Rather than statically provisioning infrastructure for peak load — and paying for it even during quiet periods — autoscaling ensures your system expands when traffic rises and contracts when it falls.
The primary metrics that drive autoscaling decisions are CPU utilization and memory consumption, though modern systems can scale on virtually any observable signal: queue depth, request latency, custom application metrics, and even external event sources.
There are two fundamental approaches to scaling: horizontal and vertical.
Horizontal Scaling (Scale Out)
Horizontal scaling — also called scaling out — describes the process of spawning new compute resources to distribute load. In practice, this means launching new copies of your application: additional pods in Kubernetes, new virtual machine instances, or additional container replicas. When traffic drops, those extra instances are terminated.
This is the approach most cloud native systems favour. It's highly flexible, works well with stateless services, and doesn't require downtime to apply. It also aligns naturally with microservices architecture, where each service can scale independently based on its own load.
Vertical Scaling (Scale Up)
Vertical scaling — scaling up — changes the size of the underlying hardware or container resource allocation. Instead of adding more instances, you give each existing instance more CPU cores or memory.
Vertical scaling works within certain physical limits. For bare-metal servers, you're constrained by the maximum hardware specs of the machine. For virtual machines and containers, the ceiling is typically the size of the largest available instance type or node. Once you hit that ceiling, vertical scaling alone isn't enough.
Key distinction: Horizontal scaling adds capacity by adding instances; vertical scaling adds capacity by enlarging existing ones. In most cloud native workloads, horizontal scaling is preferred for flexibility and resilience, with vertical scaling used to right-size individual workloads.
Kubernetes Autoscaling: HPA, VPA, and KEDA
In Kubernetes environments, autoscaling operates at multiple levels simultaneously.
The Horizontal Pod Autoscaler (HPA) is the most commonly used mechanism. It continuously monitors metrics like CPU and memory through the Kubernetes Metrics Server and adjusts the number of pod replicas accordingly. HPA runs its reconciliation loop every 15 seconds, using the formula: desiredReplicas = ceil(currentReplicas × currentMetric / targetMetric). You configure it by specifying a target utilization level and a minimum and maximum replica count.
The Vertical Pod Autoscaler (VPA) takes the opposite approach — rather than adding more pods, it analyses the resource usage of existing containers and adjusts their CPU and memory requests and limits. Where HPA assumes your pods are correctly sized and adds more of them, VPA assumes you have the right number of pods and focuses on getting their individual resource allocation right. One important caveat: running HPA and VPA together on CPU and memory can cause conflicts, since both controllers compete over the same resources. The recommended pattern is to use KEDA on custom metrics if you need both.
KEDA (Kubernetes Event-Driven Autoscaler) is a CNCF graduated project that extends HPA by enabling scaling based on external event sources — not just CPU percentages. Need to scale workers based on Kafka queue depth? The number of messages in a Redis list? A custom Prometheus metric? KEDA handles all of these. Under the hood, KEDA dynamically generates an HPA resource and feeds it custom metrics from external event sources — it enhances Kubernetes rather than replacing existing components.
Used together, these tools can reduce costs by 30–50% compared to statically provisioned clusters while maintaining high service-level objectives.
Configuring Autoscaling Correctly
Configuring autoscaling environments requires specifying minimum and maximum instance, VM, or container counts, along with the metric that triggers scaling. Getting this right isn't trivial — it requires running many near-production load tests and analyzing how your application behaves and how load balances across instances as it scales.
Some common pitfalls to avoid:
- Setting maxReplicas too low — if peak traffic demands 100 pods but your ceiling is 20, autoscaling becomes ineffective and the service degrades under load.
- Not defining resource requests on containers — HPA cannot calculate percentage usage without requests specified in the container spec.
-
Ignoring stabilization windows — without configuring
stabilizationWindowSecondson scale-down, your HPA may scale up and down repeatedly in response to short spikes, a behaviour called "flapping."
Scaling your application and underlying infrastructure properly increases both the availability and resilience of your services — a principle that holds in cloud native environments and traditional ones alike.
Serverless: Abstracting Infrastructure Away
If containers and Kubernetes abstract away physical servers, serverless computing abstracts away almost everything else — networks, virtual machines, operating systems, and load balancers. You provide the application code; the cloud provider selects the right environment to run it, allocates resources, scales it, and handles all the operational overhead.
This dramatically lowers the barrier to deploying applications. A developer building a simple web service or API no longer needs to configure infrastructure — they write and deploy code, and the platform handles the rest.
All major cloud providers offer commercial serverless runtimes: AWS Lambda, Google Cloud Functions, and Azure Functions are the most widely used. These also include subsets of serverless known as Function as a Service (FaaS), where individual functions are deployed and invoked on demand rather than running persistently.
Serverless places particular emphasis on on-demand provisioning — resources are allocated when a function is invoked and released immediately after it completes. This means you pay only for what you actually use, measured in milliseconds of execution time rather than hours of uptime.
The Cold Start Problem
One tradeoff worth understanding: serverless functions that haven't been invoked recently may experience a cold start — a brief delay as the platform provisions a new execution environment. For latency-sensitive applications, this is an important design consideration. Techniques like provisioned concurrency (on AWS Lambda) or keeping functions warm with scheduled pings can mitigate this.
Knative: Open Source Serverless on Kubernetes
Proprietary serverless offerings from cloud providers are powerful, but they come with a significant downside: vendor lock-in. Moving an application from AWS Lambda to Google Cloud Functions requires rethinking the entire deployment model.
Knative is the open source answer to this problem. Built on top of Kubernetes, Knative extends existing application platforms with serverless capabilities — allowing you to run serverless workloads on any Kubernetes cluster, whether on-premises, on a public cloud, or in a hybrid environment.
Knative handles infrastructure concerns including autoscaling (including scale to zero, where idle services consume no resources), routing, event delivery, and container building — letting developers focus on business logic rather than Kubernetes internals.
In October 2025, Knative officially graduated as a CNCF project, marking its readiness for widespread production use. Major cloud providers and enterprises including Alibaba Cloud and Scaleway have already adopted it to power serverless functions, AI inference models, and scalable automation platforms. The graduation also signals Knative's expanding role in AI workloads — an increasingly important use case as organizations run GPU-accelerated inference on Kubernetes.
CloudEvents: Solving the Vendor Lock-In Problem for Events
Many cloud providers and vendors offer proprietary event formats, making it difficult to move event-driven workloads between platforms or integrate across ecosystems.
To address this, the CloudEvents project was founded by CNCF. It provides a vendor-neutral specification for structuring event data in a common format, regardless of where the event originates or where it's consumed. The core idea is augmentation, not replacement — any existing event message flow can be slightly modified to adopt CloudEvents, meaning CloudEvents-enabled infrastructure can work seamlessly alongside non-CloudEvents infrastructure.
CloudEvents has developed SDKs in nine programming languages to help developers create and process events. Within the CNCF ecosystem, it is already adopted by Argo, Falco, Harbor, Knative, and Serverless Workflow, making it a foundational piece of the cloud native eventing layer.
Why this matters: Without a common event format, every integration between services or platforms requires custom glue code. CloudEvents standardizes this, much like how HTTP standardized web communication.
Putting It Together: Autoscaling + Serverless in Practice
These two concepts complement each other well in real-world cloud native architectures:
- A Kubernetes-based microservice might use HPA to scale pods horizontally based on CPU, KEDA to scale workers based on Kafka queue depth, and VPA to right-size resource requests automatically.
- A serverless function handles event-driven tasks (image processing, notifications, webhooks) without any persistent infrastructure — scaling from zero to thousands of concurrent executions in seconds.
- Knative bridges these two worlds — bringing serverless behaviour (scale to zero, event-driven execution) to teams who want to stay on Kubernetes without committing to a proprietary FaaS platform.
- CloudEvents ensures that events flowing between these components — and across cloud providers — follow a consistent, interoperable format.
Together, these patterns give engineering teams the tools to build systems that are not just scalable, but cost-efficient, resilient, and portable by design.
Learnings from the Linux Foundation's cloud native curriculum.
Top comments (0)