AKS and Near-Bare-Metal Workloads: What Platform Teams Can Responsibly Plan For

Every few months a team shows up with a workload that has opinions about hardware. GPU inference, high-frequency packet processing, latency-sensitive financial systems — something that makes a shared-tenant VM feel like a traffic jam. The conversation usually ends up at "can we get bare-metal nodes on AKS?"

The honest answer right now: partially, with caveats, and you should not trust anyone who gives you a confident full answer without pointing at current documentation. This post is about what you can design around today, what requires validation before you commit to it, and where the gaps are that will bite you if you skip due diligence.

A note on sources: The AKS core concepts and product overview documentation available at time of writing covers cluster and workload fundamentals¹² but does not address bare-metal or near-bare-metal node pool specifics. Claims in that category are marked NEEDS_VALIDATION throughout. Everything else is grounded in general AKS behavior that is broadly consistent across the platform.

---

Why "Bare-Metal on AKS" Is Complicated Terminology

AKS runs on Azure VMs. Azure VMs run on Azure's physical fleet. The question is how much of the hypervisor overhead and hardware sharing model you can escape.

Azure offers VM SKU families that are designed to minimize virtualization overhead — isolated VM sizes that occupy an entire physical host, giving you a single-tenant hardware environment. Whether specific SKUs in this category are available as AKS node pool targets depends on regional availability and AKS's supported VM size list. NEEDS_VALIDATION — check the current AKS supported VM sizes documentation for your target region before designing around a specific SKU.

What you get with isolated or near-bare-metal SKUs, if available:

• Predictable NUMA topology — no neighbor noise on the physical host
• Full access to hardware features — relevant for SR-IOV, DPDK, and accelerated networking scenarios (NEEDS_VALIDATION for specific feature compatibility with AKS CNI configurations)
• Single-tenant physical isolation — meaningful for compliance frameworks that require it

What you do NOT automatically get:

• A different version of Kubernetes
• Different node lifecycle behavior
• Escape from AKS's node auto-repair or OS upgrade mechanics

That last point is where platform teams usually get surprised.

---

Node Lifecycle Is Still AKS's Domain

AKS manages node health and upgrades at the node pool level¹. This does not change because you picked a specialty VM SKU. If auto-repair decides a node is unhealthy and reprovisioned it, the replacement VM must come from available capacity of the same SKU. For isolated VM families, that capacity is often constrained — regionally and numerically.

The practical consequence: your cluster auto-repair and upgrade assumptions need re-evaluation for specialty node pools.

For standard node pools on commodity SKUs, AKS node upgrade behavior is well-documented and we've covered it in depth in our prior upgrade series (#2, #42, #50). The short version is that AKS handles node image upgrades via cordon-drain-replace, and you control the surge buffer and max unavailable settings. That mechanical behavior is the same regardless of SKU. What changes is the risk profile:

• Capacity risk: If your isolated SKU has low regional availability, a surge node during upgrade may not provision. The upgrade stalls.
• Drain timing risk: Pods on high-performance hardware often have longer graceful termination windows. Make sure your terminationGracePeriodSeconds reflects that.
• OS upgrade compatibility: Node OS image updates on specialty hardware SKUs may lag behind standard SKU rollouts. NEEDS_VALIDATION — check AKS release notes for your target SKU family.

---

Node Pool Isolation Strategy

If you're mixing standard workloads and hardware-sensitive workloads in the same cluster (which is a reasonable cost efficiency decision), node pool isolation is non-negotiable. This is standard AKS practice¹ and applies regardless of whether the specialty pool is "bare-metal adjacent" or not.

The pattern:

1. Separate node pools by workload class — one pool for standard platform services, one for hardware-intensive workloads
2. Taint specialty node pools — prevent accidental scheduling of standard workloads onto expensive hardware
3. Use node selectors and tolerations explicitly — don't rely on the absence of a node being "wrong enough" to avoid it
4. Set resource requests and limits that reflect actual hardware — on high-performance SKUs with full NUMA access, misconfigured limits can be worse than no limits

Here's a minimal node pool configuration pattern for a tainted specialty pool:

# Create a specialty node pool with a taint to prevent accidental scheduling
# Replace <SKU_NAME> with your validated target SKU
az aks nodepool add \
  --resource-group myRG \
  --cluster-name myCluster \
  --name specialtypool \
  --node-vm-size <SKU_NAME> \ # NEEDS_VALIDATION: confirm SKU is AKS-supported
  --node-count 2 \
  --node-taints workload-class=hardware-intensive:NoSchedule \
  --labels workload-class=hardware-intensive \
  --zones 1 2 3

And the corresponding pod spec side:

apiVersion: v1
kind: Pod
metadata:
  name: hardware-sensitive-workload
spec:
  tolerations:
    - key: "workload-class"
      operator: "Equal"
      value: "hardware-intensive"
      effect: "NoSchedule"
  nodeSelector:
    workload-class: hardware-intensive
  containers:
    - name: app
      image: your-registry/your-image # pin your digest, don't use latest
      resources:
        requests:
          cpu: "8"
          memory: "32Gi"
        limits:
          cpu: "8"
          memory: "32Gi"

---

Concrete Steps Before Committing to a Specialty Node Pool

1. Validate SKU availability in your target region. Run az vm list-skus --location <region> --size <sku-prefix> --output table and cross-reference against the AKS supported VM sizes list. Do not assume a SKU that works for standalone VMs is available for AKS node pools.

2. Check current AKS release notes for the SKU family. Look specifically for known issues with node auto-repair, OS image compatibility, or accelerated networking feature support. NEEDS_VALIDATION — this is release-cadence dependent.

3. Model your capacity headroom. For upgrade surge and auto-repair replacement, you need available quota and available physical capacity. For isolated SKUs, physical capacity is the harder constraint. File a support ticket to confirm capacity commitments if the workload is business-critical.

4. Test node pool upgrade behavior in a non-production cluster first. Don't discover that your specialty SKU has zero surge capacity during a production Kubernetes version upgrade. The upgrade series we've covered previously (#50 is the most current approved version) gives you the tooling — apply the same runbook to a test specialty pool before you need it for real.

5. Define your eviction and rescheduling strategy. If a node is auto-repaired and the replacement takes longer than expected (capacity contention), what happens to your workload? PodDisruptionBudgets should reflect the actual tolerance of the workload, not the default.

6. Align with your golden path guardrails. If your platform has golden path templates (covered in #25), specialty node pools should either be an explicit supported variant or explicitly out-of-scope. The worst outcome is an undocumented workaround that every team discovers independently and implements differently.

---

What Not To Do

• Don't build production dependency on a feature that's in preview. AKS preview features can change or be dropped. If a specific bare-metal adjacent capability is currently in preview, treat it as R&D, not production infrastructure.
• Don't assume SKU-level isolation means network-level isolation. Physical host isolation and network security are different axes. Your CNI configuration and network policies still matter.
• Don't skip the PodDisruptionBudget. On hardware-intensive workloads, the instinct is often "this thing is stateful and complex, let's not let Kubernetes touch it." That instinct leads to nodes that can't be drained and upgrades that time out.

---

The Responsible Summary

AKS can host workloads on VM SKUs designed to reduce virtualization overhead and provide single-tenant hardware access. The exact SKUs available, their compatibility with specific AKS networking and acceleration features, and their behavior under node lifecycle operations all require validation against current documentation before you commit. NEEDS_VALIDATION on the specifics is not a cop-out — it's the difference between a platform that works and a platform that works until it doesn't.

What you can design now: isolation strategy, taint/toleration patterns, upgrade runbooks, and capacity planning processes. Get those right and the hardware specifics slot in cleanly once validated.

---

Sources

Architecture

flowchart TD
    A[Platform Team Evaluation] --> B{Workload Requirements}
    B --> C[Standard Latency\nGeneral Purpose VMs]
    B --> D[Low Latency / Hardware Intensive\nSpecialty VM SKUs]
    D --> E[Validate SKU Availability\nby Region]
    E --> F[Dedicated Node Pool\nIsolation Strategy]
    F --> G[Review Auto-Repair &\nOS Upgrade Behavior]
    G --> H[Node Pool Taints & Tolerations\nfor Workload Scheduling]
    C --> H
    H --> I[Mixed Cluster:\nStandard + Specialty Pools]
    I --> J[Ongoing: Monitor AKS\nRelease Notes & SKU Docs]

---

1. Azure Kubernetes Service (AKS) Core Concepts ↩

2. What is Azure Kubernetes Service (AKS)? ↩