From A10 to M60: An Architect's Journey into Azure GPU VM Sizing for Kubernetes Inference Workloads

How an unexpected regional constraint forced us to deeply understand Azure GPU VM families, naming conventions, and workload fit.

Introduction

As architects, we often assume that infrastructure decisions are straightforward:

"The workload is already running successfully in Region A. Let's deploy the same Kubernetes workload in Region B."

That's exactly what we thought.

Our workload consisted of a Visual Element Detection (VED) service hosted on Kubernetes. The application uses a PyTorch model to analyze images and detect various visual elements in an image file. The service was already running successfully on a node pool backed by Azure's NVads_A10_v5 GPU VMs.
Then we hit an unexpected challenge.
The target region did not offer NVads_A10_v5 instances.
What looked like a simple deployment exercise became a deep dive into Azure GPU virtual machine families, GPU architectures, VM naming conventions, and workload characteristics.
This article shares what I learned in the hope that it helps others who find themselves evaluating Azure GPU SKUs for AI inference workloads. I am relatively new to the world of MLOps, Model deployments, GPU Workloads etc and equally interested and excited to learn more on this front.

---

The Workload

Before discussing VM selection, let's understand the workload characteristics:

Model Type        : PyTorch
Model Size        : less than 200 MB (.pth)
Image Resolution  : ~2000 x 2000
Expected Throughput: 5-7 requests/sec
Platform          : AKS (Kubernetes)
Workload Type     : Inference only

This is important because GPU sizing should always start from the workload and not from the VM catalog.

Step 1: Understanding Azure GPU VM Families

Many engineers first encounter Azure GPU machines through names like:

NV12s_v3
NV6ads_A10_v5
NC4as_T4_v3
ND96isr_H100_v5

The naming can be intimidating.
The first breakthrough was understanding that Azure organizes GPU VMs into three primary families:

N-Series
├── NV
├── NC
└── ND

NV Series – Visualization and Graphics

NV-series VMs are designed primarily for:

• Virtual Desktop Infrastructure (VDI)
• Graphics rendering
• CAD/CAM workloads
• Remote workstations
• Visualization workloads
• Image processing

Examples:
NV12s_v32
NV24s_v33
NV6ads_A10_v54

Typical GPUs:
NVIDIA Tesla M60
NVIDIA A10
AMD MI25

Historically, these SKUs were optimized for graphics workloads, although many organizations now use them for lightweight AI inference workloads.

NC Series – Compute

NC-series VMs are optimized for:

• AI/ML workloads
• PyTorch
• TensorFlow
• CUDA applications
• HPC workloads
• Inference APIs

Examples:
NC4as_T4_v32
NC8as_T4_v33
NC_A100_v44
NCads_H100_v5

Typical GPUs:
T4
V100
A100
H100

For pure AI workloads, NC is often the most natural fit.

ND Series – Deep Learning at Scale

ND-series is designed for:

• Multi-GPU training
• Large Language Model training
• Distributed AI
• NVLink-enabled workloads
• InfiniBand-connected GPU clusters

Examples:
ND_A100_v42
ND_H100_v5

If you're training LLMs, ND is your friend.
If you're serving a 200 MB inference model, ND is probably overkill.

Step 2: Decoding Azure VM Names

One of the biggest learnings from this exercise was understanding Azure's VM naming scheme.
Let's decode the VM we were already using:
NV6ads_A10_v5
Breaking it apart:

Component	Meaning
NV	Visualization family
6	vCPU count
a	AMD processor
d	Local temporary disk
s	Premium SSD support
A10	NVIDIA A10 GPU
v5	Generation 5

Once you learn the modifiers, every Azure VM becomes easier to understand.

Common Suffixes You'll Encounter
as - NC4as_T4_v3
ads - NV6ads_A10_v5
adms - NV36adms_A10_v5
These memory-optimized variants generally provide significantly more RAM than their standard counterparts.

Common Letters we will usually see -

Letter	Meaning
a	AMD CPU
b	Higher storage bandwidth
d	Local/temp disk
e	Confidential/encrypted capabilities
m	Memory optimized
n	Network optimized
p	ARM processor
r	RDMA / InfiniBand
s	Premium SSD support

Step 3: Understanding Fractional GPUs

One detail many people miss about NVads_A10_v5 is that you may not receive a full GPU.
For example:
NV6ads_A10_v5 provides: 1/6th of an NVIDIA A10 ≈ 4 GB VRAM
Azure uses GPU partitioning to expose slices of the physical A10 GPU.

Conceptually:

A10 GPU (24 GB)
 ├─ VM1 -> 4 GB
 ├─ VM2 -> 4 GB
 ├─ VM3 -> 4 GB
 ├─ VM4 -> 4 GB
 ├─ VM5 -> 4 GB
 └─ VM6 -> 4 GB
``

This creates cost-effective GPU options, especially for inference workloads.

Step 4: Evaluating NVv3 as an Alternative

Because the A10-based SKUs were unavailable in our target region, we began evaluating:
NV12s_v3, NV24s_v3 and NV48s_v3
These machines use the much older - NVIDIA Tesla M60 GPU.
The key specifications:

SKU	GPUs	GPU Memory
NV12s_v3	1 × M60	16 GB
NV24s_v3	2 × M60	32 GB
NV48s_v3	4 × M60	64 GB

At first glance, moving from an A10 to an M60 looked risky.
However, after analyzing the workload, we realized something important:
The model itself was only less than 200MB so memory wasn't the challenge.
The real challenge was 2000 x 2000 image processing which drives compute consumption much more than model size.

Step 5: Right-Sizing the Kubernetes Node Pool

A common temptation in infrastructure planning is:

"Let's buy the biggest VM and move on."

We resisted that temptation.
For our workload:

• 5–7 requests/sec
• 163 MB model

• Inference only
  the most logical starting point became:
  NV12s_v3
  Why?

• 16 GB GPU memory

• Sufficient CPU capacity

• Lower cost

• Easier horizontal scaling in Kubernetes

Rather than using:
1 × NV48s_v3, we preferred the Kubernetes-native approach:

• Multiple smaller nodes
• Horizontal Pod Autoscaling
• Cluster Autoscaling This usually provides better resilience and resource efficiency.

Key Takeaways

If you're evaluating Azure GPU machines for inference workloads, remember:
Understand the family first
NV = Visualization
NC= Compute
ND= Deep Learning
Understand the suffixes
as = AMD + Premium SSD
ads = AMD + Local Disk + Premium SSD
adms = AMD + Local Disk + Memory Optimized + Premium SSD
Don't focus only on GPU names
A larger GPU doesn't automatically mean you need a larger VM.
Consider:

• Model size
• Input resolution
• Throughput requirements
• Concurrency levels
• Kubernetes scaling strategy

Start with measurement using Benchmark:
nvidia-smi

Monitor:

• GPU utilization
• Memory usage
• Inference latency
• Requests/sec

Then scale based on evidence.

Final Thoughts

What began as a regional availability issue became a valuable learning opportunity.
We started with a simple question:

"What should we use if NVads_A10_v5 isn't available?"
We ended up gaining a much deeper understanding of:

1. Azure GPU VM families
2. GPU partitioning
3. A10 vs M60 trade-offs
4. AKS node pool sizing
5. Azure VM naming conventions

As architects, these are the moments that help us move beyond simply selecting SKUs and toward making informed infrastructure decisions based on workload characteristics.
Sometimes the best architecture lessons come from constraints rather than from choice.

Have you had to migrate inference workloads between Azure regions and deal with GPU availability differences? I'd love to hear what SKUs and strategies worked for your teams.