When you submit a job to an HPC cluster, you might request 4 CPUs, 8 CPUs, or perhaps an entire compute node.
But what does that actually mean?
Does one CPU mean one physical processor? Does one core mean one CPU? And where do threads fit into the picture?
Understanding these terms makes it much easier to understand how Slurm allocates resources and how your application uses them.
1. What is a CPU?
A CPU is the processor that executes instructions. In an HPC compute node, you might find one or two physical processor packages installed in the server.
For example, a server could have two Intel Xeon processors or two AMD EPYC processors.
Each physical processor contains multiple cores. These cores allow the processor to execute several tasks in parallel.
Think of the physical processor as a building and the cores as individual workers inside it.
2. What is a core?
A core is an independent processing unit within a physical processor.
Imagine a compute node with two processors, each containing 32 physical cores.
The total number of physical cores is:
2 processors × 32 cores = 64 physical cores.
This gives the node 64 physical cores available to execute workloads.
More cores can help applications run faster when they can divide their work into parallel tasks. However, adding cores does not automatically make every application faster.
Some applications are limited by memory bandwidth, storage performance, communication overhead, or work that cannot be parallelised.
3. What is a thread?
The word thread can mean two different things in HPC.
The first is a hardware thread. Modern Intel and AMD processors can support simultaneous multithreading, allowing one physical core to expose multiple logical CPUs to the operating system.
Intel calls its implementation Hyper Threading. AMD commonly refers to its implementation as SMT.
For example, if a server has 64 physical cores and each core supports two hardware threads, Linux may report 128 logical CPUs.
These are not 128 physical cores. They are 128 hardware execution contexts sharing 64 physical cores.
The second meaning is a software thread. An application can create multiple software threads to divide its work. These threads are scheduled onto the available hardware threads by the operating system.
The distinction matters because requesting more CPUs does not necessarily mean your application will create more software threads. Your application must be designed or configured to use them.
4. How does this work on Intel and AMD systems?
Both Intel Xeon and AMD EPYC processors follow the same basic concepts, although their architectures and features vary by processor generation and model.
Imagine an Intel based server with two Xeon processors. Each processor has 32 physical cores, giving the server 64 physical cores in total.
Now imagine an AMD based server with two EPYC processors. Each processor has 48 physical cores, giving the server 96 physical cores in total.
If both systems support two hardware threads per core, Linux could report 128 logical CPUs on the Intel server and 192 on the AMD server.
These are illustrative examples, not specifications for particular processor models.
Notice that the number of logical CPUs is higher than the number of physical cores. This is because Intel Hyper Threading and AMD SMT allow each physical core to expose two hardware threads to the operating system.
However, more logical CPUs do not automatically mean better HPC performance. The actual results depend on the application, memory bandwidth, processor architecture, and how efficiently the workload uses available resources.
That is why HPC administrators need to understand both the hardware and the workload before deciding how resources should be allocated.
5. What does this mean for an HPC job?
Suppose you submit a simple Slurm job requesting four CPUs.
Your job script might look like this:
#!/bin/bash
#SBATCH --job-name=cpu-test
#SBATCH --ntasks=1
#SBATCH --cpus-per-task=4
#SBATCH --time=00:05:00
echo "Running on $(hostname)"
echo "Allocated CPUs: $SLURM_CPUS_PER_TASK"
sleep 60
Submit it using:
sbatch cpu-test.sh
Here is what the important settings mean.
--ntasks=1 requests one task, which is commonly one process.
--cpus-per-task=4 requests four CPUs for that task.
This is useful for an application that uses one process with multiple software threads, such as an appropriately configured OpenMP application.
However, the script above only runs sleep. It does not create four computational threads or perform parallel calculations. It simply demonstrates the resource request.
Slurm allocates resources according to the cluster configuration and scheduling policy. Whether those CPUs correspond to distinct physical cores or hardware threads depends on the node configuration, Slurm settings, and allocation policy.
6. How can you check the actual hardware?
On a Linux compute node, run:
lscpu
Look for these fields:
CPU(s):
Thread(s) per core:
Core(s) per socket:
Socket(s):
For example:
CPU(s): 128
Thread(s) per core: 2
Core(s) per socket: 32
Socket(s): 2
This indicates 128 logical CPUs, two hardware threads per core, 32 cores per processor package, and two processor packages.
To see how Linux maps logical CPUs to physical cores, use:
lscpu -e
You can also inspect the resources Slurm recognises with:
scontrol show node compute01
Compare the operating system view with Slurm's view. They should make sense together, although Slurm may intentionally allocate only part of the available hardware.
The key takeaway
A physical processor contains cores. A core can expose one or more hardware threads. Applications can create software threads that execute on those hardware resources.
When working with HPC, remember that the number of logical CPUs is not necessarily the number of physical cores, and the number of CPUs requested from Slurm is not necessarily the number of software threads your application will use.
Understanding that difference is one of the first steps towards understanding how HPC jobs actually run.
Top comments (0)