A virtual server benchmark is easy to run and surprisingly hard to interpret. A single attractive score can be caused by a short turbo window, a warm cache, an idle host, or a storage queue that does not resemble the application. A disappointing score can be equally misleading: perhaps another guest briefly competed for CPU time, package updates were running, or the test measured throughput when the workload actually depends on latency.
This article describes a small, repeatable method for evaluating a Linux VDS on KVM. It is not a ranking system and it does not produce one universal “performance number.” The goal is to separate three questions:
- Is the guest receiving predictable CPU time?
- How does one guest vCPU behave, and how well do several vCPUs scale?
- What latency does storage deliver at both low and moderate queue depth?
The method deliberately favors evidence that can be retained and reviewed: metadata, warm-up rules, repeated runs, latency percentiles, and raw output.
Start with a test contract
Before installing a benchmark, write down what decision the result must support. A web worker, a compilation runner, and a nightly database report stress different parts of a system. A Ryzen VDS configuration table can serve as one example of the source profile: record the listed vCPU count, memory, and storage allocation as test inputs, not as evidence of the outcome. Then add the workload’s runtime, concurrency, and I/O pattern before choosing any synthetic test.
Record the environment at the beginning of every test session:
date -u
uname -a
lscpu
nproc
free -h
lsblk -o NAME,TYPE,SIZE,FSTYPE,MOUNTPOINTS
findmnt -no SOURCE,FSTYPE,OPTIONS /
Also record the VDS plan, vCPU count, memory, guest kernel, filesystem, virtualization-visible CPU model, benchmark tool versions, test-file path and size, and the UTC start and end times. Do not infer the physical host topology from lscpu inside the guest. KVM exposes a virtual topology, and a provider may map or migrate vCPUs without making the underlying layout observable.
Choose the measurement window in advance. Avoid backups, package upgrades, log rotation, and application deployments unless those activities are intentionally part of the test. For a production guest, set explicit load and latency stop conditions. A benchmark that harms the workload it is meant to evaluate has failed operationally even if its data is technically valid.
Observe scheduling before measuring CPU
In a KVM guest, a runnable vCPU still needs a host CPU thread on which to run. Linux reports time when the guest wanted to run but the hypervisor scheduled something else as %steal. Capture a quiet baseline, then capture the same counters while the CPU test runs:
mkdir -p results
mpstat -P ALL 1 60 | tee results/mpstat-baseline.txt
vmstat 1 60 | tee results/vmstat-baseline.txt
mpstat -P ALL matters because an aggregate can hide one unstable vCPU. Save per-vCPU %usr, %sys, %iowait, and %steal. vmstat adds runnable tasks (r), blocked tasks (b), context switches, and a second view of CPU state.
Steal time is evidence, not a verdict. One non-zero sample may be harmless, and low steal does not prove that the CPU is dedicated. Look for coincidence: did throughput fall in the same seconds that steal rose? Does the pattern repeat in several runs or at different times of day? A sustained or recurrent correlation is more useful than a screenshot of one peak.
Do not interpret %iowait as disk latency. It is CPU accounting time during which a CPU was idle while I/O was outstanding. A system can have slow I/O and little iowait when other runnable work keeps the CPU busy. Storage latency must be measured from the I/O request and observed at the block layer.
Separate single-vCPU speed from scaling
A multi-threaded score mixes at least two properties: per-vCPU execution speed and the scheduler’s ability to run several vCPUs concurrently. Measure them separately. sysbench is convenient, but its version and parameters must be retained because scores from different builds or prime limits are not directly comparable.
First warm the code path, then run one worker pinned to one guest vCPU:
taskset -c 0 sysbench cpu --threads=1 --time=20 \
--cpu-max-prime=20000 run >/dev/null
taskset -c 0 sysbench cpu --threads=1 --time=60 \
--cpu-max-prime=20000 run | tee results/cpu-1t-run01.txt
taskset prevents the process from moving among guest vCPUs, which removes one source of variation. It does not pin the workload to a physical Ryzen core on the host; only the hypervisor operator can make that guarantee. Repeat the timed run at least five times. Rotate the selected guest CPU in a separate experiment if you want to detect an unusually noisy vCPU, but do not silently mix those results into the primary series.
Then test the intended concurrency. For a four-vCPU guest:
sysbench cpu --threads=4 --time=20 \
--cpu-max-prime=20000 run >/dev/null
sysbench cpu --threads=4 --time=60 \
--cpu-max-prime=20000 run | tee results/cpu-4t-run01.txt
Run mpstat -P ALL 1 in another shell during both tests. Report events per second and total events, plus the median, minimum, and maximum across repetitions. Scaling efficiency can be expressed as:
efficiency = multi-thread throughput / (single-thread throughput × worker count)
Efficiency below 100% is normal because of scheduling, shared caches, memory bandwidth, and benchmark overhead. The useful signal is reproducibility. Stable single-thread results with volatile multi-thread results and matching steal spikes suggest scheduling contention. Stable results at both thread counts make that explanation less likely.
Do not compare a 30-second result on one server with a 10-minute result on another. Boost behavior, thermal state, host power policy, and contention can change over time. Keep duration, worker count, affinity policy, tool version, and warm-up identical.
Measure storage without turning a read test into a write test
Use fio only against a dedicated test file that already exists and was provisioned during a maintenance window. Never point a casual benchmark at a database file, a mounted block device, or an unknown path. Verify the target before every run:
realpath /srv/benchmark/fio-test.bin
stat /srv/benchmark/fio-test.bin
findmnt -T /srv/benchmark/fio-test.bin
fio --version
The following low-queue-depth test is read-only. --readonly is a safety check that rejects write or trim workloads; it is worth keeping even when --rw=randread is already specified.
fio --name=randread-4k-q1 \
--filename=/srv/benchmark/fio-test.bin \
--readonly --rw=randread --bs=4k \
--ioengine=libaio --direct=1 \
--iodepth=1 --numjobs=1 \
--time_based=1 --runtime=90 --ramp_time=20 \
--randrepeat=1 --randseed=20260919 \
--lat_percentiles=1 --percentile_list=50:95:99:99.9 \
--group_reporting --output-format=json \
--output=results/fio-4k-q1-run01.json
--direct=1 reduces page cache effects, but it does not bypass every cache in the storage path. --ramp_time=20 gives the system a warm-up interval before statistics are collected. A fixed random seed makes the access sequence repeatable; that improves controlled comparison but can favor an upstream cache on later runs. State explicitly whether the objective is cold behavior, warmed steady state, or both. Do not claim “raw disk” performance from a virtual guest.
Low queue depth is useful for latency-sensitive operations. Add a separate test at a queue depth and job count that resemble the application—for example QD32 for a deliberately concurrent workload—but do not replace QD1 with it. High concurrency can produce impressive IOPS while individual requests wait longer.
During each fio run, observe the guest block layer:
iostat -xz 1 120 | tee results/iostat-fio-q1-run01.txt
In extended iostat output, r_await is the average time for read requests, including queue and service time. aqu-sz shows the average queue length. %util can be informative, but in a virtual or parallel storage stack it is not a universal saturation gauge. Interpret these fields together with fio latency and IOPS, not in isolation.
Prefer percentiles to averages
An average conceals the tail. If 99 reads finish in 0.5 ms and one takes 100 ms, the mean does not describe the pause that a request-sensitive application experiences. Retain at least p50, p95, p99, and preferably p99.9 latency from fio. Keep the units visible: depending on the output and version, latency values may be represented in nanoseconds or microseconds.
Percentiles within a run answer “how were individual requests distributed?” Repetitions answer a different question: “how stable is the environment across runs?” Do not compute a persuasive-looking p99 from only five aggregate scores. For each scenario, report the median result across at least five runs, along with its minimum and maximum. Preserve the within-run p95 and p99 from every raw JSON file.
Run the series in more than one time window if neighbor activity is part of the risk being evaluated. Use the same sequence, or alternate scenario order to prevent every QD32 test from always inheriting the warmest cache. Note every deviation rather than deleting an inconvenient run. Exclude a run only by a rule defined before testing, such as an OS update process appearing in the log.
Keep a reproducible evidence bundle
A useful result directory is understandable months later:
results/
├── metadata.txt
├── mpstat-baseline.txt
├── cpu-1t-run01.txt
├── cpu-4t-run01.txt
├── fio-4k-q1-run01.json
├── iostat-fio-q1-run01.txt
└── notes.md
Store exact commands in notes.md. Hash the raw files after the session so later processing cannot silently change the evidence:
sha256sum results/* > results/SHA256SUMS
When comparing two VDS configurations, change one factor at a time and use the same guest image, kernel, filesystem, test-file size, duration, and tool versions. Raw data should accompany summary tables. A chart without the command and source output is an illustration, not a reproducible benchmark.
Turn observations into decisions
Avoid universal pass/fail numbers. Define thresholds from the application’s service objective, then use patterns such as these:
| Observation | Plausible interpretation | Next check |
|---|---|---|
Single-thread throughput is stable; multi-thread throughput varies with %steal
|
Host scheduling contention is affecting parallel work | Repeat in another time window and correlate per-vCPU steal with each run |
| CPU results are stable; QD1 p99 read latency is high | Latency-sensitive storage work may be at risk | Compare with an application trace and inspect r_await and queue depth |
| QD1 latency is acceptable; QD32 IOPS is high but p99 grows sharply | Storage benefits from concurrency at a tail-latency cost | Cap application concurrency and test the intended queue depth |
fio latency and iostat r_await rise together |
Delay is visible through the guest block path | Check queue growth, other guest I/O, and time-of-day repetition |
iostat is busy but the benchmark remains stable |
Another workload may share the device without yet harming the test | Identify the process and repeat in a controlled window |
| Results drift after warm-up | Boost, caching, throttling, or background activity may dominate short tests | Extend runtime and retain a time series instead of one total |
The final decision should name the workload and its tolerance: for example, “p99 4 KiB read latency must remain below the application’s measured limit while four CPU workers meet the batch deadline.” That is defensible. “This VDS scored X” is not.
A benchmark as a controlled argument
A credible VDS benchmark does not need dozens of tools. It needs a declared question, a safe target, observable scheduling, separate single- and multi-vCPU tests, storage tests at realistic queue depths, warm-up, repetitions, and raw evidence. On KVM, %steal helps explain missing CPU time; it does not explain everything. For storage, await and average latency provide context, while p95 and p99 reveal the pauses users are more likely to notice.
The most valuable outcome is not the highest number. It is a result that another engineer can reproduce, challenge, and connect to a real workload.
Disclosure
This article was drafted with AI assistance. Commands, tool behavior, and safety assumptions should be checked against the versions installed on the target system and validated in a non-production environment before use.
Top comments (0)