Reading csperf JSON: The Real Performance Artifact

Why this lesson exists

When developers glance at the colorful HTML report, they often take the numbers at face value. But the HTML is just a pretty wrapper around a machine‑readable JSON artifact that contains every metric, every timestamp, and every piece of metadata the tool captured. If you ignore the JSON, you lose the ability to audit, reproduce, or share the evidence. This episode shows how to read the JSON, why it matters, and how to turn it into trustworthy performance claims.

Recap — where we are in the series

• Ep 1: Why a Single ./a.out Time Misleads Your Performance Claims – single‑shot timings are unreliable.
• Ep 2: Warm vs Cold: Why a Single Trial Misleads Performance Claims – warm‑up and repeat runs give reproducible data.
• Ep 3: Screenshots Aren’t Evidence: Use csperf for Real Performance Artifacts – screenshots are fragile; csperf artifacts are reproducible.
• Ep 4: Comparing Across Machines: Why Metadata Matters in csperf – metadata enables honest cross‑machine comparisons.
• Ep 5: csperf: A Lightweight Observatory for Honest Performance Tracking – observatory commands give quick evidence.
• Ep 6: csperf Doctor: First Command to Diagnose Toolchain Issues – validate the toolchain before blaming performance.
• Ep 7: csperf Quickstart: 30‑Second First Win – generate a reproducible artifact in half a minute.

The misconception

Many people still think the HTML view is the definitive source of truth. They copy the numbers from the table, post them on a forum, and assume anyone can reproduce the same result. In reality, the HTML is just a rendering of the JSON. If the JSON is corrupted, incomplete, or mis‑interpreted, the HTML will mislead.

What problem csperf solves (this episode's slice)

csperf’s JSON artifact is the single source of truth. It contains:

• All raw metrics (cycles, cache misses, IPC, etc.)
• The exact compiler and backend used
• The machine’s hardware and OS metadata
• The exact command line that produced the binary By inspecting the JSON you can:
• Verify that the numbers match what you see in the HTML
• Re‑run the analysis on a different machine
• Feed the data into your own dashboards or CI pipelines

Mental model

Think of the JSON as a data lake for performance. The HTML is a dashboard that queries the lake. If you want to audit the lake, you need to look at the raw files.

Lab: install and first commands

1. Install csperf (if you haven’t already):

   pip install csperf

1. Run the matrix traversal example and output JSON:

   csperf run \
     --input examples/cpp/matrix_traversal.cpp \
     --backend cpu \
     --warmup-runs 1 \
     --repeat-runs 5 \
     --output results/first.json

Why 1 warm‑up? We want to warm the caches but keep the run short.

1. Profile the JSON to generate the HTML and CSV:

   csperf profile results/first.json

The profile command reads the JSON, produces results/first.html, results/first.csv, and results/first.xlsx.

Lab: what we ran on this machine

Item	Value
Hostname	f4c59d864117
Date	2026‑10‑03T18:00:02+05:30
OS	Linux 7.0.0‑34‑generic
CPU	AMD Ryzen 7 9700X 8‑Core
Core count	16
Backend	cpu
Warm‑up runs	1
Repeat runs	5
Compiler	clang++ (LLVM 18)
Optimization	-O3
Language	C++17

Results (real numbers only)

The JSON artifact (csperf/first.json) contains the following key metrics:

Metric	Value
execution_time_ms	5.1566
execution_time_summary_ms.count	5
execution_time_summary_ms.min	5.15
execution_time_summary_ms.max	5.169
execution_time_summary_ms.mean	5.1566
execution_time_summary_ms.median	5.153
execution_time_summary_ms.stdev	0.007701
cpu_cycles	6,664,368
ref_cycles	4,577,974
frontend_stall_cycles	1,310,706
instruction_count	12,226,149
branch_instructions	1,706,962
branch_mispredictions	13,217
cache_references	586,576
cache_misses	98,637
l1_cache_misses	235,506
l1_icache_misses	33,111
dtlb_load_misses	4,534
itlb_load_misses	286
amd_l2_ic_dc_miss_in_l2	84,825
ipc	1.834555
l2_cache_misses	84,825

These numbers are the truth that the HTML table is derived from.

How to read the artifacts

1. Open the JSON in a text editor or a JSON viewer.
2. Locate the metrics section – this is where all raw numbers live.
3. Cross‑check with the HTML – the HTML table should mirror the values in metrics.
4. Use the hardware section to understand the environment: CPU model, OS, and available counters.
5. Inspect the pipeline to see the exact compiler steps and backend used.
6. Leverage the artifacts section to find the binary and CSV for further analysis.

If you need to programmatically extract a metric, you can use jq:

jq '.metrics.execution_time_ms' results/first.json

Common mistakes (teacher checklist)

Mistake	Why it matters	Fix
Relying only on the HTML table	HTML can be stale or mis‑rendered	Always read the JSON first
Ignoring the hardware metadata	Different CPUs can skew results	Compare cpu_model and cpu_architecture
Skipping the pipeline section	You may not know which compiler flags were used	Verify commands and compiler_steps
Using the JSON without a schema	Manual parsing can introduce bugs	Use the csperf CLI or a JSON schema validator
Assuming a single run is enough	Warm‑up and repeat runs are essential	Keep warmup-runs ≥1 and repeat-runs ≥3

Try this next (homework)

Run the same matrix traversal example on a different backend (e.g., --backend llvm) and compare the JSON artifacts. Look for differences in cpu_cycles, ipc, and cache metrics. Do this tonight — Episode 9 starts by assuming you did.

Closing

You now know that the JSON artifact is the source of truth for performance claims. By reading it, you can audit, reproduce, and share honest evidence. Next, we’ll dive into choosing the right backend to surface the metrics that matter.

The series so far

• Ep 1: Why a Single ./a.out Time Misleads Your Performance Claims
• Ep 2: Warm vs Cold: Why a Single Trial Misleads Performance Claims
• Ep 3: Screenshots Aren’t Evidence: Use csperf for Real Performance Artifacts
• Ep 4: Comparing Across Machines: Why Metadata Matters in csperf
• Ep 5: csperf: A Lightweight Observatory for Honest Performance Tracking
• Ep 6: csperf Doctor: First Command to Diagnose Toolchain Issues
• Ep 7: csperf Quickstart: 30‑Second First Win
• Ep 8: Reading csperf JSON: The Real Performance Artifact (this article)